Progressive visual analytics (PVA) allows analysts to maintain their flow during otherwise long-running computations by producing early, incomplete results that refine over time, for example, by running the computation over smaller partitions of the data. These partitions are created using sampling, whose goal it isto draw samples of the dataset such that the progressive visualization becomes as useful as possible as soon as possible. What makes the visualization useful depends on the analysis task and, accordingly, some task-specific sampling methods have been proposed for PVA to address this need. However, as analysts see more and more of their data during the progression, the analysis task at hand often changes, which means that analysts need to restart the computation to switch the sampling method, causing them to lose their analysis flow. This poses a clear limitation to the proposed benefits of PVA. Hence, we propose a pipeline for PVA-sampling that allows tailoring the data partitioning to analysis scenarios by switching out modules in a way that does not require restarting the analysis. To that end, we characterize the problem of PVA-sampling, formalize the pipeline in terms of data structures, discuss on-the-fly tailoring, and present additional examples demonstrating its usefulness.

Tailorable Sampling

Tailorable sampling for progressive visual analytics

We present a novel multi-level representation of time series called OM3 that facilitates efficient interactive progressive visualization of large data stored in a database and supports various interactions such as resizing, panning, zooming, and visual query. Based on our proposed line-segment aggregation, this representation can produce error-free line visualizations that preserve the shape of a time series in windows of arbitrary sizes. To reduce the interaction latency, we develop an incremental tree-based query strategy to support progressive visualizations, allowing a finer control on the accuracy-time tradeoff. We quantitatively compare OM3 with state-of-the-art methods, including a method implemented on a leading time-series database InfluxDB, in two settings with databases residing either in the local area network or on the cloud. Results show that OM3 maintains a low latency within 300 ms on the web browser and a high data reduction ratio regardless of the data size (ranging from millions to billions of records), achieving around 1,000 times faster than the state-of-the-art methods on the largest dataset experimented with.

OM3: An Ordered Multi-level Min-Max Representation for Interactive
                  Progressive Visualization of Time Series

Temporal evolving set data are time-varying and growing ubiquitous in person re-identification, parameter choice, and streaming data analysis. We construct a workflow to analyze and explore the inconspicuous pattern between multiple nonadjacent sets in temporal evolving set data. We propose a progressive timeline layout algorithm based on a mathematical optimization model to place the set element after data update, our layout algorithm can calculate the coordinates of elements in a short time and preserve the distance ratio between elements. To relax the visual clutter when visualizing sets’ relationships, we design two types of pattern enhancement strategies and their combinations: optimization-based pattern enhancement strategy and design-based pattern enhancement strategy. We conduct a comprehensive evaluation to verify and compare our pattern enhancement strategies including a quantitative experiment, two case studies, and an informal user study. The results show that our pattern enhancement strategies can effectively help users identify inconspicuous patterns. Our workflow and strategies show broad application prospects and we hope it could be a fundamental component in data projection pattern mining and streaming data analysis.

Temporal Evolving

Towards better pattern enhancement in temporal evolving set visualization

Recent advances in information visualization have shown that building proper structures to allow efficient lookup in the data can reduce significantly the time to build graphical representation of very large data sets, when compared to the linear scanning of the data. We present BiVis, a visualization technique that shows how such techniques can be further improved to reach a rendering time compatible with continuous interaction. To do so, we turn the lookup into an anytime algorithm compatible with a progressive visualization: a visualization presenting an approximation of the data and an estimation of the error can be displayed almost instantaneously and refined in successive frames until the error converges to zero. We also leverage the spatial coherency of the navigation: during the interaction, the state of the (possibly partial) lookup for the previous frames is reused to bootstrap the lookup for the next frame despite the view change. We show that those techniques allow the interactive exploration of out-of-core time series consisting of billions of events on commodity computers.

BiVis

BiVis: Interactive and Progressive Visualization of Billions (and
                  Counting) Items

Molecular Dynamics (MD) simulations are ubiquitous in cutting-edge physio-chemical research. They provide critical insights into how a physical system evolves over time given a model of interatomic interactions. Understanding a system’s evolution is key to selecting the best candidates for new drugs, materials for manufacturing, and countless other practical applications. With today’s technology, these simulations can encompass millions of unit transitions between discrete molecular structures, spanning up to several milliseconds of real time. Attempting to perform a brute-force analysis with data-sets of this size is not only computationally impractical, but would not shed light on the physically-relevant features of the data. Moreover, there is a need to analyze simulation ensembles in order to compare similar processes in differing environments. These problems call for an approach that is analytically transparent, computationally efficient, and flexible enough to handle the variety found in materials-based research. In order to address these problems, we introduce MolSieve, a progressive visual analytics system that enables the comparison of multiple long-duration simulations. Using MolSieve, analysts are able to quickly identify and compare regions of interest within immense simulations through its combination of control charts, data-reduction techniques, and highly informative visual components. A simple programming interface is provided which allows experts to fit MolSieve to their needs. To demonstrate the efficacy of our approach, we present two case studies of MolSieve and report on findings from domain collaborators.

MolSieve

MolSieve: A Progressive Visual Analytics System for Molecular Dynamics
                  Simulations

Many real-world applications involve analyzing time-dependent phenomena, which are intrinsically functional, consisting of curves varying over a continuum (e.g., time). When analyzing continuous data, functional data analysis (FDA) provides substantial benefits, such as the ability to study the derivatives and to restrict the ordering of data. However, continuous data inherently has infinite dimensions, and for a long time series, FDA methods often suffer from high computational costs. The analysis problem becomes even more challenging when updating the FDA results for continuously arriving data. In this paper, we present a visual analytics approach for monitoring and reviewing time series data streamed from a hardware system with a focus on identifying outliers by using FDA. To perform FDA while addressing the computational problem, we introduce new incremental and progressive algorithms that promptly generate the magnitude-shape (MS) plot, which conveys both the functional magnitude and shape outlyingness of time series data. In addition, by using an MS plot in conjunction with an FDA version of principal component analysis, we enhance the analyst's ability to investigate the visually-identified outliers. We illustrate the effectiveness of our approach with two use scenarios using real-world datasets. The resulting tool is evaluated by industry experts using real-world streaming datasets.

Hardware System Monitoring

A Visual Analytics Approach for Hardware System Monitoring with Streaming Functional Data Analysis

Urban traffic data records people’s individual movement and behavioral histories, which implies underlying human mobility patterns. How to effectively mine this pattern is the key to the building of a smart city. Traditional visual analysis methods only support exploration based on mobility patterns with fixed granularity, which limits the possibility of knowledge discovery at the beginning of analysis. However, the methods based on multi-granularity mobility patterns proposed in recent years have varying degrees of deficiencies in human perception or analysis efficiency. To balance the above problems, we design a comprehensive visual analysis system that supports both adaptive granularity refinement and complex semantic exploration of human mobility patterns. We introduce a hierarchical topic model H-NMF to extract multi-granularity traffic topics to capture mobility patterns with different levels of detail. Then, we design a progressive granularity refinement visualization method based on interactive Sankey diagram, which is more in line with the top-down cognitive process in human perception. The user can follow the visual cues provided by our system and gradually make decisions to obtain the desired granularity of traffic topics even for dataset without any prior knowledge. We further optimize the analysis efficiency of the refining process by taking the complete visual analysis process of traffic topics into consideration. The tensor decomposition model is used to extract the general spatio-temporal features of traffic topics to help users quickly obtain the basic semantics of traffic topics, which effectively reduces the analysis cost of users in the refinement. Besides, we design rich detail analysis visualizations to help users fully explore the semantics of traffic topics. All the above designs are well integrated in our visual analysis system which contains clear visualizations and user-friendly interactions. Finally, we verify the effectiveness of our method on real-world dataset with two case studies.

Hierarchical Topic Refinement

Progressive visual analysis of traffic data based on hierarchical topic refinement and detail analysis

Progressive visual analytics allows users to interact with early, partial results of long-running computations on large datasets. In this context, computational steering is often brought up as a means to prioritize the progressive computation. This is meant to focus computational resources on data subspaces of interest so as to ensure their computation is completed before all others. Yet, current approaches to select a region of the view space and then to prioritize its corresponding data subspace either require a one-to-one mapping between view and data space, or they need to establish and maintain computationally costly index structures to trace complex mappings between view and data space. We present steering-by-example, a novel interactive steering approach for progressive visual analytics, which allows prioritizing data subspaces for the progression by generating a relaxed query from a set of selected data items. Our approach works independently of the particular visualization technique and without additional index structures. First benchmark results show that steering-by-example considerably improves Precision and Recall for prioritizing unprocessed data for a selected view region, clearly outperforming random uniform sampling.

Steering-by-example

Steering-by-example for Progressive Visual Analytics

Numerical simulation is crucial in scientific research. Visualizing simulation data helps scientists understand data and discover connections between data. However, the development of supercomputers brings larger amounts of simulation data, which poses huge challenges for visualization: (1) Storing the whole data used for visualization is hard under the limitation of I/O and storage, making visualization technology impractical; (2) the post-processing visual analysis mode wastes computation resources and reduces analysis efficiency. In this work, we present an in situ progressive visual analytics system, PEViz, which contains the back-end in situ processing and the interactive front-end interface. The in situ processes using real-time analysis and visualization reduce data storage, leading to a smaller data scale for visualization. The visual analytic interface has four aspect views: the parameters view, the statistics views of the ensemble, the one-moment view of all members, and the sequence view of a single member. While the simulation analytics programs are still running, the progressive analytic system allows scientists to explore data space, such as early analysis and interactively ending abnormal analysis. In addition, the system assists users in processing analytical tasks through real-time visualization and abstracting knowledge of the focused object. We present two case studies to evaluate the effectiveness of the system.

PEViz

PEViz: an in situ progressive visual analytics system for ocean ensemble data

We conduct a user study to quantify and compare user performance for a value comparison task using four bar chart designs, where the bars show the mean values of data loaded progressively and updated every second (progressive bar charts). Progressive visualization divides different stages of the visualization pipeline—data loading, processing, and visualization—into iterative animated steps to limit the latency when loading large amounts of data. An animated visualization appearing quickly, unfolding, and getting more accurate with time, enables users to make early decisions. However, intermediate mean estimates are computed only on partial data and may not have time to converge to the true means, potentially misleading users and resulting in incorrect decisions. To address this issue, we propose two new designs visualizing the history of values in progressive bar charts, in addition to the use of confidence intervals. We comparatively study four progressive bar chart designs: with/without confidence intervals, and using near-history representation with/without confidence intervals, on three realistic data distributions. We evaluate user performance based on the percentage of correct answers (accuracy), response time, and user confidence. Our results show that, overall, users can make early and accurate decisions with 92% accuracy using only 18% of the data, regardless of the design. We find that our proposed bar chart design with only near-history is comparable to bar charts with only confidence intervals in performance, and the qualitative feedback we received indicates a preference for designs with history.

Early Decision Making

Studying Early Decision Making with Progressive Bar Charts

Existing systems dealing with the increasing volume of data series cannot guarantee interactive response times, even for fundamental tasks such as similarity search. Therefore, it is necessary to develop analytic approaches that support exploration and decision making by providing progressive results, before the final and exact ones have been computed. Prior works lack both efficiency and accuracy when applied to large-scale data series collections. We present and experimentally evaluate ProS, a new probabilistic learning-based method that provides quality guarantees for progressive nearest neighbor (NN) query answering. We develop our method for k-NN queries and demonstrate how it can be applied with the two most popular distance measures, namely Euclidean and dynamic time warping. We provide both initial and progressive estimates of the final answer that are getting better during the similarity search, as well suitable stopping criteria for the progressive queries. Moreover, we describe how this method can be used in order to develop a progressive algorithm for data series classification (based on a k-NN classifier), and we additionally propose a method designed specifically for the classification task. Experiments with several and diverse synthetic and real datasets demonstrate that our prediction methods constitute the first practical solutions to the problem, significantly outperforming competing approaches.

ProS

ProS: data series progressive k-NN similarity search and classification with probabilistic quality guarantees

As the volume of seismic observation time-series data grows larger, web-based visualization  schemes suffer from longer system response times. Although big data visualization schemes based on sampling  and filtering can greatly reduce the data scale and shorten transmission time, what it gains in speed it loses in  information. Progressive visualization has become an increasingly popular scheme because it can quickly “see”  some results without having to wait for all the data, thus enabling users to grasp a data-change trend quickly  and perceive the rules behind it. In this paper, a Cloudberry-based progressive real-time visualization schema  for earthquake big data (PVSEBD) is proposed for the first time. It greatly shortens the transmission time of  each data slice, improves the user interaction experience, and meets the long-term, large-scale visualization  needs of earthquake consultation business. Because the correctness of average aggregation function (AVG)  in progressive visualization is often not guaranteed, this paper proposes an innovative AVG translation rule  solution based on the accumulability of the COUNT and SUM aggregation functions. The experimental results  showed that PVSEBD automatically adjusts the amount of data returned each time according to size and has  a shorter response time for each interaction compared with the solution based on the web-based visualization  toolkit Portable Progressive Parallel Processing Pipeline (P5).

Earthquake Big Data

A Progressive Real-time Visualization Method for Earthquake Big Data

We present an algorithm that allows a user within a virtual environment to perform real-time unconstrained cuts or consecutive tears, i.e., progressive, continuous fractures on a deformable rigged and soft-body mesh model in high-performance 10ms. In order to recreate realistic results for different physically-principled materials such as sponges, hard or soft tissues, we incorporate a novel soft-body deformation, via a particle system layered on-top of a linear-blend skinning model. Our framework allows the simulation of realistic, surgical-grade cuts and continuous tears, especially valuable in the context of medical VR training. In order to achieve high performance in VR, our algorithms are based on Euclidean geometric predicates on the rigged mesh, without requiring any specific model pre-processing. The contribution of this work lies on the fact that current frameworks supporting similar kinds of model tearing, either do not operate in high-performance real-time or only apply to predefined tears. The framework presented allows the user to freely cut or tear a 3D mesh model in a consecutive way, under 10ms, while preserving its soft-body behaviour and/or allowing further animation.

Tearing and cutting

Progressive tearing and cutting of soft-bodies in high-performance virtual reality


With the growing popularity of social networks, it becomes increasingly important to analyze binary relationships between entities such as users or online posts. These relationships are particularly useful when the entities are location-based. The spatial dimension provides more insights on influences in social networks across different regions. In this paper, we study how to visualize geospatial relationships on a large social network for queries with ad hoc conditions (such as keyword search) that retrieves a subnetwork. We focus on a main efficiency challenge to support responsive visualization, and present a middleware-based system called GSViz that progressively answers requests and computes results incrementally. GSViz minimizes visual clutter by clustering the spatial points while considering the edges among them. It further minimizes the clutter by incrementally bundling the edges, i.e., grouping similar edges in a bundle to increase the screen's white space. The system allows user interactions such as zooming and panning. We conducted an extensive computational study on real data sets and a user study, which evaluated the system's performance and the quality of its visualization results.

GSViz

GSViz: progressive visualization of geospatial influences in social networks

The global routing network is the backbone of the Internet. However, it is quite vulnerable to attacks that cause major disruptions or routing manipulations. Prior related works have visualized routing path changes with node link diagrams, but it requires strong domain expertise to understand if a routing change between autonomous systems is suspicious. Geographic visualization has an advantage over conventional node-link diagrams by helping uncover such suspicious routes as the user can immediately see if a path is the shortest path to the target or an unreasonable detour. In this paper, we present ProBGP, a web-based progressive approach to visually analyze BGP update routes. We created a novel progressive data processing algorithm for the geographic approximation of autonomous systems and combined it with a progressively updating visualization. While the newest log data is continuously loaded, our approach also allows querying the entire log recordings since 1999. We present the usefulness of our approach with a real use case of a major route leak from June 2019. We report on multiple interviews with domain experts throughout the development. Finally, we evaluated our algorithm quantitatively against a public peering database and qualitatively against AS network maps.

ProBGP

ProBGP: Progressive Visual Analytics of Live BGP Updates

We present a pyramid-based scatterplot sampling technique to avoid overplotting and enable progressive and streaming visualization of large data. Our technique is based on a multiresolution pyramid-based decomposition of the underlying density map and makes use of the density values in the pyramid to guide the sampling at each scale for preserving the relative data densities and outliers. We show that our technique is competitive in quality with state-of-the-art methods and runs faster by about an order of magnitude. Also, we have adapted it to deliver progressive and streaming data visualization by processing the data in chunks and updating the scatterplot areas with visible changes in the density map. A quantitative evaluation shows that our approach generates stable and faithful progressive samples that are comparable to the state-of-the-art method in preserving relative densities and superior to it in keeping outliers and stability when switching frames. We present two case studies that demonstrate the effectiveness of our approach for exploring large data.

Pyramid-based Scatterplots

Pyramid-based Scatterplots Sampling for Progressive and Streaming Data Visualization

Diffusion Tensor Imaging (DTI) is a non-invasive magnetic resonance imaging technique that, combined with fiber tracking algorithms, allows the characterization and visualization of white matter structures in the brain. The resulting fiber tracts are used, for example, in tumor surgery to evaluate the potential brain functional damage due to tumor resection. The DTI processing pipeline from image acquisition to the final visualization is rather complex generating undesirable uncertainties in the final results. Most DTI visualization techniques do not provide any information regarding the presence of uncertainty. When planning surgery, a fixed safety margin around the fiber tracts is often used; however, it cannot capture local variability and distribution of the uncertainty, thereby limiting the informed decision-making process. Stochastic techniques are a possibility to estimate uncertainty for the DTI pipeline. However, it has high computational and memory requirements that make it infeasible in a clinical setting. The delay in the visualization of the results adds hindrance to the workflow. We propose a progressive approach that relies on a combination of wild-bootstrapping and fiber tracking to be used within the progressive visual analytics paradigm. We present a local bootstrapping strategy, which reduces the computational and memory costs, and provides fiber-tracking results in a progressive manner. We have also implemented a progressive aggregation technique that computes the distances in the fiber ensemble during progressive bootstrap computations. We present experiments with different scenarios to highlight the benefits of using our progressive visual analytic pipeline in a clinical workflow along with a use case and analysis obtained by discussions with our collaborators.

Diffusion Tensor Imaging

A Progressive Approach for Uncertainty Visualization in Diffusion Tensor Imaging

This article introduces progressive algorithms for the topological analysis of scalar data. Our approach is based on a hierarchical representation of the input data and the fast identification of topologically invariant vertices, which are vertices that have no impact on the topological description of the data and for which we show that no computation is required as they are introduced in the hierarchy. This enables the definition of efficient coarse-to-fine topological algorithms, which leverage fast update mechanisms for ordinary vertices and avoid computation for the topologically invariant ones. We demonstrate our approach with two examples of topological algorithms (critical point extraction and persistence diagram computation), which generate interpretable outputs upon interruption requests and which progressively refine them otherwise. Experiments on real-life datasets illustrate that our progressive strategy, in addition to the continuous visual feedback it provides, even improves run time performance with regard to non-progressive algorithms and we describe further accelerations with shared-memory parallelism. We illustrate the utility of our approach in batch-mode and interactive setups, where it respectively enables the control of the execution time of complete topological pipelines as well as previews of the topological features found in a dataset, with progressive updates delivered within interactive times.

Scalar Field Topology

A Progressive Approach to Scalar Field Topology

Progressive visualization is fast becoming a technique in the visualization community to help users interact with large amounts of data. With progressive visualization, users can examine intermediate results of complex or long running computations, without waiting for the computation to complete. While this has shown to be beneficial to users, recent research has identified potential risks. For example, users may misjudge the uncertainty in the intermediate results and draw incorrect conclusions or see patterns that are not present in the final results. In this article, we conduct a comprehensive set of studies to quantify the advantages and limitations of progressive visualization. Based on a recent report by Micallef et al. , we examine four types of cognitive biases that can occur with progressive visualization: uncertainty bias , illusion bias , control bias , and anchoring bias . The results of the studies suggest a cautious but promising use of progressive visualization – while there can be significant savings in task completion time, accuracy can be negatively affected in certain conditions. These findings confirm earlier reports of the benefits and drawbacks of progressive visualization and that continued research into mitigating the effects of cognitive biases is necessary.

Cognitive Biases

Impact of Cognitive Biases on Progressive Visualization

Progressive visualization allows users to examine intermediate results while they are further refined in the background. This makes them increasingly popular when dealing with large data and computationally expensive tasks. The characteristics of how preliminary visualizations evolve over time are crucial for efficient analysis; in particular unexpected disruptive changes between iterations can significantly hamper the user experience. This paper proposes a visualization framework to analyze the refinement behavior of progressive visualization. We particularly focus on sudden significant changes between the iterations, which we denote as popping artifacts, in reference to undesirable visual effects in the context of level of detail representations in computer graphics. Our visualization approach conveys where in image space and when during the refinement popping artifacts occur. It allows to compare across different runs of stochastic processes, and supports parameter studies for gaining further insights and tuning the algorithms under consideration. We demonstrate the application of our framework and its effectiveness via two diverse use cases with underlying stochastic processes: adaptive image space sampling, and the generation of grid layouts.

Progressive Popping

Visual Analysis of Popping in Progressive Visualization

This paper presents an efficient algorithm for the progressive approximation of Wasserstein barycenters of persistence diagrams, with applications to the visual analysis of ensemble data. Given a set of scalar fields, our approach enables the computation of a persistence diagram which is representative of the set, and which visually conveys the number, data ranges and saliences of the main features of interest found in the set. Such representative diagrams are obtained by computing explicitly the discrete Wasserstein barycenter of the set of persistence diagrams, a notoriously computationally intensive task. In particular, we revisit efficient algorithms for Wasserstein distance approximation [12,51] to extend previous work on barycenter estimation [94]. We present a new fast algorithm, which progressively approximates the barycenter by iteratively increasing the computation accuracy as well as the number of persistent features in the output diagram. Such a progressivity drastically improves convergence in practice and allows to design an interruptible algorithm, capable of respecting computation time constraints. This enables the approximation of Wasserstein barycenters within interactive times. We present an application to ensemble clustering where we revisit the k-means algorithm to exploit our barycenters and compute, within execution time constraints, meaningful clusters of ensemble data along with their barycenter diagram. Extensive experiments on synthetic and real-life data sets report that our algorithm converges to barycenters that are qualitatively meaningful with regard to the applications, and quantitatively comparable to previous techniques, while offering an order of magnitude speedup when run until convergence (without time constraint). Our algorithm can be trivially parallelized to provide additional speedups in practice on standard workstations. We provide a lightweight C++ implementation of our approach that can be used to reproduce our results.

Wasserstein Barycenters

Progressive Wasserstein Barycenters of Persistence Diagrams


We present a progressive algorithm for the Uniform Manifold Approximation and Projection (UMAP), called the Progressive UMAP. Based on the theory of Riemannian geometry and algebraic topology, UMAP is an emerging dimensionality reduction technique that offers better versatility and stability than t-SNE. Although UMAP is also more efficient than t-SNE, it still suffers from an initial delay of a few minutes to produce the first projection, which limits its use in interactive data exploration. To tackle this problem, we improve the sequential computations in UMAP by making them progressive, which allows people to incrementally append a batch of data points into the projection at the desired pace. In our experiment with the Fashion MNIST dataset, we found that Progressive UMAP could generate the first approximate projection within a few seconds while also sufficiently capturing the important structures of the high-dimensional dataset.

UMAP

Progressive Uniform Manifold Approximation and Projection

Understanding and tuning the performance of extreme-scale parallel computing systems demands a streaming approach due to the computational cost of applying offline algorithms to vast amounts of performance log data. Analyzing large streaming data is challenging because the rate of receiving data and limited time to comprehend data make it difficult for the analysts to sufficiently examine the data without missing important changes or patterns. To support streaming data analysis, we introduce a visual analytic framework comprising of three modules: data management, analysis, and interactive visualization. The data management module collects various computing and communication performance metrics from the monitored system using streaming data processing techniques and feeds the data to the other two modules. The analysis module automatically identifies important changes and patterns at the required latency. In particular, we introduce a set of online and progressive analysis methods for not only controlling the computational costs but also helping analysts better follow the critical aspects of the analysis results. Finally, the interactive visualization module provides the analysts with a coherent view of the changes and patterns in the continuously captured performance data. Through a multi-faceted case study on performance analysis of parallel discrete-event simulation, we demonstrate the effectiveness of our framework for identifying bottlenecks and locating outliers.

Streaming performance

A Visual Analytics Framework for Reviewing Streaming Performance Data

Existing systems dealing with the increasing volume of data series cannot guarantee interactive response times, even for fundamental tasks such as similarity search. Therefore, it is necessary to develop analytic approaches that support exploration and decision making by providing progressive results, before the final and exact ones have been computed. Prior works lack both efficiency and accuracy when applied to large-scale data series collections. We present and experimentally evaluate a new probabilistic learning-based method that provides quality guarantees for progressive Nearest Neighbor (NN) query answering. We provide both initial and progressive estimates of the final answer that are getting better during the similarity search, as well suitable stopping criteria for the progressive queries. Experiments with synthetic and diverse real datasets demonstrate that our prediction methods constitute the first practical solution to the problem, significantly outperforming competing approaches.

Similarity search 2

Data series progressive similarity search with probabilistic quality guarantees

Understanding data through interactive visualization, also known as visual analytics, is a common and necessary practice in modern data science. However, as data sizes have increased at unprecedented rates, the computation latency of visualization systems becomes a significant hurdle to visual analytics. The goal of this dissertation is to design a series of systems for progressive visual analytics (PVA)—a visual analytics paradigm that can provide intermediate results during computation and allow visual exploration of these results—to address the scalability hurdle. To support the interactive exploration of data with billions of records, we first introduce SwiftTuna, an interactive visualization system with scalable visualization and computation components. Our performance benchmark demonstrates that it can handle data with four billion records, giving responsive feedback every few seconds without precomputation. Second, we present PANENE, a progressive algorithm for the Approximate k-Nearest Neighbor (AKNN) problem. PANENE brings useful machine learning methods into visual analytics, which has been challenging due to their long initial latency resulting from AKNN computation. In particular, we accelerate t-Distributed Stochastic Neighbor Embedding (t-SNE), a popular non-linear dimensionality reduction technique, which enables the responsive visualization of data with a few hundred columns. Each of these two contributions aims to address the scalability issues stemming from a large number of rows or columns in data, respectively. Third, from the users' perspective, we focus on improving the trustworthiness of intermediate knowledge gained from uncertain results in PVA. We propose a novel PVA concept, Progressive Visual Analytics with Safeguards, and introduce PVA-Guards, safeguards people can leave on uncertain intermediate knowledge that needs to be verified. We also present a proof-of-concept system, ProReveal, designed and developed to integrate seven safeguards into progressive data exploration. Our user study demonstrates that people not only successfully created PVA-Guards on ProReveal but also voluntarily used PVA-Guards to manage the uncertainty of their knowledge. Finally, summarizing the three studies, we discuss design challenges for progressive systems as well as future research agendas for PVA.

PhD Jaemin Jo

PhD Thesis: Designing Progressive Visualization Systems for Exploring Large-scale Data

In this paper, we study the problem of interactive cleaning for progressive visualization (ICPV): Given a bad visualization V , it is to obtain a “cleaned” visualization V  whose distance is far from V , under a given (small) budget w.r.t. human cost. In ICPV, a system interacts with a user iteratively. During each iteration, it asks the user a data cleaning question such as “how to clean detected errors x?”, and takes value updates from the user to clean V . Conventional wisdom typically picks a single question (e.g., “Are SIGMOD conference and SIGMOD the same?”) with the maximum expected benefit in each iteration. We propose to use a composite question – i.e., a group of single questions to be treated as one question – in each iteration (for example, Are SIGMOD conference in t1 and SIGMOD in t2 the same value, and are t1 and t2 duplicates?). A composite question is presented to the user as a small connected graph through a novel GUI that the user can directly operate on. We propose algorithms to select the best composite question in each iteration. Experiments on real-world datasets verify that composite questions are more effective than asking single questions in isolation w.r.t. the human cost.

Interactive Cleaning

Interactive Cleaning for Progressive Visualization through Composite
                  Questions

COVID-19 data analysis has become a prominent activity in the last year. The use of different models for predicting the spread of the disease, while providing very useful insights on the epidemics, are not inherently designed for interactive analysis, with time for a single computation ranging in the tens of seconds. In order to overcome this limitation, this paper proposes three techniques for progressive visualization of Susceptible-Infectious-Recovered (SIR) models data, that govern the trade-off between time and quality of the intermediate results. The techniques are quantitatively evaluated showing promising results.

Epidemiological Models

Progressive Visualization of Epidemiological Models for COVID-19 Visual Analysis

Exploring a voluminous graph dataset visually is a challenging task due to the sheer amount of data and the lack of structure to rely on during the navigation. Indra, our framework for large-scale graph data, provides responsive visual analytics over large-scale graph datasets. In this study, we propose a novel graph indexing scheme that pivots the view of the graph to a hierarchical structure while preserving the semantic importance of vertices within the user's analysis scenario. Indra allows users to compare and track multiple aspects of sub-graphs by supporting linked multi-views and multi-resolution operations such as drill-in and roll-ups. We have performed a set of empirical benchmarks profiling Indra and these demonstrate that several operations are executed with sub-second latency to effectively support interactive visual analytics.

Sub-Graph Explorations

Rapid, Progressive Sub-Graph Explorations for Interactive Visual Analytics over Large-Scale Graph Datasets

This paper presents a visual tool, AVIATOR, that integrates the progressive visual analytics paradigm in the IR evaluation process. This tool serves to speed-up and facilitate the performance assessment of retrieval models enabling a result analysis through visual facilities. AVIATOR goes one step beyond the common "compute wait visualize" analytics paradigm, introducing a continuous evaluation mechanism that minimizes human and computational resource consumption.

Incremental Evaluation

A progressive visual analytics tool for incremental experimental evaluation

Network traffic log data is a key data source for forensic analysis of cybersecurity incidents. Packet Captures (PCAPs) are the raw information directly gathered from the network device. As the bandwidth and connections to other hosts rise, this data becomes very large quickly. Malware analysts and administrators are using this data frequently for their analysis. However, the currently most used tool Wireshark is displaying the data as a table, making it difficult to get an overview and focus on the significant parts. Also, the process of loading large files into Wireshark takes time and has to be repeated each time the file is closed. We believe that this problem poses an optimal setting for a client-server infrastructure with a progressive visual analytics approach. The processing can be outsourced to the server while the client is progressively updated. In this paper we present NetCapVis, an web-based progressive visual analytics system where the user can upload PCAP files, set initial filters to reduce the data before uploading and then instantly interact with the data while the rest is progressively loaded into the visualizations.

NetCapVis

Netcapvis: Web-based progressive visual analytics for network packet captures

We present a new visual exploration concept - Progressive Visual Analytics with Safeguards - that helps people manage the uncertainty arising from progressive data exploration. Despite its potential benefits, intermediate knowledge from progressive analytics can be incorrect due to various machine and human factors, such as a sampling bias or misinterpretation of uncertainty. To alleviate this problem, we introduce PVA-Guards, safeguards people can leave on uncertain intermediate knowledge that needs to be verified, and derive seven PVA-Guards based on previous visualization task taxonomies. PVA-Guards provide a means of ensuring the correctness of the conclusion and understanding the reason when intermediate knowledge becomes invalid. We also present ProReveal, a proof-of-concept system designed and developed to integrate the seven safeguards into progressive data exploration. Finally, we report a user study with 14 participants, which shows people voluntarily employed PVA-Guards to safeguard their findings and ProReveal's PVA-Guard view provides an overview of uncertain intermediate knowledge. We believe our new concept can also offer better consistency in progressive data exploration, alleviating people's heterogeneous interpretation of uncertainty.

Proreveal

ProReveal: Progressive Visual Analytics With Safeguards

We present P5, a web-based visualization toolkit that combines declarative visualization grammar and GPU computing for progressive data analysis and visualization. To interactively analyze and explore big data, progressive analytics and visualization methods have recently emerged. Progressive visualizations of incrementally refining results have the advantages of allowing users to steer the analysis process and make early decisions. P5 leverages declarative grammar for specifying visualization designs and exploits GPU computing to accelerate progressive data processing and rendering. The declarative specifications can be modified during progressive processing to create different visualizations for analyzing the intermediate results. To enable user interactions for progressive data analysis, P5 utilizes the GPU to automatically aggregate and index data based on declarative interaction specifications to facilitate effective interactive visualization. We demonstrate the effectiveness and usefulness of P5 through a variety of example applications and several performance benchmark tests.

P5: Portable Progressive Parallel Processing Pipelines for Interactive Data Analysis and Visualization

Effective data analysis ideally requires the analyst to have high expertise as well as high knowledge of the data. Even with such familiarity, manually pursuing all potential hypotheses and exploring all possible views is impractical. We present DataSite, a proactive visual analytics system where the burden of selecting and executing appropriate computations is shared by an automatic server-side computation engine. Salient features identified by these automatic background processes are surfaced as notifications in a feed timeline. DataSite effectively turns data analysis into a conversation between analyst and computer, thereby reducing the cognitive load and domain knowledge requirements. We validate the system with a user study comparing it to a recent visualization recommendation system, yielding significant improvement, particularly for complex analyses that existing analytics systems do not support well.

DataSite

DataSite: Proactive visual data exploration with computation of insight-based recommendations


A key component in using Progressive Visual Analytics (PVA) is to be able to gauge the quality of intermediate analysis outcomes. This is necessary in order to decide whether a current partial outcome is already good enough to cut a long-running computation short and to proceed. To aid in this process, we propose ten fundamental quality indicators that can be computed and displayed to gain a better understanding of the progress of the progression and of the stability and certainty of an intermediate outcome. We further highlight the use of these fundamental indicators to derive other quality indicators, and we show how to apply the indicators in two use cases

Quality indicators

On quality indicators for progressive visual analytics

We live in an era where data is abundant and growing rapidly; databases storing big data sprawl past memory and computation limits, and across distributed systems. New hardware and software systems have been built to sustain this growth in terms of storage management and predictive computation. However, these infrastructures, while good for data at scale, do not well support exploratory data analysis (EDA) as, for instance, commonly used in Visual Analytics. EDA allows human users to make sense of data with little or no known model on this data and is essential in many application domains, from network security and fraud detection to epidemiology and preventive medicine. Data exploration is done through an iterative loop where analysts interact with data through computations that return results, usually shown with visualizations, which in turn are interacted with by the analyst again. Due to human cognitive constraints, exploration needs highly responsive system response times: at 500 ms, users change their querying behavior; past five or ten seconds, users abandon tasks or lose attention. As datasets grow and computations become more complex, response time suffers. To address this problem, a new computation paradigm has emerged in the last decade under several names: online aggregation in the database community; progressive, incremental, or iterative visualization in other communities. It consists of splitting long computations into a series of approximate results improving with time; in this process, partial or approximate results are then rapidly returned to the user and can be interacted with in a fluent and iterative fashion. With the increasing growth in data, such progressive data analysis approaches will become one of the leading paradigms for data exploration systems, but it also will require major changes in the algorithms, data structures, and visualization tools. This Dagstuhl Seminar was set out to discuss and address these challenges, by bringing together researchers from the different involved research communities: database, visualization, and machine learning. Thus far, these communities have often been divided by a gap hindering joint efforts in dealing with forthcoming challenges in progressive data analysis and visualization. The seminar gave a platform for these researchers and practitioners to exchange their ideas, experience, and visions, jointly develop strategies to deal with challenges, and create a deeper awareness of the implications of this paradigm shift. The implications are technical, but also human--both perceptual and cognitive--and the seminar provided a holistic view of the problem by gathering specialists from all the communities.

Dagstuhl Seminar

Progressive Data Analysis and Visualization

The amount of generated and analyzed data is ever increasing, and processing such large data sets can take too long in situations where time-to-decision or fluid data exploration are critical. Progressive visual analytics (PVA) has recently emerged as a potential solution that allows users to analyze intermediary results during the computation without waiting for the computation to complete. However, there has been limited consideration on how these techniques impact the user. Based on discussions from a Dagstuhl seminar held in October 2018, this paper characterizes PVA users by their common roles, their main tasks, and their distinct focus of analysis. It further discusses cognitive biases that play a particular role in PVA. This work will help PVA visualization designers in devising systems that are tailored for their specific target users and their characteristics

Human User

The Human User in Progressive Visual Analytics

Progressive visualization offers a great deal of promise for big data visualization; however, current progressive visualization systems do not allow for continuous interaction. What if users want to see more confident results on a subset of the visualization? This can happen when users are in exploratory analysis mode but want to ask some directed questions of the data as well. In a progressive visualization system, the online aggregation algorithm determines the database sampling rate and resulting convergence rate, not the user. In this paper, we extend a recent method in online aggregation, called Wander Join, that is optimized for queries that join tables, one of the most computationally expensive operations. This extension leverages importance sampling to enable user-driven sampling when data joins are in the query. We applied user interaction techniques that allow the user to view and adjust the convergence rate, providing more transparency and control over the online aggregation process. By leveraging importance sampling, our extension of Wander Join also allows for stratified sampling of groups when there is data distribution skew. We also improve the convergence rate of filtering queries, but with additional overhead costs not needed in the original Wander Join algorithm.

Wander Join

Selective Wander Join: Fast Progressive Visualizations for Data Joins

Time series data are increasing at a dramatic rate, yet their analysis remains highly relevant in a wide range of human activities. Due to their volume, existing systems dealing with time series data cannot guarantee interactive response times, even for fundamental tasks such as similarity search. Therefore , in this paper, we present our vision to develop analytic approaches that support exploration and decision making by providing progressive results, before the final and exact ones have been computed. We demonstrate through experiments that providing first approximate and then progressive answers is useful (and necessary) for similarity search queries on very large time series data. Our findings indicate that there is a gap between the time the most similar answer is found and the time when the search algorithm terminates, resulting in inflated waiting times without any improvement. We present preliminary ideas on computing probabilistic estimates of the final results that could help users decide when to stop the search process, i.e., deciding when improvement in the final answer is unlikely, thus eliminating waiting time. Finally, we discuss two additional challenges: how to compute efficiently these probabilistic estimates, and how to communicate them to users.

Similarity search

Progressive similarity search on time series data

Most previous fusion strategies either fail to fully utilize temporal information or cost too much time, and how to effectively fuse temporal information from consecutive frames plays an important role in video super-resolution (SR). In this study, we propose a novel progressive fusion network for video SR, which is designed to make better use of spatio-temporal information and is proved to be more efficient and effective than the existing direct fusion, slow fusion or 3D convolution strategies. Under this progressive fusion framework, we further introduce an improved non-local operation to avoid the complex motion estimation and motion compensation (ME&MC) procedures as in previous video SR approaches. Extensive experiments on public datasets demonstrate that our method surpasses state-of-the-art with 0.96 dB in average, and runs about 3 times faster, while requires only about half of the parameters.

Video Super-Resolution

Progressive Fusion Video Super-Resolution Network via Exploiting Non-Local Spatio-Temporal Correlations


This paper presents a model for the progressive visualization and exploration of the structure of large datasets. That is, an abstraction on different components and relations which provide means for constructing a visual representation of a dataset’s structure, with continuous system feedback and enabled user interactions for computational steering, in spite of size. In this context, the structure of a dataset is regarded as the distance or neighborhood relationships among its data points. Size, on the other hand, is defined in terms of the number of data points. To prove the validity of the model, a proof-of-concept was developed as a Visual Analytics library for Apache Zeppelin and Apache Spark. Moreover, nine user studies where carried in order to assess the usability of the library. The results from the user studies show that the library is useful for visualizing and understanding the emerging cluster patterns, for identifying relevant features, and for estimating the number of clusters k.

Structure Model

A Model for the Progressive Visualization of Multidimensional Data
                  Structure

Consider a multi-dimensional spatio-temporal (ST) dataset where each entry is a numerical measure defined by the corresponding temporal, spatial and other domain-specific dimensions. A typical approach to explore such data utilizes interactive visualizations with multiple coordinated views. Each view displays the aggregated measures along one or two dimensions. By brushing on the views, analysts can obtain detailed information. However, this approach often cannot provide sufficient guidance for analysts to identify patterns hidden within subsets of data. Without a priori hypotheses, analysts need to manually select and iterate through different slices to search for patterns, which can be a tedious and lengthy process. In this work, we model multidimensional ST data as tensors and propose a novel piecewise rank-one tensor decomposition algorithm which supports automatically slicing the data into homogeneous partitions and extracting the latent patterns in each partition for comparison and visual summarization. The algorithm optimizes a quantitative measure about how faithfully the extracted patterns visually represent the original data. Based on the algorithm we further propose a visual analytics framework that supports a top-down, progressive partitioning workflow for level-of-detail multidimensional ST data exploration. We demonstrate the general applicability and effectiveness of our technique on three datasets from different application domains: regional sales trend analysis, customer traffic analysis in department stores, and taxi trip analysis with origin-destination (OD) data. We further interview domain experts to verify the usability of the prototype.

TPFlow

TPFlow: Progressive Partition and Multidimensional Pattern Extraction for Large-Scale Spatio-Temporal Data Analysis

The progressive visual analytics (PVA) paradigm has been proposed to describe visual analytics systems whose main goal is to reach a thorough coupling between the analyst and her system by getting rid of waiting periods classically encountered during data process- ing. PVA systems use algorithms that both provide intermediate results throughout their execution, and are steerable by the analyst to change the strategy used to perform the remaining computation. Our focus is on progressive sequential pattern mining, as in the seminal work of Stolper et al. [30]. Here we go further mainly by considering the temporal nature of patterns related to their occur- rences. We propose a pattern-oriented data model, a pattern analysis task model, and guidelines for designing progressive pattern min- ing algorithms. We introduce PPMT, a tool to support an analyst progressively explore activity traces, based on a modification of the GSP algorithm. We evaluate our proposal on the technical perfor- mances of our progressive algorithm, and on the effect of steering on analysts’ performances.

Sequential Pattern Mining

Progressive sequential pattern mining: steerable visual exploration of patterns with PPMT


Progressive Visual Analytics (PVA) has gained increasing attention over the past years. It brings the user into the loop during otherwise long-running and non-transparent computations by producing intermediate partial results. These partial results can be shown to the user for early and continuous interaction with the emerging end result even while it is still being computed. Yet as clear-cut as this fundamental idea seems, the existing body of literature puts forth various interpretations and instantiations that have created a research domain of competing terms, various definitions, as well as long lists of practical requirements and design guidelines spread across different scientific communities. This makes it more and more difficult to get a succinct understanding of PVA’s principal concepts, let alone an overview of this increasingly diverging field. The review and discussion of PVA presented in this paper address these issues and provide (1) a literature collection on this topic, (2) a conceptual characterization of PVA, as well as (3) a consolidated set of practical recommendations for implementing and using PVA-based visual analytics solutions.

Characterization

A review and characterization of progressive visual analytics

We present PANENE, a progressive algorithm for approximate nearest neighbor indexing and querying. Although the use of k-nearest neighbor (KNN) libraries is common in many data analysis methods, most KNN algorithms can only be queried when the whole dataset has been indexed, i.e., they are not online. Even the few online implementations are not progressive in the sense that the time to index incoming data is not bounded and cannot satisfy the latency requirements of progressive systems. This long latency has significantly limited the use of many machine learning methods, such as t-SNE, in interactive visual analytics. PANENE is a novel algorithm for Progressive Approximate k-NEarest NEighbors, enabling fast KNN queries while continuously indexing new batches of data. Following the progressive computation paradigm, PANENE operations can be bounded in time, allowing analysts to access running results within an interactive latency. PANENE can also incrementally build and maintain a cache data structure, a KNN lookup table, to enable constant-time lookups for KNN queries. Finally, we present three progressive applications of PANENE, such as regression, density estimation, and responsive t-SNE, opening up new opportunities to use complex algorithms in interactive systems.


PANENE

PANENE: A Progressive Algorithm for Indexing and Querying Approximate k-Nearest Neighbors

Digital Humanities (DH) research and practice is subject to uncertainty during the life cycle of any project. Even in non data-oriented cases, analysts and other stakeholders need to make decisions without being aware of the level of uncertainty associated to the data being transformed by the computational tools used to enable the kind of novel work of humanists pursued within DH. We examine in this paper the literature that have characterized the types and sources of uncertainty in other fields, with the intent of establishing a foundation upon which build novel computational tools supporting the decision-making under uncertainty processes that DH is currently facing. We propose the use of progressive visual analytics as a feasible means to manage decision-making under uncertainty, which may help tackling some challenges related to the elimination or mitigation of uncertainty in DH, that otherwise would tamper the quality of the yielded results.

Decision-making

Toward supporting decision-making under uncertainty in digital humanities with progressive visualization

Entity Resolution (ER) is the task of finding entity profiles that correspond to the same real-world entity. Progressive ER aims to efficiently resolve large datasets when limited time and/or computational resources are available. In practice, its goal is to provide the best possible partial solution by approximating the optimal comparison order of the entity profiles. So far, Progressive ER has only been examined in the context of structured (relational) data sources, as the existing methods rely on schema knowledge to save unnecessary comparisons: they restrict their search space to similar entities with the help of schema-based blocking keys (i.e., signatures that represent the entity profiles). As a result, these solutions are not applicable in Big Data integration applications, which involve large and heterogeneous datasets, such as relational and RDF databases, JSON files, Web corpus etc. To cover this gap, we propose a family of schema-agnostic Progressive ER methods, which do not require schema information, thus applying to heterogeneous data sources of any schema variety. First, we introduce two naïve schema-agnostic methods, showing that straightforward solutions exhibit a poor performance that does not scale well to large volumes of data. Then, we propose four different advanced methods. Through an extensive experimental evaluation over 7 real-world, established datasets, we show that all the advanced methods outperform to a significant extent both the naïve and the state-of-the-art schema-based ones. We also investigate the relative performance of the advanced methods, providing guidelines on the method selection.

Entity Resolution

Schema-agnostic Progressive Entity Resolution

We present an efficient framework that can generate a coherent paragraph to describe a given video. Previous works on video captioning usually focus on video clips. They typically treat an entire video as a whole and generate the caption conditioned on a single embedding. On the contrary, we consider videos with rich temporal structures and aim to generate paragraph descriptions that can preserve the story flow while being coherent and concise. Towards this goal, we propose a new approach, which produces a descriptive paragraph by assembling temporally localized descriptions. Given a video, it selects a sequence of distinctive clips and generates sentences thereon in a coherent manner. Particularly, the selection of clips and the production of sentences are done jointly and progressively driven by a recurrent network -- what to describe next depends on what have been said before. Here, the recurrent network is learned via self-critical sequence training with both sentence-level and paragraph-level rewards. On the ActivityNet Captions dataset, our method demonstrated the capability of generating high-quality paragraph descriptions for videos. Compared to those by other methods, the descriptions produced by our method are often more relevant, more coherent, and more concise.

Video Descriptions

Move Forward and Tell: A Progressive Generator of Video Descriptions

Digital Humanities

Progressive Visual Analytics aims at improving the interactivity in existing analytics techniques by means of visualization as well as interaction with intermediate results. One key method for data analysis is dimensionality reduction, for example, to produce 2D embeddings that can be visualized and analyzed efficiently. t-Distributed Stochastic Neighbor Embedding (tSNE) is a well-suited technique for the visualization of high-dimensional data. tSNE can create meaningful intermediate results but suffers from a slow initialization that constrains its application in Progressive Visual Analytics. We introduce a controllable tSNE approximation (A-tSNE), which trades off speed and accuracy, to enable interactive data exploration. We offer real-time visualization techniques, including a density-based solution and a Magic Lens to inspect the degree of approximation. With this feedback, the user can decide on local refinements and steer the approximation level during the analysis. We demonstrate our technique with several datasets, in a real-world research scenario and for the real-time analysis of high-dimensional streams to illustrate its effectiveness for interactive data analysis.

Steerable tSNE

Approximated and user steerable tSNE for progressive visual analytics


Progressive visual analytics (PVA) has emerged in recent years to manage the latency of data analysis systems. When analysis is performed progressively, rough estimates of the results are generated quickly and are then improved over time. Analysts can therefore monitor the progression of the results, steer the analysis algorithms, and make early decisions if the estimates provide a convincing picture. In this article, we describe interface design guidelines for helping users understand progressively updating results and make early decisions based on progressive estimates. To illustrate our ideas, we present a prototype PVA tool called InsightsFeed for exploring Twitter data at scale. As validation, we investigate the tradeoffs of our tool when exploring a Twitter dataset in a user study. We report the usage patterns in making early decisions using the user interface, guiding computational methods, and exploring different subsets of the dataset, compared to sequential analysis without progression.

Steering the craft

Steering the Craft: UI Elements and Visualizations for Supporting Progressive Visual Analytics

Data visualization is an effective mechanism for identifying trends, insights, and anomalies in data. On large datasets, however, generating visualizations can take a long time, delaying the extraction of insights, hampering decision making, and reducing exploration time. One solution is to use online sampling-based schemes to generate visualizations faster while improving the displayed estimates incrementally, eventually converging to the exact visualization computed on the entire data. However, the intermediate visualizations are approximate, and often fluctuate drastically, leading to potentially incorrect decisions. We propose sampling-based incremental visualization algorithms that reveal the "salient" features of the visualization quickly---with a 46× speedup relative to baselines---while minimizing error, thus enabling rapid and error-free decision making. We demonstrate that these algorithms are optimal in terms of sample complexity, in that given the level of interactivity, they generate approximations that take as few samples as possible. We have developed the algorithms in the context of an incremental visualization tool, titled IncVisage, for trendline and heatmap visualizations. We evaluate the usability of IncVisage via user studies and demonstrate that users are able to make effective decisions with incrementally improving visualizations, especially compared to vanilla online-sampling based schemes.

I’ve Seen “Enough”

I've seen "enough": incrementally improving visualizations to support rapid decision making

When using data-mining tools to analyze big data, users often need tools to support the understanding of individual data attributes and control the analysis progress. This requires the integration of data-mining algorithms with interactive tools to manipulate data and analytical process. This is where visual analytics can help. More than simple visualization of a dataset or some computation results, visual analytics provides users an environment to iteratively explore different inputs or parameters and see the corresponding results. In this research, we explore a design of progressive visual analytics to support the analysis of categorical data with a data-mining algorithm, Apriori. Our study focuses on executing data mining techniques step-by-step and showing intermediate result at every stage to facilitate sense-making. Our design, called Pattern Discovery Tool, targets for a medical dataset. Starting with visualization of data properties and immediate feedback of users’ inputs or adjustments, Pattern Discovery Tool could help users detect interesting patterns and factors effectively and efficiently. Afterward, further analyses such as statistical methods could be conducted to test those possible theories.

Pattern Discovery

Pattern discovery: A progressive visual analytic design to support categorical data analysis

Deep neural networks are now rivaling human accuracy in several pattern recognition problems. Compared to traditional classifiers, where features are handcrafted, neural networks learn increasingly complex features directly from the data. Instead of handcrafting the features, it is now the network architecture that is manually engineered. The network architecture parameters such as the number of layers or the number of filters per layer and their interconnections are essential for good performance. Even though basic design guidelines exist, designing a neural network is an iterative trial-and-error process that takes days or even weeks to perform due to the large datasets used for training. In this paper, we present DeepEyes, a Progressive Visual Analytics system that supports the design of neural networks during training. We present novel visualizations, supporting the identification of layers that learned a stable set of patterns and, therefore, are of interest for a detailed analysis. The system facilitates the identification of problems, such as superfluous filters or layers, and information that is not being captured by the network. We demonstrate the effectiveness of our system through multiple use cases, showing how a trained network can be compressed, reshaped and adapted to different problems.

Deepeyes

DeepEyes: Progressive Visual Analytics for Designing Deep Neural Networks

For interactive exploration of large-scale data, a preprocessing scheme (e.g., data cubes) has often been used to summarize the data and provide low-latency responses. However, such a scheme suffers from a prohibitively large amount of memory footprint as more dimensions are involved in querying, and a strong prerequisite that specific data structures have to be built from the data before querying. In this paper, we present SwiftTuna, a holistic system that streamlines the visual information seeking process on large-scale multidimensional data. SwiftTuna exploits an in-memory computing engine, Apache Spark, to achieve both scalability and performance without building precomputed data structures. We also present a novel interactive visualization technique, tailed charts, to facilitate large-scale multidimensional data exploration. To support responsive querying on large-scale data, SwiftTuna leverages an incremental processing approach, providing immediate low-fidelity responses (i.e., prompt responses) as well as delayed high-fidelity responses (i.e., incremental responses). Our performance evaluation demonstrates that SwiftTuna allows data exploration of a real-world dataset with four billion records while preserving the latency between incremental responses within a few seconds.

Swifttuna

SwiftTuna: Responsive and incremental visual exploration of large-scale multidimensional data

One of the key advantages of visual analytics is its capability to leverage both humans's visual perception and the power of computing. A big obstacle in integrating machine learning with visual analytics is its high computing cost. To tackle this problem, this paper presents PIVE (Per-Iteration Visualization Environment) that supports real-time interactive visualization with machine learning. By immediately visualizing the intermediate results from algorithm iterations, PIVE enables users to quickly grasp insights and interact with the intermediate output, which then affects subsequent algorithm iterations. In addition, we propose a widely-applicable interaction methodology that allows efficient incorporation of user feedback into virtually any iterative computational method without introducing additional computational cost. We demonstrate the application of PIVE for various dimension reduction algorithms such as multidimensional scaling and t-SNE and clustering and topic modeling algorithms such as k-means and latent Dirichlet allocation.

PIVE

PIVE: per-iteration visualization environment for real-time interactions with dimension reduction and clustering


Sampling is becoming an essential tool for scalable interactive visual analysis. After outlining prior work by the database community on sampling for visualization of aggregation queries, this article considers how these results might be improved and extended to a broader setting. The goal is to better understand how users interact with sampling to enable wider adoption of sampling for scalable visual analytics.

Sampling for scalable VA

Sampling for scalable visual analytics

The emerging field of Progressive Visual Analytics (PVA in what follows) deals with the objective of progressively create the final visualization through a series of intermediate visual results, affected by a degree of uncertainty and, in some cases, a non monotonic behaviour. According to that, it is a critical issue providing the user with no confusing visualization and that results in a novel point of view on stability and consistency. This position paper deals with the novel and challenging issues that PVA poses in term of visual stability and consistency, providing a preliminary framework in which this problem can be contextualized, measured, and formalized. In particular, the framework proposes a set of metrics, able to explore both data and visual changes; a preliminary case study demonstrates their applicability and advantages in adequately representing data changes in a visualization.

Visual Stability

On Visual Stability and Visual Consistency for Progressive Visual Analytics

Big data, complex computations, and the need for fluent interaction, are the well known enemies of Visual Analytics. They can seriously impair the fluent interactive back and forth between computational analysis and human analyst that happens using the visualizations that a Visual Analytics solution provides for supporting this combined analysis. The emerging Progressive Visual Analytics (PVA) took the field and wage a war against such enemies, providing a way around this conundrum by iteratively computing partial results of increasing quality, that constitute a natural means for providing the analyst with early and continuous interaction with the initial results even while the whole process is still far from being completed. However, this promising solution has several implications that must be dealt with, requiring to address different issues. This paper has the goal of providing a travel guide about the practical usage of PVA, discussing the motivations that call for its usage, the possible strategies that can be used to solve the problem and the fee that requires to be payed in order to use this approach. In order to provide a concrete discussion, the paper is driven by a concrete and intriguing example, an intractable PVA solution developed for Telecom Italia Mobile (TIM)

Dark Side

The dark side of progressive visual analytics

We present a progressive algorithm for approximate k-nearest neighbor search. Although the use of k-nearest neighbor libraries (KNN) is common in many data analysis methods, most KNN algorithms can only be run when the whole dataset has been indexed, i.e., they are not online. Even the few online implementations are not progressive in the sense that the time to index incoming data is not bounded and can exceed the latency required by progressive systems. This latency significantly restricts the interactivity of visualization systems especially when dealing with large-scale data. We improve traditional k-d trees for progressive approximate k-nearest neighbor search, enabling fast KNN queries while continuously indexing new batches of data when necessary. Following the progressive computation paradigm, our progressive k-d tree is bounded in time, allowing analysts to access ongoing results within an interactive latency. We also present performance benchmarks to compare online and progressive k-d trees.

k-d tree

A Progressive k-d tree for Approximate k-Nearest Neighbors

—Visual exploratory analysis of large-scale databases often relies on precomputed query results in order to guarantee interactivity with the visualization system. This is especially true when the query requires joining tables across the database since join is one of the most computationally expensive operations and in the worst case requires joining every row of one table to every row of the other table. Advances in approximate query processing enable quick look summary statistics, but with limitations to the types and complexities of the queries. A recent advancement in approximate query processing is a technique called Wander Join, that performs random walks across these joins, resulting in faster convergence on aggregation values. However, this online aggregation technique is not tailored for visualization tasks that often involve filtering on one or more conditions or viewing aggregation values across multiple groups such as in bar charts. In both cases, the convergence rate is slowed since more samples are needed in order to find records that pass the filters, resulting in the user waiting longer for a confident result. To address this issue, we propose a generalization of the Wander Join algorithm that improves the convergence rate for visualization queries involving filtering and viewing aggregation. We implemented this improved version of Wander Join that we call Load-n-Go and compared it to the original, specifically in the context of visual analysis tasks. Our evaluation finds that our algorithm outperforms Wander Join by reducing the sample complexity. Load-n-Go requires up to 50% fewer samples for group by queries, and up to 85% fewer samples for filtering queries. Such reduced sampling complexity can represent up to 2x and 6x speedups respectively for visual exploratory systems using the Load-n-Go algorithm.

Load-n-Go

Load-n-Go: Fast Approximate Join Visualizations That Improve Over Time

Visual analytics is entering a period of renewed growth due to a shift in focus from static to streaming data applications. In this article, the authors illustrate several challenges arising from this pivot and suggest potential avenues for future exploration.

Streaming Data

Rethinking Visual Analytics for Streaming Data Applications

Analysts need interactive speed for exploratory analysis, but big data systems are often slow. With sampling, data systems can produce approximate answers fast enough for exploratory visualization, at the cost of accuracy and trust. We propose optimistic visualization, which approaches these issues from a user experience perspective. This method lets analysts explore approximate results interactively, and provides a way to detect and recover from errors later. Pangloss implements these ideas. We discuss design issues raised by optimistic visualization systems. We test this concept with five expert visualizers in a laboratory study and three case studies at Microsoft. Analysts reported that they felt more confident in their results, and used optimistic visualization to check that their preliminary results were correct.

Trust, but Verify

Trust, but Verify: Optimistic Visualizations of Approximate Queries for Exploring Big Data

In interactive data analysis processes, the dialogue between the human and the computer is the enabling mechanism that can lead to actionable observations about the phenomena being investigated. It is of paramount importance that this dialogue is not interrupted by slow computational mechanisms that do not consider any known temporal human-computer interaction characteristics that prioritize the perceptual and cognitive capabilities of the users. In cases where the analysis involves an integrated computational method, for instance to reduce the dimensionality of the data or to perform clustering, such non-optimal processes are often likely. To remedy this, progressive computations, where results are iteratively improved, are getting increasing interest in visual analytics. In this paper, we present techniques and design considerations to incorporate progressive methods within interactive analysis processes that involve high-dimensional data. We define methodologies to facilitate processes that adhere to the perceptual characteristics of users and describe how online algorithms can be incorporated within these. A set of design recommendations and according methods to support analysts in accomplishing high-dimensional data analysis tasks are then presented. Our arguments and decisions here are informed by observations gathered over a series of analysis sessions with analysts from finance. We document observations and recommendations from this study and present evidence on how our approach contribute to the efficiency and productivity of interactive visual analysis sessions involving high-dimensional data.

Interactive Analytics Processes 

Designing Progressive and Interactive Analytics Processes for High-Dimensional Data Analysis

The stated goal for visual data exploration is to operate at a rate that matches the pace of human data analysts, but the ever increasing amount of data has led to a fundamental problem: datasets are often too large to process within interactive time frames. Progressive analytics and visualizations have been proposed as potential solutions to this issue. By processing data incrementally in small chunks, progressive systems provide approximate query answers at interactive speeds that are then refined over time with increasing precision. We study how progressive visualizations affect users in exploratory settings in an experiment where we capture user behavior and knowledge discovery through interaction logs and think-aloud protocols. Our experiment includes three visualization conditions and different simulated dataset sizes. The visualization conditions are: (1) blocking, where results are displayed only after the entire dataset has been processed; (2) instantaneous, a hypothetical condition where results are shown almost immediately; and (3) progressive, where approximate results are displayed quickly and then refined over time. We analyze the data collected in our experiment and observe that users perform equally well with either instantaneous or progressive visualizations in key metrics, such as insight discovery rates and dataset coverage, while blocking visualizations have detrimental effects.

Exploratory analysis

How progressive visualizations affect exploratory analysis.

Exploring data requires a fast feedback loop from the analyst to the system, with a latency below about 10 seconds because of human cognitive limitations. When data becomes large or analysis becomes complex, sequential computations can no longer be completed in a few seconds and data exploration is severely hampered. This article describes a novel computation paradigm called Progressive Computation for Data Analysis or more concisely Progressive Analytics, that brings at the programming language level a low-latency guarantee by performing computations in a progressive fashion. Moving this progressive computation at the language level relieves the programmer of exploratory data analysis systems from implementing the whole analytics pipeline in a progressive way from scratch, streamlining the implementation of scalable exploratory data analysis systems. This article describes the new paradigm through a prototype implementation called ProgressiVis, and explains the requirements it implies through examples.

Computation Paradigm

Progressive Analytics: A Computation Paradigm for Exploratory Data Analysis

In the age of Big Data, the real-time interactive visualization is a challenge due to latency of executing calculation over terabytes (even, petabytes) datasets. The execution of an operation has to finish before its outcome is displayed, which would be an issue in those scenarios where low-latency responses are required. To address such a requirement, this paper introduces a new approach for real-time visualization of extremely large data-at-rest as well as data-in-motion by showing intermediate results as soon as they become available. This should allow the data analyst to take decisions in real-time.

Real-Time Big Data

An Incremental Approach for Real-Time Big Data Visual Analytics

Tensorflow

Playground Tensorflow

We present a novel technique to generate transformations between arbitrary volumes, providing both expressive distances and smooth interpolates. In contrast to conventional morphing or warping approaches, our technique requires no user guidance, intermediate representations (like extracted features), or blending, and imposes no restrictions regarding shape or structure. Our technique operates directly on the volumetric data representation, and while linear programming approaches could solve the underlying problem optimally, their polynomial complexity makes them infeasible for high-resolution volumes. We therefore propose a progressive refinement approach designed for parallel execution that is able to quickly deliver approximate results that are iteratively improved toward the optimum. On this basis, we further present a new approach for the streaming selection of time steps in temporal data that allows for the reconstruction of the full sequence with a user-specified error bound. We finally demonstrate the utility of our technique for different applications, compare our approach against alternatives, and evaluate its characteristics with a variety of different data sets.

Volume-to-Volume

Progressive Direct Volume-to-Volume Transformation

We present an online visual analytics approach to helping users explore and understand hierarchical topic evolution in high-volume text streams. The key idea behind this approach is to identify representative topics in incoming documents and align them with the existing representative topics that they immediately follow (in time). To this end, we learn a set of streaming tree cuts from topic trees based on user-selected focus nodes. A dynamic Bayesian network model has been developed to derive the tree cuts in the incoming topic trees to balance the fitness of each tree cut and the smoothness between adjacent tree cuts. By connecting the corresponding topics at different times, we are able to provide an overview of the evolving hierarchical topics. A sedimentation-based visualization has been designed to enable the interactive analysis of streaming text data from global patterns to local details. We evaluated our method on real-world datasets and the results are generally favorable.