VA + Embeddings STAR: A State-of-the-Art Report on the Use of Embeddings in Visual Analytics

Over the past years, an increasing number of publications in information visualization, especially within the field of visual analytics, have mentioned the term “embedding” when describing the computational approach. Within this context, embeddings are usually (relatively) low-dimensional, distributed representations of various data types (such as texts or graphs), and since they have proven to be extremely useful for a variety of data analysis tasks across various disciplines and fields, they have become widely used. Existing visualization approaches aim to either support exploration and interpretation of the embedding space through visual representation and interaction, or aim to use embeddings as part of the computational pipeline for addressing downstream analytical tasks. To the best of our knowledge, this is the first survey that takes a detailed look at embedding methods through the lens of visual analytics, and the purpose of our survey article is to provide a systematic overview of the state of the art within the emerging field of embedding visualization. We design a categorization scheme for our approach, analyze the current research frontier based on peer-reviewed publications, and discuss existing trends, challenges, and potential research directions for using embeddings in the context of visual analytics. Furthermore, we provide an interactive survey browser for the collected and categorized survey data, which currently includes 122 entries that appeared between 2007 and 2023.     

Beyond Alternative Text and tables: Comparative Analysis of Visualization Tools and Accessibility Methods

Modern visualization software and programming libraries have made data visualization construction easier for everyone. However, the extent of accessibility design they support for blind and low-vision people is relatively unknown. It is also unclear how they can improve chart content accessibility beyond conventional alternative text and data tables. To address these issues, we examined the current accessibility features in popular visualization tools, revealing limited support for the standard accessibility methods and scarce support for chart content exploration. Next, we investigate two promising accessibility approaches that provide off-the-shelf solutions for chart content accessibility: structured navigation and conversational interaction. We present a comparative evaluation study and discuss what to consider when incorporating them into visualization tools.     

RectEuler: Visualizing Intersecting Sets using Rectangles

Euler diagrams are a popular technique to visualize set-typed data. However, creating diagrams using simple shapes remains a challenging problem for many complex, real-life datasets. To solve this, we propose RectEuler: a flexible, fully-automatic method using rectangles to create Euler-like diagrams. We use an efficient mixed-integer optimization scheme to place set labels and element representatives (e.g., text or images) in conjunction with rectangles describing the sets. By defining appropriate constraints, we adhere to well-formedness properties and aesthetic considerations. If a dataset cannot be created within a reasonable time or at all, we iteratively split the diagram into multiple components until a drawable solution is found. Redundant encoding of the set membership using dots and set lines improves the readability of the diagram. Our web tool lets users see how the layout changes throughout the optimization process and provides interactive explanations. For evaluation, we perform quantitative and qualitative analysis across different datasets and compare our method to state-of-the-art Euler diagram generation methods.     

Classifier-Guided Visual Correction of Noisy Labels for Image Classification Tasks

Training data plays an essential role in modern applications of machine learning. However, gathering labeled training data is time-consuming. Therefore, labeling is often outsourced to less experienced users, or completely automated. This can introduce errors, which compromise valuable training data, and lead to suboptimal training results. We thus propose a novel approach that uses the power of pretrained classifiers to visually guide users to noisy labels, and let them interactively check error candidates, to iteratively improve the training data set. To systematically investigate training data, we propose a categorization of labeling errors into three different types, based on an analysis of potential pitfalls in label acquisition processes. For each of these types, we present approaches to detect, reason about, and resolve error candidates, as we propose measures and visual guidance techniques to support machine learning users. Our approach has been used to spot errors in well-known machine learning benchmark data sets, and we tested its usability during a user evaluation. While initially developed for images, the techniques presented in this paper are independent of the classification algorithm, and can also be extended to many other types of training data.     

Quantitative Comparison of Time-Dependent Treemaps

Rectangular treemaps are often the method of choice to visualize large hierarchical datasets. Nowadays such datasets are available over time, hence there is a need for (a) treemaps that can handle time-dependent data, and (b) corresponding quality criteria that cover both a treemap's visual quality and its stability over time. In recent years a wide variety of (stable) treemapping algorithms has been proposed, with various advantages and limitations. We aim to provide insights to researchers and practitioners to allow them to make an informed choice when selecting a treemapping algorithm for specific applications and data. To this end, we perform an extensive quantitative evaluation of rectangular treemaps for time-dependent data. As part of this evaluation we propose a novel classification scheme for time-dependent datasets. Specifically, we observe that the performance of treemapping algorithms depends on the characteristics of the datasets used. We identify four potential representative features that characterize time-dependent hierarchical datasets and classify all datasets used in our experiments accordingly. We experimentally test the validity of this classification on more than 2000 datasets, and analyze the relative performance of 14 state-of-the-art rectangular treemapping algorithms across varying features. Finally, we visually summarize our results with respect to both visual quality and stability to aid users in making an informed choice among treemapping algorithms. All datasets, metrics, and algorithms are openly available to facilitate reuse and further comparative studies.     

QUESTO: Interactive Construction of Objective Functions for Classification Tasks

Building effective classifiers requires providing the modeling algorithms with information about the training data and modeling goals in order to create a model that makes proper tradeoffs. Machine learning algorithms allow for flexible specification of such meta-information through the design of the objective functions that they solve. However, such objective functions are hard for users to specify as they are a specific mathematical formulation of their intents. In this paper, we present an approach that allows users to generate objective functions for classification problems through an interactive visual interface. Our approach adopts a semantic interaction design in that user interactions over data elements in the visualization are translated into objective function terms. The generated objective functions are solved by a machine learning solver that provides candidate models, which can be inspected by the user, and used to suggest refinements to the specifications. We demonstrate a visual analytics system QUESTO for users to manipulate objective functions to define domain-specific constraints. Through a user study we show that QUESTO helps users create various objective functions that satisfy their goals.     

SEEVis: A Smart Emergency Evacuation Plan Visualization System with Data-Driven Shot Designs

Despite the significance of tracking human mobility dynamics in a large-scale earthquake evacuation for an effective first response and disaster relief, the general understanding of evacuation behaviors remains limited. Numerous individual movement trajectories, disaster damages of civil engineering, associated heterogeneous data attributes, as well as complex urban environment all obscure disaster evacuation analysis. Although visualization methods have demonstrated promising performance in emergency evacuation analysis, they cannot effectively identify and deliver the major features like speed or density, as well as the resulting evacuation events like congestion or turn-back. In this study, we propose a shot design approach to generate customized and narrative animations to track different evacuation features with different exploration purposes of users. Particularly, an intuitive scene feature graph that identifies the most dominating evacuation events is first constructed based on user-specific regions or their tracking purposes on a certain feature. An optimal camera route, i.e., a storyboard is then calculated based on the previous user-specific regions or features. For different evacuation events along this route, we employ the corresponding shot design to reveal the underlying feature evolution and its correlation with the environment. Several case studies confirm the efficacy of our system. The feedback from experts and users with different backgrounds suggests that our approach indeed helps them better embrace a comprehensive understanding of the earthquake evacuation.     

WYTIWYR: A User Intent-Aware Framework with Multi-modal Inputs for Visualization Retrieval

Retrieving charts from a large corpus is a fundamental task that can benefit numerous applications such as visualization recommendations. The retrieved results are expected to conform to both explicit visual attributes (e.g., chart type, colormap) and implicit user intents (e.g., design style, context information) that vary upon application scenarios. However, existing example-based chart retrieval methods are built upon non-decoupled and low-level visual features that are hard to interpret, while definition-based ones are constrained to pre-defined attributes that are hard to extend. In this work, we propose a new framework, namely WYTIWYR (What-You-Think-Is-What-You-Retrieve), that integrates user intents into the chart retrieval process. The framework consists of two stages: first, the Annotation stage disentangles the visual attributes within the query chart; and second, the Retrieval stage embeds the user's intent with customized text prompt as well as bitmap query chart, to recall targeted retrieval result. We develop aprototype WYTIWYR system leveraging a contrastive language-image pre-training (CLIP) model to achieve zero-shot classification as well as multi-modal input encoding, and test the prototype on a large corpus with charts crawled from the Internet. Quantitative experiments, case studies, and qualitative interviews are conducted. The results demonstrate the usability and effectiveness of our proposed framework.     

Canis: A High-Level Language for Data-Driven Chart Animations

In this paper, we introduce Canis, a high-level domain-specific language that enables declarative specifications of data-driven chart animations. By leveraging data-enriched SVG charts, its grammar of animations can be applied to the charts created by existing chart construction tools. With Canis, designers can select marks from the charts, partition the selected marks into mark units based on data attributes, and apply animation effects to the mark units, with the control of when the effects start. The Canis compiler automatically synthesizes the Lottie animation JSON files [Aira], which can be rendered natively across multiple platforms. To demonstrate Canis’ expressiveness, we present a wide range of chart animations. We also evaluate its scalability by showing the effectiveness of our compiler in reducing the output specification size and comparing its performance on different platforms against D3.     

Knowledge-Assisted Comparative Assessment of Breast Cancer using Dynamic Contrast-Enhanced Magnetic Resonance Imaging

Breast perfusion data are dynamic medical image data that depict perfusion characteristics of the investigated tissue. These data consist of a series of static datasets that are acquired at different time points and aggregated into time intensity curves (TICs) for each voxel. The characteristics of these TICs provide important information about a lesion's composition, but their analysis is time-consuming due to their large number. Subsequently, these TICs are used to classify a lesion as benign or malignant. This lesion scoring is commonly done manually by physicians and may therefore be subject to bias. We propose an approach that addresses both of these problems by combining an automated lesion classification with a visual confirmatory analysis, especially for uncertain cases. Firstly, we cluster the TICs of a lesion using ordering points to identify the clustering structure (OPTICS) and then visualize these clusters. Together with their relative size, they are added to a library. We then model fuzzy inference rules by using the lesion's TIC clusters as antecedents and its score as consequent. Using a fuzzy scoring system, we can suggest a score for a new lesion. Secondly, to allow physicians to confirm the suggestion in uncertain cases, we display the TIC clusters together with their spatial distribution and allow them to compare two lesions side by side. With our knowledge-assisted comparative visual analysis, physicians can explore and classify breast lesions. The true positive prediction accuracy of our scoring system achieved 71.4 % in one-fold cross-validation using 14 lesions.     

A Visual Analytics Approach to Facilitate Crime Hotspot Analysis

Computer-based technology has played a significant role in crime prevention over the past 30 years, especially with the popularization of spatial databases and crime mapping systems. Police departments frequently use hotspot analysis to identify regions that should be a priority in receiving preventive resources. Practitioners and researchers agree that tracking crime over time and identifying its geographic patterns are vital information for planning efficiently. Frequently, police departments have access to systems that are too complicated and excessively technical, leading to modest usage. By working closely together with domain experts from police agencies of two different countries, we identified and characterized five domain tasks inherent to the hotspot analysis problem and developed SHOC, a visualization tool that strives for simplicity and ease of use in helping users to perform all the domain tasks. SHOC is included in a visual analytics system that allows users without technical expertise to annotate, save, and share analyses. We also demonstrate that our system effectively supports the completion of the domain tasks in two different real-world case studies.     

Invertible Paradoxic Loop Structures for Transformable Design

We present an interactive tool compatible with existing software (Rhino/Grasshopper) to design ring structures with a paradoxic mobility, which are self-collision-free over the complete motion cycle. Our computational approach allows non-expert users to create these invertible paradoxic loops with six rotational joints by providing several interactions that facilitate design exploration. In a first step, a rational cubic motion is shaped either by means of a four pose interpolation procedure or a motion evolution algorithm. By using the representation of spatial displacements in terms of dual-quaternions, the associated motion polynomial of the resulting motion can be factored in several ways, each corresponding to a composition of three rotations. By combining two suitable factorizations, an arrangement of six rotary axes is achieved, which possesses a 1-parametric mobility. In the next step, these axes are connected by links in a way that the resulting linkage is collision-free over the complete motion cycle. Based on an algorithmic solution for this problem, collision-free design spaces of the individual links are generated in a post-processing step. The functionality of the developed design tool is demonstrated in the context of an architectural and artistic application studied in a master-level studio course. Two results of the performed design experiments were fabricated by the use of computer-controlled machines to achieve the necessary accuracy ensuring the mobility of the models.     

ParaDime: A Framework for Parametric Dimensionality Reduction

ParaDime is a framework for parametric dimensionality reduction (DR). In parametric DR, neural networks are trained to embed high-dimensional data items in a low-dimensional space while minimizing an objective function. ParaDime builds on the idea that the objective functions of several modern DR techniques result from transformed inter-item relationships. It provides a common interface for specifying these relations and transformations and for defining how they are used within the losses that govern the training process. Through this interface, ParaDime unifies parametric versions of DR techniques such as metric MDS, t-SNE, and UMAP. It allows users to fully customize all aspects of the DR process. We show how this ease of customization makes ParaDime suitable for experimenting with interesting techniques such as hybrid classification/embedding models and supervised DR. This way, ParaDime opens up new possibilities for visualizing high-dimensional data.     

PAVED: Pareto Front Visualization for Engineering Design

Design problems in engineering typically involve a large solution space and several potentially conflicting criteria. Selecting a compromise solution is often supported by optimization algorithms that compute hundreds of Pareto-optimal solutions, thus informing a decision by the engineer. However, the complexity of evaluating and comparing alternatives increases with the number of criteria that need to be considered at the same time. We present a design study on Pareto front visualization to support engineers in applying their expertise and subjective preferences for selection of the most-preferred solution. We provide a characterization of data and tasks from the parametric design of electric motors. The requirements identified were the basis for our development of PAVED, an interactive parallel coordinates visualization for exploration of multi-criteria alternatives. We reflect on our user-centered design process that included iterative refinement with real data in close collaboration with a domain expert as well as a summative evaluation in the field. The results suggest a high usability of our visualization as part of a real-world engineering design workflow. Our lessons learned can serve as guidance to future visualization developers targeting multi-criteria optimization problems in engineering design or alternative domains.     

Orchard: Exploring Multivariate Heterogeneous Networks on Mobile Phones

People are becoming increasingly sophisticated in their ability to navigate information spaces using search, hyperlinks, and visualization. But, mobile phones preclude the use of multiple coordinated views that have proven effective in the desktop environment (e.g., for business intelligence or visual analytics). In this work, we propose to model information as multivariate heterogeneous networks to enable greater analytic expression for a range of sensemaking tasks while suggesting a new, list-based paradigm with gestural navigation of structured information spaces on mobile phones. We also present a mobile application, called Orchard, which combines ideas from both faceted search and interactive network exploration in a visual query language to allow users to collect facets of interest during exploratory navigation. Our study showed that users could collect and combine these facets with Orchard, specifying network queries and projections that would only have been possible previously using complex data tools or custom data science.     

Phase Space Projection of Dynamical Systems

Dynamical systems are commonly used to describe the state of time-dependent systems. In many engineering and control problems, the state space is high-dimensional making it difficult to analyze and visualize the behavior of the system for varying input conditions. We present a novel dimensionality reduction technique that is tailored to high-dimensional dynamical systems. In contrast to standard general purpose dimensionality reduction algorithms, we use energy minimization to preserve properties of the flow in the high-dimensional space. Once the projection operator is optimized, further high-dimensional trajectories are projected easily. Our 3D projection maintains a number of useful flow properties, such as critical points and flow maps, and is optimized to match geometric characteristics of the high-dimensional input, as well as optional user constraints. We apply our method to trajectories traced in the phase spaces of second-order dynamical systems, including finite-sized objects in fluids, the circular restricted three-body problem and a damped double pendulum. We compare the projections with standard visualization techniques, such as PCA, t-SNE and UMAP, and visualize the dynamical systems with multiple coordinated views interactively, featuring a spatial embedding, projection to subspaces, our dimensionality reduction and a seed point exploration tool.     

Co-creating Visualizations: A First Evaluation with Social Science Researchers

Co-creation is a design method where designers and domain experts work together to develop a product. In this paper, we present and evaluate the use of co-creation to design a visual information system with social science researchers in order to explore and analyze their data. Co-creation proposes involving the future users in the design process to ensure that they play a critical role in the design, and to increase the chances of long-term adoption. We evaluated the co-creation process through surveys, interviews and a user study. According to the participants’ feedback, they felt listened to through co-creation, and considered the methodology helpful to develop visualizations that support their research in the near future. However, participation was far from perfect, particularly early career researchers showed limited interest in participating because they did not see the process as beneficial for their research publication goals. We summarize benefits and limitations of co-creation, together with our recommendations, as lessons learned.     

The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations

Machine learning (ML) models are nowadays used in complex applications in various domains, such as medicine, bioinformatics, and other sciences. Due to their black box nature, however, it may sometimes be hard to understand and trust the results they provide. This has increased the demand for reliable visualization tools related to enhancing trust in ML models, which has become a prominent topic of research in the visualization community over the past decades. To provide an overview and present the frontiers of current research on the topic, we present a State-of-the-Art Report (STAR) on enhancing trust in ML models with the use of interactive visualization. We define and describe the background of the topic, introduce a categorization for visualization techniques that aim to accomplish this goal, and discuss insights and opportunities for future research directions. Among our contributions is a categorization of trust against different facets of interactive ML, expanded and improved from previous research. Our results are investigated from different analytical perspectives: (a) providing a statistical overview, (b) summarizing key findings, (c) performing topic analyses, and (d) exploring the data sets used in the individual papers, all with the support of an interactive web-based survey browser. We intend this survey to be beneficial for visualization researchers whose interests involve making ML models more trustworthy, as well as researchers and practitioners from other disciplines in their search for effective visualization techniques suitable for solving their tasks with confidence and conveying meaning to their data.