WYTIWYR: A User Intent-Aware Framework with Multi-modal Inputs for Visualization Retrieval

Retrieving charts from a large corpus is a fundamental task that can benefit numerous applications such as visualization recommendations. The retrieved results are expected to conform to both explicit visual attributes (e.g., chart type, colormap) and implicit user intents (e.g., design style, context information) that vary upon application scenarios. However, existing example-based chart retrieval methods are built upon non-decoupled and low-level visual features that are hard to interpret, while definition-based ones are constrained to pre-defined attributes that are hard to extend. In this work, we propose a new framework, namely WYTIWYR (What-You-Think-Is-What-You-Retrieve), that integrates user intents into the chart retrieval process. The framework consists of two stages: first, the Annotation stage disentangles the visual attributes within the query chart; and second, the Retrieval stage embeds the user's intent with customized text prompt as well as bitmap query chart, to recall targeted retrieval result. We develop aprototype WYTIWYR system leveraging a contrastive language-image pre-training (CLIP) model to achieve zero-shot classification as well as multi-modal input encoding, and test the prototype on a large corpus with charts crawled from the Internet. Quantitative experiments, case studies, and qualitative interviews are conducted. The results demonstrate the usability and effectiveness of our proposed framework.     

Been There,Seen That: Visualization of Movement and 3D Eye Tracking Data from Real-World Environments

The distribution of visual attention can be evaluated using eye tracking, providing valuable insights into usability issues and interaction patterns. However, when used in real, augmented, and collaborative environments, new challenges arise that go beyond desktop scenarios and purely virtual environments. Toward addressing these challenges, we present a visualization technique that provides complementary views on the movement and eye tracking data recorded from multiple people in real-world environments. Our method is based on a space-time cube visualization and a linked 3D replay of recorded data. We showcase our approach with an experiment that examines how people investigate an artwork collection. The visualization provides insights into how people moved and inspected individual pictures in their spatial context over time. In contrast to existing methods, this analysis is possible for multiple participants without extensive annotation of areas of interest. Our technique was evaluated with a think-aloud experiment to investigate analysis strategies and an interview with domain experts to examine the applicability in other research fields.     

Doppler Volume Rendering: A Dynamic,Piecewise Linear Spectral Representation for Visualizing Astrophysics Simulations

We present a novel approach for rendering volumetric data including the Doppler effect of light. Similar to the acoustic Doppler effect, which is caused by relative motion between a sound emitter and an observer, light waves also experience compression or expansion when emitter and observer exhibit relative motion. We account for this by employing spectral volume rendering in an emission–absorption model, with the volumetric matter moving according to an accompanying vector field, and emitting and attenuating light at wavelengths subject to the Doppler effect. By introducing a novel piecewise linearear representation of the involved light spectra, we achieve accurate volume rendering at interactive frame rates. We compare our technique to rendering with traditional point-based spectral representation, and demonstrate its utility using a simulation of galaxy formation.     

Mini-VLAT: A Short and Effective Measure of Visualization Literacy

The visualization community regards visualization literacy as a necessary skill. Yet, despite the recent increase in research into visualization literacy by the education and visualization communities, we lack practical and time-effective instruments for the widespread measurements of people's comprehension and interpretation of visual designs. We present Mini-VLAT, a brief but practical visualization literacy test. The Mini-VLAT is a 12-item short form of the 53-item Visualization Literacy Assessment Test (VLAT). The Mini-VLAT is reliable (coefficient omega = 0.72) and strongly correlates with the VLAT. Five visualization experts validated the Mini-VLAT items, yielding an average content validity ratio (CVR) of 0.6. We further validate Mini-VLAT by demonstrating a strong positive correlation between study participants' Mini-VLAT scores and their aptitude for learning an unfamiliar visualization using a Parallel Coordinate Plot test. Overall, the Mini-VLAT items showed a similar pattern of validity and reliability as the 53-item VLAT. The results show that Mini-VLAT is a psychometrically sound and practical short measure of visualization literacy.     

SEEVis: A Smart Emergency Evacuation Plan Visualization System with Data-Driven Shot Designs

Despite the significance of tracking human mobility dynamics in a large-scale earthquake evacuation for an effective first response and disaster relief, the general understanding of evacuation behaviors remains limited. Numerous individual movement trajectories, disaster damages of civil engineering, associated heterogeneous data attributes, as well as complex urban environment all obscure disaster evacuation analysis. Although visualization methods have demonstrated promising performance in emergency evacuation analysis, they cannot effectively identify and deliver the major features like speed or density, as well as the resulting evacuation events like congestion or turn-back. In this study, we propose a shot design approach to generate customized and narrative animations to track different evacuation features with different exploration purposes of users. Particularly, an intuitive scene feature graph that identifies the most dominating evacuation events is first constructed based on user-specific regions or their tracking purposes on a certain feature. An optimal camera route, i.e., a storyboard is then calculated based on the previous user-specific regions or features. For different evacuation events along this route, we employ the corresponding shot design to reveal the underlying feature evolution and its correlation with the environment. Several case studies confirm the efficacy of our system. The feedback from experts and users with different backgrounds suggests that our approach indeed helps them better embrace a comprehensive understanding of the earthquake evacuation.     

LOCALIS: Locally-adaptive Line Simplification for GPU-based Geographic Vector Data Visualization

Visualization of large vector line data is a core task in geographic and cartographic systems. Vector maps are often displayed at different cartographic generalization levels, traditionally by using several discrete levels-of-detail (LODs). This limits the generalization levels to a fixed and predefined set of LODs, and generally does not support smooth LOD transitions. However, fast GPUs and novel line rendering techniques can be exploited to integrate dynamic vector map LOD management into GPU-based algorithms for locally-adaptive line simplification and real-time rendering. We propose a new technique that interactively visualizes large line vector datasets at variable LODs. It is based on the Douglas-Peucker line simplification principle, generating an exhaustive set of line segments whose specific subsets represent the lines at any variable LOD. At run time, an appropriate and view-dependent error metric supports screen-space adaptive LOD levels and the display of the correct subset of line segments accordingly. Our implementation shows that we can simplify and display large line datasets interactively. We can successfully apply line style patterns, dynamic LOD selection lenses, and anti-aliasing techniques to our line rendering.     

Visual Gaze Labeling for Augmented Reality Studies

Augmented Reality (AR) provides new ways for situated visualization and human-computer interaction in physical environments. Current evaluation procedures for AR applications rely primarily on questionnaires and interviews, providing qualitative means to assess usability and task solution strategies. Eye tracking extends these existing evaluation methodologies by providing indicators for visual attention to virtual and real elements in the environment. However, the analysis of viewing behavior, especially the comparison of multiple participants, is difficult to achieve in AR. Specifically, the definition of areas of interest (AOIs), which is often a prerequisite for such analysis, is cumbersome and tedious with existing approaches. To address this issue, we present a new visualization approach to define AOIs, label fixations, and investigate the resulting annotated scanpaths. Our approach utilizes automatic annotation of gaze on virtual objects and an image-based approach that also considers spatial context for the manual annotation of objects in the real world. Our results show, that with our approach, eye tracking data from AR scenes can be annotated and analyzed flexibly with respect to data aspects and annotation strategies.     

VISITOR: Visual Interactive State Sequence Exploration for Reinforcement Learning

Understanding the behavior of deep reinforcement learning agents is a crucial requirement throughout their development. Existing work has addressed the identification of observable behavioral patterns in state sequences or analysis of isolated internal representations; however, the overall decision-making of deep-learning RL agents remains opaque. To tackle this, we present VISITOR, a visual analytics system enabling the analysis of entire state sequences, the diagnosis of singular predictions, and the comparison between agents. A sequence embedding view enables the multiscale analysis of state sequences, utilizing custom embedding techniques for a stable spatialization of the observations and internal states. We provide multiple layers: (1) a state space embedding, highlighting different groups of states inside the state-action sequences, (2) a trajectory view, emphasizing decision points, (3) a network activation mapping, visualizing the relationship between observations and network activations, (4) a transition embedding, enabling the analysis of state-to-state transitions. The embedding view is accompanied by an interactive reward view that captures the temporal development of metrics, which can be linked directly to states in the embedding. Lastly, a model list allows for the quick comparison of models across multiple metrics. Annotations can be exported to communicate results to different audiences. Our two-stage evaluation with eight experts confirms the effectiveness in identifying states of interest, comparing the quality of policies, and reasoning about the internal decision-making processes.     

Slice and Dice: A Physicalization Workflow for Anatomical Edutainment

During the last decades, anatomy has become an interesting topic in education—even for laymen or schoolchildren. As medical imaging techniques become increasingly sophisticated, virtual anatomical education applications have emerged. Still, anatomical models are often preferred, as they facilitate 3D localization of anatomical structures. Recently, data physicalizations (i.e., physical visualizations) have proven to be effective and engaging—sometimes, even more than their virtual counterparts. So far, medical data physicalizations involve mainly 3D printing, which is still expensive and cumbersome. We investigate alternative forms of physicalizations, which use readily available technologies (home printers) and inexpensive materials (paper or semi-transparent films) to generate crafts for anatomical edutainment. To the best of our knowledge, this is the first computer-generated crafting approach within an anatomical edutainment context. Our approach follows a cost-effective, simple, and easy-to-employ workflow, resulting in assemblable data sculptures (i.e., semi-transparent sliceforms). It primarily supports volumetric data (such as CT or MRI), but mesh data can also be imported. An octree slices the imported volume and an optimization step simplifies the slice configuration, proposing the optimal order for easy assembly. A packing algorithm places the resulting slices with their labels, annotations, and assembly instructions on a paper or transparent film of user-selected size, to be printed, assembled into a sliceform, and explored. We conducted two user studies to assess our approach, demonstrating that it is an initial positive step towards the successful creation of interactive and engaging anatomical physicalizations.     

Don't Peek at My Chart: Privacy-preserving Visualization for Mobile Devices

Data visualizations have been widely used on mobile devices like smartphones for various tasks (e.g., visualizing personal health and financial data), making it convenient for people to view such data anytime and anywhere. However, others nearby can also easily peek at the visualizations, resulting in personal data disclosure. In this paper, we propose a perception-driven approach to transform mobile data visualizations into privacy-preserving ones. Specifically, based on human visual perception, we develop a masking scheme to adjust the spatial frequency and luminance contrast of colored visualizations. The resulting visualization retains its original information in close proximity but reduces visibility when viewed from a certain distance or farther away. We conducted two user studies to inform the design of our approach (N=16) and systematically evaluate its performance (N=18), respectively. The results demonstrate the effectiveness of our approach in terms of privacy preservation for mobile data visualizations.     

Extraction of Distinguished Hyperbolic Trajectories for 2D Time-Dependent Vector Field Topology

This paper does two main contributions to 2D time-dependent vector field topology. First, we present a technique for robust, accurate, and efficient extraction of distinguished hyperbolic trajectories (DHT), the generative structures of 2D time-dependent vector field topology. It is based on refinement of initial candidate curves. In contrast to previous approaches, it is robust because the refinement converges for reasonably close initial candidates, it is accurate due to its adaptive scheme, and it is efficient due to its high convergence speed. Second, we provide a detailed evaluation and discussion of previous approaches for the extraction of DHTs and time-dependent vector field topology in general. We demonstrate the utility of our approach using analytical flows, as well as data from computational fluid dynamics.     

RectEuler: Visualizing Intersecting Sets using Rectangles

Euler diagrams are a popular technique to visualize set-typed data. However, creating diagrams using simple shapes remains a challenging problem for many complex, real-life datasets. To solve this, we propose RectEuler: a flexible, fully-automatic method using rectangles to create Euler-like diagrams. We use an efficient mixed-integer optimization scheme to place set labels and element representatives (e.g., text or images) in conjunction with rectangles describing the sets. By defining appropriate constraints, we adhere to well-formedness properties and aesthetic considerations. If a dataset cannot be created within a reasonable time or at all, we iteratively split the diagram into multiple components until a drawable solution is found. Redundant encoding of the set membership using dots and set lines improves the readability of the diagram. Our web tool lets users see how the layout changes throughout the optimization process and provides interactive explanations. For evaluation, we perform quantitative and qualitative analysis across different datasets and compare our method to state-of-the-art Euler diagram generation methods.     

VisuaLint: Sketchy In Situ Annotations of Chart Construction Errors

Chart construction errors, such as truncated axes or inexpressive visual encodings, can hinder reading a visualization, or worse, imply misleading facts about the underlying data. These errors can be caught by critical readings of visualizations, but readers must have a high level of data and design literacy and must be paying close attention. To address this issue, we introduce VisuaLint: a technique for surfacing chart construction errors in situ. Inspired by the ubiquitous red wavy underline that indicates spelling mistakes, visualization elements that contain errors (e.g., axes and legends) are sketchily rendered and accompanied by a concise annotation. VisuaLint is unobtrusive — it does not interfere with reading a visualization — and its direct display establishes a close mapping between erroneous elements and the expression of error. We demonstrate five examples of VisualLint and present the results of a crowdsourced evaluation (N = 62) of its efficacy. These results contribute an empirical baseline proficiency for recognizing chart construction errors, and indicate near-universal difficulty in error identification. We find that people more reliably identify chart construction errors after being shown examples of VisuaLint, and prefer more verbose explanations for unfamiliar or less obvious flaws.     

PAVED: Pareto Front Visualization for Engineering Design

Design problems in engineering typically involve a large solution space and several potentially conflicting criteria. Selecting a compromise solution is often supported by optimization algorithms that compute hundreds of Pareto-optimal solutions, thus informing a decision by the engineer. However, the complexity of evaluating and comparing alternatives increases with the number of criteria that need to be considered at the same time. We present a design study on Pareto front visualization to support engineers in applying their expertise and subjective preferences for selection of the most-preferred solution. We provide a characterization of data and tasks from the parametric design of electric motors. The requirements identified were the basis for our development of PAVED, an interactive parallel coordinates visualization for exploration of multi-criteria alternatives. We reflect on our user-centered design process that included iterative refinement with real data in close collaboration with a domain expert as well as a summative evaluation in the field. The results suggest a high usability of our visualization as part of a real-world engineering design workflow. Our lessons learned can serve as guidance to future visualization developers targeting multi-criteria optimization problems in engineering design or alternative domains.     

SeqDynamics: Visual Analytics for Evaluating Online Problem-solving Dynamics

Problem-solving dynamics refers to the process of solving a series of problems over time, from which a student's cognitive skills and non-cognitive traits and behaviors can be inferred. For example, we can derive a student's learning curve (an indicator of cognitive skill) from the changes in the difficulty level of problems solved, or derive a student's self-regulation patterns (an example of non-cognitive traits and behaviors) based on the problem-solving frequency over time. Few studies provide an integrated overview of both aspects by unfolding the problem-solving process. In this paper, we present a visual analytics system named SeqDynamics that evaluates students ‘problem-solving dynamics from both cognitive and non-cognitive perspectives. The system visualizes the chronological sequence of learners’ problem-solving behavior through a set of novel visual designs and coordinated contextual views, enabling users to compare and evaluate problem-solving dynamics on multiple scales. We present three scenarios to demonstrate the usefulness of SeqDynamics on a real-world dataset which consists of thousands of problem-solving traces. We also conduct five expert interviews to show that SeqDynamics enhances domain experts’ understanding of learning behavior sequences and assists them in completing evaluation tasks efficiently.     

DRLViz: Understanding Decisions and Memory in Deep Reinforcement Learning

We present DRLViz, a visual analytics interface to interpret the internal memory of an agent (e.g. a robot) trained using deep reinforcement learning. This memory is composed of large temporal vectors updated when the agent moves in an environment and is not trivial to understand due to the number of dimensions, dependencies to past vectors, spatial/temporal correlations, and co-correlation between dimensions. It is often referred to as a black box as only inputs (images) and outputs (actions) are intelligible for humans. Using DRLViz, experts are assisted to interpret decisions using memory reduction interactions, and to investigate the role of parts of the memory when errors have been made (e.g. wrong direction). We report on DRLViz applied in the context of video games simulators (ViZDoom) for a navigation scenario with item gathering tasks. We also report on experts evaluation using DRLViz, and applicability of DRLViz to other scenarios and navigation problems beyond simulation games, as well as its contribution to black box models interpretability and explain-ability in the field of visual analytics.     

Warehouse Vis: A Visual Analytics Approach to Facilitating Warehouse Location Selection for Business Districts

Selecting a proper warehouse location serving to satisfy the demands of the goods from a certain business area is important to a successful retail business. However, the large solution space, uncertain traffic conditions, and varying business preferences impose great challenges on warehouse location selection. Conventional approaches mainly summarize relevant evaluation criteria and compile them into an analysis report to facilitate rapid data absorption but fail to support a comprehensive and joint decision-making process in warehouse location selection. In this paper, we propose a visual analytics approach to facilitating warehouse location selection. We first visually centralize relevant information of warehouses and adapts a widely-used methodology to efficiently rank warehouse candidates. We then design a delivering estimation model based on massive logistics trajectories to resolve the uncertainty issue of traffic conditions of warehouses. Based on these techniques, an interactive framework is proposed to generate and explore the candidate warehouses. We conduct a case study and a within-subject study with baseline systems to assess the efficacy of our system. Experts ‘feedback also suggests that our approach indeed helps them better tackle the problem of finding an ideal warehouse in the field of retail logistics management.     

FlexEvent: going beyond Case-Centric Exploration and Analysis of Multivariate Event Sequences

In many domains, multivariate event sequence data is collected focused around an entity (the case). Typically, each event has multiple attributes, for example, in healthcare a patient has events such as hospitalization, medication, and surgery. In addition to the multivariate events, also the case (a specific attribute, e.g., patient) has associated multivariate data (e.g., age, gender, weight). Current work typically only visualizes one attribute per event (label) in the event sequences. As a consequence, events can only be explored from a predefined case-centric perspective. However, to find complex relations from multiple perspectives (e.g., from different case definitions, such as doctor), users also need an event- and attribute-centric perspective. In addition, support is needed to effortlessly switch between and within perspectives. To support such a rich exploration, we present FlexEvent: an exploration and analysis method that enables investigation beyond a fixed case-centric perspective. Based on an adaptation of existing visualization techniques, such as scatterplots and juxtaposed small multiples, we enable flexible switching between different perspectives to explore the multivariate event sequence data needed to answer multi-perspective hypotheses. We evaluated FlexEvent with three domain experts in two use cases with sleep disorder and neonatal ICU data that show our method facilitates experts in exploring and analyzing real-world multivariate sequence data from different perspectives.