FlexEvent: going beyond Case-Centric Exploration and Analysis of Multivariate Event Sequences

In many domains, multivariate event sequence data is collected focused around an entity (the case). Typically, each event has multiple attributes, for example, in healthcare a patient has events such as hospitalization, medication, and surgery. In addition to the multivariate events, also the case (a specific attribute, e.g., patient) has associated multivariate data (e.g., age, gender, weight). Current work typically only visualizes one attribute per event (label) in the event sequences. As a consequence, events can only be explored from a predefined case-centric perspective. However, to find complex relations from multiple perspectives (e.g., from different case definitions, such as doctor), users also need an event- and attribute-centric perspective. In addition, support is needed to effortlessly switch between and within perspectives. To support such a rich exploration, we present FlexEvent: an exploration and analysis method that enables investigation beyond a fixed case-centric perspective. Based on an adaptation of existing visualization techniques, such as scatterplots and juxtaposed small multiples, we enable flexible switching between different perspectives to explore the multivariate event sequence data needed to answer multi-perspective hypotheses. We evaluated FlexEvent with three domain experts in two use cases with sleep disorder and neonatal ICU data that show our method facilitates experts in exploring and analyzing real-world multivariate sequence data from different perspectives.     

VA-TRAC: Geospatial Trajectory Analysis for Monitoring,Identification, and Verification in Fishing Vessel Operations

In order to ensure sustainability, fishing operations are governed by many rules and regulations that restrict the use of certain techniques and equipment, specify the species and size offish that can be harvested, and regulate commercial activities based on licensing schemes. As the world's second largest exporter offish and seafood products, Norway invests a significant amount of effort into maintaining natural ecosystem dynamics by ensuring compliance with its constantly evolving science-based regulatory body. This paper introduces VA-TRAC, a geovisual analytics application developed in collaboration with the Norwegian Directorate of Fisheries in order to address this complex task. Our approach uses automatic methods to identify possible catch operations based on fishing vessel trajectories, embedded in an interactive web-based visual interface used to explore the results, compare them with licensing information, and incorporate the analysts’ domain knowledge into the decision making process. We present a data and task analysis based on a close collaboration with domain experts, and the design and implementation of VA-TRAC to address the identified requirements.     

Boxer: Interactive Comparison of Classifier Results

Machine learning practitioners often compare the results of different classifiers to help select, diagnose and tune models. We present Boxer, a system to enable such comparison. Our system facilitates interactive exploration of the experimental results obtained by applying multiple classifiers to a common set of model inputs. The approach focuses on allowing the user to identify interesting subsets of training and testing instances and comparing performance of the classifiers on these subsets. The system couples standard visual designs with set algebra interactions and comparative elements. This allows the user to compose and coordinate views to specify subsets and assess classifier performance on them. The flexibility of these compositions allow the user to address a wide range of scenarios in developing and assessing classifiers. We demonstrate Boxer in use cases including model selection, tuning, fairness assessment, and data quality diagnosis.     

A Visual Analytics Approach to Facilitate Crime Hotspot Analysis

Computer-based technology has played a significant role in crime prevention over the past 30 years, especially with the popularization of spatial databases and crime mapping systems. Police departments frequently use hotspot analysis to identify regions that should be a priority in receiving preventive resources. Practitioners and researchers agree that tracking crime over time and identifying its geographic patterns are vital information for planning efficiently. Frequently, police departments have access to systems that are too complicated and excessively technical, leading to modest usage. By working closely together with domain experts from police agencies of two different countries, we identified and characterized five domain tasks inherent to the hotspot analysis problem and developed SHOC, a visualization tool that strives for simplicity and ease of use in helping users to perform all the domain tasks. SHOC is included in a visual analytics system that allows users without technical expertise to annotate, save, and share analyses. We also demonstrate that our system effectively supports the completion of the domain tasks in two different real-world case studies.     

VISITOR: Visual Interactive State Sequence Exploration for Reinforcement Learning

Understanding the behavior of deep reinforcement learning agents is a crucial requirement throughout their development. Existing work has addressed the identification of observable behavioral patterns in state sequences or analysis of isolated internal representations; however, the overall decision-making of deep-learning RL agents remains opaque. To tackle this, we present VISITOR, a visual analytics system enabling the analysis of entire state sequences, the diagnosis of singular predictions, and the comparison between agents. A sequence embedding view enables the multiscale analysis of state sequences, utilizing custom embedding techniques for a stable spatialization of the observations and internal states. We provide multiple layers: (1) a state space embedding, highlighting different groups of states inside the state-action sequences, (2) a trajectory view, emphasizing decision points, (3) a network activation mapping, visualizing the relationship between observations and network activations, (4) a transition embedding, enabling the analysis of state-to-state transitions. The embedding view is accompanied by an interactive reward view that captures the temporal development of metrics, which can be linked directly to states in the embedding. Lastly, a model list allows for the quick comparison of models across multiple metrics. Annotations can be exported to communicate results to different audiences. Our two-stage evaluation with eight experts confirms the effectiveness in identifying states of interest, comparing the quality of policies, and reasoning about the internal decision-making processes.     

Human–Computer Collaboration for Visual Analytics: an Agent-based Framework

The visual analytics community has long aimed to understand users better and assist them in their analytic endeavors. As a result, numerous conceptual models of visual analytics aim to formalize common workflows, techniques, and goals leveraged by analysts. While many of the existing approaches are rich in detail, they each are specific to a particular aspect of the visual analytic process. Furthermore, with an ever-expanding array of novel artificial intelligence techniques and advances in visual analytic settings, existing conceptual models may not provide enough expressivity to bridge the two fields. In this work, we propose an agent-based conceptual model for the visual analytic process by drawing parallels from the artificial intelligence literature. We present three examples from the visual analytics literature as case studies and examine them in detail using our framework. Our simple yet robust framework unifies the visual analytic pipeline to enable researchers and practitioners to reason about scenarios that are becoming increasingly prominent in the field, namely mixed-initiative, guided, and collaborative analysis. Furthermore, it will allow us to characterize analysts, visual analytic settings, and guidance from the lenses of human agents, environments, and artificial agents, respectively.     

Tac-Anticipator: Visual Analytics of Anticipation Behaviors in Table Tennis Matches

Anticipation skill is important for elite racquet sports players. Successful anticipation allows them to predict the actions of the opponent better and take early actions in matches. Existing studies of anticipation behaviors, largely based on the analysis of in-lab behaviors, failed to capture the characteristics of in-situ anticipation behaviors in real matches. This research proposes a data-driven approach for research on anticipation behaviors to gain more accurate and reliable insight into anticipation skills. Collaborating with domain experts in table tennis, we develop a complete solution that includes data collection, the development of a model to evaluate anticipation behaviors, and the design of a visual analytics system called Tac-Anticipator. Our case study reveals the strengths and weaknesses of top table tennis players' anticipation behaviors. In a word, our work enriches the research methods and guidelines for visual analytics of anticipation behaviors.     

DRLViz: Understanding Decisions and Memory in Deep Reinforcement Learning

We present DRLViz, a visual analytics interface to interpret the internal memory of an agent (e.g. a robot) trained using deep reinforcement learning. This memory is composed of large temporal vectors updated when the agent moves in an environment and is not trivial to understand due to the number of dimensions, dependencies to past vectors, spatial/temporal correlations, and co-correlation between dimensions. It is often referred to as a black box as only inputs (images) and outputs (actions) are intelligible for humans. Using DRLViz, experts are assisted to interpret decisions using memory reduction interactions, and to investigate the role of parts of the memory when errors have been made (e.g. wrong direction). We report on DRLViz applied in the context of video games simulators (ViZDoom) for a navigation scenario with item gathering tasks. We also report on experts evaluation using DRLViz, and applicability of DRLViz to other scenarios and navigation problems beyond simulation games, as well as its contribution to black box models interpretability and explain-ability in the field of visual analytics.     

Teru Teru Bōzu: Defensive Raincloud Plots

Univariate visualizations like histograms, rug plots, or box plots provide concise visual summaries of distributions. However, each individual visualization may fail to robustly distinguish important features of a distribution, or provide sufficient information for all of the relevant tasks involved in summarizing univariate data. One solution is to juxtapose or superimpose multiple univariate visualizations in the same chart, as in Allen et al.'s [APW^*19] “raincloud plots.” In this paper I examine the design space of raincloud plots, and, through a series of simulation studies, explore designs where the component visualizations mutually “defend” against situations where important distribution features are missed or trivial features are given undue prominence. I suggest a class of “defensive” raincloud plot designs that provide good mutual coverage for surfacing distributional features of interest.     

LINGO : Visually Debiasing Natural Language Instructions to Support Task Diversity

Cross-task generalization is a significant outcome that defines mastery in natural language understanding. Humans show a remarkable aptitude for this, and can solve many different types of tasks, given definitions in the form of textual instructions and a small set of examples. Recent work with pre-trained language models mimics this learning style: users can define and exemplify a task for the model to attempt as a series of natural language prompts or instructions. While prompting approaches have led to higher cross-task generalization compared to traditional supervised learning, analyzing ‘bias’ in the task instructions given to the model is a difficult problem, and has thus been relatively unexplored. For instance, are we truly modeling a task, or are we modeling a user's instructions? To help investigate this, we develop LINGO, a novel visual analytics interface that supports an effective, task-driven workflow to (1) help identify bias in natural language task instructions, (2) alter (or create) task instructions to reduce bias, and (3) evaluate pre-trained model performance on debiased task instructions. To robustly evaluate LINGO, we conduct a user study with both novice and expert instruction creators, over a dataset of 1,616 linguistic tasks and their natural language instructions, spanning 55 different languages. For both user groups, LINGO promotes the creation of more difficult tasks for pre-trained models, that contain higher linguistic diversity and lower instruction bias. We additionally discuss how the insights learned in developing and evaluating LINGO can aid in the design of future dashboards that aim to minimize the effort involved in prompt creation across multiple domains.     

A Fully Integrated Pipeline for Visual Carotid Morphology Analysis

Analyzing stenoses of the internal carotids – local constrictions of the artery – is a critical clinical task in cardiovascular disease treatment and prevention. For this purpose, we propose a self-contained pipeline for the visual analysis of carotid artery geometries. The only inputs are computed tomography angiography (CTA) scans, which are already recorded in clinical routine. We show how integrated model extraction and visualization can help to efficiently detect stenoses and we provide means for automatic, highly accurate stenosis degree computation. We directly connect multiple sophisticated processing stages, including a neural prediction network for lumen and plaque segmentation and automatic global diameter computation. We enable interactive and retrospective user control over the processing stages. Our aims are to increase user trust by making the underlying data validatable on the fly, to decrease adoption costs by minimizing external dependencies, and to optimize scalability by streamlining the data processing. We use interactive visualizations for data inspection and adaption to guide the user through the processing stages. The framework was developed and evaluated in close collaboration with radiologists and neurologists. It has been used to extract and analyze over 100 carotid bifurcation geometries and is built with a modular architecture, available as an extendable open-source platform.     

RectEuler: Visualizing Intersecting Sets using Rectangles

Euler diagrams are a popular technique to visualize set-typed data. However, creating diagrams using simple shapes remains a challenging problem for many complex, real-life datasets. To solve this, we propose RectEuler: a flexible, fully-automatic method using rectangles to create Euler-like diagrams. We use an efficient mixed-integer optimization scheme to place set labels and element representatives (e.g., text or images) in conjunction with rectangles describing the sets. By defining appropriate constraints, we adhere to well-formedness properties and aesthetic considerations. If a dataset cannot be created within a reasonable time or at all, we iteratively split the diagram into multiple components until a drawable solution is found. Redundant encoding of the set membership using dots and set lines improves the readability of the diagram. Our web tool lets users see how the layout changes throughout the optimization process and provides interactive explanations. For evaluation, we perform quantitative and qualitative analysis across different datasets and compare our method to state-of-the-art Euler diagram generation methods.     

QUESTO: Interactive Construction of Objective Functions for Classification Tasks

Building effective classifiers requires providing the modeling algorithms with information about the training data and modeling goals in order to create a model that makes proper tradeoffs. Machine learning algorithms allow for flexible specification of such meta-information through the design of the objective functions that they solve. However, such objective functions are hard for users to specify as they are a specific mathematical formulation of their intents. In this paper, we present an approach that allows users to generate objective functions for classification problems through an interactive visual interface. Our approach adopts a semantic interaction design in that user interactions over data elements in the visualization are translated into objective function terms. The generated objective functions are solved by a machine learning solver that provides candidate models, which can be inspected by the user, and used to suggest refinements to the specifications. We demonstrate a visual analytics system QUESTO for users to manipulate objective functions to define domain-specific constraints. Through a user study we show that QUESTO helps users create various objective functions that satisfy their goals.     

VA + Embeddings STAR: A State-of-the-Art Report on the Use of Embeddings in Visual Analytics

Over the past years, an increasing number of publications in information visualization, especially within the field of visual analytics, have mentioned the term “embedding” when describing the computational approach. Within this context, embeddings are usually (relatively) low-dimensional, distributed representations of various data types (such as texts or graphs), and since they have proven to be extremely useful for a variety of data analysis tasks across various disciplines and fields, they have become widely used. Existing visualization approaches aim to either support exploration and interpretation of the embedding space through visual representation and interaction, or aim to use embeddings as part of the computational pipeline for addressing downstream analytical tasks. To the best of our knowledge, this is the first survey that takes a detailed look at embedding methods through the lens of visual analytics, and the purpose of our survey article is to provide a systematic overview of the state of the art within the emerging field of embedding visualization. We design a categorization scheme for our approach, analyze the current research frontier based on peer-reviewed publications, and discuss existing trends, challenges, and potential research directions for using embeddings in the context of visual analytics. Furthermore, we provide an interactive survey browser for the collected and categorized survey data, which currently includes 122 entries that appeared between 2007 and 2023.