Visual Analytics for the Exploration of Tumor Tissue Characterization

Tumors are heterogeneous tissues consisting of multiple regions with distinct characteristics. Characterization of these intra‐tumor regions can improve patient diagnosis and enable a better targeted treatment. Ideally, tissue characterization could be performed non‐invasively, using medical imaging data, to derive per voxel a number of features, indicative of tissue properties. However, the high dimensionality and complexity of this imaging‐derived feature space is prohibiting for easy exploration and analysis ‐ especially when clinical researchers require to associate observations from the feature space to other reference data, e.g., features derived from histopathological data. Currently, the exploratory approach used in clinical research consists of juxtaposing these data, visually comparing them and mentally reconstructing their relationships. This is a time consuming and tedious process, from which it is difficult to obtain the required insight. We propose a visual tool for: (1) easy exploration and visual analysis of the feature space of imaging‐derived tissue characteristics and (2) knowledge discovery and hypothesis generation and confirmation, with respect to reference data used in clinical research. We employ, as central view, a 2D embedding of the imaging‐derived features. Multiple linked interactive views provide functionality for the exploration and analysis of the local structure of the feature space, enabling linking to patient anatomy and clinical reference data. We performed an initial evaluation with ten clinical researchers. All participants agreed that, unlike current practice, the proposed visual tool enables them to identify, explore and analyze heterogeneous intra‐tumor regions and particularly, to generate and confirm hypotheses, with respect to clinical reference data.     

Visual Verification of Cancer Staging for Therapy Decision Support

It is generally accepted practice that each cancer patient case should be discussed in a clinical expert meeting, the so‐called tumor board. A central role in finding the best therapy options for patients with solid tumors plays the Tumor, lymph Node, and Metastasis staging (TNM staging). Correctness of TNM staging has a significant impact on the therapy choice and hence on the patient's post‐therapeutic quality of life or even survival. If inconsistencies in the TNM staging occur, possible explanations and solutions must be found based on the complex patient records, which takes the costly time of (multiple) physicians. We propose a more efficient visual analysis component, which supports a physician in verifying the given TNM staging before forwarding it to the tumor board. Our component comprises a Bayesian network model of the TNM staging process. Using information from the patient records and Bayesian inference, the models computes a patient‐specific TNM staging, which is then explored and compared to the given staging by means of a graph‐based visualization. Our component is implemented in a research prototype that supports an understanding of the model computations, allows for a fast identification of important influencing factors, and facilitates a quick detection of differences between two TNM stagings. We evaluated our component with five physicians, each studying 20 cases of laryngeal cancer.     

Glyph‐Based Comparative Stress Tensor Visualization in Cerebral Aneurysms

We present the first visualization tool that enables a comparative depiction of structural stress tensor data for vessel walls of cerebral aneurysms. Such aneurysms bear the risk of rupture, whereas their treatment also carries considerable risks for the patient. Medical researchers emphasize the importance of analyzing the interaction of morphological and hemodynamic information for the patient‐specific rupture risk evaluation and treatment analysis. Tensor data such as the stress inside the aneurysm walls characterizes the interplay between the morphology and blood flow and seems to be an important rupture‐prone criterion. We use different glyph‐based techniques to depict local stress tensors simultaneously and compare their applicability to cerebral aneurysms in a user study. We thus offer medical researchers an effective visual exploration tool to assess the aneurysm rupture risk. We developed a GPU‐based implementation of our techniques with a flexible interactive data exploration mechanism. Our depictions are designed in collaboration with domain experts, and we provide details about the evaluation.     

Visual Support for Interactive Post‐Interventional Assessment of Radiofrequency Ablation Therapy

Percutaneous radiofrequency (RF) ablation is a minimally invasive, image‐guided therapy for the treatment of liver tumors. The assessment of the ablation area (coagulation) is performed to verify the treatment success as an essential part of the therapy. Traditionally, pre‐ and post‐interventional CT images are used to visually compare the shape, size, and position of tumor and coagulation. In this work, we present a novel visualization as well as a navigation tool, the so‐called tumor map. The tumor map is a pseudo‐cylindrical mapping of the tumor surface onto a 2D image. It is used for a combined visualization of all ablation zones of the tumor to allow a reliable therapy assessment. Additionally, the tumor map serves as an interactive tool for intuitive navigation within the 3D volume rendering of the tumor vicinity as well as with familiar 2D viewers.     

Visualization and Quantification for Interactive Analysis of Neural Connectivity in Drosophila

Neurobiologists investigate the brain of the common fruit fly Drosophila melanogaster to discover neural circuits and link them to complex behaviour. Formulating new hypotheses about connectivity requires potential connectivity information between individual neurons, indicated by overlaps of arborizations of two or more neurons. As the number of higher order overlaps (i.e. overlaps of three or more arborizations) increases exponentially with the number of neurons under investigation, visualization is impeded by clutter and quantification becomes a burden. Existing solutions are restricted to visual or quantitative analysis of pairwise overlaps, as they rely on precomputed overlap data. We present a novel tool that complements existing methods for potential connectivity exploration by providing for the first time the possibility to compute and visualize higher order arborization overlaps on the fly and to interactively explore this information in both its spatial anatomical context and on a quantitative level. Qualitative evaluation by neuroscientists and non‐experts demonstrated the utility and usability of the tool.     

Navigation and Exploration of Interconnected Pathways

Visualizing pathways, i. e. models of cellular functional networks, is a challenging task in computer assisted biomedicine. Pathways are represented as large collections of interwoven graphs, with complex structures present in both the individual graphs and their interconnections. This situation requires the development of novel visualization techniques to allow efficient visual exploration. We present the Caleydo framework, which incorporates a number of approaches to handle such pathways. Navigation in the network of pathways is facilitated by a hierarchical approach which dynamically selects a working set of individual pathways for closer inspection. These pathways are interactively rendered together with visual interconnections in a 2.5D view using graphics hardware acceleration. The layout of individual graphs is not computed automatically, but taken from the KEGG and BioCarta databases, which use layouts that life scientists are familiar with. Therefore they encode essential meta‐information. While the KEGG and BioCarta pathways use a pre‐defined layout, interactions such as linking+brushing, neighborhood search or detail on demand are still fully interactive in Caleydo. We have evaluated Caleydo with pathologists working on the determination of unknown gene functions. Informal experiences confirm that Caleydo is useful in both generating and validating such hypotheses. Even though the presented techniques are applied to medical pathways, the proposed way of interaction is not limited to cellular processes and therefore has the potential to open new possibilities in other fields of application.     

Cell lineage visualisation

Cell lineages describe the developmental history of Cell populations and are produced by combining time‐lapse imaging and image processing. Biomedical researchers study Cell lineages to understand fundamental processes such as Cell differentiation and the pharmacodynamic action of anticancer agents. Yet, the interpretation of Cell lineages is hindered by their complexity and insufficient capacity for visual analysis. We present a novel approach for interactive visualisation of Cell lineages. Based on an understanding of Cellular biology and live‐Cell imaging methodology, we identify three requirements: multimodality (Cell lineages combine spatial, temporal, and other properties), symmetry (related to lineage branching structure), and synchrony (related to temporal alignment of Cellular events). We address these by combining visual summaries of the spatiotemporal behaviour of an arbitrary number of lineages, including variation from average behaviour, with node‐link representations that emphasise the presence or absence of symmetry and synchrony. We illustrate the merit of our approach by presenting a real‐world case study where the cytotoxic action of the anticancer drug topotecan was determined.     

An Intuitive Interface for Interactive High Quality Image‐Based Modeling

We present the design of an interactive image‐based modeling tool that enables a user to quickly generate detailed 3D models with texture from a set of calibrated input images. Our main contribution is an intuitive user interface that is entirely based on simple 2D painting operations and does not require any technical expertise by the user or difficult pre‐processing of the input images. One central component of our tool is a GPU‐based multi‐view stereo reconstruction scheme, which is implemented by an incremental algorithm, that runs in the background during user interaction so that the user does not notice any significant response delay.     

Dynamic Multi‐View Exploration of Shape Spaces

Statistical shape modeling is a widely used technique for the representation and analysis of the shapes and shape variations present in a population. A statistical shape model models the distribution in a high dimensional shape space, where each shape is represented by a single point. We present a design study on the intuitive exploration and visualization of shape spaces and shape models. Our approach focuses on the dual‐space nature of these spaces. The high‐dimensional shape space represents the population, whereas object space represents the shape of the 3D object associated with a point in shape space. A 3D object view provides local details for a single shape. The high dimensional points in shape space are visualized using a 2D scatter plot projection, the axes of which can be manipulated interactively. This results in a dynamic scatter plot, with the further extension that each point is visualized as a small version of the object shape that it represents. We further enhance the population‐object duality with a new type of view aimed at shape comparison. This new “shape evolution view” visualizes shape variability along a single trajectory in shape space, and serves as a link between the two spaces described above. Our three‐view exploration concept strongly emphasizes linked interaction between all spaces. Moving the cursor over the scatter plot or evolution views, shapes are dynamically interpolated and shown in the object view. Conversely, camera manipulation in the object view affects the object visualizations in the other views. We present a GPU‐accelerated implementation, and show the effectiveness of the three‐view approach using a number of real‐world cases. In these, we demonstrate how this multi‐view approach can be used to visually explore important aspects of a statistical shape model, including specificity, compactness and reconstruction error.     

PhysioEx: Visual Analysis of Physiological Event Streams

In this work, we introduce a novel visualization technique, the Temporal Intensity Map, which visually integrates data values over time to reveal the frequency, duration, and timing of significant features in streaming data. We combine the Temporal Intensity Map with several coordinated visualizations of detected events in data streams to create PhysioEx, a visual dashboard for multiple heterogeneous data streams. We have applied PhysioEx in a design study in the field of neonatal medicine, to support clinical researchers exploring physiologic data streams. We evaluated our method through consultations with domain experts. Results show that our tool provides deep insight capabilities, supports hypothesis generation, and can be well integrated into the workflow of clinical researchers.     

InSpectr: Multi‐Modal Exploration,Visualization, and Analysis of Spectral Data

This paper addresses the increasing demand in industry for methods to analyze and visualize multimodal data involving a spectral modality. Two data modalities are used: high‐resolution X‐ray computed tomography (XCT) for structural characterization and low‐resolution X‐ray fluorescence (XRF) spectral data for elemental decomposition. We present InSpectr, an integrated tool for the interactive exploration and visual analysis of multimodal, multiscalar data. The tool has been designed around a set of tasks identified by domain experts in the fields of XCT and XRF. It supports registered single scalar and spectral datasets optionally coupled with element maps and reference spectra. InSpectr is instantiating various linked views for the integration of spatial and non‐spatial information to provide insight into an industrial component's structural and material composition: views with volume renderings of composite and individual 3D element maps visualize global material composition; transfer functions defined directly on the spectral data and overlaid pie‐chart glyphs show elemental composition in 2D slice‐views; a representative aggregated spectrum and spectra density histograms are introduced to provide a global overview in the spectral view. Spectral magic lenses, spectrum probing and elemental composition probing of points using a pie‐chart view and a periodic table view aid the local material composition analysis. Two datasets are investigated to outline the usefulness of the presented techniques: a 3D virtually created phantom with a brass metal alloy and a real‐world 2D water phantom with insertions of gold, barium, and gadolinium. Additionally a detailed user evaluation of the results is provided.     

Visual Analysis of Defects in Glass Fiber Reinforced Polymers for 4DCT Interrupted In situ Tests

Material engineers use interrupted in situ tensile testing to investigate the damage mechanisms in composite materials. For each subsequent scan, the load is incrementally increased until the specimen is completely fractured. During the interrupted in situ testing of glass fiber reinforced polymers (GFRPs) defects of four types are expected to appear: matrix fracture, fiber/matrix debonding, fiber pull‐out, and fiber fracture. There is a growing demand for the detection and analysis of these defects among the material engineers. In this paper, we present a novel workflow for the detection, classification, and visual analysis of defects in GFRPs using interrupted in situ tensile tests in combination with X‐ray Computed Tomography. The workflow is based on the automatic extraction of defects and fibers. We introduce the automatic Defect Classifier assigning the most suitable type to each defect based on its geometrical features. We present a visual analysis system that integrates four visualization methods: 1) the Defect Viewer highlights defects with visually encoded type in the context of the original CT image, 2) the Defect Density Maps provide an overview of the defect distributions according to type in 2D and 3D, 3) the Final Fracture Surface estimates the material fracture's location and displays it as a 3D surface, 4) the 3D Magic Lens enables interactive exploration by combining detailed visualizations in the region of interest with overview visualizations as context. In collaboration with material engineers, we evaluate our solution and demonstrate its practical applicability.     

Visual Analysis of Missing Values in Longitudinal Cohort Study Data

Attrition or dropout is the most severe missingness problem in longitudinal cohort study data where some participants do not show up for follow-up examinations. Dropouts result in biased data and cause the reduction of 1ata set size. Moreover, they limit the power of statistical analysis and the validity of study findings. Visualization can play a strong role in analysing and displaying the missingness patterns. In this work, we present VIVID, a framework for the v isual analysis of mi ssing v alues i n cohort study d ata. VIVID is inspired by discussions with epidemiologists and adds visual components to their current statistics-based approaches. VIVID provides functions for exploration, imputation and validity check of imputations. The main focus of this paper is multiple imputation to fix the missing data.     

Visual Analysis of Spatio‐Temporal Data: Applications in Weather Forecasting

Weather conditions affect multiple aspects of human life such as economy, safety, security, and social activities. For this reason, weather forecast plays a major role in society. Currently weather forecasts are based on Numerical Weather Prediction (NWP) models that generate a representation of the atmospheric flow. Interactive visualization of geo‐spatial data has been widely used in order to facilitate the analysis of NWP models. This paper presents a visualization system for the analysis of spatio‐temporal patterns in short‐term weather forecasts. For this purpose, we provide an interactive visualization interface that guides users from simple visual overviews to more advanced visualization techniques. Our solution presents multiple views that include a timeline with geo‐referenced maps, an integrated webmap view, a forecast operation tool, a curve‐pattern selector, spatial filters, and a linked meteogram. Two key contributions of this work are the timeline with geo‐referenced maps and the curve‐pattern selector. The latter provides novel functionality that allows users to specify and search for meaningful patterns in the data. The visual interface of our solution allows users to detect both possible weather trends and errors in the weather forecast model. We illustrate the usage of our solution with a series of case studies that were designed and validated in collaboration with domain experts.     

Visualizing Probabilistic Multi‐Phase Fluid Simulation Data using a Sampling Approach

Eulerian Method of Moment (MoM) solvers are gaining popularity for multi‐phase CFD simulation involving bubbles or droplets in process engineering. Because the actual positions of bubbles are uncertain, the spatial distribution of bubbles is described by scalar fields of moments, which can be interpreted as probability density functions. Visualizing these simulation results and comparing them to physical experiments is challenging, because neither the shape nor the distribution of bubbles described by the moments lend themselves to visual interpretation. In this work, we describe a visualization approach that provides explicit instances of the bubble distribution and produces bubble geometry based on local flow properties. To facilitate animation, the instancing of the bubble distribution provides coherence over time by advancing bubbles between time steps and updating the distribution. Our approach provides an intuitive visualization and enables direct visual comparison of simulation results to physical experiments.     

Interactive Regression Lens for Exploring Scatter Plots

Data analysis often involves finding models that can explain patterns in data, and reduce possibly large data sets to more compact model‐based representations. In Statistics, many methods are available to compute model information. Among others, regression models are widely used to explain data. However, regression analysis typically searches for the best model based on the global distribution of data. On the other hand, a data set may be partitioned into subsets, each requiring individual models. While automatic data subsetting methods exist, these often require parameters or domain knowledge to work with. We propose a system for visual‐interactive regression analysis for scatter plot data, supporting both global and local regression modeling. We introduce a novel regression lens concept, allowing a user to interactively select a portion of data, on which regression analysis is run in interactive time. The lens gives encompassing visual feedback on the quality of candidate models as it is interactively navigated across the input data. While our regression lens can be used for fully interactive modeling, we also provide user guidance suggesting appropriate models and data subsets, by means of regression quality scores. We show, by means of use cases, that our regression lens is an effective tool for user‐driven regression modeling and supports model understanding.     

Exploring Different Parameters to Assess Left Ventricle Global and Regional Functional Analysis from Coronary CT Angiography

Coronary CT angiography is widely used in clinical practice for the assessment of coronary artery disease. Several studies have shown that the same exam can also be used to assess left ventricle (LV) function. Even though coronary CT angiography provides data concerning multiple cardiac phases, along the cardiac cycle, LV function is usually evaluated using just the end‐systolic and end‐diastolic phases. This unused wealth of data, mostly due to its complexity and the lack of proper tools, has still to be explored to assess if further insight is possible regarding regional LV functional analysis. Furthermore, different parameters can be computed to characterize LV function and though some are well known by clinicians others still need to be tested concerning their value in clinical scenarios. Based on these premises, we present several parameters characterizing global and regional LV function, computed for several cardiac phases over one cardiac cycle. The data provided by the computed parameters is shown using a set of visualizations allowing synchronized visual exploration of the different data. The main purpose is to provide means for clinicians to explore the data and gather insight over their meaning and their correlation with each other and with diagnosis outcomes.     

GuideME: Slice‐guided Semiautomatic Multivariate Exploration of Volumes

Multivariate volume visualization is important for many applications including petroleum exploration and medicine. State‐of‐the‐art tools allow users to interactively explore volumes with multiple linked parameter‐space views. However, interactions in the parameter space using trial‐and‐error may be unintuitive and time consuming. Furthermore, switching between different views may be distracting. In this paper, we propose GuideME: a novel slice‐guided semiautomatic multivariate volume exploration approach. Specifically, the approach comprises four stages: attribute inspection, guided uncertainty‐aware lasso creation, automated feature extraction and optional spatial fine tuning and visualization. Throughout the exploration process, the user does not need to interact with the parameter views at all and examples of complex real‐world data demonstrate the usefulness, efficiency and ease‐of‐use of our method.     

Visual Analysis of Biomolecular Cavities: State of the Art

In this report we review and structure the branch of molecular visualization that is concerned with the visual analysis of cavities in macromolecular protein structures. First the necessary background, the domain terminology, and the goals of analytical reasoning are introduced. Based on a comprehensive collection of relevant research works, we present a novel classification for cavity detection approaches and structure them into four distinct classes: grid‐based, Voronoi‐based, surface‐based, and probe‐based methods. The subclasses are then formed by their combinations. We match these approaches with corresponding visualization technologies starting with direct 3D visualization, followed with non‐spatial visualization techniques that for example abstract the interactions between structures into a relational graph, straighten the cavity of interest to see its profile in one view, or aggregate the time sequence into a single contour plot. We also discuss the current state of methods for the visual analysis of cavities in dynamic data such as molecular dynamics simulations. Finally, we give an overview of the most common tools that are actively developed and used in the structural biology and biochemistry research. Our report is concluded by an outlook on future challenges in the field.