Dynamic Visibility‐Driven Molecular Surfaces

Molecular surface representations are an important tool for the visual analysis of molecular structure and function. In this paper, we present a novel method for the visualization of dynamic molecular surfaces based on the Gaussian model. In contrast to previous approaches, our technique does not rely on the construction of intermediate representations such as grids or triangulated surfaces. Instead, it operates entirely in image space, which enables us to exploit visibility information to efficiently skip unnecessary computations. With this visibility‐driven approach, we can visualize dynamic high‐quality surfaces for molecules consisting of millions of atoms. Our approach requires no preprocessing, allows for the interactive adjustment of all properties and parameters, and is significantly faster than previous approaches, while providing superior quality.     

A General Illumination Model for Molecular Visualization

Several visual representations have been developed over the years to visualize molecular structures, and to enable a better understanding of their underlying chemical processes. Today, the most frequently used atom‐based representations are the Space‐filling, the Solvent Excluded Surface, the Balls‐and‐Sticks, and the Licorice models. While each of these representations has its individual benefits, when applied to large‐scale models spatial arrangements can be difficult to interpret when employing current visualization techniques. In the past it has been shown that global illumination techniques improve the perception of molecular visualizations; unfortunately existing approaches are tailored towards a single visual representation. We propose a general illumination model for molecular visualization that is valid for different representations. With our illumination model, it becomes possible, for the first time, to achieve consistent illumination among all atom‐based molecular representations. The proposed model can be further evaluated in real‐time, as it employs an analytical solution to simulate diffuse light interactions between objects. To be able to derive such a solution for the rather complicated and diverse visual representations, we propose the use of regression analysis together with adapted parameter sampling strategies as well as shape parametrization guided sampling, which are applied to the geometric building blocks of the targeted visual representations. We will discuss the proposed sampling strategies, the derived illumination model, and demonstrate its capabilities when visualizing several dynamic molecules.     

DimSUM: Dimension and Scale Unifying Map for Visual Abstraction of DNA Origami Structures

We present a novel visualization concept for DNA origami structures that integrates a multitude of representations into a Dimension and Scale Unifying Map (DimSUM). This novel abstraction map provides means to analyze, smoothly transition between, and interact with many visual representations of the DNA origami structures in an effective way that was not possible before. DNA origami structures are nanoscale objects, which are challenging to model in silico. In our holistic approach we seamlessly combine three‐dimensional realistic shape models, two‐dimensional diagrammatic representations, and ordered alignments in one‐dimensional arrangements, with semantic transitions across many scales. To navigate through this large, two‐dimensional abstraction map we highlight locations that users frequently visit for certain tasks and datasets. Particularly interesting viewpoints can be explicitly saved to optimize the workflow. We have developed DimSUM together with domain scientists specialized in DNA nanotechnology. In the paper we discuss our design decisions for both the visualization and the interaction techniques. We demonstrate two practical use cases in which our approach increases the specialists’ understanding and improves their effectiveness in the analysis. Finally, we discuss the implications of our concept for the use of controlled abstraction in visualization in general.     

Bladder Runner: Visual Analytics for the Exploration of RT‐Induced Bladder Toxicity in a Cohort Study

We present the Bladder Runner, a novel tool to enable detailed visual exploration and analysis of the impact of bladder shape variation on the accuracy of dose delivery, during the course of prostate cancer radiotherapy (RT). Our tool enables the investigation of individual patients and cohorts through the entire treatment process, and it can give indications of RT‐induced complications for the patient. In prostate cancer RT treatment, despite the design of an initial plan prior to dose administration, bladder toxicity remains very common. The main reason is that the dose is delivered in multiple fractions over a period of weeks, during which, the anatomical variation of the bladder – due to differences in urinary filling – causes deviations between planned and delivered doses. Clinical researchers want to correlate bladder shape variations to dose deviations and toxicity risk through cohort studies, to understand which specific bladder shape characteristics are more prone to side effects. This is currently done with Dose‐Volume Histograms (DVHs), which provide limited, qualitative insight. The effect of bladder variation on dose delivery and the resulting toxicity cannot be currently examined with the DVHs. To address this need, we designed and implemented the Bladder Runner, which incorporates visualization strategies in a highly interactive environment with multiple linked views. Individual patients can be explored and analyzed through the entire treatment period, while inter‐patient and temporal exploration, analysis and comparison are also supported. We demonstrate the applicability of our presented tool with a usage scenario, employing a dataset of 29 patients followed through the course of the treatment, across 13 time points. We conducted an evaluation with three clinical researchers working on the investigation of RT‐induced bladder toxicity. All participants agreed that Bladder Runner provides better understanding and new opportunities for the exploration and analysis of the involved cohort data.     

Visual Attention for Rendered 3D Shapes

Understanding the attentional behavior of the human visual system when visualizing a rendered 3D shape is of great importance for many computer graphics applications. Eye tracking remains the only solution to explore this complex cognitive mechanism. Unfortunately, despite the large number of studies dedicated to images and videos, only a few eye tracking experiments have been conducted using 3D shapes. Thus, potential factors that may influence the human gaze in the specific setting of 3D rendering, are still to be understood. In this work, we conduct two eye‐tracking experiments involving 3D shapes, with both static and time‐varying camera positions. We propose a method for mapping eye fixations (i.e., where humans gaze) onto the 3D shapes with the aim to produce a benchmark of 3D meshes with fixation density maps, which is publicly available. First, the collected data is used to study the influence of shape, camera position, material and illumination on visual attention. We find that material and lighting have a significant influence on attention, as well as the camera path in the case of dynamic scenes. Then, we compare the performance of four representative state‐of‐the‐art mesh saliency models in predicting ground‐truth fixations using two different metrics. We show that, even combined with a center‐bias model, the performance of 3D saliency algorithms remains poor at predicting human fixations. To explain their weaknesses, we provide a qualitative analysis of the main factors that attract human attention. We finally provide a comparison of human‐eye fixations and Schelling points and show that their correlation is weak.     

The State of the Art in Sentiment Visualization

Visualization of sentiments and opinions extracted from or annotated in texts has become a prominent topic of research over the last decade. From basic pie and bar charts used to illustrate customer reviews to extensive visual analytics systems involving novel representations, sentiment visualization techniques have evolved to deal with complex multidimensional data sets, including temporal, relational and geospatial aspects. This contribution presents a survey of sentiment visualization techniques based on a detailed categorization. We describe the background of sentiment analysis, introduce a categorization for sentiment visualization techniques that includes 7 groups with 35 categories in total, and discuss 132 techniques from peer‐reviewed publications together with an interactive web‐based survey browser. Finally, we discuss insights and opportunities for further research in sentiment visualization. We expect this survey to be useful for visualization researchers whose interests include sentiment or other aspects of text data as well as researchers and practitioners from other disciplines in search of efficient visualization techniques applicable to their tasks and data.     

Learning‐Based Animation of Clothing for Virtual Try‐On

This paper presents a learning‐based clothing animation method for highly efficient virtual try‐on simulation. Given a garment, we preprocess a rich database of physically‐based dressed character simulations, for multiple body shapes and animations. Then, using this database, we train a learning‐based model of cloth drape and wrinkles, as a function of body shape and dynamics. We propose a model that separates global garment fit, due to body shape, from local garment wrinkles, due to both pose dynamics and body shape. We use a recurrent neural network to regress garment wrinkles, and we achieve highly plausible nonlinear effects, in contrast to the blending artifacts suffered by previous methods. At runtime, dynamic virtual try‐on animations are produced in just a few milliseconds for garments with thousands of triangles. We show qualitative and quantitative analysis of results.     

A Review of Guidance Approaches in Visual Data Analysis: A Multifocal Perspective

Visual data analysis can be envisioned as a collaboration of the user and the computational system with the aim of completing a given task. Pursuing an effective system‐user integration, in which the system actively helps the user to reach his/her analysis goal has been focus of visualization research for quite some time. However, this problem is still largely unsolved. As a result, users might be overwhelmed by powerful but complex visual analysis systems which also limits their ability to produce insightful results. In this context, guidance is a promising step towards enabling an effective mixed‐initiative collaboration to promote the visual analysis. However, the way how guidance should be put into practice is still to be unravelled. Thus, we conducted a comprehensive literature research and provide an overview of how guidance is tackled by different approaches in visual analysis systems. We distinguish between guidance that is provided by the system to support the user, and guidance that is provided by the user to support the system. By identifying open problems, we highlight promising research directions and point to missing factors that are needed to enable the envisioned human‐computer collaboration, and thus, promote a more effective visual data analysis.     

Visual Analysis of protein‐ligand interactions

The analysis of protein‐ligand interactions is complex because of the many factors at play. Most current methods for visual analysis provide this information in the form of simple 2D plots, which, besides being quite space hungry, often encode a low number of different properties. In this paper we present a system for compact 2D visualization of molecular simulations. It purposely omits most spatial information and presents physical information associated to single molecular components and their pairwise interactions through a set of 2D InfoVis tools with coordinated views, suitable interaction, and focus+context techniques to analyze large amounts of data. The system provides a wide range of motifs for elements such as protein secondary structures or hydrogen bond networks, and a set of tools for their interactive inspection, both for a single simulation and for comparing two different simulations. As a result, the analysis of protein‐ligand interactions of Molecular Simulation trajectories is greatly facilitated.     

Interactive Volumetric Visual Analysis of Glycogen‐derived Energy Absorption in Nanometric Brain Structures

Digital acquisition and processing techniques are changing the way neuroscience investigation is carried out. Emerging applications range from statistical analysis on image stacks to complex connectomics visual analysis tools targeted to develop and test hypotheses of brain development and activity. In this work, we focus on neuroenergetics, a field where neuroscientists analyze nanoscale brain morphology and relate energy consumption to glucose storage in form of glycogen granules. In order to facilitate the understanding of neuroenergetic mechanisms, we propose a novel customized pipeline for the visual analysis of nanometric‐level reconstructions based on electron microscopy image data. Our framework supports analysis tasks by combining i) a scalable volume visualization architecture able to selectively render image stacks and corresponding labelled data, ii) a method for highlighting distance‐based energy absorption probabilities in form of glow maps, and iii) a hybrid connectivitybased and absorption‐based interactive layout representation able to support queries for selective analysis of areas of interest and potential activity within the segmented datasets. This working pipeline is currently used in a variety of studies in the neuroenergetics domain. Here, we discuss a test case in which the framework was successfully used by domain scientists for the analysis of aging effects on glycogen metabolism, extracting knowledge from a series of nanoscale brain stacks of rodents somatosensory cortex.     

IGM‐Vis: Analyzing Intergalactic and Circumgalactic Medium Absorption Using Quasar Sightlines in a Cosmic Web Context

We introduce IGM‐Vis, a novel astrophysics visualization and data analysis application for investigating galaxies and the gas that surrounds them in context with their larger scale environment, the Cosmic Web. Environment is an important factor in the evolution of galaxies from actively forming stars to quiescent states with little, if any, discernible star formation activity. The gaseous halos of galaxies (the circumgalactic medium, or CGM) play a critical role in their evolution, because the gas necessary to fuel star formation and any gas expelled from widely observed galactic winds must encounter this interface region between galaxies and the intergalactic medium (IGM). We present a taxonomy of tasks typically employed in IGM/CGM studies informed by a survey of astrophysicists at various career levels, and demonstrate how these tasks are facilitated via the use of our visualization software. Finally, we evaluate the effectiveness of IGM‐Vis through two in‐depth use cases that depict real‐world analysis sessions that use IGM/CGM data.     

On‐Site Example‐Based Material Appearance Acquisition

We present a novel example‐based material appearance modeling method suitable for rapid digital content creation. Our method only requires a single HDR photograph of a homogeneous isotropic dielectric exemplar object under known natural illumination. While conventional methods for appearance modeling require prior knowledge on the object shape, our method does not, nor does it recover the shape explicitly, greatly simplifying on‐site appearance acquisition to a lightweight photography process suited for non‐expert users. As our central contribution, we propose a shape‐agnostic BRDF estimation procedure based on binary RGB profile matching. We also model the appearance of materials exhibiting a regular or stationary texture‐like appearance, by synthesizing appropriate mesostructure from the same input HDR photograph and a mesostructure exemplar with (roughly) similar features. We believe our lightweight method for on‐site shape‐agnostic appearance acquisition presents a suitable alternative for a variety of applications that require plausible “rapid‐appearance‐modeling”.     

Visual‐Interactive Preprocessing of Multivariate Time Series Data

Pre‐processing is a prerequisite to conduct effective and efficient downstream data analysis. Pre‐processing pipelines often require multiple routines to address data quality challenges and to bring the data into a usable form. For both the construction and the refinement of pre‐processing pipelines, human‐in‐the‐loop approaches are highly beneficial. This particularly applies to multivariate time series, a complex data type with multiple values developing over time. Due to the high specificity of this domain, it has not been subject to in‐depth research in visual analytics. We present a visual‐interactive approach for preprocessing multivariate time series data with the following aspects. Our approach supports analysts to carry out six core analysis tasks related to pre‐processing of multivariate time series. To support these tasks, we identify requirements to baseline toolkits that may help practitioners in their choice. We characterize the space of visualization designs for uncertainty‐aware pre‐processing and justify our decisions. Two usage scenarios demonstrate applicability of our approach, design choices, and uncertainty visualizations for the six analysis tasks. This work is one step towards strengthening the visual analytics support for data pre‐processing in general and for uncertainty‐aware pre‐processing of multivariate time series in particular.     

On‐the‐Fly Power‐Aware Rendering

Power saving is a prevailing concern in desktop computers and, especially, in battery‐powered devices such as mobile phones. This is generating a growing demand for power‐aware graphics applications that can extend battery life, while preserving good quality. In this paper, we address this issue by presenting a real‐time power‐efficient rendering framework, able to dynamically select the rendering configuration with the best quality within a given power budget. Different from the current state of the art, our method does not require precomputation of the whole camera‐view space, nor Pareto curves to explore the vast power‐error space; as such, it can also handle dynamic scenes. Our algorithm is based on two key components: our novel power prediction model, and our runtime quality error estimation mechanism. These components allow us to search for the optimal rendering configuration at runtime, being transparent to the user. We demonstrate the performance of our framework on two different platforms: a desktop computer, and a mobile device. In both cases, we produce results close to the maximum quality, while achieving significant power savings.     

A CNN‐based Flow Correction Method for Fast Preview

Eulerian‐based smoke simulations are sensitive to the initial parameters and grid resolutions. Due to the numerical dissipation on different levels of the grid and the nonlinearity of the governing equations, the differences in simulation resolutions will result in different results. This makes it challenging for artists to preview the animation results based on low‐resolution simulations. In this paper, we propose a learning‐based flow correction method for fast previewing based on low‐resolution smoke simulations. The main components of our approach lie in a deep convolutional neural network, a grid‐layer feature vector and a special loss function. We provide a novel matching model to represent the relationship between low‐resolution and high‐resolution smoke simulations and correct the overall shape of a low‐resolution simulation to closely follow the shape of a high‐resolution down‐sampled version. We introduce the grid‐layer concept to effectively represent the 3D fluid shape, which can also reduce the input and output dimensions. We design a special loss function for the fluid divergence‐free constraint in the neural network training process. We have demonstrated the efficacy and the generality of our approach by simulating a diversity of animations deviating from the original training set. In addition, we have integrated our approach into an existing fluid simulation framework to showcase its wide applications.     

Illustrative Multivariate Visualization for Geological Modelling

In this paper, we present a novel illustrative multivariate visualization for geological modelling to assist geologists and reservoir engineers in visualizing multivariate datasets in superimposed representations, in contrast to the single‐attribute visualizations supported by commercial software. Our approach extends the use of decals from a single surface to 3D irregular grids, using the layering concept to represent multiple attributes. We also build upon prior work to augment the design and implementation of different geological attributes (namely, rock type, porosity, and permeability). More specifically, we propose a new sampling strategy to generate decals for porosity on the geological grid, a hybrid visualization for permeability which combines 2D decals and 3D ellipsoid glyphs, and a perceptually‐based design that allows us to visualize additional attributes (e.g., oil saturation) while avoiding visual interference between layers. Furthermore, our visual design draws from traditional geological illustrations, facilitating the understanding and communication between interdisciplinary teams. An evaluation by domain experts highlights the potential of our approach for geological modelling and interpretation in this complex domain.     

Visualizing for the Non‐Visual: Enabling the Visually Impaired to Use Visualization

The majority of visualizations on the web are still stored as raster images, making them inaccessible to visually impaired users. We propose a deep‐neural‐network‐based approach that automatically recognizes key elements in a visualization, including a visualization type, graphical elements, labels, legends, and most importantly, the original data conveyed in the visualization. We leverage such extracted information to provide visually impaired people with the reading of the extracted information. Based on interviews with visually impaired users, we built a Google Chrome extension designed to work with screen reader software to automatically decode charts on a webpage using our pipeline. We compared the performance of the back‐end algorithm with existing methods and evaluated the utility using qualitative feedback from visually impaired users.     

Object Partitioning for Support‐Free 3D‐Printing

Fused deposition modeling based 3D‐printing is becoming increasingly popular due to it's low‐cost and simple operation and maintenance. While it produces rugged prints made from a wide range of materials, it suffers from an inherent printing limitation where it cannot produce overhanging surfaces of non‐trivial size. This limitation can be handled by constructing temporary support‐structures, however this solution involves additional material costs, longer print time, and often a fair amount of labor in removing it. In this paper we present a new method for partitioning general solid objects into a small number of parts that can be printed with no support. The partitioning is computed by applying a sequence of cutting‐planes that split the object recursively. Unlike existing algorithms, the planes are not chosen at random, rather they are derived from shape analysis routines that identify and resolve various commonly‐found geometric configurations. In addition, we guide this search by a revised set of conditions that both ensure the objects' printability as well as realistically model the printing capabilities of the printer at hand. Evaluation of the new method demonstrates its ability to efficiently obtain support‐free partitionings typically containing fewer parts compared to existing methods that rely on support‐structures.     

Terrain Super‐resolution through Aerial Imagery and Fully Convolutional Networks

Despite recent advances in surveying techniques, publicly available Digital Elevation Models (DEMs) of terrains are low‐resolution except for selected places on Earth. In this paper we present a new method to turn low‐resolution DEMs into plausible and faithful high‐resolution terrains. Unlike other approaches for terrain synthesis/amplification (fractal noise, hydraulic and thermal erosion, multi‐resolution dictionaries), we benefit from high‐resolution aerial images to produce highly‐detailed DEMs mimicking the features of the real terrain. We explore different architectures for Fully Convolutional Neural Networks to learn upsampling patterns for DEMs from detailed training sets (high‐resolution DEMs and orthophotos), yielding up to one order of magnitude more resolution. Our comparative results show that our method outperforms competing data amplification approaches in terms of elevation accuracy and terrain plausibility.     

Robust Reference Frame Extraction from Unsteady 2D Vector Fields with Convolutional Neural Networks

Robust feature extraction is an integral part of scientific visualization. In unsteady vector field analysis, researchers recently directed their attention towards the computation of near‐steady reference frames for vortex extraction, which is a numerically challenging endeavor. In this paper, we utilize a convolutional neural network to combine two steps of the visualization pipeline in an end‐to‐end manner: the filtering and the feature extraction. We use neural networks for the extraction of a steady reference frame for a given unsteady 2D vector field. By conditioning the neural network to noisy inputs and resampling artifacts, we obtain numerically stabler results than existing optimization‐based approaches. Supervised deep learning typically requires a large amount of training data. Thus, our second contribution is the creation of a vector field benchmark data set, which is generally useful for any local deep learning‐based feature extraction. Based on Vatistas velocity profile, we formulate a parametric vector field mixture model that we parameterize based on numerically‐computed example vector fields in near‐steady reference frames. Given the parametric model, we can efficiently synthesize thousands of vector fields that serve as input to our deep learning architecture. The proposed network is evaluated on an unseen numerical fluid flow simulation.