Topology Aware Data‐Driven Inverse Kinematics

Creating realistic human movement is a time consuming and labour intensive task. The major difficulty is that the user has to edit individual joints while maintaining an overall realistic and collision free posture. Previous research suggests the use of data‐driven inverse kinematics, such that one can focus on the control of a few joints, while the system automatically composes a natural posture. However, as a common problem of kinematics synthesis, penetration of body parts is difficult to avoid in complex movements. In this paper, we propose a new data‐driven inverse kinematics framework that conserves the topology of the synthesizing postures. Our system monitors and regulates the topology changes using the Gauss Linking Integral (GUI), such that penetration can be efficiently prevented. As a result, complex motions with tight body movements, as well as those involving interaction with external objects, can be simulated with minimal manual intervention. Experimental results show that using our system, the user can create high quality human motion in real‐time by controlling a few joints using a mouse or a multi‐touch screen. The movement generated is both realistic and penetration free. Our system is best applied for interactive motion design in computer animations and games.     

DTI in Context: Illustrating Brain Fiber Tracts In Situ

We present an interactive illustrative visualization method inspired by traditional pen‐and‐ink illustration styles. Specifically, we explore how to provide context around DTI fiber tracts in the form of surfaces of the brain, the skull, or other objects such as tumors. These contextual surfaces are derived from either segmentation data or generated using interactive iso‐surface extraction and are rendered with a flexible, slice‐based hatching technique, controlled with ambient occlusion. This technique allows us to produce a consistent and frame‐coherent appearance with precise control over the lines. In addition, we provide context through cutting planes onto which we render gray matter with stippling. Together, our methods not only facilitate the interactive exploration and illustration of brain fibers within their anatomical context but also allow us to produce high‐quality images for print reproduction. We provide evidence for the success of our approach with an informal evaluation with domain experts.     

DYVERSO: A Versatile Multi‐Phase Position‐Based Fluids Solution for VFX

Many impressive fluid simulation methods have been presented in research papers before. These papers typically focus on demonstrating particular innovative features, but they do not meet in a comprehensive manner the production demands of actual VFX pipelines. VFX artists seek methods that are flexible, efficient, robust and scalable, and these goals often conflict with each other. In this paper, we present a multi‐phase particle‐based fluid simulation framework, based on the well‐known Position‐Based Fluids (PBF) method, designed to address VFX production demands. Our simulation framework handles multi‐phase interactions robustly thanks to a modified constraint formulation for density contrast PBF. And, it also supports the interaction of fluids sampled at different resolutions. We put special care on data structure design and implementation details. Our framework highlights cache‐efficient GPU‐friendly data structures, an improved spatial voxelization technique based on Z‐index sorting, tuned‐up simulation algorithms and two‐way‐coupled collision handling based on VDB fields. Altogether, our fluid simulation framework empowers artists with the efficiency, scalability and versatility needed for simulating very diverse scenes and effects.     

Advances in Interaction with 3D Environments

Various interaction techniques have been developed for interactive 3D environments. This paper presents an up‐to‐date and comprehensive review of the state of the art of non‐immersive interaction techniques for Navigation, Selection & Manipulation, and System Control, including a basic introduction to the topic, the challenges and an examination of a number of popular approaches. We also introduce 3D Interaction Testbed (3DIT) to firstly allow a ‘hands‐on' understanding of 3D interaction principles, and secondly to create an open platform for defining evaluation methods, stimuli as well as representative tasks akin to those found in other disciplines of science. We hope that this survey can aid both researchers and developers of interactive 3D applications in having a clearer overview of the topic and in particular can be useful for practitioners and researchers that are new to the field of interactive 3D graphics.     

Context Aware Terrain Visualization for Wayfinding and Navigation

To assist wayfinding and navigation, the display of maps and driving directions on mobile devices is nowadays commonplace. While existing system can naturally exploit GPS information to facilitate orientation, the inherently limited screen space is often perceived as a drawback compared to traditional street maps as it constrains the perception of contextual information. Moreover, occlusion issues add to this problem if the environment is shown from the popular egocentric perspective. In this paper we describe an interactive visualization system that addresses these problems by reallocating the available screen space. At the heart of our system are three novel visualization techniques: First, we propose a non‐standard perspective that allows to blend between the familiar pedestrian perspective and a standard map depiction with reduced occlusion. Second, we derive an efficient deformation technique that allows an interactive allocation of screen space to areas of interest like e.g. nearby touristic attractions. Finally, a path adaptive isometric perspective is proposed that reveals otherwise hidden facades in top‐down views. We describe efficient implementations of all techniques and exemplify our interactive system on real world urban models.     

Real‐Time Oil Painting on Mobile Hardware

This paper presents a realistic digital oil painting system, specifically targeted at the real‐time performance on highly resource‐constrained portable hardware such as tablets and iPads. To effectively use the limited computing power, we develop an efficient adaptation of the shallow water equations that models all the characteristic properties of oil paint. The pigments are stored in a multi‐layered structure to model the peculiar nature of pigment mixing in oil paint. The user experience ranges from thick shape‐retaining strokes to runny diluted paint that reacts naturally to the gravity set by tablet orientation. Finally, the paint is rendered in real time using a combination of carefully chosen efficient rendering techniques. The virtual lighting adapts to the tablet orientation, or alternatively, the front‐facing camera captures the lighting environment, which leads to a truly immersive user experience. Our proposed features are evaluated via a user study. In our experience, our system enables artists to quickly try out ideas and compositions anywhere when inspiration strikes, in a truly ubiquitous way. They do not need to carry expensive and messy oil paint supplies.     

Flexible Splicing of Upper‐Body Motion Spaces on Locomotion

This paper presents an efficient technique for synthesizing motions by stitching, or splicing, an upper‐body motion retrieved from a motion space on top of an existing lower‐body locomotion of another motion. Compared to the standard motion splicing problem, motion space splicing imposes new challenges as both the upper and lower body motions might not be known in advance. Our technique is the first motion (space) splicing technique that propagates temporal and spatial properties of the lower‐body locomotion to the newly generated upper‐body motion and vice versa. Whereas existing techniques only adapt the upper‐body motion to fit the lower‐body motion, our technique also adapts the lower‐body locomotion based on the upper body task for a more coherent full‐body motion. In this paper, we will show that our decoupled approach is able to generate high‐fidelity full‐body motion for interactive applications such as games.     

Deformable Motion: Squeezing into Cluttered Environments

We present an interactive method that allows animated characters to navigate through cluttered environments. Our characters are equipped with a variety of motion skills to clear obstacles, narrow passages, and highly constrained environment features. Our control method incorporates a behavior model into well‐known, standard path planning algorithms. Our behavior model, called deformable motion, consists of a graph of motion capture fragments. The key idea of our approach is to add flexibility on motion fragments such that we can situate them into a cluttered environment via constraint‐based formulation. We demonstrate our deformable motion for realtime interactive navigation and global path planning in highly constrained virtual environments.     

Interactive Generation of Realistic Facial Wrinkles from Sketchy Drawings

Synthesizing facial wrinkles has been tackled either by a long process of manual sculpting on 3D models, or using automatic methods that do not allow for user interaction or artistic expression. In this paper, we propose a method that accepts interactive sketchy drawings depicting wrinkle patterns, and synthesizes realistic looking wrinkles on faces. The method inherits the simplicity of sketching, making it possible for artists as well as novice users to generate realistic facial detail very efficiently, allowing fast preview for physical makeup, or aging simulations for fun and professional applications. All strokes are used to infer the wrinkles, retaining the expressiveness of the sketches and realism of the final result at the same time. This is achieved by designing novel multi‐scale statistics tailored to the wrinkle geometry and coupled to the sketch interpretation method. The statistics capture the cross‐sectional profiles of wrinkles at different scales and parts of a face. The strokes are augmented with the statistics extracted from given example face models, and applied to an input face model interactively. The interface gives the user control over the shapes and scales of wrinkles via sketching while adding extra details required for realism automatically.     

Computer‐Suggested Facial Makeup

Finding the best makeup for a given human face is an art in its own right. Experienced makeup artists train for years to be skilled enough to propose a best‐fit makeup for an individual. In this work we propose a system that automates this task. We acquired the appearance of 56 human faces, both without and with professional makeup. To this end, we use a controlled‐light setup, which allows to capture detailed facial appearance information, such as diffuse reflectance, normals, subsurface‐scattering, specularity, or glossiness. A 3D morphable face model is used to obtain 3D positional information and to register all faces into a common parameterization. We then define makeup to be the change of facial appearance and use the acquired database to find a mapping from the space of human facial appearance to makeup. Our main application is to use this mapping to suggest the best‐fit makeup for novel faces that are not in the database. Further applications are makeup transfer, automatic rating of makeup, makeup‐training, or makeup‐exaggeration. As our makeup representation captures a change in reflectance and scattering, it allows us to synthesize faces with makeup in novel 3D views and novel lighting with high realism. The effectiveness of our approach is further validated in a user‐study.     

Guided Image Filtering for Interactive High‐quality Global Illumination

Interactive computation of global illumination is a major challenge in current computer graphics research. Global illumination heavily affects the visual quality of generated images. It is therefore a key attribute for the perception of photo‐realistic images. Path tracing is able to simulate the physical behaviour of light using Monte Carlo techniques. However, the computational burden of this technique prohibits interactive rendering times on standard commodity hardware in high‐quality. Trying to solve the Monte Carlo integration with fewer samples results in characteristic noisy images. Global illumination filtering methods take advantage of the fact that the integral for neighbouring pixels may be very similar. Averaging samples of similar characteristics in screen‐space may approximate the correct integral, but may result in visible outliers. In this paper, we present a novel path tracing pipeline based on an edge‐aware filtering method for the indirect illumination which produces visually more pleasing results without noticeable outliers. The key idea is not to filter the noisy path traced images but to use it as a guidance to filter a second image composed from characteristic scene attributes that do not contain noise by default. We show that our approach better approximates the Monte Carlo integral compared to previous methods. Since the computation is carried out completely in screen‐space it is therefore applicable to fully dynamic scenes, arbitrary lighting and allows for high‐quality path tracing at interactive frame rates on commodity hardware.     

Paint Mesh Cutting

We present a novel progressive painting‐based mesh cut out tool, called Paint Mesh Cutting, for interactive mesh segmentation. Different from the previous user interfaces, the user only needs to draw a single stroke on the foreground region and then obtains the desired cutting part at an interactive rate. Moreover, the user progressively paints the region of interest using a brush and has the instant feedback on cutting results as he/she drags the mouse. This is achieved by efficient local graph‐cut based optimizations based on the Gaussian mixture models (GMM) on the shape diameter function (SDF) metric of the shape. We demonstrate a number of various examples to illustrate the flexibility and applicability of our system and present a user study that supports the advantages of our user interface.     

Pre‐computed Gathering of Multi‐Bounce Glossy Reflections

Recent work in interactive global illumination addresses diffuse and moderately glossy indirect lighting effects, but high‐frequency effects such as multi‐bounce reflections on highly glossy surfaces are often ignored. Accurately simulating such effects is important to convey the realistic appearance of materials such as chrome and shiny metal. In this paper, we present an efficient method for visualizing multi‐bounce glossy reflections at interactive rates under environment lighting. Our main contribution is a pre‐computation–based method which efficiently gathers subsequent highly glossy reflection passes modelled with a non‐linear transfer function representation based on the von Mises–Fisher distribution. We show that our gathering method is superior to scattered sampling. To exploit the sparsity of the pre‐computed data, we apply perfect spatial hashing. As a result, we are able to visualize multi‐bounce glossy reflections at interactive rates at a low pre‐computation cost.     

Decoupled Space and Time Sampling of Motion and Defocus Blur for Unified Rendering of Transparent and Opaque Objects

We propose a unified rendering approach that jointly handles motion and defocus blur for transparent and opaque objects at interactive frame rates. Our key idea is to create a sampled representation of all parts of the scene geometry that are potentially visible at any point in time for the duration of a frame in an initial rasterization step. We store the resulting temporally‐varying fragments (t‐fragments) in a bounding volume hierarchy which is rebuild every frame using a fast spatial median construction algorithm. This makes our approach suitable for interactive applications with dynamic scenes and animations. Next, we perform spatial sampling to determine all t‐fragments that intersect with a specific viewing ray at any point in time. Viewing rays are sampled according to the lens uv‐sampling for depth‐of‐field effects. In a final temporal sampling step, we evaluate the predetermined viewing ray/t‐fragment intersections for one or multiple points in time. This allows us to incorporate all standard shading effects including transparency. We describe the overall framework, present our GPU implementation, and evaluate our rendering approach with respect to scalability, quality, and performance.     

Semi‐Automatic Editing of Graphs with Customized Layouts

Usually visualization is applied to gain insight into data. Yet consuming the data in form of visual representation is not always enough. Instead, users need to edit the data, preferably through the same means used to visualize them. In this work, we present a semi‐automatic approach to visual editing of graphs. The key idea is to use an interactive EditLens that defines where an edit operation affects an already customized and established graph layout. Locally optimal node positions within the lens and edge routes to connected nodes are calculated according to different criteria. This spares the user much manual work, but still provides sufficient freedom to accommodate application‐dependent layout constraints. Our approach utilizes the advantages of multi‐touch gestures, and is also compatible with classic mouse and keyboard interaction. Preliminary user tests have been conducted with researchers from bio‐informatics who need to manually maintain a slowly, but constantly growing molecular network. As the user feedback indicates, our solution significantly improves the editing procedure applied so far.     

Restoration of traditional Chinese shadow play‐Piying art from tangible interaction

Piying, the world's intangible culture heritage, is a characteristic Chinese folk shadow play and one of the origins of modern movie. The spirit of traditional Piying is to express rich emotion and stories through impromptu action change by professional artists. Now Piying gradually fades away in people's lives, encountering the risk of extinction. We focus on transforming the traditional Piying play into an interactive system, which can be seamlessly performed by ordinary people and bring a strong sense of immersion to the users. Two interactive systems were developed, with Kinect‐based interaction or sensors, users can create own digital Piying animations by employing their body movement as input. A field study was presented with 20 students in a primary school. The result showed that our system was far more effective in emotion induction and Piying understanding than traditional one. We demonstrated this system in a charitable foundation and a workshop of a tangible conference. It is also honored to be collected by China's intangible cultural heritage network. Copyright © 2013 John Wiley & Sons, Ltd.     

Supporting the creative drive: investigating paralinguistic voice as a mode of interaction for artists with upper limb disabilities

For people with upper limb disabilities visual art is an important activity that allows for expression of individuality and independence. They show remarkable endurance, patience and determination to adapt their remaining capabilities to create visual art. There are significant advantages of digital technologies in assisting artists with upper limb disabilities. Paralinguistic voice recognition technologies have proven to be a particularly promising mode of interaction. Despite these benefits, technological support for people with upper limb disabilities to create visual art is scarce. This paper reports on a number of case studies of several artists with upper limb disabilities. These case studies illustrate the struggles they face to be creative and also show the significant advantages of digital technologies in assisting such artists. An investigation into people’s ability to use the volume of their voice to control cursor movement to create drawings on the screen is also reported. With motivation, training and practise, use of volume to control drawing tasks shows great promise. It is believed that paralinguistic voice has wider implications beyond assisting artists with upper limb disabilities, such as: an alternative mode of interaction for disabled people to perform tasks other than creating visual art, alternative mode of interaction for hands busy environments and as a voice training system for people with speech impairments.     

An Interactive Visual Analytics System for Bridge Management

Bridges deteriorate over their life cycles and require continuous maintenance to ensure their structural integrity, and in turn, the safety of the public. Maintaining bridges is a multi‐faceted operation that requires both domain knowledge and analytics techniques over large data sources. Although most existing bridge management systems (BMS) are very efficient at data storage, they are not as effective at providing analytical capabilities or as flexible at supporting different inspection technologies. In this paper, we present a visual analytics system that extends the capability of current BMSs. Based on a nation‐wide survey and our interviews with bridge managers, we designed our system to be customizable so that it can provide interactive exploration, information correlation, and domain‐oriented data analysis. When tested by bridge managers of the U.S. Department of Transportation, we validated that our system provides bridge managers with the necessary features for performing in‐depth analysis of bridges from a variety of perspectives that are in accordance to their typical workflow.     

AppFusion: Interactive Appearance Acquisition Using a Kinect Sensor

We present an interactive material acquisition system for average users to capture the spatially varying appearance of daily objects. While an object is being scanned, our system estimates its appearance on‐the‐fly and provides quick visual feedback. We build the system entirely on low‐end, off‐the‐shelf components: a Kinect sensor, a mirror ball and printed markers. We exploit the Kinect infra‐red emitter/receiver, originally designed for depth computation, as an active hand‐held reflectometer, to segment the object into clusters of similar specular materials and estimate the roughness parameters of BRDFs simultaneously. Next, the diffuse albedo and specular intensity of the spatially varying materials are rapidly computed in an inverse rendering framework, using data from the Kinect RGB camera. We demonstrate captured results of a range of materials, and physically validate our system.