Integrating discrete events and continuous head movements for video-based interaction techniques

Human head gestures can potentially trigger different commands from the list of available options in graphical user interfaces or in virtual and smart environments. However, continuous tracking techniques are limited in generating discrete events which could be used to execute a predefined set of commands. In this article, we discuss a possibility to encode a set of discrete events by integrating continuous head movements and crossing-based interaction paradigm. A set of commands can be encoded through specific sequences of crossing points when a head-mouse cursor such as a scaled pointer interacts with a graphical object. The goal of the present experiment was testing the perceptual-motor performance of novices in target acquisition tasks using a subset of round head gestures and symbolic icons designating eight types of directional head movements. We have demonstrated that the novices can equally well execute round head gestures in clockwise and counter-clockwise directions by making two crossings for about 2 s or three crossings for about 3 s. None of the participants reported neck strain or other problems after 360 trials performed during a 40-min test in each of 5 days.     

3D segmentation of abdominal CT imagery with graphical models,conditional random fields and learning

Probabilistic graphical models have had a tremendous impact in machine learning and approaches based on energy function minimization via techniques such as graph cuts are now widely used in image segmentation. However, the free parameters in energy function-based segmentation techniques are often set by hand or using heuristic techniques. In this paper, we explore parameter learning in detail. We show how probabilistic graphical models can be used for segmentation problems to illustrate Markov random fields (MRFs), their discriminative counterparts conditional random fields (CRFs) as well as kernel CRFs. We discuss the relationships between energy function formulations, MRFs, CRFs, hybrids based on graphical models and their relationships to key techniques for inference and learning. We then explore a series of novel 3D graphical models and present a series of detailed experiments comparing and contrasting different approaches for the complete volumetric segmentation of multiple organs within computed tomography imagery of the abdominal region. Further, we show how these modeling techniques can be combined with state of the art image features based on histograms of oriented gradients to increase segmentation performance. We explore a wide variety of modeling choices, discuss the importance and relationships between inference and learning techniques and present experiments using different levels of user interaction. We go on to explore a novel approach to the challenging and important problem of adrenal gland segmentation. We present a 3D CRF formulation and compare with a novel 3D sparse kernel CRF approach we call a relevance vector random field. The method yields state of the art performance and avoids the need to discretize or cluster input features. We believe our work is the first to provide quantitative comparisons between traditional MRFs with edge-modulated interaction potentials and CRFs for multi-organ abdominal segmentation and the first to explore the 3D adrenal gland segmentation problem. Finally, along with this paper we provide the labeled data used for our experiments to the community.     

Parallel use of hand gestures and force‐input device for interacting with 3d and virtual reality environments

User interfaces of current 3D and virtual reality environments require highly interactive input/output (I/O) techniques and appropriate input devices, providing users with natural and intuitive ways of interacting. This paper presents an interaction model, some techniques, and some ways of using novel input devices for 3D user interfaces. The interaction model is based on a tool‐object syntax, where the interaction structure syntactically simulates an action sequence typical of a human's everyday life: One picks up a tool and then uses it on an object. Instead of using a conventional mouse, actions are input through two novel input devices, a hand‐ and a force‐input device. The devices can be used simultaneously or in sequence, and the information they convey can be processed in a combined or in an independent way by the system. The use of a hand‐input device allows the recognition of static poses and dynamic gestures performed by a user's hand. Hand gestures are used for selecting, or acting as, tools and for manipulating graphical objects. A system for teaching and recognizing dynamic gestures, and for providing graphical feedback for them, is described.     

The FlowVizMenu and parallel scatterplot matrix: hybrid multidimensional visualizations for network exploration

A standard approach for visualizing multivariate networks is to use one or more multidimensional views (for example, scatterplots) for selecting nodes by various metrics, possibly coordinated with a node-link view of the network. In this paper, we present three novel approaches for achieving a tighter integration of these views through hybrid techniques for multidimensional visualization, graph selection and layout. First, we present the FlowVizMenu, a radial menu containing a scatterplot that can be popped up transiently and manipulated with rapid, fluid gestures to select and modify the axes of its scatterplot. Second, the FlowVizMenu can be used to steer an attribute-driven layout of the network, causing certain nodes of a node-link diagram to move toward their corresponding positions in a scatterplot while others can be positioned manually or by force-directed layout. Third, we describe a novel hybrid approach that combines a scatterplot matrix (SPLOM) and parallel coordinates called the Parallel Scatterplot Matrix (P-SPLOM), which can be used to visualize and select features within the network. We also describe a novel arrangement of scatterplots called the Scatterplot Staircase (SPLOS) that requires less space than a traditional scatterplot matrix. Initial user feedback is reported.     

Dynamic visualization of coexpression in systems genetics data

Biologists hope to address grand scientific challenges by exploring the abundance of data made available through modern microarray technology and other high-throughput techniques. The impact of this data, however, is limited unless researchers can effectively assimilate such complex information and integrate it into their daily research; interactive visualization tools are called for to support the effort. Specifically, typical studies of gene co-expression require novel visualization tools that enable the dynamic formulation and fine-tuning of hypotheses to aid the process of evaluating sensitivity of key parameters. These tools should allow biologists to develop an intuitive understanding of the structure of biological networks and discover genes residing in critical positions in networks and pathways. By using a graph as a universal representation of correlation in gene expression, our system employs several techniques that when used in an integrated manner provide innovative analytical capabilities. Our tool for interacting with gene co-expression data integrates techniques such as: graph layout, qualitative subgraph extraction through a novel 2D user interface, quantitative subgraph extraction using graph-theoretic algorithms or by compound queries, dynamic level-of-detail abstraction, and template-based fuzzy classification. We demonstrate our system using a real-world workflow from a large-scale, systems genetics study of mammalian gene co-expression.     

基于自适应光标的图形用户界面输入效率优化

提高图形用户界面(graphical user interface)的输入效率,是人机交互中的一项重要研究内容.已有的研究包括点击增强技术和自适应界面技术,前者改变光标的控制方式或呈现方式,后者改变界面上控件的位置布局,但两种技术都存在不足.通过分析界面操作,提出了图形用户界面输入效率的评价模型;然后,在此基础上提出一种人机交互效率优化技术:自适应光标.它以自适应的方式,有选择地对界面上用户可能访问的控件通过点击增强技术支持,实现快速访问.该方法既解决了以往的自适应界面技术因频繁调整控件布局而给用户带来额外认知成本的问题,也克服了点击增强技术仅适用于稀疏控件布局的限制.为了检验其可用性,在控件较多的Visual Studio上实现了自适应光标技术.实验结果表明,使用自适应光标技术可以将获取目标的时间缩短27.7％,显著提高了图形用户界面的输入效率.     

Multidimensional Projections for Visual Analysis of Social Networks

Visual analysis of social networks is usually based on graph drawing algorithms and tools.However,social networks are a special kind of graph in the sense that interpretation of displayed relationships is heavily dependent on context.Context,in its turn,is given by attributes associated with graph elements,such as individual nodes,edges,and groups of edges,as well as by the nature of the connections between individuals.In most systems,attributes of individuals and communities are not taken into consideration during graph layout,except to derive weights for force-based placement strategies.This paper proposes a set of novel tools for displaying and exploring social networks based on attribute and connectivity mappings.These properties are employed to layout nodes on the plane via multidimensional projection techniques.For the attribute mapping,we show that node proximity in the layout corresponds to similarity in attribute,leading to easiness in locating similar groups of nodes.The projection based on connectivity yields an initial placement that forgoes force-based or graph analysis algorithm,reaching a meaningful layout in one pass.When a force algorithm is then applied to this initial mapping,the final layout presents better properties than conventional force-based approaches.Numerical evaluations show a number of advantages of pre-mapping points via projections.User evaluation demonstrates that these tools promote ease of manipulation as well as fast identification of concepts and associations which cannot be easily expressed by conventional graph visualization alone.In order to allow better space usage for complex networks,a graph mapping on the surface of a sphere is also implemented.     

GRAFEDIT: An interactive general-purpose graphics editor

GRAFEDIT is a general-purpose interactive graphical editor. It provides means to create, modify, transform, copy, store and retrieve almost any kind of two-dimensional line diagrams on a CRT terminal. The editor is easy to use even by unexperienced persons. A set of simple and yet powerful commands allows the user to conduct a natural, efficient, and productive dialogue with GRAFEDIT. In response to the commands, the editor displays various guiding messages or prompts for specific actions, and executes the user's instructions. GRAFEDIT is implemented in the MIRA-2D language, a graphical PASCAL extension based on abstract graphical types.The applications for GRAFEDIT are numerous. Among these are business administration, marketing, advertising, graphical design, technical and industrial drawing, electronic design, teaching and audio-visual presentations.     

Navigation and Exploration of Interconnected Pathways

Visualizing pathways, i. e. models of cellular functional networks, is a challenging task in computer assisted biomedicine. Pathways are represented as large collections of interwoven graphs, with complex structures present in both the individual graphs and their interconnections. This situation requires the development of novel visualization techniques to allow efficient visual exploration. We present the Caleydo framework, which incorporates a number of approaches to handle such pathways. Navigation in the network of pathways is facilitated by a hierarchical approach which dynamically selects a working set of individual pathways for closer inspection. These pathways are interactively rendered together with visual interconnections in a 2.5D view using graphics hardware acceleration. The layout of individual graphs is not computed automatically, but taken from the KEGG and BioCarta databases, which use layouts that life scientists are familiar with. Therefore they encode essential meta‐information. While the KEGG and BioCarta pathways use a pre‐defined layout, interactions such as linking+brushing, neighborhood search or detail on demand are still fully interactive in Caleydo. We have evaluated Caleydo with pathologists working on the determination of unknown gene functions. Informal experiences confirm that Caleydo is useful in both generating and validating such hypotheses. Even though the presented techniques are applied to medical pathways, the proposed way of interaction is not limited to cellular processes and therefore has the potential to open new possibilities in other fields of application.     

Exploring gene causal interactions using an enhanced constraint-based method

DNA microarray provides a powerful basis for analysis of gene expression. Bayesian networks, which are based on directed acyclic graphs (DAGs) and can provide models of causal influence, have been investigated for gene regulatory networks. The difficulty with this technique is that learning the Bayesian network structure is an NP-hard problem, as the number of DAGs is superexponential in the number of genes, and an exhaustive search is intractable. In this paper, we propose an enhanced constraint-based approach for causal structure learning. We integrate with graphical Gaussian modeling and use its independence graph as an input of our constraint-based causal learning method. We also present graphical decomposition techniques to further improve the performance. Our enhanced method makes it feasible to explore causal interactions among genes interactively. We have tested our methodology using two microarray data sets. The results show that the technique is both effective and efficient in exploring causal structures from microarray data.     

Using adjacency matrices to lay out larger small-world networks

Many networks exhibit small-world properties. The structure of a small-world network is characterized by short average path lengths and high clustering coefficients. Few graph layout methods capture this structure well which limits their effectiveness and the utility of the visualization itself. Here we present an extension to our novel graphTPP layout method for laying out small-world networks using only their topological properties rather than their node attributes. The Watts–Strogatz model is used to generate a variety of graphs with a small-world network structure. Community detection algorithms are used to generate six different clusterings of the data. These clusterings, the adjacency matrix and edgelist are loaded into graphTPP and, through user interaction combined with linear projections of the adjacency matrix, graphTPP is able to produce a layout which visually separates these clusters. These layouts are compared to the layouts of two force-based techniques. graphTPP is able to clearly separate each of the communities into a spatially distinct area and the edge relationships between the clusters show the strength of their relationship. As a secondary contribution, an edge-grouping algorithm for graphTPP is demonstrated as a means to reduce visual clutter in the layout and reinforce the display of the strength of the relationship between two communities.     

From visual data exploration to visual data mining: a survey

We survey work on the different uses of graphical mapping and interaction techniques for visual data mining of large data sets represented as table data. Basic terminology related to data mining, data sets, and visualization is introduced. Previous work on information visualization is reviewed in light of different categorizations of techniques and systems. The role of interaction techniques is discussed, in addition to work addressing the question of selecting and evaluating visualization techniques. We review some representative work on the use of information visualization techniques in the context of mining data. This includes both visual data exploration and visually expressing the outcome of specific mining algorithms. We also review recent innovative approaches that attempt to integrate visualization into the DM/KDD process, using it to enhance user interaction and comprehension.     

Automatic modeling of cluttered multi‐room floor plans from panoramic images

We present a novel and light‐weight approach to capture and reconstruct structured 3D models of multi‐room floor plans. Starting from a small set of registered panoramic images, we automatically generate a 3D layout of the rooms and of all the main objects inside. Such a 3D layout is directly suitable for use in a number of real‐world applications, such as guidance, location, routing, or content creation for security and energy management. Our novel pipeline introduces several contributions to indoor reconstruction from purely visual data. In particular, we automatically partition panoramic images in a connectivity graph, according to the visual layout of the rooms, and exploit this graph to support object recovery and rooms boundaries extraction. Moreover, we introduce a plane‐sweeping approach to jointly reason about the content of multiple images and solve the problem of object inference in a top‐down 2D domain. Finally, we combine these methods in a fully automated pipeline for creating a structured 3D model of a multi‐room floor plan and of the location and extent of clutter objects. These contribution make our pipeline able to handle cluttered scenes with complex geometry that are challenging to existing techniques. The effectiveness and performance of our approach is evaluated on both real‐world and synthetic models.     

IMMIView: a multi-user solution for design review in real-time

IMMIView is an interactive system that relies on multiple modalities and multi-user interaction to support collaborative design review. It was designed to offer natural interaction in visualization setups such as large-scale displays, head mounted displays or TabletPC computers. To support architectural design, our system provides content creation and manipulation, 3D scene navigation and annotations. Users can interact with the system using laser pointers, speech commands, body gestures and mobile devices. In this paper, we describe how we design a system to answer architectural user requirements. In particular, our system takes advantage of multiple modalities to provide a natural interaction for design review. We also propose a new graphical user interface adapted to architectural user tasks, such as navigation or annotations. The interface relies on a novel stroke-based interaction supported by simple laser pointers as input devices for large-scale displays. Furthermore, input devices such as speech and body tracking allow IMMIView to support multiple users. Moreover, they allow each user to select different modalities according to their preference and modality adequacy for the user task. We present a multi-modal fusion system developed to support multi-modal commands on a collaborative, co-located, environment, i.e. with two or more users interacting at the same time, on the same system. The multi-modal fusion system listens to inputs from all the IMMIView modules in order to model user actions and issue commands. The multiple modalities are fused based on a simple rule-based sub-module developed in IMMIView and presented in this paper. User evaluation performed over IMMIView is presented. The results show that users feel comfortable with the system and suggest that users prefer the multi-modal approach to more conventional interactions, such as mouse and menus, for the architectural tasks presented.     

Displaying Trace Files

Computers generate trace files containing reports on system performance, status and faults. To analyze these trace files more efficiently, we have developed a graphical technique embodied in an interactive system for displaying large trace files. Our system uses abstraction, color, aggregation, filtering, interaction, and a drill-down capability to find patterns among the reports. We apply our system and technique to analyze command accounting trace files from a Unix compute server, showing what commands were executed, by which users, when, and how long the commands ran. We identify resource intensive commands, sequences of commands initiated by a compilations, and commands run with super-user permissions.     

MultiClusterTree: Interactive Visual Exploration of Hierarchical Clusters in Multidimensional Multivariate Data

Visual analytics of multidimensional multivariate data is a challenging task because of the difficulty in understanding metrics in attribute spaces with more than three dimensions. Frequently, the analysis goal is not to look into individual records but to understand the distribution of the records at large and to find clusters of records with similar attribute values. A large number of (typically hierarchical) clustering algorithms have been developed to group individual records to clusters of statistical significance. However, only few visualization techniques exist for further exploring and understanding the clustering results. We propose visualization and interaction methods for analyzing individual clusters as well as cluster distribution within and across levels in the cluster hierarchy. We also provide a clustering method that operates on density rather than individual records. To not restrict our search for clusters, we compute density in the given multidimensional multivariate space. Clusters are formed by areas of high density. We present an approach that automatically computes a hierarchical tree of high density clusters. To visually represent the cluster hierarchy, we present a 2D radial layout that supports an intuitive understanding of the distribution structure of the multidimensional multivariate data set. Individual clusters can be explored interactively using parallel coordinates when being selected in the cluster tree. Furthermore, we integrate circular parallel coordinates into the radial hierarchical cluster tree layout, which allows for the analysis of the overall cluster distribution. This visual representation supports the comprehension of the relations between clusters and the original attributes. The combination of the 2D radial layout and the circular parallel coordinates is used to overcome the overplotting problem of parallel coordinates when looking into data sets with many records. We apply an automatic coloring scheme based on the 2D radial layout of the hierarchical cluster tree encoding hue, saturation, and value of the HSV color space. The colors support linking the 2D radial layout to other views such as the standard parallel coordinates or, in case data is obtained from multidimensional spatial data, the distribution in object space.     

Dual-Finger 3D Interaction Techniques for mobile devices

Three-dimensional capabilities on mobile devices are increasing, and the interactivity is becoming a key feature of these tools. It is expected that users will actively engage with the 3D content, instead of being passive consumers. Because touch-screens provide a direct means of interaction with 3D content by directly touching and manipulating 3D graphical elements, touch-based interaction is a natural and appealing style of input for 3D applications. However, developing 3D interaction techniques for handheld devices using touch-screens is not a straightforward task. One issue is that when interacting with 3D objects, users occlude the object with their fingers. Furthermore, because the user’s finger covers a large area of the screen, the smallest size of the object users can touch is limited. In this paper, we first inspect existing 3D interaction techniques based on their performance with handheld devices. Then, we present a set of precise Dual-Finger 3D Interaction Techniques for a small display. Finally, we present the results of an experimental study, where we evaluate the usability, performance, and error rate of the proposed and existing 3D interaction techniques.     

Sketching and Composing Widgets for 3D Manipulation

We present an interface for 3D object manipulation in which standard transformation tools are replaced with transient 3D widgets invoked by sketching context‐dependent strokes. The widgets are automatically aligned to axes and planes determined by the user's stroke. Sketched pivot‐points further expand the interaction vocabulary. Using gestural commands, these basic elements can be assembled into dynamic, user‐constructed 3D transformation systems. We supplement precise widget interaction with techniques for coarse object positioning and snapping. Our approach, which is implemented within a broader sketch‐based modeling system, also integrates an underlying “widget history” to enable the fluid transfer of widgets between objects. An evaluation indicates that users familiar with 3D manipulation concepts can be taught how to efficiently use our system in under an hour.     

The Personal Acoustics Lab (PAL): a microcomputer-based system for digital signal acquisition, analysis, and synthesis

A new, integrated digital signal processing (DSP) system, the Personal Acoustics Lab (PAL), is described. This microcomputer-based system is suitable for analogue signal digitization at rates from several samples per hour to 150,000 samples per second in 12- or 16-bit words. Data may be acquired on one to sixteen single-ended A/D, or one to eight double-ended A/D channels in bipolar or unipolar modes. Digitized data may be reconverted to analogue signals using one or two D/A channels. An external clock and trigger and two bidirectional digital ports are provided. Integrated PAL-ILS software commands perform all necessary DSP functions, including: data editing, time- and frequency-domain graphical display, plotting, filtering, Fourier and Hilbert transforms, linear predictive coding, auto- and cross-correlation, and summary statistics. The system is suitable for biological and engineering DSP applications. Output from selected PAL-ILS software commands is illustrated using a bioacoustical example.     

ArchiDNA: An interactive system for creating 2D and 3D conceptual drawings in architectural design

We present an interactive system called ArchiDNA for creating 2D and 3D conceptual drawings in architectural design. We developed a novel principle of shape generation called match-and-attach by analyzing drawing styles of a contemporary architect, Peter Eisenman. The process consists of user interaction techniques and a set of rules that decide how one or more shapes attach to another shape. One key ingredient of our process is a unique concept for the interactive semi-automatic shape generation that uses the combination of algorithmic rules of a computer and designers’ manual inputs. These techniques enable designers to use CAD software in the early stages of architectural designs to explore conceptual building forms. ArchiDNA dynamically responds to drawing inputs, configures 2D shapes, and converts them to 3D shapes in a similar style. We intend to complement existing CAD software and computational drawing pipelines for intuitive 2D and 3D conceptual drawing creation.