Simulation in contexts involving an interactive table and tangible objects

By using an interactive table, it is possible to interact with several people (decision-makers) in a simultaneous and collaborative way, around the table, during a simulation session. Thanks to the RFID technology with which the table is fitted, it is possible to give tangible objects a unique identity to include and to consider them in the simulation. The paper describes a context model, which takes into consideration the specificities related to interactive tables. The TangiSense interactive table is presented; it is connected to a multi-agent system making it possible to give the table a certain level of adaptation: each tangible object can be associated to an agent which can bring roles to the object (i.e., the roles are the equivalent of a set of behaviors). The multi-agent system proposed in this paper is modeled according to an architecture adapted to the exploitation of tangible and virtual objects during simulation on an interactive table. A case study is presented; it concerns a simulation of road traffic management. The illustrations give an outline of the potentialities of the simulation system as regards the context-awareness aspect, following both the actions of the decision-makers implied in simulation, and the agents composing the road traffic simulation.     

A dual-representation strategy for the virtual assembly of thin deformable objects

The two main objectives of virtual assembly are: (1) to train assembly-operators through virtual assembly models, and (2) to simultaneously evaluate products for ease-of-assembly. The focus of this paper is on developing computational techniques for virtual assembly of thin deformable beam and plate-like objects. To meet the objectives of virtual assembly, the underlying computational technique must: (1) be carried out at a high frame-rate (>20 frames/second), (2) be accurate (<5% error in deformation and force estimation), (3) be conducive to collision detection, and (4) support rapid design evaluations. We argue in this paper that popular computational techniques such as 3-D finite element analysis, boundary element analysis and classic beam/plate/shell analysis fail to meet these requirements. We therefore propose a new class of dual representation techniques for virtual assembly of thin solids, where the geometry is retained in its full 3-D form, while the underlying physics is dimensionally reduced, delivering: (1) high computational efficiency and accuracy (over 20 frames per second with <1% deformation error), and (2) direct CAD model processing, i.e., the CAD model is not geometrically simplified, and 3-D finite element mesh is not generated. In particular, a small-size stiffness matrix with about 300 degrees of freedom per deformable object is generated directly from a coarse surface triangulation, and its LU-decomposition is then exploited during real-time simulation. The accuracy and efficiency of the proposed method is established through numerical experiments and a case study.     

A computational model of perceptual saliency for 3D objects in virtual environments

When giving directions to the location of an object, people typically use other attractive objects as reference, that is, reference objects. With the aim to select proper reference objects, useful for locating a target object within a virtual environment (VE), a computational model to identify perceptual saliency is presented. Based on the object’s features with the major stimulus for the human visual system, three basic features of a 3D object (i.e., color, size, and shape) are individually evaluated and then combined to get a degree of saliency for each 3D object in a virtual scenario. An experiment was conducted to evaluate the extent to which the proposed measure of saliency matches with the people’s subjective perception of saliency; the results showed a good performance of this computational model.     

Building interactive virtual environments for simulated training in medicine using VRML and Java/JavaScript

Medicine is a difficult thing to learn. Experimenting with real patients should not be the only option; simulation deserves a special attention here. Virtual Reality Modelling Language (VRML) as a tool for building virtual objects and scenes has a good record of educational applications in medicine, especially for static and animated visualisations of body parts and organs. However, to create computer simulations resembling situations in real environments the required level of interactivity and dynamics is difficult to achieve. In the present paper we describe some approaches and techniques which we used to push the limits of the current VRML technology further toward dynamic 3D representation of virtual environments (VEs). Our demonstration is based on the implementation of a virtual baby model, whose vital signs can be controlled from an external Java application. The main contributions of this work are: (a) outline and evaluation of the three-level VRML/Java implementation of the dynamic virtual environment, (b) proposal for a modified VRML Timesensor node, which greatly improves the overall control of system performance, and (c) architecture of the prototype distributed virtual environment for training in neonatal resuscitation comprising the interactive virtual newborn, active bedside monitor for vital signs and full 3D representation of the surgery room.     

Performance of new participants in virtual environments: The Nottingham tool for assessment of interaction in virtual environments (NAÏVE)

There is a need for an assessment tool which reliably distinguishes levels of participant performance in virtual environments (VEs) built within virtual reality (VR) systems. Such screening might be of potential users amongst a company's staff or might be carried out by human factors experimenters prior to the start of experiments in order to provide a base-line of participant competences. The Nottingham Tool for Assessment for Interaction in Virtual Environments (NAÏVE) comprises a set of VE tasks and related tests, with appropriate performance criteria levels, covering the main aspects of navigation (viewpoint) control and object manipulation and operation. Trials with test participants enabled performance levels to be set to distinguish good, adequate and poor performers and tests to be distinguished according to whether performance in the general population is evenly spread or is skewed towards success or failure.     

A multimodal virtual reality interface for 3D interaction with VTK

The object-oriented visualization Toolkit (VTK) is widely used for scientific visualization. VTK is a visualization library that provides a large number of functions for presenting three-dimensional data. Interaction with the visualized data is controlled with two-dimensional input devices, such as mouse and keyboard. Support for real three-dimensional and multimodal input is non-existent. This paper describes VR-VTK: a multimodal interface to VTK on a virtual environment. Six degree of freedom input devices are used for spatial 3D interaction. They control the 3D widgets that are used to interact with the visualized data. Head tracking is used for camera control. Pedals are used for clutching. Speech input is used for application commands and system control. To address several problems specific for spatial 3D interaction, a number of additional features, such as more complex interaction methods and enhanced depth perception, are discussed. Furthermore, the need for multimodal input to support interaction with the visualization is shown. Two existing VTK applications are ported using VR-VTK to run in a desktop virtual reality system. Informal user experiences are presented. Arjan J. F. Kok is an assistant professor at the Department of Computer Science at the Open University of the Netherlands. He studied Computer Science at the Delft University of Technology, The Netherlands. He received his Ph.D. from the same university. He worked as a Scientist for TNO (Netherlands Organization for Applied Scientific Research) and as assistant professor at the Eindhoven University of Technology before he joined the Open University. His research interests are visualization, virtual reality, and computer graphics. Robert van Liere studied Computer Science at the Delft University of Technology, the Netherlands. He received his Ph.D. with the thesis “Studies in Interactive Scientific Visualization” at the University of Amsterdam. Since 1985, he has worked at CWI, the Center for Mathematics and Computer Science in Amsterdam in which he is the head of CWI’s visualization research group. Since 2004, he holds a part-time position as full professor at the Eindhoven University of Technology. His research interests are in interactive data visualization and virtual reality. He is a member of IEEE.     

A scalable interest management scheme for distributed virtual environments

With the expansion of the internet and its bandwidth, distributed virtual environment (DVE) applications have become more prevalent. In DVE applications, users frequently crowd in a specific place, and a key aspect to consider is how to provide interactive performance for users. However, existing approaches using multicast require users to receive uninteresting messages. Even though recent works have addressed fine‐grained filtering, they still incur other drawbacks in terms of assigning lots of multicast addresses or handling overhead of multicast groups. This makes the system less scalable as the number of users increases. In this paper, we propose a new scalable filtering scheme that reduces not only the number of messages during interaction in a region and among neighboring regions, but also the number of multicast addresses without significant computational overhead. Interest management in a region dynamically creates groups of users with the same interests. While members communicate with each other with high fidelity, a representative sends information to non‐members with low frequency. For interaction among neighboring regions, we propose a sub‐region concept to select only a subset of users from the neighboring regions based on proximity, the distribution of the users' locations, and the viewing direction of a user. Copyright © 2007 John Wiley & Sons, Ltd.     

Badumna: A decentralised network engine for virtual environments

Many of today’s virtual environment applications, such as massively multiplayer online games, involve a large and rapidly changing set of users. The server-based architecture of these applications makes its hard for them to support the dynamic user base and provide a scalable solution. Badumna, a scalable network engine for large-scale virtual environments, was developed to address these issues. The approach is based on a decentralised architecture using a structured peer-to-peer network. In order to ensure a highly scalable and efficient design, Badumna uses techniques that include neighbour introductions and the formation of dynamic bounded regions to reduce the load on the peer-to-peer network. The performance of Badumna is evaluated using extensive simulations. Results indicate that Badumna is able to scale to a large number of users without placing any additional overheads on the network that degrade performance.     

Traffic modeling flows in a developing urban infrastructure with a software suite for creating interactive virtual environments

We present a design paradigm for a cloud-based service for interactive modeling of traffic flows in a developing urban infrastructure. The mathematical forecasting model for the load of a transportation network is constructed as a result of synthesizing a gravitational model that describes origin-destination trips and a multimodal traffic equilibrium problem with elastic demand. The search for traffic equilibrium reduces to solving a variational inequality. Three-dimensional visualization of the transportation network and adjacent infrastructure is implemented with the virtual environment’s declarative model.     

Peltier Haptic Interface (PHI) for improved sensation of touch in virtual environments

In this report we describe an advanced virtual reality glove that we are developing, called the Peltier Haptic Interface (PHI), which will provide improved sensation of touch in virtual environments. PHI will provide force/pressure feedback that can be varied independently on each finger, as well as temperature sensation that can be varied non-uniformly over the whole hand. The combination of these sensations will provide a more realistic sense of touch and significantly increase the realism of virtual environments. PHI will find extensive applications in biomedical simulations, teaching, industrial line training, and many other areas.     

A distributed platform for personalized advertising in digital interactive TV environments

Advertising plays an important role in modern free markets. Furthermore, advertising is moving towards the establishment of one-to-one marketing relationships. Thus, personalized advertisement is currently considered as a hot topic in product promotion as it can be proved beneficial for all the key players, such as the advertisers, the advertised companies, as well as the consumers. Interactive TV and WWW can provide the means for personalized advertising. But of course, special systems and platforms for personalization must be first developed. This paper proposes a prototype system which efficiently achieves the personalization of the advertisements in the environment of digital interactive TV. Thus, the environment for the exploitation of the proposed system are examined, the details in design and implementation are given, while extensive operation testing and evaluation are provided proving its high applicability in real business environments.     

A hyper elasticity method for interactive virtual design of hearing aids

We present a computational efficient method for isotropic hyper elasticity based on functional analysis. By selecting a class of shape functions, we arrive at a computational scheme which yields very sparse tensors. This enables fast computations of the hyper elastic energy potential and its derivatives. We achieve efficiency and performance through the use of shape functions that are linear in their parameters and through rotation into the eigenspace of the right Cauchy–Green strain tensor. This makes near real time evaluation of hyper elasticity of complex meshes on CPU relatively easy to implement. The approach does not rely on a specific shape function or material model but offers a general framework for isotropic hyper elasticity. The method is aimed at interactive and accurate non-linear hyper elastic modeling for a wide range of industrial virtual design applications, which we exemplify by insertion of hearing aid domes into the ear canal. We validate the method for tetrahedral meshes with linear shape functions with an Ogden material model by comparing simulations to deformations of real material. We illustrate the use of other shape functions and models using uniform cubic B-splines in combination with Riemannian elasticity.     

The roles of sensory modalities in collaborative virtual environments (CVEs)

This study was conducted to assess the effects of sensorial modalities on user performance, perception, and behavior in collaborative virtual environments (CVEs). Participants played a CVE game, air hockey, together with a remote partner under different sensory modality conditions, depending on the type of sensory feedback provided: visual-only (V), visual–haptic (V + H), and visual–haptic–audio feedback (V + H + A). Three types of measurements were used as dependent variables: (1) task performance measured as playing time, (2) user perception including the sense of presence, the sense of togetherness, and perceived collaboration, and (3) behavior measurement including the amount of force applied and the mallet deviation. Results of the study indicated that the task performance, perception, and user behavior in CVEs can be affected due to supported sensory modalities. Therefore, the multiple sensory information types that are required to perform the task at hand should be provided to effectively support collaboration between people in CVEs. The outcomes of this research should have a broad impact on multimodal user interaction, including research on physiological, psychophysical, and psychological mechanisms underlying human perception on multisensory feedback in CVEs.     

A multimodal approach for 3D face modeling and recognition using 3D deformable facial mask

We present a multimodal approach for face modeling and recognition. The algorithm uses three cameras to capture stereo images, two frontal and one profile, of the face. 2D facial features are extracted from one of the frontal images and a dense disparity map is computed from the two frontal images. Using the extracted 2D features and their corresponding disparities, we compute their 3D coordinates. We next align a low resolution 3D mesh model to the 3D features, re-project its vertices onto the frontal 2D image and adjust its profile silhouette vertices using the profile view image. We increase the resolution of the resulting 2D model at its center region to obtain a facial mask model covering distinctive features of the face. The 2D coordinates of the vertices, along with their disparities, result in a deformed 3D mask model specific to a given subject’s face. Our method integrates information from the extracted facial features from the 2D image modality with information from the 3D modality obtained from the stereo images. Application of the models in 3D face recognition, for 112 subjects, validates the algorithm with a 95% identification rate and 92% verification rate at 0.1% false acceptance rate.

A virtual environment simulator for mechanical system dynamics with online interactive control

Virtual prototyping is an effective tool in the development of mechanical product. Physically accurate simulation of multi-body mechanical system enables designers to investigate, explore, and experience the performance and behavior of an evolving product and thus reduce the number of physical prototypes needed. For the sake of better support for designers to manipulate the simulation, we have developed a dynamic simulation package of mechanical system, which provides an interactive control during the simulation process. The package incorporates physical behaviors and dynamic interactions by dynamics based modeling approach in multi-body mechanical system. In the package, user’s actions (e.g., loading a model, picking and dragging objects, steering objects during simulation process, etc.) are based on an ATN task management, and a multi-modal interface is provided which supports 2D desktop devices and 3D VR devices. The main contribution of the simulator is providing supports that allow users’ interactive manipulation in the simulation loop. During the simulation process, users can modify the constraints between components, apply force/torque to interested components and change the parameters of forces/torques. The simulator automatically updates the dynamic model of the mechanical system in real time, and then continuously simulates the behavior of the model under the current condition in the loop. An example of dynamic simulation of a vehicle is implemented and the simulation result is compared to that of ADAMS to verify the correctness and accuracy of the simulator. With the real-time interaction, solution and visualization of simulation model, the package affords better support for designers to participate in the simulation interactively and effectively.     

A model of (en)action to approach embodiment: a cornerstone for the design of virtual environments for learning

This paper presents a model of (en)action from a conceptual and theoretical point of view. This model is used to provide solid bases to overcome the complexity of designing virtual environments for learning (VEL). It provides a common grounding for trans-disciplinary collaborations where embodiment can be perceived as the cornerstone of the project. Where virtual environments are concerned, both computer scientists and educationalists have to deal with the learner/user’s body; therefore the model provides tools with which to approach both human actions and learning processes within a threefold model. It is mainly based on neuroscientific research, including enaction and the neurophysiology of action.     

A multi-resolution statistical deformable model (MISTO) for soft-tissue organ reconstruction

While active shape model (ASM) has been increasingly adopted in the medical domain, there are issues that need to be addressed for it to be applicable in practice. Among them, the small sample size problem and how to represent the variation of the clutter of surroundings are two of the challenges. In this paper, to overcome these problems, we propose a novel multi-resolution statistical deformable model and the associated techniques for the reconstruction of soft-tissue organs such as livers. To address the small sample size problem, we define a multi-resolution integrated model for soft-tissue organs called MISTO that is able to capture the most significant deformations from a small training set as well as to generate representative variation modes of the organ shapes. To deal with the complex surroundings of the model surface or landmark points in the underlying medical images during model deformation, we propose to apply multi-resolution appearance models which allows the surrounding visual context of the model surface points to be learnt and characterized automatically from the training samples. By combining the powerful shape models and the resulting context constraints, the object segmentation and reconstruction process can be carried out very robustly. Furthermore, to avoid the local minima during model optimization, we develop an adaptive deformation strategy such that the more stable parts of the surface are moved prior to the rest of the model surface. The experimental and validation results verify that our proposed approaches can be successfully and robustly applied to the reconstruction of the soft-tissue organs such as the human liver. The major contributions of our approaches are that we extend the traditional ASM to address open problems associated with reconstructing significantly deformable three-dimensional anatomies in cluttered surrounding, and we propose effective ways to formulate the perceptual knowledge of the anatomies and make use of it in the process of model construction and deformation for medical reconstruction.     

A domain-oriented end-user design environment for generating interactive 3D virtual chemistry experiments

Three-dimensional virtual worlds are potentially feasible for building virtual educational environments. However, educators face technical challenges to apply these technologies because creating virtual educational environments based on virtual worlds demands 3D modeling and programming skills. This paper proposes a method to lower the technical barriers through domain-oriented interfaces with languages and environments that are familiar to educators. A domain-oriented end-user design environment, iVirtualWorld, is designed and developed to implement the proposed method in a specific educational domain, namely introductory chemistry experiments. This web-based environment provides end-users with domain-oriented building blocks, which can be assembled to create 3D virtual chemistry experiments. A usability evaluation and a comparative case study are designed to evaluate the system among chemistry educators, who are the target audience. The usability evaluation contains a task requiring participants to create a 3D virtual chemistry experiment and a voluntary semi-structured interview. The case study compares a virtual experiment generated using iVirtualWorld with an experiment in a commercial virtual chemistry laboratory system. The results show that 1) the domain-oriented end-user design environment enables participants to generate the 3D virtual chemistry experiment within 30 min; 2) participants gain confidence on creating 3D virtual experiments by themselves using iVirtualWorld; 3) participants confirm the usefulness of applying the system in introductory chemistry education; and 4) iVirtualWorld is considered more intuitive and straightforward for students to focus on finishing the experiment without being distracted than the commercial virtual chemistry laboratory system. Areas that can benefit from the system most and areas where the system is less effective are identified by participants. The responses also reveal the limitations of the current system and suggest directions for future improvement.