Creating interactive virtual auditory environments

We survey sound rendering techniques by comparing them to visual image rendering and describe several approaches for performing sound rendering in virtual auditory environments.     

Image-based interactive exploration of real-world environments

Interactive scene walkthroughs have long been an important computer graphics application area. More recently, researchers have developed techniques for constructing photorealistic 3D architectural models from real-world images. We present an image-based rendering system that brings us a step closer to a compelling sense of being there. Whereas many previous systems have used still photography and 3D scene modeling, we avoid explicit 3D reconstruction because it tends to be brittle. Our system is not the first to propose interactive video-based tours. We believe, however, that our system is the first to deliver fully interactive, photorealistic image-based tours on a personal computer at or above broadcast video resolutions and frame rates. Moreover, to our knowledge, no other tour provides the same rich set of interactions or visually complex environments.     

Knowledge-based interaction protocols for intelligent interactive environments

Intelligent interactive environments are systems whose main features are related to intelligence, interactivity and location. Intelligent interactive environments are implemented in a variety of platforms, such as distributed sensors, actuators and processors for ambient intelligence; distributed and mobile reasoning entities in virtual worlds and augmented reality; and distributed, multiuser information systems such as social networks and service-oriented systems directed to the dissemination of public services and retailing. In the present article, we introduce JamSession, a tool to specify and execute workflow choreographies, as well as to monitor and control heterogeneous resources, which can be specially useful for the specification and implementation of the coordination of real-time, multimodal and multimedia activities. JamSession is used to specify and execute knowledge-based interaction protocols, which are formally grounded and can be formally analysed and verified using a straightforward and user-friendly graphical language. A knowledge-based interaction protocol determines how disparate resources can work cooperatively to build complex behaviour in intelligent interactive environments.     

Socially aware robot navigation system in human interactive environments

In social environments, humans mostly stay in social interactive groups with their daily activities. A mobile service robot must be aware of not only human individuals but also social interactive groups, and then behave safely and socially (politely and, respectively) in human interactive environments. In this paper, we propose a social reactive control (SRC) that enables a mobile service robot to navigate safely and socially in the human interactive environments. The SRC is derived by incorporating both states of individuals (position, orientation, motion, and human field of view) and social interactive groups (group’s types, group’s centre, group’s radius, and group’s velocity) into the conventional social force model . The SRC can be combined with a conventional path planning technique to generate a socially aware robot navigation system that is capable of controlling mobile service robots to traverse with socially acceptable behaviours. We validate the effectiveness of the proposed social reactive control through a series of real-world experiments.     

QoS and resource management in distributed interactive multimedia environments

Quality of Service (QoS) is becoming an integral part of current ubiquitous Distributed Interactive Multimedia Environments (DIMEs) because of their high resource and real-time interactivity demands. It is highly influenced by the management techniques of available resources in these cyber-physical environments. We consider QoS and resource management influenced by two most important resources; the computing (CPU) and networking resources. In this paper, we survey existing DIME-relevant QoS and resource management techniques for these two resources, present their taxonomy, compare them, and show their impacts on DIMEs. Finally, we discuss appropriateness of those techniques in a sample DIME scenario.     

The design space of dynamic interactive virtual environments

Virtual environments have become a key component of many fields and the critical component of virtual reality applications. Due to their virtual nature, they can accommodate an infinite number of possibilities. A theoretical work is presented, which decomposes those innumerous possibilities into concepts to help clarify the vast design space and provide insights into future applied research. We propose that what makes environments interesting and engaging is having worlds that are both active and reactive. This article explores the manifestations of those actions and reactions in what we term: dynamic components and interactions. We term worlds containing these dynamic interactive virtual environments (DIVE). An analysis of each component time was performed, with the purpose of providing a theoretical understanding of the respective design spaces. Initially, we collected the myriad possibilities of each component, e.g., the possible kinds of interactions. We point to examples throughout the field to ground and explain concepts presented. We then categorized of each area into taxonomies. The result of the analyses provides insights into the design space of virtual environments, exposes several avenues of research that are yet underexplored, and provides better understandings of ways in which DIVE creation can be supported.     

Characterization of server performance bottlenecks in distributed interactive simulation environments

Servers running distributed simulation applications need to process a large number of small packets at the network level. This becomes a burden on the server CPUs and exhibits itself as high CPU utilization. In this paper, first, we confirm this phenomenon, then, we utilize performance analyzer tools from major processor vendors to characterize the major sources of CPU usage. In addition, the experiments run on real hardware reveal server bottlenecks within the context of new architectural features such as hyper-threading, hyper-transport and multi-core processors.     

A distributed platform for personalized advertising in digital interactive TV environments

Advertising plays an important role in modern free markets. Furthermore, advertising is moving towards the establishment of one-to-one marketing relationships. Thus, personalized advertisement is currently considered as a hot topic in product promotion as it can be proved beneficial for all the key players, such as the advertisers, the advertised companies, as well as the consumers. Interactive TV and WWW can provide the means for personalized advertising. But of course, special systems and platforms for personalization must be first developed. This paper proposes a prototype system which efficiently achieves the personalization of the advertisements in the environment of digital interactive TV. Thus, the environment for the exploitation of the proposed system are examined, the details in design and implementation are given, while extensive operation testing and evaluation are provided proving its high applicability in real business environments.     

Multiparty gaze preservation through perspective switching for interactive elearning environments

Multimedia Tools and Applications - Existing live tele-teaching systems enable eye-contact between interacting participants, however, they are often incomplete as they neglect finer levels of...     

Next generation session management for 3D teleimmersive interactive environments

In the recent past we have seen a boom of distributed interactive multimedia environments which use multiple correlated media sensors, multi-view displays, and advanced haptic-audio-visual user interfaces for teleimmersive gaming, business meetings and other collaborative activities. However, when we investigate the emerging teleimmersive environments closer, we realize that their overall session management, including end-to-end session setup, establishment and run-time management are not appropriate for the new demands that these environments present. These environments are cyber-physical rooms that demand (a) large scale of multi-sensory devices across geographically-distributed locations and (b) interaction with each other in synchronous and real-time manner. To deal with the new teleimmersive demands, we present a new session management design with (a) session initiation protocol(s) that understand media correlations, (b) view-based multi-stream topology establishment among multiple parties, (c) efficient, light-weight and distributed session monitoring with querying and debugging capabilities, (d) flexible view-based session adaptation with efficient topology adjustments, and (e) light-weighted and consistent session tear-down protocols. The presented design of the next generation session management protocols, services, algorithms and data structures is based on our extensive experiences with building 3D teleimmersive interactive systems, experimenting with high impact teleimmersive applications and deploying such environments at various venues.     

Towards perceptual fidelity: Slant perception in real and interactive virtual environments

An innovative motoric measure of slant based on gait is proposed as the angle between the foot and the walking surface during walking. This work investigates whether the proposed action-based measure is affected by factors such as material and inclination of the walking surface. Experimental studies were conducted in a real environment set-up and in its virtual simulation counterpart evaluating behavioural fidelity and user performance in ecologically-valid simulations. In the real environment, the measure slightly overestimated the inclined path whereas in the virtual environment it slightly underestimated the inclined path. The results imply that the proposed slant measure is modulated by motoric caution. Since the “reality” of the synthetic environment was relatively high, performance results should have revealed the same degree of caution as in the real world, however, that was not the case. People become more cautious when the ground plane was steep, slippery, or virtual.     

基于注意力感知和语义感知的RGB-D室内图像语义分割算法

近年来,全卷积神经网络有效提升了语义分割任务的准确率.然而,由于室内环境的复杂性,室内场景语义分割仍然是一个具有挑战性的问题.随着深度传感器的出现,人们开始考虑利用深度信息提升语义分割效果.以往的研究大多简单地使用等权值的拼接或求和操作来融合RGB特征和深度特征,未能充分利用RGB特征与深度特征之间的互补信息.本文提出...     

3D sound interactive environments for blind children problem solving skills

Audio-based virtual environments have been increasingly used to foster cognitive and learning skills. A number of studies have also highlighted that the use of technology can help learners to develop effective skills such as motivation and self-esteem. This study presents the design and usability of 3D interactive environments for children with visual disabilities to help them solve problems in Chilean geography and culture. We introduce AudioChile, a virtual environment that can be navigated through 3D sound to enhance spatiality and immersion throughout the environment. 3D sound is used to orientate, avoid obstacles, and identify the positions of various characters and objects within the environment. We have found during the usability evaluation that sound can be fundamental for attention and motivation purposes during interaction. Learners identified and clearly discriminated environmental sounds to solve everyday problems, spatial orientation, and laterality.