Cost-effective haptic-based networked virtual environments with high-resolution tiled display

In this paper, a haptic-based NVE (networked virtual environment) supporting high-resolution tiled display is proposed with a resource management scheme for future home applications such as a haptic-based networked game. Although NVEs with haptic interaction and immersive display have been developed in previous studies, they usually require expensive system configurations. In order to implement a cost-effective system, our proposed design includes a tiled display supporting high-resolution with several low-cost LCDs. The display nodes of the tiled display also have low processing and storage resource requirements because they only need to handle display-ready pixels received from the application node. The proposed resource management scheme, called as an object-based display scheme, reduces the resource requirements of the application node that would otherwise require significant resources to manage all the display nodes and to provide a user with haptic feedback. The proposed scheme segments graphic scenes into several pixelated virtual objects and assigns different frame rates to each object according to the interest of user. Experimental results confirm that the proposed scheme reduces the required network bandwidth and appropriately assigns the limited processing resources to graphic and haptic renderings for a high-level visual and haptic quality.     

Prediction-based methods for teleoperation across delayed networks

The remote nature of telepresence scenarios can be seen as a strongpoint and also as a weakness. Although it enables the remote control of robots in dangerous or inaccessible environments, it necessarily involves some kind of communication mechanism for the transmission of control signals. This communication mechanism necessarily involves adverse network effects such as delay. Three mechanisms aimed at improving the effects of network delay are presented in this paper: (1) Motion prediction to partially compensate for network delays, (2) force prediction to learn a local force model, thereby reducing dependency on delayed force signals, and (3) haptic data compression to reduce the required bandwidth of high frequency data. The utilized motion prediction scheme was shown to improve operator performance, but had no influence on operator immersion. The force prediction provided haptic feedback through synchronous forces from the local model, thereby stabilizing the control loop. The developed haptic data compression scheme reduced the number of packets sent across the network by 90%, while improving the quality of the haptic feedback.     

Efficient triangulation for P2P networked virtual environments

Peer-to-peer (P2P) architectures have recently become a popular design choice for building scalable Networked Virtual Environments (NVEs). In P2P-based NVEs, system and data management is distributed among all participating users. Towards this end, a Delaunay Triangulation can be used to provide connectivity between the different NVE users depending on their positions in the virtual world. However, a Delaunay Triangulation clearly suffers from high maintenance cost as it is subject to high connection change rate due to continuous users’ movement. In this paper, we propose a new triangulation algorithm that provides network connectivity to support P2P NVEs while dramatically decreasing maintenance overhead by reducing the number of connection changes due to users’ insertion and movement. Performance evaluations show that our solution drastically reduces overlay maintenance cost in highly dynamic NVEs. More importantly, and beyond its quantitative advantages, this work questions the well accepted Delaunay Triangulation as a reference means for providing connectivity in NVEs, and paves the way for more research towards more practical alternatives for NVE applications.     

Priority-based haptic event filtering for transmission and error control in networked virtual environments

A data filtering scheme is proposed for transmission and error control of haptic events in haptic-based network virtual environments; this scheme is called as priority-based haptic event filtering. Because a high update rate of approximately 1 kHz is required for haptic rendering, sophisticated transmission rate control and reduction schemes are necessary for the haptic events. Although existing schemes can reduce the transmission rate without any perception impairment, they are very sensitive to packet losses. In this paper, we prioritize the haptic events according to the delay and loss effects. Utilizing the proposed haptic event prioritization, the proposed filtering scheme adapts the transmission rate and updates the predicted loss rate according to the current network state. Our simulation and experiment results confirm that the proposed scheme can effectively select important haptic events and guarantee an improved haptic interaction quality over a bandwidth-limited lossy network than existing transmission schemes tailored for networked haptics.     

Experimental evaluation of haptic data communication with prediction

The transmission of haptic data is relatively challenging in multimedia communication. In this research study, the methods are presented for exploiting the properties of human haptic perception for data reduction of haptic data transmission. Packet-switched communication of haptic data is characterized by high packet rates on the communication channel. The quality of the internet-based haptic tele-control/tele-presence systems is highly dependent on the quality of the communication channel between the operator and the remote site, and on the delay jitter in the data exchange. The proposed research work is evaluated experimentally using a Geomagic Touch (previously PHANTOM Sensable Omni) haptic device with a sphere as a virtual model. Four experiments were conducted to evaluate the proposed research study. In the first experiment the JND Weber’s law is applied on sent force values only, while in the second experiment, the force calculation algorithm has been modified to include human movement velocity. The third experiment discusses the use of JND on the sent velocity values. The evaluation of the human’s perception shows that the proposed modification to the basic dead-band approach highly reduces the number of sent packets with minimal disturbance in haptic feeling. Further enhancements using prediction techniques have also been introduced in the fourth experimental evaluation. The linear predictions are added to the above proposed reduction methods. Combining the dead-band approach with a fast, configurable and accurate prediction algorithm enables a significant reduction in the amount of data sent across the network. The reduction is estimated to be 85%, while preserving the original data structure.     

Hybrid client-server architecture and control techniques for collaborative product development using haptic interfaces

In this paper, a collaborative product development and prototyping framework is proposed by using distributed haptic interfaces along with deformable objects modeling. Collaborative Virtual Environment (CVE) is a promising technique for industrial product development and virtual prototyping. Network control problems such as network traffic and network delay in communication have greatly limited collaborative virtual environment applications. The problems become more difficult when high-update-rate haptic interfaces and computation intensive deformable objects modeling are integrated into CVEs for intuitive manipulation and enhanced realism. A hybrid network architecture is proposed to balance the computational burden of haptic rendering and deformable object simulation. Adaptive artificial time compensation is used to reduce the time discrepancy between the server and the client. Interpolation and extrapolation approaches are used to synchronize graphic and haptic data transmitted over the network. The proposed techniques can be used for collaborative product development, virtual assembly, remote product simulation and other collaborative virtual environments where both haptic interfaces and deformable object models are involved.     

Adaptive Control for State Synchronization of Nonlinear Haptic Telerobotic Systems with Asymmetric Varying Time Delays

In this paper, we introduce a new adaptive controller design scheme for nonlinear telerobotic systems with varying time delays where the delays and their variation rates are unknown. The designed controller has the ability to synchronize the state behaviors of the local and the remote robots. In this paper, asymptotic stability in the presence of varying time delays is of interest. Using the proposed controller, asymptotic stability of the bilateral telerobotic system subject to any bounded yet unknown varying delay with a bounded yet unknown rate of change can be guaranteed. Besides the varying time delay, the proposed adaptive controller has the ability to adapt to the parameter variations in the local and the remote robots’ dynamics. It is shown that position and velocity errors between the local and the remote manipulators converge to the zero asymptotically, thus ensuring teleoperation transparency. Experimental and simulation results with a pair of PHANToM haptic devices and a pair of planar manipulators under varying time delays in the communication channel demonstrate the effectiveness of the proposed scheme.     

Neural and fuzzy computation techniques for playout delay adaptation in VoIP networks

Playout delay adaptation algorithms are often used in real time voice communication over packet-switched networks to counteract the effects of network jitter at the receiver. Whilst the conventional algorithms developed for silence-suppressed speech transmission focused on preserving the relative temporal structure of speech frames/packets within a talkspurt (intertalkspurt adaptation), more recently developed algorithms strive to achieve better quality by allowing for playout delay adaptation within a talkspurt (intratalkspurt adaptation). The adaptation algorithms, both intertalkspurt and intratalkspurt based, rely on short term estimations of the characteristics of network delay that would be experienced by up-coming voice packets. The use of novel neural networks and fuzzy systems as estimators of network delay characteristics are presented in this paper. Their performance is analyzed in comparison with a number of traditional techniques for both inter and intratalkspurt adaptation paradigms. The design of a novel fuzzy trend analyzer system (FTAS) for network delay trend analysis and its usage in intratalkspurt playout delay adaptation are presented in greater detail. The performance of the proposed mechanism is analyzed based on measured Internet delays.     

Adaptive predictive functional control for networked control systems with random delays

Nowadays, more and more field devices are connected to the central controller through a serial communication network such as fieldbus or industrial Ethernet. Some of these serial communication networks like controller area network (CAN) or industrial Ethernet will introduce random transfer delays into the networked control systems (NCS), which causes control performance degradation and even system instability. To address this problem, the adaptive predictive functional control algorithm is derived by applying the concept of predictive functional control to a discrete state space model with variable delay. The method of estimating the network-induced delay is also proposed to facilitate the control algorithm implementing. Then, an NCS simulation research based on TrueTime simulator is carried out to validate the proposed control algorithm. The numerical simulations show that the proposed adaptive predictive functional control algorithm is effective for NCS with random delays.     

Finding the Latent Semantics of Haptic Interaction Research: A Systematic Literature Review of Haptic Interaction Using Content Analysis and Network Analysis

As interest in multimodal and tangible interfaces is increasing in the field of human–robot interaction and virtual reality, haptics has been researched across areas such as engineering, computer science, psychology, and neuroscience. The main objective of the study was to construct a comprehensive review of the current haptic‐related literature based on quantitative data derived from content analysis and network analysis. Using the results of content analysis and network analysis of 6,000 research articles on haptic interaction, the haptic‐related literature was classified into two categories: 1) studies on technologies providing haptic stimuli and 2) studies on the human perception of haptic stimuli. Emotions in haptic feedback and haptic perception characteristics of various body sites were identified as potential research topics for further investigation. Greater research effort on understanding human haptic sensation and perception using the proposed systematic approach could accelerate the development of haptic interaction technology.     

具有通信时延的AQM控制算法的稳定性

具有通信时延的AQM控制算法是一个复杂的动态非线性反馈系统．为了调查通信时延对Internet服务质量的影响,借助广义Nyquist判据研究了具有通信时延的AQM策略的稳定性．通过对网络系统的传递函数的分析,得到了各通信回路时延相同和时延不同条件下网络系统在平衡点的稳定性判据．这些结论表明,影响Internet性能的关键因素之一的通信时延在Internet通信过程中起着重要的作用．最后仿真验证了该判据的有效性．     

Stability analysis of model-based networked distributed control systems

Networked distributed control systems (NDCSs) face serious challenges such as delays and packet dropouts induced by the communication network employed to connect local controllers of interacting subsystems. These two network-induced shortcomings may degrade the performance or even destabilize NDCSs. This paper is concerned with the problem of stability analysis and stabilization of the NDCSs, featuring both random delay and random packet loss in their communication networks. A model-based networked distributed control framework is proposed to stabilize the NDCS consisting of discrete-time subsystems interconnected through their states. In this control framework, to compensate for the adverse effects of these two network-induced shortcomings, an interaction estimator is provided in each local controller; in addition to a main control unit. This estimator uses the explicit model of the subsystems to estimate the evolution of the states of interacting subsystems, when information about their actual values is not available. A model for the NDCS subject to both random packet loss and random delay is developed. By providing a 3-step interaction estimating algorithm, the closed-loop model-based networked distributed control system (MB-NDCS) is formulated as a time-dependent impulsive system. Then, a quadratic Lyapunov function is constructed to derive a linear matrix inequality (LMI) based sufficient condition for stability analysis of the overall impulsive system. Finally, an illustrative example of a network of interconnected chemical reactors with recycle is presented to show the effectiveness of the proposed approach.     

含时延约束的多智能体系统二分一致性

针对多智能体系统中信息交互存在通信时延这一约束,在无向符号图拓扑结构下分别研究了含固定时延和时变时延的一阶多智能体系统二分一致性问题。通过设计相应的控制协议,使得该系统收敛到两个模值相同但符号不同的状态。在稳定性分析中,利用广义Nyquist准则的方法,得到含固定时延多智能体系统实现二分一致性的充分条件;对含时变时延系统构造包含三重积分项的Lyapunov函数,利用积分不等式和线性矩阵不等式理论,并结合自由矩阵的方法得到含时变时延多智能体系统实现二分一致性的充分条件。最后,数值仿真验证了所得结论的有效性和正确性。     

Guidelines for haptic interpersonal communication applications: an exploration of foot interaction styles

A new method for researching haptic interaction styles is presented, based on a layered interaction model and a classification of existing devices. The method is illustrated by designing a new foot interaction device. The aim of which is to enhance non-verbal communication over a computer network. A layered protocols interaction model allows to consider all aspects of the haptic communication process: the intention to perform an action, limitations of the human body, and specifications of the communication device and the network. We demonstrate how this model can be used to derive design-guidelines by analyzing and classifying existing communication devices. By designing and evaluating a foot interaction device, we not only demonstrate that feet are suited for personal, concealed communication over a network, but also show the added value of the design-guidelines. Results of user tests provide clues for designing stimuli for foot interaction and indicate applications of foot communication devices.     

Digital neural emulators using tree accumulation and communicationstructures

Three digital artificial neural network processors suitable for the emulation of fully interconnected neural networks are proposed. The processors use N(2) multipliers and an arrangement of tree structures that provide the communication and accumulation function either individually or in a combined manner using communicating adder trees. The performance for the emulation of an N-neuron network for all processors is achieved in 2log(2)N+C time units, where C is a constant equal to the multiplication, neuron activation, and internal fixed delays. The feasibility and characteristics of the proposed configurations to emulate single and/or multiple neural networks simultaneously are discussed, and a comparison with recently proposed neurocomputer architectures is reported.     

Nonlinear trilateral teleoperation stability analysis subjected to time-varying delays

A trilateral teleoperation system facilitates the collaboration of two users to share control of a single robot in a remote environment. While various applications of shared-control trilateral haptic teleoperation systems have recently emerged, they have mostly been studied in the context of single-DOF, LTI robotic systems. On the other hand, robotic manipulators with multiple degrees of freedom (DOF) and therefore nonlinear dynamics have recently found many applications such as in robotic-assisted surgery and therapy, space exploration and navigation systems. In this paper, considering the full nonlinear dynamical models of multi-DOF robots, stability analysis of a dual-user haptic teleoperation system is considered over a communication network subjected to asymmetrical time varying delays and through a dominance factor suitable for trainer–trainee applications. Stability in free motion and contact motion and asymptotic position tracking of the trilateral haptic teleoperation system in free motion are proven via Lyapunov stability analysis and Barbalat's lemma where operators and the environment are assumed to be passive. Simulation and experimental results concerning robot position tracking and user-perceived forces for three 2-DOF robots and experimental analysis of user-perceived stiffnesses for three 3-DOF robots validate the theoretical findings pertaining to the system stability and demonstrate the efficiency of the proposed controller.     

Slingshot 3D: A synchronous haptic-audio-video game

Recent advances in multimedia and human computer interaction technologies have paved the way for rich contents across multiple media such as haptics, audio, and video. This paper introduces a multimodal game named the Slingshot 3D Game: an interactive and synchronous haptic-audio-video shooter game over the Internet network. The game incorporates two types of haptic feedback: tactile feedback using a haptic jacket and kinesthetic feedback using the Novint Falcon haptic interface. Furthermore, the game utilizes a depth camera to track the player's (upper) body movements and detect collisions between the player's body and the shot projectile. To promote availability and cost, the game uses the Internet network as the communication medium between the players, by utilizing the Admux communication framework [10]. The game design and implementation are detailed in this paper. Both the player performance analysis and the user satisfaction analysis have shown that the incorporation of synchronous haptic-video multimedia has enhanced the perception of player presence and the overall quality of performance.     

HE-MAN: Hierarchical Management for Vehicular Delay-Tolerant Networks

Vehicular Ad Hoc Networks (VANETs) are mobile networks that extend over vast areas and have intense node mobility. These characteristics lead to frequent delays and disruptions. A solution is to employ the Delay Tolerant Network (DTN) paradigm. However, the frequent disruptions as well as the delay and reliability constraints of certain VANET applications hinder the employment of both conventional and DTN-based management architectures. We present the HiErarchical MANagement (HE-MAN) architecture, which considers the specificities of Vehicular Delay-Tolerant Network (VDTN) management. The HE-MAN architecture implements a hierarchical management topology in order to take advantage of local communication opportunities for monitoring and configuration tasks. The proposed techniques for network clustering, monitoring, and configuration are evaluated using simulations, and results show that the proposed architecture successfully organizes the VDTN in relatively stable clusters, leading to more intelligent and efficient management of VDTN nodes through the usage of middle-level managers.     

多媒体实时播放系统时延特性的研究

随着计算机技术和通信技术的发展，多媒体数据网络实时播放成为一项新的技术研究热点。本文通过对实际系统的分析和测试。指出了多媒体实时播放系统时延主要由压缩卡时延，网络时延、软件解压时延构成，叙述了它们的来由，机制和特点，并通过设计一种新系统证明了网络时延时其中最主要的原因，同时提出了一些减少时延的基本方法。     

Accelerating average consensus by using the information of second-order neighbours with communication delays

This article investigates the problem of accelerating average consensus in undirected and connected networks. The protocol using the information of second-order neighbours with communication delays is proposed and the delay effects on stability and the convergence speed are analysed, respectively, under an assumption about the network topologies. It is proved that, for appropriate communication delays, networks reach average consensus faster under the proposed protocol than the standard protocol using only the information of first-order neighbours. Finally, a simulation example is presented to illustrate the proposed results.