Usability Evaluation of VR Interface for Mobile Robot Teleoperation

This article presents the results from research in which 3 different remote control interfaces were compared to assess the impact of interface structure on the performance of the operator for remotely controlled mobile inspection robots. The primary control interface of a mobile robot consists of a head-mounted display, data gloves for gripper control, joystick for movement control of the robot platform, and a motion tracking system for measuring head orientation and hand position. In order to compare different control interfaces, an additional system, based on a Liquid Crystal Display monitor and joystick, was prepared. Results of this study show that the use of virtual reality techniques in the interfaces of mobile inspection robots increases operator productivity, the level of spatial presence, and distance evaluation while facilitating the execution of tasks, as well as improving and speeding up their execution and reducing the operator’s time needed to adapt to the control interface. The latter is achieved with the increased level of intuitive control while ensuring comfort.     

Online turnover‐free control for a mobile agent with a terrain prediction sensor

This paper proposes a turnover‐free control method for a teleoperated mobile agent (or vehicle) moving through uneven terrain. The teleoperated agent is primarily driven by an operator at a remote site and is able to react autonomously when a possible turnover is predicted. In order to predict the turnover, a low‐cost terrain prediction sensor has been developed using a camera vision with a structured laser light. Since it is difficult for an operator to predict the reactive motion of the agent, a force reflection technique with a force feedback joystick is employed to intuitively recognize the inconsistency between the intended motion and the reactive motion of the agent. Finally, to verify the feasibility and effectiveness of the proposed method, experiments with the ROBHAZ‐DT (actual mobile agent) have been carried out. In the experiments, the operator could recognize the reactive motion of the agent for turnover prevention through force reflection while the agent was moving on slopped terrain. © 2006 Wiley Periodicals, Inc.     

A Dynamic Gesture Language and Graphical Feedback for Interaction in a 3D User Interface

In user interfaces of modern systems, users get the impression of directly interacting with application objects. In 3D based user interfaces, novel input devices, like hand and force input devices, are being introduced. They aim at providing natural ways of interaction. The use of a hand input device allows the recognition of static poses and dynamic gestures performed by a user's hand. This paper describes the use of a hand input device for interacting with a 3D graphical application. A dynamic gesture language, which allows users to teach some hand gestures, is presented. Furthermore, a user interface integrating the recognition of these gestures and providing feedback for them, is introduced. Particular attention has been spent on implementing a tool for easy specification of dynamic gestures, and on strategies for providing graphical feedback to users' interactions. To demonstrate that the introduced 3D user interface features, and the way the system presents graphical feedback, are not restricted to a hand input device, a force input device has also been integrated into the user interface.     

Off-road machine controls: investigating the risk of carpal tunnel syndrome

Occupationally induced hand and wrist repetitive strain injuries (RSI) such as carpal tunnel syndrome (CTS) are a growing problem in North America. The purpose of this investigation was to apply a modification of the wrist flexion/ extension models of Armstrong and Chaffin (1978, 1979) to determine if joystick controller use in off-road machines could contribute to the development of CTS. A construction equipment cab in the laboratory was instrumented to allow force, displacement and angle measurements from 10 operators while they completed an approximately 30-min joystick motion protocol. The investigation revealed that both the external fingertip and predicted internal wrist forces resulting from the use of these joysticks were very low, indicating that the CTS risk associated with this factor was slight. However, the results also indicated that, particularly for the 'forward' and 'left' right side motions and for all left side motions, force was exerted by other portions of the fingers and hand, thereby under-predicting the tendon tension and internal wrist forces. Wrist angles observed were highest for motions that moved the joysticks to the sides rather than front to back. Thus, the 'right' and 'left' motions for both hands posed a higher risk for CTS development. When the right hand moved into the 'right' position and the left hand moved into the 'left' position, the wrist went into extension in both cases. Results indicate that neither learning nor fatigue affected the results.     

Off-road machine controls: investigating the risk of carpal tunnel syndrome

Occupationally induced hand and wrist repetitive strain injuries (RSI) such as carpal tunnel syndrome (CTS) are a growing problem in North America. The purpose of this investigation was to apply a modification of the wrist flexion/extension models of Armstrong and Chaffin (1978, 1979) to determine if joystick controller use in oV-road machines could contribute to the development of CTS. A construction equipment cab in the laboratory was instrumented to allow force, displacement and angle measurements from 10 operators while they completed an ? 30-min joystick motion protocol. The investigation revealed that both the external fingertip and predicted internal wrist forces resulting from the use of these joysticks were very low, indicating that the CTS risk associated with this factor was slight. However, the results also indicated that, particularly for the ‘forward’ and ‘left’ right side motions and for all left side motions, force was exerted by other portions of the fingers and hand, thereby under-predicting the tendon tension and internal wrist forces. Wrist angles observed were highest for motions that moved the joysticks to the sides rather than front to back. Thus, the ‘right’ and ‘left’ motions for both hands posed a higher risk for CTS development. When the right hand moved into the ‘right’ position and the left hand moved into the ‘left’ position, the wrist went into extension in both cases. Results indicate that neither learning nor fatigue aVected the results.     

基于虚拟仪器技术的六分力测试系统设计

为解决目前犁体外载的测定装置普遍存在数据采集精度不高和数据记录方法落后等问题,利用虚拟仪器技术开发六分力测试系统,实现了犁体外载的多参数实时数据采集、处理、显示、保存及控制等功能。基于LabVIEW图形化编程软件,研究并开发了数据采集处理系统,设计测试与控制过程的虚拟前面板并得出数据分析结果。虚拟仪器在犁体外载六分力测试上的开发应用,有效提高了试验过程的自动化和智能化水平及测试数据效率和精度。     

Physical demands of overhead crane operation

Recent studies have suggested that ergonomic factors may contribute to risks experienced by overhead crane operators. However, there are few studies that provide a comprehensive overview of the physical demands of overhead crane operation. This study aimed to provide this information by quantifying muscular, postural, and upper limb movement demands of overhead crane operation including examination of muscle activation and trunk posture by task. Trunk posture, upper limb movement demands and muscle activation in the trunk and upper limbs were quantified for seven overhead crane operators. Trunk posture was quantified using trunk angle and joystick motion requirements were determined using camera data. Muscle activation was measured bilaterally using surface EMG for the upper trapezii, anterior deltoids, posterior deltoids, biceps brachii, triceps brachii, flexor carpi radialis and erector spinae. Lastly, joystick force requirements were assessed using a spring scale. High upper limb and trunk muscle loading were observed when compared to joystick use in other heavy machinery, in part due to the forward, trunk-flexed position required to adequately view the workspace, and the increased force requirements of the joysticks. Joystick input force requirements were 9–31 N for the right-hand joystick and 11–40 N for the left-hand joystick. Operators maintained a forward trunk flexion (>20°) for all subtasks which suggests that trunk posture might play a role in sustained trunk muscle activation. Results suggest that the primary issue with overhead crane cab operation is upper limb and trunk muscle loading. Results confirm the need to investigate muscle load reduction strategies such as camera systems to help reduce the need for trunk flexion. Other design modification suggestions include reducing the joystick input force and displacement requirements coupled with potentially distributing the machine functions more evenly across the right and left controllers.     

Neuro-fuzzy and model-based motion control for mobile manipulator among dynamic obstacles

This paper focuses on autonomous motion control of a nonholonomic platform with a robotic arm, which is called mobile manipulator. It serves in transportation of loads in imperfectly known industrial environments with unknown dynamic obstacles. A union of both procedures is used to solve the general problems of collision-free motion. The problem of collision-free motion for mobile manipulators has been approached from two directions, Planning and Reactive Control. The dynamic path planning can be used to solve the problem of locomotion of mobile platform, and reactive approaches can be employed to solve the motion planning of the arm. The execution can generate the commands for the servo-systems of the robot so as to follow a given nominal trajectory while reacting in real-time to unexpected events. The execution can be designed as an Adaptive Fuzzy Neural Controller. In real world systems, sensor-based motion control becomes essential to deal with model uncertainties and unexpected obstacles.     

面向人机工程学测试的虚拟手仿真模型

以人机工程学测试为目标提出虚拟手仿真模型,建立虚拟手运动学模型。提出一种曲线拟合的校准方法,以获取精确的数据,更好地控制虚拟手的运动。建立弹簧模型计算手与被抓取物体之间的受力,以视觉渲染的形式将其反馈给用户。实验结果表明,虚拟手可自然地抓取三维虚拟物体,视觉反馈的使用可免于购买昂贵的力反馈设备。     

An Approach to Platform Independent Real-Time Programming: (1) Formal Description

A method for extending programming languages with timing constructs is proposed. It enables the specification of timing constraints in programs. The approach is not language specific and the extension can be included in many existing programming languages. With the extension it is possible to write real-time program components that can be proven correct independently of the properties of the machine that is used for their execution. It therefore provides a similar abstraction from the execution platform as is normal in non-real-time programming. The presented approach distinguishes two phases in system construction: (1) a platform-independent programming phase that includes the expression of timing requirements, and (2) an implementation phase where all platform dependencies are addressed. The latter only differs from a normal compilation in the sense that a feasible schedule must be found to execute the program.     

MicroMAIS: executing and orchestrating Web services on constrained mobile devices

Mobile devices with their more and more powerful resources allow the development of mobile information systems in which services are not only provided by traditional systems but also autonomously executed and controlled in the mobile devices themselves. Services distributed on autonomous mobile devices allow both the development of cooperative applications without a back‐end infrastructure and the development of applications blending distributed and centralized services. In this paper, we propose MicroMAIS: an integrated platform for supporting the execution of Web service‐based applications natively on a mobile device. The MicroMAIS platform is composed of mAS and μ‐BPEL. The former allows the execution of a single Web service, whereas the latter permits the orchestration of several Web services according to the WS‐BPEL standard. Copyright © 2011 John Wiley & Sons, Ltd.     

Linguistic reflection in Java

Reflective systems allow their own structures to be altered from within. Here we are concerned with a style of reflection, called linguistic reflection, which is the ability of a running program to generate new program fragments and to integrate these into its own execution. In particular, we describe how this kind of reflection may be provided in the compiler-based, strongly typed object-oriented programming language Java. The advantages of the programming technique include attaining high levels of genericity and accommodating system evolution. These advantages are illustrated by an example taken from persistent programming, which shows how linguistic reflection allows functionality (program code) to be generated on demand (Just-In-Time) from a generic specification and integrated into the evolving running program. The technique is evaluated against alternative implementation approaches with respect to efficiency, safety and ease of use. © 1998 John Wiley & Sons, Ltd.     

Networked multiplayer cooperative interaction using decoupled motion control method in a shared virtual environment with haptic,visual and movement feedback

This paper focuses on multiplayer cooperative interaction in a shared haptic environment based on a local area network. Decoupled motion control, which allows one user to manipulate a haptic interface to control only one‐dimensional movement of an avatar, is presented as a new type haptic‐based cooperation among multiple users. Users respectively move an avatar along one coordinate axis so that the motion of the avatar is the synthesis of movements along all axes. It is different from previous haptic cooperation where all users can apply forces on an avatar along any direction to move it, the motion of which completely depends on the resultant force. A novel concept of movement feedback is put forward where one user can sense other users’ hand motions through his or her own haptic interface. The concept can also be explained wherein one person who is required to move a virtual object along only one axis can also feel the motions of the virtual object along other axes. Movement feedback, which is a feeling of motion, differs from force feedback, such as gravity, collision force and resistance. A spring‐damper force model is proposed for the computation of motion feedback to implement movement transmission among users through haptic devices. Experimental results validate that movement feedback is beneficial for performance enhancement of such kind of haptic‐based cooperation, and the effect of movement feedback in performance improvement is also evaluated by all subjects.Copyright © 2012 John Wiley & Sons, Ltd.     

基于RTW的空间机器人遥操作实时仿真系统

搭建了一套实时仿真系统以作为空间机器人遥操作的研究平台.系统以两台PC机分别模拟主从端的运行情况,主端PC连接通用力反馈游戏手柄作为主手输入设备,从端PC挂接嵌入式处理器(PC/104)用作空间机械手的实时仿真;主从端信号传输时延根据TCP/IP协议由软件模拟产生.利用Madab中的RTW/xPC Target工具箱将空间机械手模型编译为实时可执行代码,下载到嵌入式处理器(PC/104)实时执行,保证了手柄输入和空间机械手Simulink模型间的时钟同步.结合给定模型数据,完成了空问机械手实时位置跟踪实验,证明了系统搭建方案的可行性.系统对空间机器人遥操作的研究起到了很好的支撑作用.     

An instrumented glove for grasp specification invirtual-reality-based point-and-direct telerobotics

Hand posture and force, which define aspects of the way an object is grasped, are features of robotic manipulation. A means for specifying these grasping “flavors” has been developed that uses an instrumented glove equipped with joint and force sensors. The new grasp specification system will be used at the Pennsylvania State University (Penn State) in a Virtual Reality based Point-and-Direct (VR-PAD) robotics implementation. Here, an operator gives directives to a robot in the same natural way that human may direct another. Phrases such as “put that there” cause the robot to define a grasping strategy and motion strategy to complete the task on its own. In the VR-PAD concept, pointing is done using virtual tools such that an operator can appear to graphically grasp real items in live video. Rather than requiring full duplication of forces and kinesthetic movement throughout a task as is required in manual telemanipulation, hand posture and force are now specified only once. The grasp parameters then become object flavors. The robot maintains the specified force and hand posture flavors for an object throughout the task in handling the real workpiece or item of interest     

一种Stewart结构六维力／力矩传感器参数辨识研究

介绍了一种大量程Stewart结构六维力／力矩传感器系统．建立了传感器力／力矩测量数学模型,构建了基于运动学逆解的优化目标函数．提出了一种新的参数标定法,称作分支轮换标定法．基于所研发的六维力／力矩传感器系统进行了实验研究,实验结果表明分支轮换法可以有效地辨识出传感器的结构参数,提高了测量精度．该方法可用于类似结构的六维力／力矩传感器参数标定．     

Provision and maintenance of presence and immersion in hand‐held virtual reality through motion based interaction

Hand‐held devices are also becoming computationally more powerful and being equipped with special sensors and non‐traditional displays for diverse applications aside from just making phone calls. As such, it raises the question of whether realizing virtual reality, providing a minimum level of immersion and presence, might be possible on a hand‐held device capable of only relatively “small” display. In this paper, we propose that motion based interaction can widen the perceived field of view (FOV) more than the actual physical FOV, and in turn, increase the sense of presence and immersion up to a level comparable to that of a desktop or projection display based VR systems. We have implemented a prototype hand‐held VR platform and conducted two experiments to verify our hypothesis. Our experimental study has revealed that when a motion based interaction was used, the FOV perceived by the user for the small hand held device was significantly greater than (around 50%) the actual. Other larger display platforms using the conventional button or mouse/keyboard interface did not exhibit such a phenomenon. In addition, the level of user felt presence in the hand‐held platform was higher than or comparable to those in VR platforms with larger displays. We hypothesize that this phenomenon is related to and analogous to the way the human vision system compensates for differences in acuity resolution in the eye/retina through the saccadic activity. The paper demonstrates the distinct possibility of realizing reasonable virtual reality even with devices with a small visual FOV and limited processing power. Copyright © 2010 John Wiley & Sons, Ltd.     

A Study of Reactional Force Compensation Based on Three-Degree-of Freedom Parallel Platforms

Reaction compensation is necessary for space robot applications because reactional forces/torques will cause undesired movement of the spacecraft. Because the reactional torques can be compensated by the existing torque balancing device in the spacecraft, an additional reaction compensating device is necessary to compensate the reactional forces. In this article we study two types of reactional force compensating devices based on three-degree-of freedom, parallel platforms. The first type has three R-P-S legs while the second type has three R-R-P-S leg and a passive R-R-P leg. A three-degree of freedom serial manipulator is used to generate reactional forces, which are to be compensated by the paralle platforms. The kinematics and dynamics of both platforms are analyzed and closed form inverse kinematics solutions are derived. We then design a reactional force compensating device that satisfies the strict volume constraint in a spacecraft. The first type of parallel platform is found to require very long legs due to large orientational motion at certain positions. The second type has smooth motion in both position and orientation, and therefore its size can be very compact. It is concluded that the second type of parallel platform has great potential to be used as a compact reactional force compensating device. © 1995 John Wiley & Sons, Inc.