Automatic Viewpoint Planning in Teleoperation of a Mobile Robot

Camera viewpoint has significant impact on operators situation awareness in teleoperation. This paper presents a method for automatic optimal positioning of a single camera for a remotely navigated mobile robot in systems with a controllable camera platform. The proposed algorithm continuously adjusts the camera view of the workspace based on the task circumstances, allowing the operator to focus mainly on navigation and manipulation. The workspace and motion limits of the camera system and the location of the obstacles are taken into consideration in the camera view planning by formulating and solving a constrained optimization problem in real-time. A head tracking system enables the operator to use his/her head movements as an extra control input to guide the camera placement, if and when necessary. The proposed viewpoint control framework has been implemented and evaluated in a teleoperation experiment with a mobile robot. Results of a user study comparing this approach to two other common viewpoint control strategies are also reported.     

一种基于轨迹分布模式的网络机器人用户意图识别方法

在Internet移动机器人遥操作系统中,网络本身的不确定性是影响系统性能和用户操作效果的一个重要因素。提出了一种新的基于轨迹分布模式的方法,能对用户意图进行建模和识别,利用网络机器人的本身的自主性削弱网络不确定性给系统带来的不良影响。通过实验验证,发现这种方法能够较好地识别出用户的控制意图,并根据识别结果自主执行用户期望完成的任务,从而改善了用户的操作体验和机器人接受用户操作的表现,提高了遥操作系统的效率。     

遥自主移动机器人系统设计及实现

为了解决移动机器人在特定环境下自主性不强的问题,构建了自主移动机器人的遥操作控制系统。通过无线网络传输的通信方式实现了经过透视解算展开后的全景图像的传输和基于USB操纵杆的多功能远程控制平台的开发。详细介绍了机器人利用超声波传感器进行自主模糊避障的算法,利用USB操纵杆对机器人遥操作的程序实现,以及全景摄像头透视解算和视频压缩的方法。实验结果表明,构建的遥操作控制系统可以实现良好的人机交互,使移动机器人的自主性更强,更加智能化。     

A user study of command strategies for mobile robot teleoperation

This article presents a user study of mobile robot teleoperation. Performance of speed, position and combined command strategies in combination with text, visual and haptic feedback information were evaluated by experiments. Two experimental tasks were designed as follows: positioning of mobile robot and navigation in complex environment. Time for task completion and motion accuracy were measured and compared for different command strategies and types of feedback. Role of haptic, text and visual feedback information in combination with described command strategies is outlined.     

Remote navigation of a mobile robot in an RFID-augmented environment

In the current article, we address the problem of constructing radiofrequency identification (RFID)-augmented environments for mobile robots and the issues related to creating user interfaces for efficient remote navigation with a mobile robot in such environments. First, we describe an RFID-based positioning and obstacle identification solution for remotely controlled mobile robots in indoor environments. In the robot system, an architecture specifically developed by the authors for remotely controlled robotic systems was tested in practice. Second, using the developed system, three techniques for displaying information about the position and movements of a remote robot to the user were compared. The experimental visualization techniques displayed the position of the robot on an indoor floor plan augmented with (1) a video view from a camera attached to the robot, (2) display of nearby obstacles (identified using RFID technology) on the floor plan, and (3) both features. In the experiment, test subjects controlled the mobile robot through predetermined routes as quickly as possible avoiding collisions. The results suggest that the developed RFID-based environment and the remote control system can be used for efficient control of mobile robots. The results from the comparison of the visualization techniques showed that the technique without a camera view (2) was the fastest, and the number of steering motions made was smallest using this technique, but it also had the highest need for physical human interventions. The technique with both additional features (3) was subjectively preferred by the users. The similarities and differences between the current results and those found in the literature are discussed.     

新型嵌入式实时数字影像系统设计和实现

嵌入式数字影像系统广泛地应用于数码相机、数码摄像机、照相手机等消费电子产品中,而采用CMOS Sensor的嵌入式系统目前主要使用于摄像头、拍照手机、低档数码相机等低分辨率、非实时、专用系统中。该文提供了一种可使用2百万像素以上CMOS Sensor与ARM—Core的嵌入式实时数字影像系统设计与实现,这个设计可以应用于中、高档数码相机、数码摄像机和Smart phone等产品,也可以替代现有采用CCD Srnsor的产品,并具有图像质量好、低功耗、低成本、高软件通用性、电路设计简单等优点。     

移动远程视频监控系统

为了扩大单个摄像头的视频监控范围及灵活性, 设计了一种可远程操控的移动视频监控系统. 该系统由四个模块组成, 基于Arduino系统的智能车搭载有摄像头, 接收用户指令, 用于移动视频的采集; 嵌入式Linux系统通过V4L2接口实现对视频数据的实时采集, 一方面将数据通过网络发送至转发服务器, 另一方面将来自用户的控制指令转发至智能车; 服务器则用于转发视频至客户端以及转发用户控制指令至Linux系统; 基于Android的移动端呈现监控视频并提供用户控制界面. 与现有系统相比, 该系统可使用单摄像头实现无死角监控.     

A New Difficulty Index for Teleoperated Robots Driving through Obstacles

Teleoperation allows humans to reach environments that would otherwise be too difficult or dangerous. The distance between the human operator and remote robot introduces a number of issues that can negatively impact system performance including degraded and delayed information exchange between the robot and human. Some operation scenarios and environments can tolerate these degraded conditions, while others cannot. However, little work has been done to investigate how factors such as communication delay, automation, and environment characteristics interact to affect teleoperation system performance. This paper presents results from a user study analyzing the effects of teleoperation factors including communication delay, autonomous assistance, and environment layout on user performance. A mobile robot driving task is considered in which subjects drive a robot to a goal location around obstacles as quickly (minimize time) and safely (avoid collisions) as possible. An environment difficulty index (ID) is defined in the paper and is shown to be able to predict the average time it takes for the human to drive the robot to a goal location with different obstacle configurations. The ID is also shown to predict the path chosen by the human better than travel time along that path.     

基于Android的MJPEG网络摄像机设计

针对智能家居场景需求,设计了基于ARM9处理器Linux操作系统的网络摄像机。深入研究了MJPEG视频压缩算法,阐述了算法实现过程,并设计出基于Android操作系统的3G手机网络摄像机客户端软件。本系统通过实地安装与测试达到了预期效果,突破了传统网络摄像机客户端固定地域的限制,有良好的应用前景。     

Design and development of a semi‐autonomous agricultural vineyard sprayer: Human–robot interaction aspects

This article presents the design aspects and development processes to transform a general‐purpose mobile robotic platform into a semi‐autonomous agricultural robot sprayer focusing on user interfaces for teleoperation. The hardware and the software modules that must be installed onto the system are described, with particular emphasis on human–robot interaction. Details of the technology are given focusing on the user interface aspects. Two laboratory experiments and two studies in the field to evaluate the usability of the user interface provide evidence for the increased usability of a prototype robotic system. Specifically, the study aimed to empirically evaluate the type of target selection input device mouse and digital pen outperformed Wiimote in terms of usability. A field experiment evaluated the effect of three design factors: (a) type of screen output, (b) number of views, (c) type of robot control input device. Results showed that participants were significantly more effective but less efficient when they had multiple views, than when they had a single view. PC keyboard was also found to significantly outperform PS3 gamepad in terms of interaction efficiency and perceived usability. Heuristic evaluations of different user interfaces were also performed using research‐based HRI heuristics. Finally, a study on participants’ overall user experience found that the system was evaluated positively on the User Experience Questionnaire scales.     

Human-centered control scheme for delayed bilateral teleoperation of mobile robots

Teleoperation task performance strongly depends on how well the human operator’s commands are executed. In this paper, we propose a control scheme for delayed bilateral teleoperation of mobile robots that considers user’s commands execution in order to achieve a high-performance teleoperation system in some important aspects like time to complete the task, safety, and operator dependence. We describe some evaluation metrics that allow us to address these aspects and a quantitative metric is proposed and incorporated in the control scheme to compensate wrong commands. A force feedback is applied to the master at the local site as a haptic cue. In addition, the system stability is analyzed taking into consideration the master and remote robot dynamic models and the asymmetric time-varying delays of the communication channel. Multiple human-in-the-loop simulations were carried out and the results of the evaluation metrics were discussed. Additionally, we present experiments where a user teleoperates a mobile robot via the Internet connection between Argentina and Italy.     

Robot, asker of questions

Collaborative control is a teleoperation system model based on human–robot dialogue. With this model, the robot asks questions to the human in order to obtain assistance with cognition and perception. This enables the human to function as a resource for the robot and help to compensate for limitations of autonomy. To understand how collaborative control influences human–robot interaction, we performed a user study based on contextual inquiry (CI). The study revealed that: (1) dialogue helps users understand problems encountered by the robot and (2) human assistance is a limited resource that must be carefully managed.     

远程遥控移动机械手的可视化实时控制

采用OTG接口连接摇杆手柄的Android手机和无线视频传输方案,来实现对移动机械手的可视化实时控制。本设计以OMAP4430开发板和Android手机为核心,通过响应摇杆水平、竖直方向的移动事件和手柄按键事件来控制机械手的地表移动及机械手马达操作,同时借助车载USB摄像头拍摄720P视频,经H264编码后用无线网卡发送到Android手机,解码后显示在控制界面上。实验结果表明,可视化远程遥控机械手能够实现全方位移动、捉取地面目标等功能,同时无线视频传输帧率达到28fps,满足实时视频传输要求。     

Network Distributed Multi-Functional Robotic System Supporting the Elderly and Disabled People

We have been developing a network distributed Human-Assistance Robotic System in order to improve care cost and the QoL (Quality of Life) of the elderly people in the population-aging society. We developed multi-functional robotic system and implemented several CORBA application servers to provide some basic services to aid the aged or disabled. A novel method of localization of mobile robot with a camera and RFID (Radio Frequency Identification) technology is proposed as it is inexpensive, flexible and easy to use in practical environment. A video/audio conference system is also developed to improve the interaction among the users, switch robot manipulating privilege with the help of a centralized user management server, and enable a web-user to get a better understanding of what is going on in the local environment. Considering multi-type user of the developed, we have implemented multiple HRIs (Human Robot Interfaces) that enable different user to control robot systems easily.     

Socio-technical aspects of remote media control for a NG9-1-1 system

9-1-1 emergency calls mostly involve distress situations that cause people to panic while trying to answer questions or follow instructions given by a dispatcher. To obtain precious information with the least user intervention and reduced stress on the user, there is a need for the dispatcher to have a better control and understanding of the condition or situation at the other end. The increasing growth of smartphones embedded with camera, speaker phone, GPS, microphone and various other sensors, extends their usage from merely making calls to life saving gadgets during critical situations. By integrating these sensor rich smartphones and the rapidly growing VoIP technology, we propose a VoIP based Next Generation 9-1-1 (NG9-1-1) system for remote media control. Specifically, we use Session Initiation Protocol (SIP) in the implementation of the system using a mobile and a PC client. The proposed system on the mobile client accounted for less than 25% of CPU utilization even with video transmission. The average network utilization was about 10 and 72 kbps for audio and video, respectively. With these encouraging results, we believe the proposed remote media control system will facilitate information acquisition and decision making in emergency situations.     

Automatic concrete tunnel inspection robot system

This paper describes the development of an automatic concrete tunnel inspection system by an autonomous mobile robot. This work was motivated by the accidents in recent years that were caused by falling parts of the inner wall of concrete tunnels. This brought about serious damage to national property and led to the great concern throughout the whole society. Here, we proposed a non-destructive automatic tunnel inspection method. In this method, we aim to inspect the tunnel automatically and completely at high speed by using non-destructive sensors. For the non-destructive sensors, we employ 24 ultrasonic range sensors and six video cameras. These sensors are mounted on the same plane in the shape of a semi-ring. This ultrasonic range sensor video camera semi-ring is called a URS/VC semi-ring. This URS/VC semi-ring is mounted on an autonomous mobile robot to inspect the concrete tunnel. Experiment results show that this system can detect deformed inner walls at divisions of 14 mm when the robot moves at 20 mm/s.     

基于主动视觉和超声的机器人目标跟踪系统

运动目标跟踪技术是未知环境下移动机器人研究领域的一个重要研究方向。该文提出了一种基于主动视觉和超声信息的移动机器人运动目标跟踪设计方法,利用一台SONY EV-D31彩色摄像机、自主研制的摄像机控制模块、图像采集与处理单元等构建了主动视觉系统。移动机器人采用了基于行为的分布式控制体系结构,利用主动视觉锁定运动目标,通过超声系统感知外部环境信息,能在未知的、动态的、非结构化复杂环境中可靠地跟踪运动目标。实验表明机器人具有较高的鲁棒性,运动目标跟踪系统运行可靠。     

Teleoperation of robot arm with position measurement via angle-pixel characteristic and visual supporting function

In this paper, a teleoperation system of a robot arm with position measurement function and visual supporting function is developed. The working robot arm is remotely controlled by the manual operation of the human operator and the autonomous control via visual servo. The visual servo employs the template matching technique. The position measurement is realized using a stereo camera based on the angle-pixel characteristic. The visual supporting function to give the human operator useful information about the teleoperation is also provided. The usefulness of the proposed teleoperation system is confirmed through experiments using an industrial articulated robot arm.     

Remote Visual Servoing of a Robot Manipulator via Internet2

Visual servoing is a powerful approach to enlarge the applications of robotic systems by incorporating visual information into the control system. On the other hand, teleoperation – the use of machines in a remote way – is increasing the number of applications in many domains. This paper presents a remote visual servoing system using only partial camera calibration and exploiting the high bandwidth of Internet2 to stream video information. The underlying control scheme is based on the image-based philosophy for direct visual servoing – computing the applied torque inputs to the robot based in error signals defined in the image plane – and evoking a velocity field strategy for guidance. The novelty of this paper is a remote visual servoing with the following features: (1) full camera calibration is unnecessary, (2) direct visual servoing does not neglect the robot nonlinear dynamics, and (3) the novel velocity field control approach is utilized. Experiments carried out between two laboratories demonstrated the effectiveness of the application. Work partially supported by CONACyT grant 45826 and CUDI.