基于目标检测的机器人手眼标定方法

智能协作机器人依赖视觉系统感知未知环境中的动态工作空间定位目标,实现机械臂对目标对象的自主抓取回收作业。RGB-D相机可采集场景中的彩色图和深度图,获取视野内任意目标三维点云,辅助智能协作机器人感知周围环境。为获取抓取机器人与RGB-D相机坐标系之间的转换关系,提出基于yolov3目标检测神经网络的机器人手眼标定方法。将3D打印球作为标靶球夹持在机械手末端,使用改进的yolov3目标检测神经网络实时定位标定球的球心,计算机械手末端中心在相机坐标系下的3D位置,同时运用奇异值分解方法求解机器人与相机坐标系转换矩阵的最小二乘解。在6自由度UR5机械臂和Intel RealSense D415深度相机上的实验结果表明,该标定方法无需辅助设备,转换后的空间点位置误差在2 mm以内,能较好满足一般视觉伺服智能机器人的抓取作业要求。     

基于RGB-D相机的室内环境3D地图创建

RGB-D相机（如微软的Kinect）能够在获取彩色图像的同时得到每个像素的深度信息,在移动机器人三维地图创建方向具有广泛应用。本文设计了一种利用RGB-D相机进行机器人自定位及创建室内场景三维模型的方法,该方法首先由RGB-D相机获取周围环境的连续帧信息;其次提取并匹配连续帧间的SURF特征点,通过特征点的位置变化计算机器人的位姿并结合非线性最小二乘优化算法最小化对应点的双向投影误差;最后结合关键帧技术及观察中心法将相机观测到的三维点云依据当前位姿投影到全局地图。本文选择三个不同的场景试验了该方法,并对比了不同特征点下该方法的效果,试验中本文方法在轨迹长度为5.88m情况下误差仅为0.023,能够准确地创建周围环境的三维模型。     

A multimodal language to communicate with life-supporting robots through a touch screen and a speech interface

This article proposes a multimodal language to communicate with life-supporting robots through a touch screen and a speech interface. The language is designed for untrained users who need support in their daily lives from cost-effective robots. In this language, the users can combine spoken and pointing messages in an interactive manner in order to convey their intentions to the robots. Spoken messages include verb and noun phrases which describe intentions. Pointing messages are given when the user’s finger touches a camera image, a picture containing a robot body, or a button on a touch screen at hand which convey a location in their environment, a direction, a body part of the robot, a cue, a reply to a query, or other information to help the robot. This work presents the philosophy and structure of the language.     

激光—相机系统语义栅格建图和路径规划

目的 SLAM（simultaneous localization and mapping）是移动机器人在未知环境进行探索、感知和导航的关键技术。激光SLAM测量精确，便于机器人导航和路径规划，但缺乏语义信息。而视觉SLAM的图像能提供丰富的语义信息，特征区分度更高，但其构建的地图不能直接用于路径规划和导航。为了实现移动机器人构建语义地图并在地图上进行路径规划，本文提出一种语义栅格建图方法。方法 建立可同步获取激光和语义数据的激光-相机系统，将采集的激光分割数据与目标检测算法获得的物体包围盒进行匹配，得到各物体对应的语义激光分割数据。将连续多帧语义激光分割数据同步融入占据栅格地图。对具有不同语义类别的栅格进行聚类，得到标注物体类别和轮廓的语义栅格地图。此外，针对语义栅格地图发布导航任务，利用路径搜索算法进行路径规划，并对其进行改进。结果 在实验室走廊和办公室分别进行了语义栅格建图的实验，并与原始栅格地图进行了比较。在语义栅格地图的基础上进行了路径规划，并采用了语义赋权算法对易移动物体的路径进行对比。结论 多种环境下的实验表明本文方法能获得与真实环境一致性较高、标注环境中物体类别和轮廓的语义栅格地图，且实验硬件结构简单、成本低、性能良好，适用于智能化机器人的导航和路径规划。     

Fast RGB-D people tracking for service robots

Service robots have to robustly follow and interact with humans. In this paper, we propose a very fast multi-people tracking algorithm designed to be applied on mobile service robots. Our approach exploits RGB-D data and can run in real-time at very high frame rate on a standard laptop without the need for a GPU implementation. It also features a novel depth-based sub-clustering method which allows to detect people within groups or even standing near walls. Moreover, for limiting drifts and track ID switches, an online learning appearance classifier is proposed featuring a three-term joint likelihood. We compared the performances of our system with a number of state-of-the-art tracking algorithms on two public datasets acquired with three static Kinects and a moving stereo pair, respectively. In order to validate the 3D accuracy of our system, we created a new dataset in which RGB-D data are acquired by a moving robot. We made publicly available this dataset which is not only annotated by hand, but the ground-truth position of people and robot are acquired with a motion capture system in order to evaluate tracking accuracy and precision in 3D coordinates. Results of experiments on these datasets are presented, showing that, even without the need for a GPU, our approach achieves state-of-the-art accuracy and superior speed.     

Web interfaces for mobile robots in public places

The growth of the World Wide Web provides unique opportunities to bring robots closer to people. The vision behind such endeavors ranges from relatively simple web-based inspections and surveillance applications to highly versatile applications that use robots connected to the web to establish a remote telepresence in dynamic and populated environments. In the latter scenario, robots play the role of a physical mediator, enabling remote people to acquire information, explore, manipulate, communicate, and interact physically with people far away. The article describes a series of web interfaces designed to remotely operate mobile robots in public places through the web. The design of these interfaces specifically addresses issues such as low bandwidth of interconnections, control brokering, and shared control as well as interaction with people in the robot's environment, which arise naturally in applications with web-based robot control. The interfaces have been tested extensively using two deployed service robots, which were installed as interactive tour guides in two museums. The article also discusses trade-offs and limitations of web-based robots that interact with people in populated public places     

Assessing the effect of persuasive robots interactive social cues on users’ psychological reactance,liking, trusting beliefs and compliance

ABSTRACT

Research in the field of social robotics suggests that enhancing social cues in robots can elicit more social responses in users. It is however not clear how users respond socially to persuasive social robots and whether such reactions will be more pronounced when the robots feature more interactive social cues. In the current research, we examine social responses towards persuasive attempts provided by a robot featuring different numbers of interactive social cues. A laboratory experiment assessed participants’ psychological reactance, liking, trusting beliefs and compliance toward a persuasive robot that either presented users with: no interactive social cues (random head movements and random social praises), low number of interactive social cues (head mimicry), or high number of interactive social cues (head mimicry and proper timing for social praise). Results show that a persuasive robot with the highest number of interactive social cues invoked lower reactance and was liked more than the robots in the other two conditions. Furthermore, results suggest that trusting beliefs towards persuasive robots can be enhanced by utilizing praise as presented by social robots in no interactive social cues and high number of interactive social cues conditions. However, interactive social cues did not contribute to higher compliance.     

基于Kinect的改进移动机器人视觉SLAM

针对传统ICP（iterative closest points,迭代最近点算法）存在易陷入局部最优、匹配误差大等问题,提出了一种新的欧氏距离和角度阈值双重限制方法,并在此基础上构建了基于Kinect的室内移动机器人RGB-D SLAM（simultaneous localization and mapping）系统。首先,使用Kinect获取室内环境的彩色信息和深度信息,通过图像特征提取与匹配,结合相机内参与像素点深度值,建立三维点云对应关系;然后,利用RANSAC（random sample consensus）算法剔除外点,完成点云的初匹配;采用改进的点云配准算法完成点云的精匹配;最后,在关键帧选取中引入权重,结合g2o（general graph optimization）算法对机器人位姿进行优化。实验证明该方法的有效性与可行性,提高了三维点云地图的精度,并估计出了机器人运行轨迹。     

基于ROS的服务机器人云端协同计算框架

随着机器人应用技术的发展,服务机器人走进人们的生活日渐成为可能. 但机器人本身计算能力有限,同时仅靠自身的传感器接收的信息也有一定的局限性. 现有的机器人还不足以胜任面对复杂场景的应对能力,也不能够满足人们对服务机器人的期待. 本文设计的云机器人计算框架（cloud robot computing framework, CRCF）通过云端,将智能家居以及其他智能硬件与机器人相结合,为机器人提供更多更广的信息. 同时,CRCF通过互联网结合了其它第三方的云端应用API,为机器人提供更多的服务功能. CRCF框架旨在利用云端的大数据处理能力提升机器人的计算和存储能力,并结合第三方云端应用服务和智能硬件设备来拓展机器人的信息来源和服务功能. 最后,本文通过远程语音控制机器人的实验,验证了CRCF系统平台在结合硬件设备以及第三方云端应用的功能和性能.     

基于云平台的智能语音交互机器人设计

现有的语音交互机器人多采用用户提问、机器人回答的单向交流方式,人机交互的智能性和灵活性较差。本文研究运用树莓派(Raspberry Pi)计算机和配套的语音板作为硬件载体,融合语音唤醒、语音识别、语音合成、自然语言处理等人工智能技术,调用科大讯飞开放云平台、在线图灵机器人,搭建一种基于云平台的智能语音交互机器人系统,并结合自主开发的本地知识库和问题库,使智能语音交互机器人能够根据不同环境与任务需求实现双向互动交流,实现由机器人采集信息和交流反馈,以提供高适应性的无接触人机语音交互服务。     

机器人技术研究进展

机器人技术的研究已从传统的工业领域扩展到医疗服务、教育娱乐、勘探勘测、生物工程、救灾救援等新领域, 并快速发展. 本文简要介绍了工业机器人、移动机器人、医疗与康复机器人和仿生机器人研究中的部分主要进展, 并通过分析和梳理, 归纳了机器人技术发展中的一些重要问题, 探讨机器人技术的发展趋势.     

基于RGB-D信息的移动机器人SLAM和路径规划方法研究与实现

移动机器人在各种辅助任务中需具备自主定位、建图、路径规划与运动控制的能力。本文利用RGB-D信息和ORB-SLAM算法进行自主定位,结合点云数据和GMapping算法建立环境栅格地图,基于二次规划方法进行平滑可解析的路径规划,并设计非线性控制器,实现了由一个运动底盘、一个RGB-D传感器和一个运算平台组成的自主移动机器人系统。经实验验证,这一系统实现了复杂室内环境下的实时定位与建图、自主移动和障碍物规避。由此,为移动机器人的推广应用提供了一个硬件结构简单、性能良好、易扩展、经济性好、开发维护方便的解决方案。     

多传感器的移动机器人可定位性估计与自定位

针对有人干扰的动态室内环境,利用可定位性估计理论提出一种RGB-D传感器辅助激光传感器的移动机器人可靠自定位方法。利用RGB-D传感器信息快速检测人的位置区域,并通过坐标转换计算激光扫描数据中的动态障碍物影响因子,结合离散化Fisher信息矩阵在线估计观测信息的可定位性矩阵;同时通过预测模型协方差矩阵评价里程计信息的可靠性,从而动态补偿观测信息对粒子集的影响。在典型含多人运动的动态室内环境中实验,结果验证了本文方法能够提高机器人自定位的准确性和可靠性。     

A hardware intelligent processing accelerator for domestic service robots

ABSTRACT

We present a method for implementing hardware intelligent processing accelerator on domestic service robots. These domestic service robots support human life; therefore, they are required to recognize environments using intelligent processing. Moreover, the intelligent processing requires large computational resources. Therefore, standard personal computers (PCs) with robot middleware on the robots do not have enough resources for this intelligent processing. We propose a ‘connective object for middleware to an accelerator (COMTA),’ which is a system that integrates hardware intelligent processing accelerators and robot middleware. Herein, by constructing dedicated architecture digital circuits, field-programmable gate arrays (FPGAs) accelerate intelligent processing. In addition, the system can configure and access applications on hardware accelerators via a robot middleware space; consequently, robotic engineers do not require the knowledge of FPGAs. We conducted an experiment on the proposed system by utilizing a human-following application with image processing, which is commonly applied in the robots. Experimental results demonstrated that the proposed system can be automatically constructed from a single-configuration file on the robot middleware and can execute the application 5.2 times more efficiently than an ordinary PC.     

A survey of socially interactive robots

This paper reviews “socially interactive robots”: robots for which social human–robot interaction is important. We begin by discussing the context for socially interactive robots, emphasizing the relationship to other research fields and the different forms of “social robots”. We then present a taxonomy of design methods and system components used to build socially interactive robots. Finally, we describe the impact of these robots on humans and discuss open issues. An expanded version of this paper, which contains a survey and taxonomy of current applications, is available as a technical report [T. Fong, I. Nourbakhsh, K. Dautenhahn, A survey of socially interactive robots: concepts, design and applications, Technical Report No. CMU-RI-TR-02-29, Robotics Institute, Carnegie Mellon University, 2002].     

A Social Robot that Stands in Line

Recent research in mobile robot navigation make it feasible to utilize autonomous robots in service fields. But, such applications require more than just navigation. To operate in a peopled environment, robots should recognize and act according to human social behavior. In this paper, we present the design and implementation of one such social behavior: a robot that stands in line much as people do. The system employs stereo vision to recognize lines of people, and uses the concept of personal space for modeling the social behavior. Personal space is used both to detect the end of a line and to determine how much space to leave between the robot and the person in front of it. Our model of personal space is based on measurements from people forming lines. We demonstrate our ideas with a mobile robot navigation system that can purchase a cup of coffee, even if people are waiting in line for service.     

The Matching Fund Project for Practical Use of Robotic Travel Aid for the Visually Impaired

Destined for the visually impaired, Robotic Travel Aid (RoTA) acts as an intelligent cart, guiding people across the streets. The 60 kg, 1-m tall mobile robot is equipped with a computer vision system, stereo camera sensor and voice interface. When moving, it is aware of its environment: it recognizes landmarks such as zebra-crossing marks or traffic signals, stopping when the light is red, and 'sees' cars or other pedestrians. In case of trouble, the robots communicates wirelessly with a service center, allowing it to give extra information on the trajectory and adapt the navigational information system. Now, a wheelchair robot is developed as the successor of RoTA, not only for the visually impaired, but also the disabled.     

Social and collaborative aspects of interaction with a service robot

To an increasing extent, robots are being designed to become a part of the lives of ordinary people. This calls for new models of the interaction between humans and robots, taking advantage of human social and communicative skills. Furthermore, human–robot relationships must be understood in the context of use of robots, and based on empirical studies of humans and robots in real settings. This paper discusses social aspects of interaction with a service robot, departing from our experiences of designing a fetch-and-carry robot for motion-impaired users in an office environment. We present the motivations behind the design of the Cero robot, especially its communication paradigm. Finally, we discuss experiences from a recent usage study, and research issues emerging from this work. A conclusion is that addressing only the primary user in service robotics is unsatisfactory, and that the focus should be on the setting, activities and social interactions of the group of people where the robot is to be used.     

Concrete Defects Inspection and 3D Mapping Using CityFlyer Quadrotor Robot

The concrete aging problem has gained more attention in recent years as more bridges and tunnels in the United States lack proper maintenance. Though the Federal Highway Administration requires these public concrete structures to be inspected regularly, on-site manual inspection by human operators is time-consuming and labor-intensive. Conventional inspection approaches for concrete inspection, using RGB image-based thresholding methods, are not able to determine metric information as well as accurate location information for assessed defects for conditions. To address this challenge, we propose a deep neural network (DNN) based concrete inspection system using a quadrotor flying robot (referred to as CityFlyer) mounted with an RGB-D camera. The inspection system introduces several novel modules. Firstly, a visual-inertial fusion approach is introduced to perform camera and robot positioning and structure 3D metric reconstruction. The reconstructed map is used to retrieve the location and metric information of the defects. Secondly, we introduce a DNN model, namely AdaNet, to detect concrete spalling and cracking, with the capability of maintaining robustness under various distances between the camera and concrete surface. In order to train the model, we craft a new dataset, i.e., the concrete structure spalling and cracking (CSSC) dataset, which is released publicly to the research community. Finally, we introduce a 3D semantic mapping method using the annotated framework to reconstruct the concrete structure for visualization. We performed comparative studies and demonstrated that our AdaNet can achieve 8.41% higher detection accuracy than ResNets and VGGs. Moreover, we conducted five field tests, of which three are manual hand-held tests and two are drone-based field tests. These results indicate that our system is capable of performing metric field inspection, and can serve as an effective tool for civil engineers.