Multimodal sensing and active continuous closed‐loop feedback for achieving reliable manipulation in the outdoor physical world

The use of field robots can greatly decrease the amount of time, effort, and associated risk compared to if human workers were to carryout certain tasks such as disaster response. However, transportability and reliability remain two main issues for most current robot systems. To address the issue of transportability, we have developed a lightweight modularizable platform named AeroArm. To address the issue of reliability, we utilize a multimodal sensing approach, combining the use of multiple sensors and sensor types, and the use of different detection algorithms, as well as active continuous closed‐loop feedback to accurately estimate the state of the robot with respect to the environment. We used Challenge 2 of the 2017 Mohammed Bin Zayed International Robotics Competition as an example outdoor manipulation task, demonstrating the capabilities of our robot system and approach in achieving reliable performance in the fields, and ranked fifth place internationally in the competition.     

Prowling autonomous mobile robot with a network camera

This research aimed to develop an autonomous mobile robot that helps various kinds of people. The evasion of obstacles is absolutely imperative so that the robot can act in a human-life environment. Therefore, we developed a robot that moves through doors and avoids obstacles with the help of images taken by a camera set on the robot. This work was presented in part at the 13th International Symposium on Artifical Life and Robotics, Oita, Japan, January 31–February 2, 2008     

Planetary exploration by a mobile robot: Mission teleprogramming and autonomous navigation

Sending mobile robots to accomplish planet exploration missions is scientifically promising and technologically challenging. We present in this paper a complete approach that encompasses the major aspects involved in the design of a robotic system for planetary exploration. It includes mission teleprogramming and supervision at a ground station, and autonomous mission execution by the remote mobile robot. We have partially implemented and validated these concepts. Experimental results illustrate the approach and the results.     

Cooperative autonomous search,grasping, and delivering in a treasure hunt scenario by a team of unmanned aerial vehicles

This paper addresses the problem of autonomous cooperative localization, grasping and delivering of colored ferrous objects by a team of unmanned aerial vehicles (UAVs). In the proposed scenario, a team of UAVs is required to maximize the reward by collecting colored objects and delivering them to a predefined location. This task consists of several subtasks such as cooperative coverage path planning, object detection and state estimation, UAV self‐localization, precise motion control, trajectory tracking, aerial grasping and dropping, and decentralized team coordination. The failure recovery and synchronization job manager is used to integrate all the presented subtasks together and also to decrease the vulnerability to individual subtask failures in real‐world conditions. The whole system was developed for the Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2017, where it achieved the highest score and won Challenge No. 3—Treasure Hunt. This paper does not only contain results from the MBZIRC 2017 competition but it also evaluates the system performance in simulations and field tests that were conducted throughout the year‐long development and preparations for the competition.     

Payload‐centric autonomy for in‐space robotic assembly of modular space structures

The paper addresses the problem of constructing large space structures (～100 m) by using autonomous robots to assemble modular components in space. We are motivated by the problem of creating space structures at a scale greater than what is feasible with a single self‐deploying design. We had two goals in this work. The first was to investigate and demonstrate the feasibility of long‐order multitask autonomy. The second was to study the balance between required tolerances in hardware design and robotic autonomy. This paper reports on a payload‐centric autonomy paradigm and presents results from laboratory demonstrations of automated assembly of structures using a multilimbed robotic platform. We present results with deployable 20 lb payloads (1 m trusses) that are robotically assembled to form a 3‐m diameter kinematically closed loop structure to subcentimeter accuracy. The robot uses its limbs to deploy the stowed modular structural components, manipulate them in free space, and assemble them via dual‐arm force control. We report on results and lessons learned from multiple successful end‐to‐end in‐lab demonstrations of autonomous truss assembly with JPL's RoboSimian robot originally developed for the Defense Advanced Research Projects Agency (DARPA). Videos of these demonstrations can be seen at https://goo.gl/muNLJp (JPL, 2017 ). Each end‐to‐end run took precisely 26 min to execute with very little variance across runs. We present changes/improvements to the RoboSimian system post‐DARPA Robotics Challenge (DRC) (Karumanchi et al., 2016 ). The new architecture has been improved with a focus on scalable autonomy as opposed to semiautonomy as required at the DRC.     

自主移动机器人足球比赛视觉定位方法综述

综述了RoboCup足球赛中全自主移动机器人基于视觉的定位技术,包括机器人自定位和多机器人协作物体定位.介绍了定位技术的发展情况与分类.从机器人环境构建形式的不同以及先验位姿和概率方法的应用与否等方面,系统地分析和比较了各种自定位方法.对于多机器人协作物体定位,阐述了静态方法和动态跟踪方法.总结了定位过程中需要重点研究的传感器模型构建、图像处理、特征匹配以及协作过程涉及的相关问题.最后就视觉定位存在的问题和技术发展趋势进行了讨论.     

Restock and straightening system for retail automation using compliant and mobile manipulation

As the retail industry keeps expanding and shortage of workers increasing, there is a need for autonomous manipulation of products to support retail operations. The increasing amount of products and customers in establishments such as convenience stores requires the automation of restocking, disposing and straightening of products. The manipulation of products needs to be time-efficient, avoid damaging products and beautify the display of products. In this paper, we propose a robotic system to restock shelves, dispose expired products, and straighten products in retail environments. The proposed mobile manipulator features a custom-made end effector with compact and compliant design to safely and effectively manipulate products in retail stores. Through experiments in a convenience store scenario, we verify the effectiveness of our system to restock, dispose and rearrange items.     

Learning-based model predictive control for improved mobile robot path following using Gaussian processes and feedback linearization

This paper proposes a high-performance path following algorithm that combines Gaussian processes (GP) based learning and feedback linearization (FBL) with model predictive control (MPC) for ground mobile robots operating in off-road terrains, referred to as GP-FBLMPC. The algorithm uses a nominal kinematic model and learns unmodeled dynamics as GP models by using observation data collected during field experiments. Extensive outdoor experiments using a Clearpath Husky A200 mobile robot show that the proposed GP-FBLMPC algorithm's performance is comparable to existing GP learning-based nonlinear MPC (GP-NMPC) methods with respect to the path following errors. The advantage of GP-FBLMPC is that it is generalizable in reducing path following errors for different paths that are not included in the GP models training process, while GP-NMPC methods only work well on exactly the same path on which GP models are trained. GP-FBLMPC is also computationally more efficient than the GP-NMPC because it does not conduct iterative optimization and requires fewer GP models to make predictions over the MPC prediction horizon loop at every time step. Field tests show the effectiveness and generalization of reducing path following errors of the GP-FBLMPC algorithm. It requires little training data to perform GP modeling before it can be used to reduce path-following errors for new, more complex paths on the same terrain (see video at https://youtu.be/tC09jJQ0OXM ).     

Terrain‐adaptive wheel speed control on the Curiosity Mars rover: Algorithm and flight results

NASA's Mars Science Laboratory Curiosity rover landed in August 2012 and began experiencing higher rates of wheel damage beginning in October 2013. While the wheels were designed to accumulate considerable damage, the unexpected damage rate raised concerns regarding wheel lifetime. In response, the Jet Propulsion Laboratory developed and deployed mobility flight software on Curiosity that reduces the forces on the wheels. The new algorithm adapts each wheel's speed to fit the terrain topography in real time, by leveraging the rover's measured attitude rates and rocker/bogie suspension angles and rates. Together with a rigid‐body kinematics model, it estimates the real‐time wheel‐terrain contact angles and commands idealized, no‐slip wheel angular rates. In addition, free‐floating “wheelies” are detected and autonomously corrected. Ground test data indicate that the forces on the wheels are reduced by 19% for leading wheels and 11% for middle leading wheels. On the ground, the required data volume increased by up to 129%, and drive duration increased by up to 25%. In flight, data collected over 3.6 km and 149 drives confirmed a reduction in wheel current, correlated with wheel torque, of 18.7%. The new algorithm proved to use fewer resources in flight than ground estimates suggested, as only a 10% increase in drive duration and double the drive data volume were experienced. These data indicate the promise of the new algorithm to extend the life of the wheels for the Curiosity rover. This paper describes the algorithm, its ground testing campaign and associated challenges, and its validation, implementation, and performance in flight.     

Force‐based perception and manipulation,the DTU team competing in MBZIRC 2017

This paper presents how the team from the Technical University of Denmark (DTU) implemented and solved the second challenge of the Mohamed Bin Zayed International Robotics Challenge. The competition was imitating a disaster scene where a robotic platform had to operate autonomously in a partly known environment to localize and manipulate a valve on a panel. To solve the given problem, the robot needs to be able to perceive the environment reliably. This is often accomplished using vision based solutions, however these might not always be feasible. Thus we show how force feedback can successfully be used as an alternative way of perception. To accomplish this the team equipped a robot arm with a force torque sensor, allowing the robot to perceive its environment through direct contact. This approach resulted in a robust solution of the task, independent of several external factors, such as lighting, which might affect a more traditional approach. First the theory and thoughts behind the implementation is presented, followed by an evaluation of the results from physical experiments and the competition itself, ultimately resulting in a robust solution which performed without errors in the competition.     

SWAMI: An autonomous mobile robot for inspection of nuclear waste storage facilities

The Stored Waste Autonomous Mobile Inspector (SWAMI) is a prototype mobile robot designed to perform autonomous inspection of nuclear and hazardous waste storage facilities. The onboard control system, consisting of three Motorola 68030-based microcomputers, controls a number of subsystem components including barcode readers, cameras, and a radiation detector. The control system software, running under the VxWorks real-time operating system, is designed toward the client-server model and is implemented in C++. GENISAS, a communication library developed by the Sandia National Laboratories, is used extensively. Much of the onboard software was generated by a custom code generation tool called Moses.     

An autonomous strawberry‐harvesting robot: Design,development, integration,and field evaluation

This paper presents an autonomous robot capable of picking strawberries continuously in polytunnels. Robotic harvesting in cluttered and unstructured environment remains a challenge. A novel obstacle‐separation algorithm was proposed to enable the harvesting system to pick strawberries that are located in clusters. The algorithm uses the gripper to push aside surrounding leaves, strawberries, and other obstacles. We present the theoretical method to generate pushing paths based on the surrounding obstacles. In addition to manipulation, an improved vision system is more resilient to lighting variations, which was developed based on the modeling of color against light intensity. Further, a low‐cost dual‐arm system was developed with an optimized harvesting sequence that increases its efficiency and minimizes the risk of collision. Improvements were also made to the existing gripper to enable the robot to pick directly into a market punnet, thereby eliminating the need for repacking. During tests on a strawberry farm, the robots first‐attempt success rate for picking partially surrounded or isolated strawberries ranged from 50% to 97.1%, depending on the growth situations. Upon an additional attempt, the pick success rate increased to a range of 75–100%. In the field tests, the system was not able to pick a target that was entirely surrounded by obstacles. This failure was attributed to limitations in the vision system as well as insufficient dexterity in the grippers. However, the picking speed improved upon previous systems, taking just 6.1 s for manipulation operation in the one‐arm mode and 4.6 s in the two‐arm mode.     

Multirepresentation,Multiheuristic A* search‐based motion planning for a free‐floating underwater vehicle‐manipulator system in unknown environment

A key challenge in autonomous mobile manipulation is the ability to determine, in real time, how to safely execute complex tasks when placed in unknown or changing world. Addressing this issue for Intervention Autonomous Underwater Vehicles (I‐AUVs), operating in potentially unstructured environment is becoming essential. Our research focuses on using motion planning to increase the I‐AUVs autonomy, and on addressing three major challenges: (a) producing consistent deterministic trajectories, (b) addressing the high dimensionality of the system and its impact on the real‐time response, and (c) coordinating the motion between the floating vehicle and the arm. The latter challenge is of high importance to achieve the accuracy required for manipulation, especially considering the floating nature of the AUV and the control challenges that come with it. In this study, for the first time, we demonstrate experimental results performing manipulation in unknown environment. The Multirepresentation, Multiheuristic A* (MR‐MHA*) search‐based planner, previously tested only in simulation and in a known a priori environment, is now extended to control Girona500 I‐AUV performing a Valve‐Turning intervention in a water tank. To this aim, the AUV was upgraded with an in‐house‐developed laser scanner to gather three‐dimensional (3D) point clouds for building, in real time, an occupancy grid map (octomap) of the environment. The MR‐MHA* motion planner used this octomap to plan, in real time, collision‐free trajectories. To achieve the accuracy required to complete the task, a vision‐based navigation method was employed. In addition, to reinforce the safety, accounting for the localization uncertainty, a cost function was introduced to keep minimum clearance in the planning. Moreover a visual‐servoing method had to be implemented to complete the last step of the manipulation with the desired accuracy. Lastly, we further analyzed the approach performance from both loose‐coupling and clearance perspectives. Our results show the success and efficiency of the approach to meet the desired behavior, as well as the ability to adapt to unknown environments.     

Design,control, and visual navigation of the DelftaCopter VTOL tail‐sitter UAV

To participate in the Outback Medical Express UAV Challenge 2016, a vehicle was designed and tested that can autonomously hover precisely, takeoff and land vertically, fly fast forward efficiently, and use computer vision to locate a person and a suitable landing location. The vehicle is a novel hybrid tail‐sitter combining a delta‐shaped biplane fixed‐wing and a conventional helicopter rotor. The rotor and wing are mounted perpendicularly to each other,and the entire vehicle pitches down to transition from hover to fast forward flight where the rotor serves as propulsion. To deliver sufficient thrust in hover while still being efficient in fast forward flight, a custom rotor system was designed. The theoretical design was validated with energy measurements, wind tunnel tests, and application in real‐world missions. A rotor‐head and corresponding control algorithm were developed to allow transitioning flight with the nonconventional rotor dynamics that are caused by the fuselage rotor interaction. Dedicated electronics were designed that meet vehicle needs and comply with regulations to allow safe flight beyond visual line of sight. Vision‐based search and guidance algorithms running on a stereo‐vision fish‐eye camera were developed and tested to locate a person in cluttered terrain never seen before. Flight tests and a competition participation illustrate the applicability of the DelftaCopter concept.     

The pros and cons of flocking in the long-range “migration” of mobile robot swarms

In this paper, we study how flocking affects the accuracy and speed of individuals in long-range “migration”. Specifically, we extend a behavior that can generate self-organized flocking in a swarm of robots to follow a homing direction sensed through the magnetic field of the Earth and evaluate how the final points reached by the flock are scattered in space and how the speed of the flock is affected. We propose that four factors influence the performance of migration, in the proposed behavior, namely: (1) averaging through heading alignment behavior, (2) disturbances caused by proximal control behavior, (3) noise in sensing the homing direction, and (4) differences in the characteristics of the individuals. Systematic experiments are conducted to evaluate the effects of these factors using both physical and simulated robots. The results show that although flocking reduces the speed of an individual, it increases the accuracy of “migration” for flocks that are larger than a certain size.     

RUR53: an unmanned ground vehicle for navigation,recognition, and manipulation

ABSTRACT

This paper proposes RUR53: an Unmanned Ground Vehicle able to navigate through, identify, and reach areas of interest. There, it can recognize, localize, and manipulate work tools to perform both indoor and outdoor complex tasks. Indeed, a wide range of sensors composes the robot and enables it to perceive vast workspaces, reach distant targets, and face the uncertainties of the real world. Precise object detection is also guaranteed, essential to manipulate objects of different shapes and materials. Moreover, a customized 3-finger gripper makes the gripping mode suitable for any lightweight object. Two modalities are proposed: autonomous and teleoperated, letting both unskilled and skilled human operators easily adapt the system to complete personalized tasks. The paper exhaustively describes RUR53 architecture and demonstrates its good performance while executing both indoor and outdoor navigation and manipulation tasks. A specific case study is described where the proposed modular architecture allows to easily switch to a semi-teleoperated mode: the 2017 Mohamed Bin Zayed International Robotics Challenge, where our team ranked third in the Grand Challenge in collaboration with the Czech Technical University in Prague, the University of Pennsylvania, and the University of Lincoln (UK).     

Precise and efficient pose estimation of stacked objects for mobile manipulation in industrial robotics challenges

Object manipulation tasks such as picking up, carrying and placing should be executed based on the information of objects which are provided by the perception system. A precise and efficient pose estimation system has been developed to address the requirements and to achieve the objectives for autonomous packaging, specifically picking up of stacked non-rigid objects. For fine pose estimation, a drawing pin shaped kernel and pinhole filtering methods are used on the roughly estimated pose of objects. The system has been applied in a realistic industrial environment as a challenging scenario for the Challenge 2 – Shop Floor Logistics and Manipulation on a mobile manipulator in the context of the European Robotics Challenges (EuRoC) project.     

High‐accuracy absolute positioning for the stationary planetary rover by integrating the star sensor and inclinometer

In this paper, we introduce a novel method for the high‐accuracy absolute position determination for planetary rovers using the star sensor and inclinometer. We describe the star sensor and inclinometer model and the alignment method for the two sensors. We deduce the compensation algorithm for the atmosphere refraction correction error in detail and provide the rover's position solution, which effectively eliminates the tilt correction error. The experimental site and hardware configuration are introduced, and the experimental steps for the one‐time positioning are described. Three field tests on Earth indicate that the accuracy of the one‐time positioning is higher than 40 m (1σ) using 8 star images and relative inclinometer measurements. Multiple positionings in one night can improve the accuracy to approximately 15 m.