Improvements to and large‐scale evaluation of a robotic kiwifruit harvester

The growing popularity of kiwifruit orchards in New Zealand is increasing the already high demand for seasonal labourers. A novel robotic kiwifruit harvester has been designed and built to help meet this demand [H. A. Williams et al. Biosystems Eng. 181 (2019), pp. 140–156]. Early evaluations of the platform have shown good results with the system capable of harvesting 51.0% of 1,456 kiwifruit in four bays with an average cycle‐time of 5.5 s/fruit. However, the harvester's high cycle‐time, and high fruit loss rate at 23.4%, prevent it from being commercially viable. This paper presents two new developments for the harvesting system, a new vision system and two new gripper variations designed to overcome the harvester's previous limitations. Furthermore, we have designed and conducted the largest real‐world evaluation of a robotic fruit harvesting system published to date with over 12,000 kiwifruit involved. The results of this trial demonstrated that our kiwifruit harvester is one of the most effective selective fruit harvesters in the world capable of successfully harvesting 86.0% of reachable fruit, and 55.8% of all kiwifruit with a cycle‐time of 2.78 s/fruit.     

Eco-programming of industrial robots for sustainable manufacturing via dynamic time scaling of trajectories

Nowadays, Industrial Robots (IRs) have become widespread in many manufacturing industries. Medium and high payload IRs cover a significant percentage of the overall factory Energy Consumption (EC). This article focuses on the IRs eco-programming to minimize the EC of a robot, being energy efficiency one of the fundamental aims of sustainable manufacturing. By leveraging well-known trajectory scaling methods, this research develops a novel, versatile, fast, and efficient process to define the IR optimal velocity/acceleration profile in time, keeping the geometry of the trajectory fixed. A complete IR system model that founds application in various types of 6 degrees of freedom articulated manipulators has been developed by considering electrical motors, actuator drive systems, and controller cabinet losses. A new optimization technique based on Dynamic Time Scaling of trajectories is presented, and the obtained results are compared with other methods used in the scientific literature. When performing critical path analysis, the EC of the robot system is estimated to be cut down, being the robot motion time fixed, by about 13% through this novel approach. The model has been validated through commercial software, and the proposed optimization algorithm has been implemented in a user-friendly interface tool.     

Active Scene Understanding via Online Semantic Reconstruction

We propose a novel approach to robot‐operated active understanding of unknown indoor scenes, based on online RGBD reconstruction with semantic segmentation. In our method, the exploratory robot scanning is both driven by and targeting at the recognition and segmentation of semantic objects from the scene. Our algorithm is built on top of a volumetric depth fusion framework and performs real‐time voxel‐based semantic labeling over the online reconstructed volume. The robot is guided by an online estimated discrete viewing score field (VSF) parameterized over the 3D space of 2D location and azimuth rotation. VSF stores for each grid the score of the corresponding view, which measures how much it reduces the uncertainty (entropy) of both geometric reconstruction and semantic labeling. Based on VSF, we select the next best views (NBV) as the target for each time step. We then jointly optimize the traverse path and camera trajectory between two adjacent NBVs, through maximizing the integral viewing score (information gain) along path and trajectory. Through extensive evaluation, we show that our method achieves efficient and accurate online scene parsing during exploratory scanning.     

Development of a sweet pepper harvesting robot

This paper presents the development, testing and validation of SWEEPER, a robot for harvesting sweet pepper fruit in greenhouses. The robotic system includes a six degrees of freedom industrial arm equipped with a specially designed end effector, RGB‐D camera, high‐end computer with graphics processing unit, programmable logic controllers, other electronic equipment, and a small container to store harvested fruit. All is mounted on a cart that autonomously drives on pipe rails and concrete floor in the end‐user environment. The overall operation of the harvesting robot is described along with details of the algorithms for fruit detection and localization, grasp pose estimation, and motion control. The main contributions of this paper are the integrated system design and its validation and extensive field testing in a commercial greenhouse for different varieties and growing conditions. A total of 262 fruits were involved in a 4‐week long testing period. The average cycle time to harvest a fruit was 24 s. Logistics took approximately 50% of this time (7.8 s for discharge of fruit and 4.7 s for platform movements). Laboratory experiments have proven that the cycle time can be reduced to 15 s by running the robot manipulator at a higher speed. The harvest success rates were 61% for the best fit crop conditions and 18% in current crop conditions. This reveals the importance of finding the best fit crop conditions and crop varieties for successful robotic harvesting. The SWEEPER robot is the first sweet pepper harvesting robot to demonstrate this kind of performance in a commercial greenhouse.     

基于T-YOLO-LITE树干检测的模型部署方法

为解决猕猴桃采摘机器人视觉导航问题,提出基于T-YOLO-LITE的猕猴桃树干检测方法。通过保留BN层、调整输入图像尺寸、修改anchor boxes、添加负样本对YOLO-LITE进行改进,并利用Movidius与OpenCV-DNN将改进后的模型分别部署在树莓派与CPU设备中。实验结果表明:通过改进优化使模型检测精度提升至59.75%;利用Movidius与OpenCV-DNN部署后,模型检测速度分别达到了2帧每秒和6帧每秒。该模型在检测精度与YOLOV2-TINY持平的情况下,检测速度为YOLOV2-TINY的两倍,并在非GPU设备上完成近实时的树干检测任务。     

Learning‐based Nonlinear Model Predictive Control to Improve Vision‐based Mobile Robot Path Tracking

This paper presents a Learning‐based Nonlinear Model Predictive Control (LB‐NMPC) algorithm to achieve high‐performance path tracking in challenging off‐road terrain through learning. The LB‐NMPC algorithm uses a simple a priori vehicle model and a learned disturbance model. Disturbances are modeled as a Gaussian process (GP) as a function of system state, input, and other relevant variables. The GP is updated based on experience collected during previous trials. Localization for the controller is provided by an onboard, vision‐based mapping and navigation system enabling operation in large‐scale, GPS‐denied environments. The paper presents experimental results including over 3 km of travel by three significantly different robot platforms with masses ranging from 50 to 600 kg and at speeds ranging from 0.35 to 1.2 m/s (associated video at http://tiny.cc/RoverLearnsDisturbances ). Planned speeds are generated by a novel experience‐based speed scheduler that balances overall travel time, path‐tracking errors, and localization reliability. The results show that the controller can start from a generic a priori vehicle model and subsequently learn to reduce vehicle‐ and trajectory‐specific path‐tracking errors based on experience.     

Autonomous robot navigation in outdoor cluttered pedestrian walkways

This paper describes an implementation of a mobile robot system for autonomous navigation in outdoor concurred walkways. The task was to navigate through nonmodified pedestrian paths with people and bicycles passing by. The robot has multiple redundant sensors, which include wheel encoders, an inertial measurement unit, a differential global positioning system, and four laser scanner sensors. All the computation was done on a single laptop computer. A previously constructed map containing waypoints and landmarks for position correction is given to the robot. The robot system's perception, road extraction, and motion planning are detailed. The system was used and tested in a 1‐km autonomous robot navigation challenge held in the City of Tsukuba, Japan, named “Tsukuba Challenge 2007.” The proposed approach proved to be robust for outdoor navigation in cluttered and crowded walkways, first on campus paths and then running the challenge course multiple times between trials and the challenge final. The paper reports experimental results and overall performance of the system. Finally the lessons learned are discussed. The main contribution of this work is the report of a system integration approach for autonomous outdoor navigation and its evaluation. © 2009 Wiley Periodicals, Inc.     

基于ROS的ICP-SLAM在嵌入式移动机器人上的实现与优化

基于激光雷达的ICP-SLAM在机器人操作系统中无法直接运行,且通过PC实现算法时,没有高效运载计算机资源,产生资源浪费,同时降低了移动机器人系统的灵活性。通过在嵌入式系统中搭建机器人操作系统,在系统中移植MRPT函数库中的ICP-SLAM算法,对算法进行优化,调整算法运行的CPU占用率,使用基于激光雷达的移动机器人进行建图。结果表明机器人操作系统生成的栅格地图可以满足机器人自主导航的需求,SLAM效果更为直观,系统更加灵活,成本和硬件配置要求大大降低。证明优化后的算法更贴近实际的使用需求。     

基于花授粉算法的PID参数优化

PID参数优化对PID控制性能起着决定性作用,针对PID参数寻优问题,提出运用一种花授粉算法（FPA）。该算法启发于自然界中花粉的传播授粉过程,以三个PID参数组成每个花粉单元的位置坐标,根据一定的全局授粉与局部授粉规则更新花粉单元的位置,使其向最优解迭代。仿真结果表明,与粒子群算法和人群搜索算法相比,花授粉算法优化参数使系统具备更短的响应时间、更高的系统控制精度以及更好的鲁棒性,为PID控制系统的参数整定提供了参考。     

Under canopy light detection and ranging‐based autonomous navigation

This paper describes a light detection and ranging (LiDAR)‐based autonomous navigation system for an ultralightweight ground robot in agricultural fields. The system is designed for reliable navigation under cluttered canopies using only a 2D Hokuyo UTM‐30LX LiDAR sensor as the single source for perception. Its purpose is to ensure that the robot can navigate through rows of crops without damaging the plants in narrow row‐based and high‐leaf‐cover semistructured crop plantations, such as corn (Zea mays) and sorghum ( Sorghum bicolor). The key contribution of our work is a LiDAR‐based navigation algorithm capable of rejecting outlying measurements in the point cloud due to plants in adjacent rows, low‐hanging leaf cover or weeds. The algorithm addresses this challenge using a set of heuristics that are designed to filter out outlying measurements in a computationally efficient manner, and linear least squares are applied to estimate within‐row distance using the filtered data. Moreover, a crucial step is the estimate validation, which is achieved through a heuristic that grades and validates the fitted row‐lines based on current and previous information. The proposed LiDAR‐based perception subsystem has been extensively tested in production/breeding corn and sorghum fields. In such variety of highly cluttered real field environments, the robot logged more than 6 km of autonomous run in straight rows. These results demonstrate highly promising advances to LiDAR‐based navigation in realistic field environments for small under‐canopy robots.     

Path planning in the presence of obstacles based on task requirements

We propose a novel and efficient scheme for planning a kinematically feasible path in the presence of obstacles according to task requirements. By employing geometrical analysis, we derive expressions to describe the relationship between the planned path, kinematic constraints, and obstacles in the robot workspace. The freedom available according to task requirements is then utilized to modify the infeasible portions of the planned path. We use a 6R (revolute) wrist-partitioned type of robot manipulator and a spherical obstacle as a case study to demonstrate the proposed scheme. We then extend our results to general wrist-partitioned types of robot manipulators and arbitrarily-shaped or multiple obstacles. © 1994 John Wiley & Sons, Inc.     

Learning Control of Robot Manipulators in Task Space

Two important properties of industrial tasks performed by robot manipulators, namely, periodicity (i.e., repetitive nature) of the task and the need for the task to be performed by the end‐effector, motivated this work. Not being able to utilize the robot manipulator dynamics due to uncertainties complicated the control design. In a seemingly novel departure from the existing works in the literature, the tracking problem is formulated in the task space and the control input torque is aimed to decrease the task space tracking error directly without making use of inverse kinematics at the position level. A repetitive learning controller is designed which “learns” the overall uncertainties in the robot manipulator dynamics. The stability of the closed‐loop system and asymptotic end‐effector tracking of a periodic desired trajectory are guaranteed via Lyapunov based analysis methods. Experiments performed on an in‐house developed robot manipulator are presented to illustrate the performance and viability of the proposed controller.     

中型组足球机器人挑球系统的设计

中型组机器人足球比赛系统是进行多智能体研究的理想实验平台。中型组足球机器人有自己的“头脑”和“手脚”,后者就是它的挑球系统。设计了基于电磁原理的中型组足球机器人挑球系统,简化了系统设计,提高了系统性能,满足中型组足球机器人的控制要求。     

A lattice-type reconfigurable robotic system for achieving gaits of chain-type robots

Traditional lattice-type reconfigurable robots can only achieve the flow-style locomotion with low efficiency. Since gaits of chain-type robots are proved to be efficient and practical, this paper presents a novel lattice distortion approach for lattice-type reconfigurable robots to achieve locomotion gaits of chain-type robots. Using this approach, the robotic system can be actuated by local lattice distortion to move as an ensemble. In this paper, a rule that makes the lattice distortion equivalent to joint rotation is presented firstly. Then, a kind of module structure is designed according to requirements of the lattice distortion. Finally, a motion planning for achieving locomotion is developed, which works well in physics-based simulations of completing a serpentine locomotion gait of a snake-like robot and a tripod gait of a hexapod robot.     

可跳跃式移动机器人机构设计及实现

构建了一个具有跳跃能力的移动式机器人．机器人在较平坦地形下采用轮式移动方式前行；遇到障碍物或沟渠时，可以进行跳跃，从而扩大运动范围．介绍了机器人机械系统的总体结构，给出了机器人的本体结构及起跳姿态，并分析了机器人的运动过程．然后，详细分析了机器人的跳跃机构、跳跃参数调节机构、倾覆翻转机构等关键机构的工作原理，给出了机构设计方案．最后，根据总体设计要求选定了机器人的一些关键参数．     

Energy‐efficient Path Planning for Solar‐powered Mobile Robots*

We explore the problem of energy‐efficient, time‐constrained path planning of a solar‐powered robot embedded in a terrestrial environment. Because of the effects of changing weather conditions, as well as sensing concerns in complex environments, a new method for solar power prediction is desirable. We present a method that uses Gaussian Process regression to build a solar map in a data‐driven fashion. Using this map and an empirical model for energy consumption, we perform dynamic programming to find energy‐minimal paths. We validate our map construction and path‐planning algorithms with outdoor experiments, and we perform simulations on our solar maps to further determine the limits of our approach. Our results show that we can effectively construct a solar map using only a simple current measurement circuit and basic GPS localization, and this solar map can be used for energy‐efficient navigation. This establishes informed solar harvesting as a viable option for extending system lifetime even in complex environments with low‐cost commercial solar panels.     

Intelligent synchronized robot/conveyor system

A robot/conveyor system is an important part of the manufacturing system. As such, the robot/conveyor system must comply with all the requirements of a modern manufacturing system: high flexibility, high efficiency, and smart reasoning processes to lead the system to its next status based on its current status and varied inputs. An expert system consisting of a knowledge base and an inference engine was developed to operate and control a robot/conveyor system. A large variety of robot/conveyor system runs were performed to explore the efficiency of its operations. These system runs explore all the parameters associated with robot/conveyor system operations: manipulators specifications and velocities, pick-up methods, types of arriving parts, their locations, and orientations. The performed runs verified a well-known fact that a robot/conveyor system based on variable pick-up locations is more efficient than a robot/conveyor system based on a single, predefined, and unchanged pick-up location. The implementations of the variable pick-up locations method is associated with the need to implement real-time complicated software algorithms to track the arriving parts and to synchronize their maneuvers with the conveyor belt. Therefore, the implementation of variable pick-up locations method must be performed only when they result in most significant benefits. The system runs show that the benefits of the variable pick-up locations approach are inversely related to manipulations' velocities. If the variable pick-up location method is incorporated with efficient and reliable tracking algorithms, most efficient robot/conveyor systems will be resulted. Therefore, future efforts should address the advance of tracking algorithms of arriving parts, thus improving the synchronization of these parts with manipulator's maneuver, in order to achieve the most efficient robot/conveyor system. ©1999 John Wiley & Sons, Inc.     

Vision techniques for on‐board detection,following, and mapping of moving targets

This study presents computer vision modules of a multi‐unmanned aerial vehicle (UAV) system, which scored gold, silver, and bronze medals at the Mohamed Bin Zayed International Robotics Challenge 2017. This autonomous system, which was running completely on board and in real time, had to address two complex tasks in challenging outdoor conditions. In the first task, an autonomous UAV had to find, track, and land on a human‐driven car moving at 15 km/hr on a figure‐eight‐shaped track. During the second task, a group of three UAVs had to find small colored objects in a wide area, pick them up, and deliver them into a specified drop‐off zone. The computer vision modules presented here achieved computationally efficient detection, accurate localization, robust velocity estimation, and reliable future position prediction of both the colored objects and the car. These properties had to be achieved in adverse outdoor environments with changing light conditions. Lighting varied from intense direct sunlight with sharp shadows cast over the objects by the UAV itself, to reduced visibility caused by overcast to dust and sand in the air. The results presented in this paper demonstrate good performance of the modules both during testing, which took place in the harsh desert environment of the central area of United Arab Emirates, as well as during the contest, which took place at a racing complex in the urban, near‐sea location of Abu Dhabi. The stability and reliability of these modules contributed to the overall result of the contest, where our multi‐UAV system outperformed teams from world’s leading robotic laboratories in two challenging scenarios.     

An efficient local approach for the path generation of robot manipulators

In this article an efficient local approach for the path generation of robot manipulators is presented. The approach is based on formulating a simple nonlinear programming problem. This problem is considered as a minimization of energy with given robot kinematics and subject to the robot requirements and a singularities avoidance constraint. From this formulation a closed form solution is derived which has the properties that allows to pursue both singularities and obstacle avoidance simultaneously; and that it can incorporate global information. These properties enable the accomplishment of the important task that while a specified trajectory in the operational space can be closely followed, also a desired joint configuration can be attained accurately at a given time. Although the proposed approach is primarily developed for redundant manipulators, its application to nonredundant manipulators is examplified by considering a particular commercial manipulator.