Development of a sweet pepper harvesting robot

This paper presents the development, testing and validation of SWEEPER, a robot for harvesting sweet pepper fruit in greenhouses. The robotic system includes a six degrees of freedom industrial arm equipped with a specially designed end effector, RGB‐D camera, high‐end computer with graphics processing unit, programmable logic controllers, other electronic equipment, and a small container to store harvested fruit. All is mounted on a cart that autonomously drives on pipe rails and concrete floor in the end‐user environment. The overall operation of the harvesting robot is described along with details of the algorithms for fruit detection and localization, grasp pose estimation, and motion control. The main contributions of this paper are the integrated system design and its validation and extensive field testing in a commercial greenhouse for different varieties and growing conditions. A total of 262 fruits were involved in a 4‐week long testing period. The average cycle time to harvest a fruit was 24 s. Logistics took approximately 50% of this time (7.8 s for discharge of fruit and 4.7 s for platform movements). Laboratory experiments have proven that the cycle time can be reduced to 15 s by running the robot manipulator at a higher speed. The harvest success rates were 61% for the best fit crop conditions and 18% in current crop conditions. This reveals the importance of finding the best fit crop conditions and crop varieties for successful robotic harvesting. The SWEEPER robot is the first sweet pepper harvesting robot to demonstrate this kind of performance in a commercial greenhouse.     

Performance improvements of a sweet pepper harvesting robot in protected cropping environments

Using robots to harvest sweet peppers in protected cropping environments has remained unsolved despite considerable effort by the research community over several decades. In this paper, we present the robotic harvester, Harvey, designed for sweet peppers in protected cropping environments that achieved a 76.5% success rate on 68 fruit (within a modified scenario) which improves upon our prior work which achieved 58% on 24 fruit and related sweet pepper harvesting work which achieved 33% on 39 fruit (for their best tool in a modified scenario). This improvement was primarily achieved through the introduction of a novel peduncle segmentation system using an efficient deep convolutional neural network, in conjunction with three‐dimensional postfiltering to detect the critical cutting location. We benchmark the peduncle segmentation against prior art demonstrating an improvement in performance with a $F_1$ score of 0.564 compared to 0.302. The robotic harvester uses a perception pipeline to detect a target sweet pepper and an appropriate grasp and cutting pose used to determine the trajectory of a multimodal harvesting tool to grasp the sweet pepper and cut it from the plant. A novel decoupling mechanism enables the gripping and cutting operations to be performed independently. We perform an in‐depth analysis of the full robotic harvesting system to highlight bottlenecks and failure points that future work could address.     

A fruit-tracking system for robotic harvesting

The fruit-tracking system described in this article estimated in real-time (60 Hz) the size and position of a valid fruit region in color images. This information was used to control the motion of a fruit-picking robot. Statistical classification of color pixels was employed to improve image processing under the variety of imaging conditions encountered in a grove. Manual specification of valid fruit colors was possible through interactive training sessions. Object-oriented aperture control allowed adjustment of image exposure based on the illumination level of a targeted fruit. This feature increased the dynamic illumination range of the tracking system by ignoring excessively dark or bright background conditions. The performance of the fruit-tracking system was evaluated with off-line tests and under actual operating conditions.This work is part of Florida Agriculture Experiment Station Journal Series No. 9576.     

Localization of spherical fruits for robotic harvesting

The orange picking robot (OPR) is a project for developing a robot that is able to harvest oranges automatically. One of the key tasks in this robotic application is to identify the fruit and to measure its location in three dimensions. This should be performed using image processing techniques which must be sufficiently robust to cope with variations in lighting conditions and a changing environment. This paper describes the image processing system developed so far to guide automatic harvesting of oranges, which here has been integrated in the first complete full-scale prototype OPR. Received: 16 April 2000 / Accepted: 19 December 2000     

A vision system based on a laser range-finder applied to robotic fruit harvesting

This paper describes a laser-based computer vision system used for automatic fruit recognition. It is based on an infrared laser range-finder sensor that provides range and reflectance images and is designed to detect spherical objects in non-structured environments. Image analysis algorithms integrate both range and reflectance information to generate four characteristic primitives which give evidence of the existence of spherical objects. The output of this vision system includes 3D position, radius and surface reflectivity of each spherical object. It has been applied to the AGRIBOT orange harvesting robot, where it has obtained good fruit detection rates and unlikely false detections.     

A field‐tested robotic harvesting system for iceberg lettuce

Agriculture provides an unique opportunity for the development of robotic systems; robots must be developed which can operate in harsh conditions and in highly uncertain and unknown environments. One particular challenge is performing manipulation for autonomous robotic harvesting. This paper describes recent and current work to automate the harvesting of iceberg lettuce. Unlike many other produce, iceberg is challenging to harvest as the crop is easily damaged by handling and is very hard to detect visually. A platform called Vegebot has been developed to enable the iterative development and field testing of the solution, which comprises of a vision system, custom end effector and software. To address the harvesting challenges posed by iceberg lettuce a bespoke vision and learning system has been developed which uses two integrated convolutional neural networks to achieve classification and localization. A custom end effector has been developed to allow damage free harvesting. To allow this end effector to achieve repeatable and consistent harvesting, a control method using force feedback allows detection of the ground. The system has been tested in the field, with experimental evidence gained which demonstrates the success of the vision system to localize and classify the lettuce, and the full integrated system to harvest lettuce. This study demonstrates how existing state‐of‐the art vision approaches can be applied to agricultural robotics, and mechanical systems can be developed which leverage the environmental constraints imposed in such environments.     

A vision-centered multi-sensor fusing approach to self-localization and obstacle perception for robotic cars

Most state-of-the-art robotic cars’ perception systems are quite different from the way a human driver understands traffic environments. First, humans assimilate information from the traffic scene mainly through visual perception, while the machine perception of traffic environments needs to fuse information from several different kinds of sensors to meet safety-critical requirements. Second, a robotic car requires nearly 100% correct perception results for its autonomous driving, while an experienced human driver works well with dynamic traffic environments, in which machine perception could easily produce noisy perception results. In this paper, we propose a vision-centered multi-sensor fusing framework for a traffic environment perception approach to autonomous driving, which fuses camera, LIDAR, and GIS information consistently via both geometrical and semantic constraints for efficient self-localization and obstacle perception. We also discuss robust machine vision algorithms that have been successfully integrated with the framework and address multiple levels of machine vision techniques, from collecting training data, efficiently processing sensor data, and extracting low-level features, to higher-level object and environment mapping. The proposed framework has been tested extensively in actual urban scenes with our self-developed robotic cars for eight years. The empirical results validate its robustness and efficiency.     

Prior notice method of robotic arm motion for suppressing a threat to a human

The ability to develop a gait with one or more legs missing is an important issue for multi-legged robots used in demining applications. Accordingly, this paper presents a three-legged gait under the assumption that one leg of a quadruped walking robot is missing. After outlining a posture classification scheme for three-legged walking, the kick-and-swing gait is proposed as a basic and reasonable gait for three-legged walking and analyzed using a simple dynamic model. Minimum energy gait planning and an active shock-absorbing method are also investigated. The validity of the proposed gait is shown based on experiments using the quadruped walking robot TITAN VIII.     

Decentralized navigation method for a robotic swarm with nonhomogeneous abilities

This paper addresses the navigation of a robotic swarm with nonhomogeneous abilities, including sensing range, maximum velocity, and acceleration. With this method, the robotic swarm moves in a two-dimensional plane, and each follower distributedly constructs and maintains local directed connection using only local information to achieve maintenance of global connectivity. We also ensure the swarm is stable when the leader moves at a constant velocity. Validity and effectiveness of the proposed control strategy are shown by theoretical analysis, experiments with real robots, and numerical simulations.     

Learning to fall: Designing low damage fall sequences for humanoid soccer robots

A methodology for the analysis and design of fall sequences of robots that minimize joint/articulation injuries, and the damage of valuable body parts is proposed. These fall sequences can be activated/triggered by the robot in case of a detected unintentional fall or an intentional fall, which are common events in humanoid soccer environments. The methodology is human-based and requires the use of a realistic simulator as development tool. The obtained results show that fall sequences designed using the proposed method produce less damage than standard, uncontrolled falls.     

AL-ProMP: Force-relevant skills learning and generalization method for robotic polishing

Skill learning in robot polishing is gaining attention and becoming a hot issue. Current studies on skill learning in robot polishing are mainly about trajectory skills, and force-relevant skills learning models are less studied. A skill learning method with good generalization and robustness is one of the elements worth investigating. In this study, a force-relevant skills learning method called arc-length probabilistic movement primitives (AL-ProMP) is proposed to improve the efficiency of robot polishing force planning. AL-ProMP learns the mapping between the contact force and polishing trajectory, and the temporal scaling factor and force scaling factor in AL-ProMP enable better robustness of force planning in speed scaling tasks and polishing tasks in different scenarios. Speed scaling is an important property for adaptation of the polishing policy. For the generalization of polishing skills to different polishing tools in robotics disc polishing tasks of unknown geometric model workpieces, a novel force scaling factor for different polishing discs is proposed according to the contact force model. In addition, polishing contact position learning provides the basis for polishing trajectory generalization. Finally, it is experimentally verified that the proposed method is effective in learning and generalizing the demonstrated skills and improving the polishing surface quality of the workpiece with unknown geometric model.     

Case-based reactive navigation: a method for on-line selection andadaptation of reactive robotic control parameters

We present a new line of research investigating on-line adaptive reactive control mechanisms for autonomous intelligent agents. We discuss a case-based method for dynamic selection and modification of behavior assemblages for a navigational system. The case-based reasoning module is designed as an addition to a traditional reactive control system, and provides more flexible performance in novel environments without extensive high level reasoning that would otherwise slow the system down. The method is implemented in the ACBARR (case-based reactive robotic) system and evaluated through empirical simulation of the system on several different environments, including "box canyon" environments known to be problematic for reactive control systems in general.     

Control architecture for human-robot integration: application to a robotic wheelchair.

Completely autonomous performance of a mobile robot within noncontrolled and dynamic environments is not possible yet due to different reasons including environment uncertainty, sensor/software robustness, limited robotic abilities, etc. But in assistant applications in which a human is always present, she/he can make up for the lack of robot autonomy by helping it when needed. In this paper, the authors propose human-robot integration as a mechanism to augment/improve the robot autonomy in daily scenarios. Through the human-robot-integration concept, the authors take a further step in the typical human-robot relation, since they consider her/him as a constituent part of the human-robot system, which takes full advantage of the sum of their abilities. In order to materialize this human integration into the system, they present a control architecture, called architecture for human-robot integration, which enables her/him from a high decisional level, i.e., deliberating a plan, to a physical low level, i.e., opening a door. The presented control architecture has been implemented to test the human-robot integration on a real robotic application. In particular, several real experiences have been conducted on a robotic wheelchair aimed to provide mobility to elderly people.     

Civilian pepper spray for self defense: Understanding user perception and impact of design on user performance

Pepper spray is widely used in the US, and is marketed as an effective self-defense device. While pepper spray can be useful in deterring an attacker, many pepper spray owners do not have any experience using it. There is a dearth of published studies that focus on civilian pepper-spray use, not to mention first time pepper-spray users. A study to analyze pepper-spray designs with first-time users, to see how the design, specifically the safety mechanism, affects response time and overall performance would be helpful. The study was conducted in 2 parts – a survey to understand user perception, and a lab experiment using a 2 × 2 randomized block design with two designs of pepper sprays (side-slide safety and flip-top safety) and the two starting locations for the pepper spray (purse or pocket) as the manipulated factors. Results revealed a significant difference in response times between the flip-top pepper-spray and the side-slide pepper-spray. There was also a significant difference in response times between using the index or thumb as the trigger finger but no significant difference in response times when the pepper-spray for location (purse or pocket). Overall, participants rated the side-slide device as the most effective self-defense device.Relevance to industryBecause there is no other research in the area of first-time pepper-spray users, let alone civilian pepper-spray users, much can be learned about how well people use these devices in a real-world situation, and when there is a highly stressful situation how well does pepper-spray allow an individual to protect him or herself. This study starts the discussion on some of these topics.     

A biological neuroprocessor for robotic guidance using a center of area method

This paper introduces a neuroinformatic system using human neuroblastoma cultures and centre of area learning for basic robotic guidance. Multielectrode Arrays Setups have been designed for direct culturing neural cells over silicon or glass substrates, providing the capability to stimulate and record simultaneously populations of neural cells. The main objective of this work will be to control a robot using this biological neuroprocessor and a simple centre of area learning scheme. The final system could be applied for testing how chemicals affect the behaviour of the robot or to establish the basis for new hybrid optogenetic learning.     

Task reconstruction method for real-time singularity avoidance for robotic manipulators

The kinematic (KS) and algorithmic singularities (AS) in controlling robotic manipulators have been investigated intensively because they are not predictable or difficult to avoid. The problem with handling these singularities is an unnecessary performance reduction in the non-singular region and the difficulty in performance tuning. In this paper, we propose a method of avoiding KS and AS by applying a task reconstruction approach while maximizing the task performance by calculating singularity measures. The proposed method is implemented by removing the component approaching the singularity calculated by using a singularity measure in real-time. The outstanding feature of the proposed task reconstruction (TR) method is that it is based on a local TR as opposed to the local joint reconstruction of many other approaches. This method has a dynamic task priority assignment feature which ensures system stability under singular regions due to the change of task priority. The TR method enables us to increase the task controller gain to improve the task performance, whereas this increase can destabilize the system for conventional algorithms in real experiments. In addition, the physical meaning of tuning parameters is very straightforward. Hence, we can maximize task performance even near the singular region while simultaneously obtaining the singularity-free motion. The advantage of the proposed method is experimentally tested by using a 7-d. o. f. spatial manipulator and the result shows that the new method improves the performance several times over the existing algorithms.     

Developing a method to plan robotic straight needle insertion using a probability-based assessment of puncture occurrence

Needle-based treatments for cancer require accurate placement of the needle tip into the target tissue. However, it is often difficult to insert a needle accurately because of the cancer displacement caused by organ deformation. Therefore, developing a planning method based on a numerical simulation that analyzes organ deformation is important for accurate needle insertion. However, predicting the puncture conditions, including the force applied to the needle is not trivial owing to marked variations in the experimental data. The purpose of this research is to develop a novel method for predicting a robust path for straight needle insertion with various puncture points. The method is based on the probabilities of the various puncture conditions and evaluates the expected accuracy of needle placement. First, a probability-based puncture condition was established and the expected accuracy of needle placement was defined. We also performed in vitro needle insertion experiments using a porcine liver. Significant variations in puncture force were observed. Accordingly, we established a probability distribution for the tissue stress caused by the puncture. An in vitro experiment was performed to measure needle placement accuracy using the optimized path. Experimental results show that the proposed method achieves a mean accuracy of 1.5?mm.     

Automated stripping: a robotic handling cell for garmentmanufacture

The cutting room is currently the most highly automated part of the garment industry's manufacturing cycle. The next step, stripping the pieces from the cutting table and sorting them for assembly, is still highly labor intensive. At De Montfort University a robotic handling cell to perform the stripping operation has been developed and demonstrated     

A closed-loop error compensation method for robotic flank milling

Errors of diverse sources prevented industrial robots from being adopted into milling applications. This paper proposes a closed-loop error compensation method for robotic flank milling of complex shaped surfaces. First, the finished surface is measured in situ by a laser tracker based measuring system without unclamping the fixture. Then, the sampled points are mapped into the model reference coordinate system and a deterministic bicubic B-spline surface is fitted to extract the systematic components of the machining errors. Finally, the compensation tool path is directly generated for the mirror symmetry points of the only-systematic-error-contained sample points. The robot motion program is converted accordingly for further machining. The experiment shows that the surface accuracy is improved significantly in terms of the profile error via the proposed error compensation process, which well validates the effectiveness of the method.     

图像椒盐噪声的自适应滤波算法研究

为有效去除严重的椒盐噪声、更好地保护图像细节,提出了一种基于改进脉冲耦合神经网络（PCNN）的自适应去噪方法。根据PCNN神经网络的点火时刻矩阵,对受噪声污染的像素进行定位,仅对噪声像素进行类中值滤波,实现了图像细节的有效保留;根据噪声强度的估计信息,自动进行滤波次数和滤波窗口尺寸的优选,实现了图像的强自适应滤波。实验表明,与传统去噪方法相比,该方法噪声去除效果好,图像细节保持完整,而且系统具有一定的泛化能力。