Object Recognition and Selective Handling by a Robot

A small flexible production cell has been built around a selectively compliant articulated robot arm. Moving on a conveyor belt, boxes marked with different labels are presented to the robot in a random order. Using a camera and a vision card, the labels on the boxes are recognized. Each one of the labels can be rotated, translated or scaled. Three different invariant feature extraction methods (signature, invariant moments of Hu and Zernike) are compared. A neural net is used to classify the labels. The task of the SCARA robot is to pick up the moving boxes and to sort them according to their labels.     

Sonar-based mobile robot navigation through supervised learning on a neural net

For mobile robot navigation in an unknown and changing environment, a reactive approach is both simple to implement and fast in response. A neural net can be trained to exhibit such a behaviour. The advantage is that, it relates the desired motion directly to the sensor inputs, obviating the need of modeling and planning. In this work, a feedforward neural net is trained to output reactive motion in response to ultrasonic range inputs, with data generated artificially on the computer screen. We develop input and output representations appropriate to this problem.A purely reactive robot, being totally insensitive to context, often gets trapped in oscillations in front of a wide object. To overcome this problem, we introduce a notion of memory into the net by including context units at the input layer. We describe the mode of training for such a net and present simulated runs of a point robot under the guidance of the trained net in various situations. We also train a neural net for the navigation of a mobile robot with a finite turning radius. The results of the numerous test runs of the mobile robot under the control of the trained neural net in simulation as well as in experiments carried out in the laboratory, are reported in this paper.     

Experimental validation and field performance metrics of a hybrid mobile robot mechanism

This paper presents the experimental validation and field testing of a novel hybrid mobile robot (HMR) system using a complete physical prototype. The mobile robot system consists of a hybrid mechanism whereby the locomotion platform and manipulator arm are designed as one entity to support both locomotion and manipulation symbiotically and interchangeably. The mechanical design is briefly described along with the related control hardware architecture based on an embedded onboard wireless communication network between the robot's subsystems, including distributed onboard power using Li‐ion batteries. The paper focuses on demonstrating through extensive experimental results the qualitative and quantitative field performance improvements of the mechanical design and how it significantly enhances mobile robot functionality in terms of the new operative locomotion and manipulation capabilities that it provides. In terms of traversing challenging obstacles, the robot was able to surmount cylindrical obstacles up to 0.6‐m diameter; cross ditches with at least 0.635‐m width; climb and descend step obstacles up to 0.7‐m height; and climb and descend stairs of different materials (wood, metal, concrete, plastic plaster, etc.), different stair riser and run sizes, and inclinations up to 60 deg. The robot also demonstrated the ability to manipulate objects up to 61 kg before and after flipping over, including pushing capacity of up to 61 kg when lifting objects from underneath. The above‐mentioned functions are critical in various challenging applications, such as search and rescue missions, military and police operations, and hazardous site inspections. © 2010 Wiley Periodicals, Inc.     

Learning separate visual representations of independently rotating objects

Individual cells that respond preferentially to particular objects have been found in the ventral visual pathway. How the brain is able to develop neurons that exhibit these object selective responses poses a significant challenge for computational models of object recognition. Typically, many objects make up a complex natural scene and are never presented in isolation. Nonetheless, the visual system is able to build invariant object selective responses. In this paper, we present a model of the ventral visual stream, VisNet, which can solve the problem of learning object selective representations even when multiple objects are always present during training. Past research with the VisNet model has shown that the network can operate successfully in a similar training paradigm, but only when training comprises many different object pairs. Numerous pairings are required for statistical decoupling between objects. In this research, we show for the first time that VisNet is capable of utilizing the statistics inherent in independent rotation to form object selective representations when training with just two objects, always presented together. Crucially, our results show that in a dependent rotation paradigm, the model fails to build object selective representations and responds as if the two objects are in fact one. If the objects begin to rotate independently, the network forms representations for each object separately.     

Invariant object recognition with trace learning and multiple stimuli present during training

Over successive stages, the ventral visual system develops neurons that respond with view, size and position invariance to objects including faces. A major challenge is to explain how invariant representations of individual objects could develop given visual input from environments containing multiple objects. Here we show that the neurons in a 1-layer competitive network learn to represent combinations of three objects simultaneously present during training if the number of objects in the training set is low (e.g. 4), to represent combinations of two objects as the number of objects is increased to for e.g. 10, and to represent individual objects as the number of objects in the training set is increased further to for e.g. 20. We next show that translation invariant representations can be formed even when multiple stimuli are always present during training, by including a temporal trace in the learning rule. Finally, we show that these concepts can be extended to a multi-layer hierarchical network model (VisNet) of the ventral visual system. This approach provides a way to understand how a visual system can, by self-organizing competitive learning, form separate invariant representations of each object even when each object is presented in a scene with multiple other objects present, as in natural visual scenes.     

CAD-based Pose Estimation Design for Random Bin Picking using a RGB-D Camera

This paper presents a CAD-based six-degrees-of-freedom (6-DoF) pose estimation design for random bin picking for multiple objects. A virtual camera generates a point cloud database for the objects using their 3D CAD models. To reduce the computational time of 3D pose estimation, a voxel grid filter reduces the number of points for the 3D cloud of the objects. A voting scheme is used for object recognition and to estimate the 6-DoF pose for different objects. An outlier filter filters out badly matching poses so that the robot arm always picks up the upper object in the bin, which increases the success rate. In a computer simulation using a synthetic scene, the average recognition rate is 97.81 % for three different objects with various poses. A series of experiments have been conducted to validate the proposed method using a Kuka robot arm. The average recognition rate for three objects is 92.39 % and the picking success rate is 89.67 %.     

An open-source multi-DOF articulated robotic educational platform for autonomous object manipulation

This research presents an autonomous robotic framework for academic, vocational and training purpose. The platform is centred on a 6 Degree Of Freedom (DOF) serial robotic arm. The kinematic and dynamic models of the robot have been derived to facilitate controller design. An on-board camera to scan the arm workspace permits autonomous applications development. The sensory system consists of position feedback from each joint of the robot and a force sensor mounted at the arm gripper. External devices can be interfaced with the platform through digital and analog I/O ports of the robot controller. To enhance the learning outcome for beginners, higher level commands have been provided. Advanced users can tailor the platform by exploiting the open-source custom-developed hardware and software architectures. The efficacy of the proposed platform has been demonstrated by implementing two experiments; autonomous sorting of objects and controller design. The proposed platform finds its potential to teach technical courses (like Robotics, Control, Electronics, Image-processing and Computer vision) and to implement and validate advanced algorithms for object manipulation and grasping, trajectory generation, path planning, etc. It can also be employed in an industrial environment to test various strategies prior to their execution on actual manipulators.     

A robot language for parts handling based on hierarchical structure of verbs

An operation-level robot language for parts handling has been developed, in which stress is laid on the systematization of the hierarchical structure of verbs (and verb phases) and the realization of and interpreter of the language for controlling a robot arm. In this research, the circumstances are defined as a robot arm, shelves (work benches), boxes and objects on a floor. In the user's programming, eighteen verbs can be utilized: [Level 3] place on, stack on, lift down, carry, direct to, put in, put out, push and draw; [Level 2] have, lift and put; [Level 1] go, raise, lower, seize (grip), detach and turn. In the system, a verb module is reduced to a sequence of the verb modules immediately one level below it, and a series of control commands to a robot controller is automatically generated.     

基于视觉与RFID的机器人自定位抓取算法

为实现机器人灵活的自定位,并使其准确地抓取物体,提出一种基于视觉与无线射频识别(RFID)技术的机器人自定位抓取算法。构建网格化环境,利用RFID技术确定机器人的初始位置、行进路线和方向,使用视觉系统获取物体的空间坐标,将其转换到手臂坐标系,采用改进的D-H模型对手臂进行建模,并给出机械臂逆解抓取算法。实验结果表明,该算法使得机器人定位的成功率达到76.7%,抓取成功率高达90%。     

Neural net robot controller with guaranteed tracking performance

A neural net (NN) controller for a general serial-link robot arm is developed. The NN has two layers so that linearity in the parameters holds, but the "net functional reconstruction error" and robot disturbance input are taken as nonzero. The structure of the NN controller is derived using a filtered error/passivity approach, leading to new NN passivity properties. Online weight tuning algorithms including a correction term to backpropagation, plus an added robustifying signal, guarantee tracking as well as bounded NN weights. The NN controller structure has an outer tracking loop so that the NN weights are conveniently initialized at zero, with learning occurring online in real-time. It is shown that standard backpropagation, when used for real-time closed-loop control, can yield unbounded NN weights if (1) the net cannot exactly reconstruct a certain required control function or (2) there are bounded unknown disturbances in the robot dynamics. The role of persistency of excitation is explored.     

Active Learning for Vision-Based Robot Grasping

Reliable vision-based grasping has proved elusive outside of controlled environments. One approach towards building more flexible and domain-independent robot grasping systems is to employ learning to adapt the robot's perceptual and motor system to the task. However, one pitfall in robot perceptual and motor learning is that the cost of gathering the learning set may be unacceptably high. Active learning algorithms address this shortcoming by intelligently selecting actions so as to decrease the number of examples necessary to achieve good performance and also avoid separate training and execution phases, leading to higher autonomy. We describe the IE-ID3 algorithm, which extends the Interval Estimation (IE) active learning approach from discrete to real-valued learning domains by combining IE with a classification tree learning algorithm (ID-3). We present a robot system which rapidly learns to select the grasp approach directions using IE-ID3 given simplified superquadric shape approximations of objects. Initial results on a small set of objects show that a robot with a laser scanner system can rapidly learn to pick up new objects, and simulation studies show the superiority of the active learning approach for a simulated grasping task using larger sets of objects. Extensions of the approach and future areas of research incorporating more sophisticated perceptual and action representation are discussed     

Invariant global motion recognition in the dorsal visual system: a unifying theory

The motion of an object (such as a wheel rotating) is seen as consistent independent of its position and size on the retina. Neurons in higher cortical visual areas respond to these global motion stimuli invariantly, but neurons in early cortical areas with small receptive fields cannot represent this motion, not only because of the aperture problem but also because they do not have invariant representations. In a unifying hypothesis with the design of the ventral cortical visual system, we propose that the dorsal visual system uses a hierarchical feedforward network architecture (V1, V2, MT, MSTd, parietal cortex) with training of the connections with a short-term memory trace associative synaptic modification rule to capture what is invariant at each stage. Simulations show that the proposal is computationally feasible, in that invariant representations of the motion flow fields produced by objects self-organize in the later layers of the architecture. The model produces invariant representations of the motion flow fields produced by global in-plane motion of an object, in-plane rotational motion, looming versus receding of the object, and object-based rotation about a principal axis. Thus, the dorsal and ventral visual systems may share some similar computational principles.     

Distributed Telerobotics System Based on Common Object Request Broker Architecture

This paper proposes using CORBA as communication architecture to integrate network-distributed software and robotic systems in support systems for the aged or disabled. The proposed method keeps system costs low and expands availability. Its high scaling and inter-operating ability allows clients and server objects that are written in different languages, run in different operating systems, and connected to different networks to inter-operate. It also enables the system to be extended and integrated with other technologies and applications distributed over the Internet. Based on CORBA, we developed hardware base including a robot arm and an omnidirectional mobile robot and application servers including a task-level robot arm control server, live feedback image server, mobile robot control server and iGPS server. By remotely controlling mobile robot to cooperate with the robot arm, the caregivers or family member can use the developed system for some basic services to the aged or disabled.     

Three-dimensional neural net for learning visuomotor coordinationof a robot arm

An extension of T. Kohonen's (1982) self-organizing mapping algorithm together with an error-correction scheme based on the Widrow-Hoff learning rule is applied to develop a learning algorithm for the visuomotor coordination of a simulated robot arm. Learning occurs by a sequence of trial movements without the need for an external teacher. Using input signals from a pair of cameras, the closed robot arm system is able to reduce its positioning error to about 0.3% of the linear dimensions of its work space. This is achieved by choosing the connectivity of a three-dimensional lattice consisting of the units of the neural net.     

Scalable Techniques from Nonparametric Statistics for Real Time Robot Learning

Locally weighted learning (LWL) is a class of techniques from nonparametric statistics that provides useful representations and training algorithms for learning about complex phenomena during autonomous adaptive control of robotic systems. This paper introduces several LWL algorithms that have been tested successfully in real-time learning of complex robot tasks. We discuss two major classes of LWL, memory-based LWL and purely incremental LWL that does not need to remember any data explicitly. In contrast to the traditional belief that LWL methods cannot work well in high-dimensional spaces, we provide new algorithms that have been tested on up to 90 dimensional learning problems. The applicability of our LWL algorithms is demonstrated in various robot learning examples, including the learning of devil-sticking, pole-balancing by a humanoid robot arm, and inverse-dynamics learning for a seven and a 30 degree-of-freedom robot. In all these examples, the application of our statistical neural networks techniques allowed either faster or more accurate acquisition of motor control than classical control engineering.     

Vision-based path following by using a neural network Guidance System

This article describes a neural network controller for guidance of a robot arm, used to model some aspects of autonomous vehicle technology. The controller uses video images with adaptive view-angles for the sensory input, and the system was configured to simulate an autonomous vehicle guidance system on a flat terrain using a high-contrast guiding path. To demonstrate the feasibility of using neural networks in this type of application, an Intelledex 405 robot fitted with a video camera and associated vision system was used. Phase I of the project consisted of a single-speed implementation and limited network training. Phase II featured a multi-speed implementation using adaptively varied view-angles based on robot arm velocity. It was shown that the neural network controller was able to control the robot arm along a path composed of path segments unlike those with which it was trained. In addition it was shown that a multi-speed implementation with adaptive view angles improved system performance. © 1994 John Wiley & Sons, Inc.     

An adaptive-learning algorithm to solve the inverse kinematics problem of a 6 D.O.F serial robot manipulator

An adoptive learning strategy using an artificial neural network ANN has been proposed here to control the motion of a 6 D.O.F manipulator robot and to overcome the inverse kinematics problem, which are mainly singularities and uncertainties in arm configurations. In this approach a network have been trained to learn a desired set of joint angles positions from a given set of end effector positions, experimental results has shown an excellent mapping over the working area of the robot, to validate the ability of the designed network to make prediction and well generalization for any set of data, a new training using different data set has been performed using the same network, experimental results has shown a good generalization for the new data sets.The proposed control technique does not require any prior knowledge of the kinematics model of the system being controlled, the basic idea of this concept is the use of the ANN to learn the characteristics of the robot system rather than to specify explicit robot system model. Any modification in the physical set-up of the robot such as the addition of a new tool would only require training for a new path without the need for any major system software modification, which is a significant advantage of using neural network technology.     

基于Petri网的PLC系统设计分析与远程监控

Petri网在离散事件的动态仿真中有着广泛的应用,而如何将Petri网应用到离散制造系统中具有重大的价值.以离散制造业中最常见的搬运机械手模型为基础,提出了它的Petri网模型设计方法,并且利用科学的转换方法将Petri网转化为梯形图,应用到在制造业中广泛使用的PLC系统中.利用这种设计方法,不仅能够轻松地将Petri网应用到实践当中,还能够对模型的死锁、生产周期和不变量进行定量的分析,给制造系统进行管理控制、故障诊断和现场监控带来方便.     

Accumulation of object representations utilising interaction of robot action and perception

We introduce a robotic-vision system which is able to extract object representations autonomously utilising a tight interaction of visual perception and robotic action within a perception action cycle [Ecological Psychology 4 (1992) 121; Algebraic Frames for the Perception and Action Cycle, 1997, 1]. Controlled movement of the object grasped by the robot enables us to compute the transformations of entities which are used to represent aspects of objects and to find correspondences of entities within an image sequence.A general accumulation scheme allows to acquire robust information from partly missing information extracted from single frames of an image sequence. Here we use this scheme with a preprocessing stage in which 3D-line segments are extracted from stereo images. However, the accumulation scheme can be used with any kind of preprocessing as long as the entities used to represent objects can be brought to correspondence by certain equivalence relations such as ‘rigid body motion’.We show that an accumulated representation can be applied within a tracking algorithm. The accumulation scheme is an important module of a vision based robot system on which we are currently working. In this system, objects are planned to be represented by different visual and tactile entities. The object representations are going to be learned autonomously. We discuss the accumulation scheme in the context of this project.