Adaptive Reibungskompensation zur Verbesserung des Folgeverhaltens bei Visual Servoing

The vollowing article surveys the application of adaptive friction compensation to improve tracking behaviour using classical methods of motion control for visual servoing. Visual servoing is the use of motion control in order to follow a moving object, e.g. by the use of a robot and a gripping device. The position of the object is captured through image processing. The survey has been carried out on an XY-linear axis system using simulations and real time measurements. It shows that the tracking behaviour with classical motion control using visual servoing can be considerably improved by adaptive friction compensation. Problems which occur using such methods of motion control with visual servoing are also displayed.     

Eye-to-hand approach on eye-in-hand configuration within real-time visual servoing

This paper presents a framework of hand-eye relation for visual servoing with more precision, mobility, and global view. Mainly, there are two types of camera utilization for visual servoing: eye-in-hand and eye-to-hand configurations. Both have own merits and drawbacks regarding to precision and view limit that oppose each other. Based on both behaviors, this paper employs a mobile manipulator as the second robot to hold the camera. Here, the camera architecture is eye-to-hand configuration for the main robot, but mainly behaves as eye-in-hand configuration for the second robot. Having this framework, the drawback of each configuration is resolved by the benefit of the other. Here, the camera becomes mobile with more precision and global view. In addition, since there is no additional camera, the vision algorithm can be kept simple. In order to gain real-time visual servoing, this paper also addresses real-time constraints on vision system and data communication between robot and vision. Here, a hexagon pattern of artificial marker with a simplified image processing is developed. A grasp positioning problem is considered with position-based dynamic look and move visual control through object pose estimation. The system performance is validated by the experimental result.     

Optimizing Deep Learning Parameters Using Genetic Algorithm for Object Recognition and Robot Grasping

The performance of deep learning (DL) networks has been increased by elaborating the network structures. However, the DL netowrks have many parameters, which have a lot of influence on the performance of the network. We propose a genetic algorithm (GA) based deep belief neural network (DBNN) method for robot object recognition and grasping purpose. This method optimizes the parameters of the DBNN method, such as the number of hidden units, the number of epochs, and the learning rates, which would reduce the error rate and the network training time of object recognition. After recognizing objects, the robot performs the pick-and-place operations. We build a database of six objects for experimental purpose. Experimental results demonstrate that our method outperforms on the optimized robot object recognition and grasping tasks.     

Visual Control of robots with changes of visibility in image features

Visual servoing concerns several fields of research including vision systems, robotics and automatic control. Visual servoing can be useful for a wide range of applications and it can be used to control many different dynamic systems (manipulator arms, mobile robots, autonomous underwater vehicles, aircraft, etc.). However, visual servoing systems are not complete efficiency due to the numerous problems that are still unresolved. In this paper a new general solution to visibility restriction associated to all visual servoing techniques based directly or un-directly on image features is presented. This solution allows the temporary disappearance of image features during the control task versus the solutions proposed until now which are based on keeping always the object in the field of view. Taking into account this concept, the theoretical bases of a continuous and stable visual servoing approach that allows the temporary presence of features in the image have been defined and developed. Using them, the camera invariant visual servoing approach has been reformulated to adapt it to the temporary disappearance of image features during the control task.     

Visual Control of robots with changes of visibility in image features

Visual servoing concerns several fields of research including vision systems, robotics and automatic control. Visual servoing can be useful for a wide range of applications and it can be used to control many different dynamic systems (manipulator arms, mobile robots, autonomous underwater vehicles, aircraft, etc.). However, visual servoing systems are not complete efficiency due to the numerous problems that are still unresolved. In this paper a new general solution to visibility restriction associated to all visual servoing techniques based directly or un-directly on image features is presented. This solution allows the temporary disappearance of image features during the control task versus the solutions proposed until now which are based on keeping always the object in the field of view. Taking into account this concept, the theoretical bases of a continuous and stable visual servoing approach that allows the temporary presence of features in the image have been defined and developed. Using them, the camera invariant visual servoing approach has been reformulated to adapt it to the temporary disappearance of image features during the control task.     

A fractional order control strategy for visual servoing systems

In this paper, a control strategy based on fractional calculus for visual servoing systems is proposed. The image-based control strategy is designed using a point features based fractional-order PI controller. A real-time visual servoing system, composed of a manipulator robot with 6 degrees of freedom (d.o.f.) with an eye-in-hand camera, is used for performance evaluation of the proposed control strategy. The image acquisition and processing, together with the computing of the image-based control law are implemented in MATLAB. Using planar static objects, real-time experiments are conducted and the results reveal that the image-based fractional-order PI controller outperforms the conventional image-based integer-order PI controller.     

Hybrid Motion Control and Planning Strategies for Visual Servoing

This paper presents two hybrid strategies for robot visual servoing. Two specific image constraints, the image singularities and image local minima, are considered in both strategies. The hybrid motion control strategy consists of a local switching control between the image-based and position-based visual servoing for direct avoidance of image singularities and image local minima. The hybrid motion planning strategy consists of an artificial potential field-based global hybrid trajectory planner, where a complete set of Cartesian, image, and robot joint constraints under a complex visual servoing scenario are considered. In this strategy, the image singularities are resolved using the damped-least-square-based joint trajectory planning, while the image local minima are evaluated only along the planned image trajectories and automatically avoided in the image-based trajectory tracking. Two global planning methods are considered. In the first method, the end-effector trajectory is directly planned with respect to the stationary target object frame, which provides a much shorter translational path compared with the local planning method. In the second method, the target trajectory is planned with respect to the current end-effector frame, which minimizes the chances of image trajectories leaving the camera field of view. Simulation and experimental results are given to demonstrate the efficiency of the two hybrid strategies.     

Stochastic re-grasp planning for vision aided capture of deforming and moving object

In the last two decades several researchers have studied the problem of grasping of a moving rigid object based on vision data. However the problem of grasping a moving and deforming object still remains unsolved. In this paper we present the development of a fast algorithm for the computation of the optimal force closure grasp points on a slowly moving and deforming object. The main focus is to find the best grasp points as the object deforms, track its position at a future instant and then transfer grasp at that location. At first the potential grasping configurations satisfying force closure are evaluated through an objective function that maximizes the grasping span while minimizing the distance between the object centroid and the intersection of the fingertip normal. A population based stochastic search strategy is adopted for computing the optimal configurations and re-localizing them as the shape undergoes translations, rotations and scaling. Experiments have been conducted to prove that the object can be tracked in real time and the optimal grasp points determined so that a three finger robot can capture it. This method works in real time so it has great potential for application in industries for grasping objects whose shapes are not clearly defined (e.g. cloth), deforming objects, or objects that are partially occluded.     

Sensor-based design of a Delta parallel robot

This paper proposes a sensor-based design methodology in order to design a Delta robot with guaranteed accuracy performance for a dedicated sensor-based controller. This sensor-based design methodology takes into account the accuracy performance of the controller in the design process in order to find optimal geometric parameters of the robot. Three types of controllers are envisaged to be applied to the Delta robot, leading to three different optimal designs: leg-direction-based visual servoing, line-based visual servoing and image moment visual servoing. Based on these three controllers, positioning error models taking into account the error of observation coming from the camera are developed, and the controller singularities are analyzed. Then, design optimization problems are formulated in order to find the optimal geometric parameters and relevant parameters of the camera for the Delta robot for each type of controller. Prototypes of Delta robots have been manufactured based on the obtained optimum design parameters in order to test the performance of the pair {robot-controller}.     

An experimental approach to robotic grasping using a connectionistarchitecture and generic grasping functions

An experimental approach to robotic grasping is presented. This approach is based on developing a generic representation of grasping rules, which allows learning them from experiments between the object and the robot. A modular connectionist design arranged in subsumption layers is used to provide a mapping between sensory inputs and robot actions. Reinforcement feedback is used to select between different grasping rules and to reduce the number of failed experiments. This is particularly critical for applications in the personal service robot environment. Simulated experiments on a 15-object database show that the system is capable of learning grasping rules for each object in a finite number of experiments as well as generalizing from experiments on one object to grasping from another     

Transferring human grasping synergies to a robot

In this paper, a system for transferring human grasping skills to a robot is presented. In order to reduce the dimensionality of the grasp postures, we extracted three synergies from data on human grasping experiments and trained a neural network with the features of the objects and the coefficients of the synergies. Then, the trained neural network was employed to control robot grasping via an individually optimized mapping between the human hand and the robot hand. As force control was unavailable on our robot hand, we designed a simple strategy for the robot to grasp and hold the objects by exploiting tactile feedback at the fingers. Experimental results demonstrated that the system can generalize the transferred skills to grasp new objects.     

A visual servoing algorithm using fuzzy logics and fuzzy-neural networks

A visual servoing algorithm is proposed for a robot with a camera in the hand to track a moving object in terms of image features and their variations, where fuzzy logics and fuzzy-neural networks are involved to learn feature Jacobian-based kinematic control law. Specifically, novel image features are suggested by employing a viewing model of the perspective projection to estimate the relative pitching and yawing angles. Such perspective projection-based features would not interact with the relative distance between the object and the camera, and, desired feature trajectories for learning the visually guided line-of-sight robot motion are obtained by measuring features by the camera in the hand not in the entire workspace, but on a single linear path along which the robot moves under the control of a commercially provided function of linear motion, and then, control actions of the camera are approximately found by fuzzy-neural networks to follow such desired feature trajectories.To show the validity of the proposed algorithm, some experimental results are illustrated, where a four-axis SCARA robot with a BW CCD camera is used.     

Passivity based visual servoing of mobile robots with dynamics compensation

This paper presents an image-based dynamic visual servoing to make a mobile robot able to track a moving object on the workspace by using a calibrated on board vision system. The stability of the proposed system is proved based on its passivity properties. A robustness analysis and an L₂-gain performance analysis are also presented. Experimental results are shown to illustrate the system performance.     

Visual servoing for constrained planar robots subject to complex friction

The theoretical framework and the experimental validation of a new image-based position-force control for planar robots are presented in this paper. This scheme produces simultaneous convergence of the constrained visual position and the contact force between the end effector and the constraint surface. Camera, robot, and the visual jacobian parameters are considered unknown. This approach is based on a new formulation of the orthogonalization principle used in the robot force control, termed the visual orthogonalization principle. This allows, under the framework of passivity, to yield a synergetic closed-loop system that fuses accordingly camera, encoder, and the force sensor signals. Furthermore, due to the technological limitations, it can be noticed that the visual servoing contact tasks are characterized by slow motion, typically with frequent velocity reversals along the constraint surface, thus, important friction problems arise at the joints and the contact points. Therefore, visual compensation of the complex dynamic joint friction and the viscous contact friction are also studied. A Linux real-time operating-system-based experimental system is implemented to visually drive a constrained direct-drive planar robot manipulator, equipped with six-axes JR3 force sensor and a digital fixed camera, thus proving the effectiveness of the proposed scheme.     

Slipping detection and avoidance based on Kalman filter

The purpose of this paper is to present the latest slipping detection and avoidance algorithms developed by the authors for application in robotic manipulation tasks. Slipping can happen not only in quasi-static conditions such as in grasping tasks but also during dynamic manipulation, therefore the availability of slip control techniques effective in both conditions, such as those proposed here, are essential in real robotic applications. A new algorithm is also proposed to estimate on-line the actual friction coefficient at the contact with the manipulated object by means of a preliminary exploration phase, thus enabling safe manipulation of objects with unknown surface properties. A detailed dynamic simulator is presented and experimentally validated on a mechatronic test bench used for proving the effectiveness of the proposed approach.     

Analyzing of flexible gripper by computational intelligence approach

Adaptive grippers should be able to detect and recognize grasping objects. To be able to do it control algorithm need to be established to control gripper tasks. Compliant underactuated mechanisms with passive behavior can be used for modelling of adaptive robotic fingers. Undearactuation is a feature which allows fully adaptability of robotic fingers for different objects. In this study gripper with two fingers was established. Finite element method (FEM) procedure was used to optimize the gripper structural topology. Kinetostatic model of the underactuated finger mechanism was analyzed. This design of the gripper has embedded sensors as part of its structure. The use of embedded sensors in a robot gripper gives the control system the ability to control input displacement of the gripper and to recognize specific shapes of the grasping objects. Since the conventional control strategy is a very challenging task, soft computing based controllers are considered as potential candidates for such an application. The sensors could be used for grasping shape detection. Given that the contact forces of the finger depend on contact position of the finger and object, it is suitable to make a prediction model for the contact forces in function of contact positions of the finger and grasping objects. The prediction of the contact forces was established by using a soft computing (computational intelligence) approach. To perform the contact forces estimation adaptive neuro-fuzzy (ANFIS) methodology was used. FEM simulations were performed in order to acquire experimental data for ANFIS training. The main goal was to apply ANFIS network in order to find correlation between sensors’ stresses and finger contact forces. Afterwards ANFIS results were compared with benchmark models (extreme learning machine (ELM), extreme learning machine with discrete wavelet algorithm (ELM-WAVELET), support vector machines (SVM), support vector machines with discrete wavelet algorithm (SVM-WAVELET), genetic programming (GP) and artificial neural network (ANN)). The reliability of these computational models was analyzed based on simulation results.     

Vision-based nonlinear tracking controllers with uncertainrobot-camera parameters

This paper considers the problem of position tracking control of planar robot manipulators via visual servoing in the presence of parametric uncertainty associated with the robot mechanical dynamics and/or the camera system. Specifically, by assuming exact knowledge of the mechanical parameters, we design an adaptive camera calibration controller that compensates for uncertain camera parameters and ensures global asymptotic position tracking. We then develop an adaptive robot controller that accounts for parametric uncertainty throughout the entire robot-camera system while producing global asymptotic position tracking. Experimental results illustrating the viability of the adaptive controllers and extensions regarding robust control and redundant robot manipulators are also included     

Extraction of Video Objects via Surface Optimization and Voronoi Order

We implement a video object segmentation system that integrates the novel concept of Voronoi Order with existing surface optimization techniques to support the MPEG-4 functionality of object-addressable video content in the form of video objects. The major enabling technology for the MPEG-4 standard are systems that compute video object segmentation, i.e., the extraction of video objects from a given video sequence. Our surface optimization formulation describes the video object segmentation problem in the form of an energy function that integrates many visual processing techniques. By optimizing this surface, we balance visual information against predictions of models with a priori information and extract video objects from a video sequence. Since the global optimization of such an energy function is still an open problem, we use Voronoi Order to decompose our formulation into a tractable optimization via dynamic programming within an iterative framework. In conclusion, we show the results of the system on the MPEG-4 test sequences, introduce a novel objective measure, and compare results against those that are hand-segmented by the MPEG-4 committee.     

Tracking tasks of massive objects by multiple robot systems with non-firm grasping

The control issue of tracking tasks of massive objects by multiple robot systems is studied in this paper. To consider an actual situation of grasping, instead of the firm grasping postulated in previous works devoted to multiple robot systems, new grasping models are built. After some general mechanical properties are given, control laws for accomplishing the tracking tasks are established. Numerical simulations are made to show the feasibility of the results obtained.