Vision-Based Task-Level Control of a Flexible-Link Manipulator

This paper discusses a vision-based approach to implement task-level control in flexible-link manipulators. The proposed approach emphasizes the advantage of using vision in the control of flexible manipulators. It is pointed out that taking advantage of the inherent robustness, implementation of an image-based visual servo can be regarded as a synthetic solution to precise task-level control of flexible manipulators. This approach is implemented in a three-dimensional flexible-link manipulator. The implementation makes good use of filters in decoupling task-level control and vibration suppression control. Moreover, we point out that although the robustness of the approach can help to overcome the difficulty in control resulting from the complex measurement of the link's elastic deformation, it lacks in capability of tip trajectory specification. This problem is analyzed in this research and it leads to the proposal for the integration of the image interpolation technique. This technique makes the proposed approach adequate for tasks involved with complex tip trajectories. For flexible-link manipulators, the proposed approach with the remedy is the first vision-based synthetic solution that attempts to make a flexible manipulator usable for a practical task.     

Wrist Camera-Based Vibration Suppression Control for a Flexible Manipulator

This paper addresses a vision-based method for estimating vibration excited in the tip of a flexible-link manipulator. In this method, estimation of vibration is achieved by observing the variation of image features projected on a wrist camera. It mimics the situation of utilizing a wrist camera in tip vibration control of a space manipulator. In space, a vision sensor can be expected to be a feasible means for measuring the elastic vibration of the space manipulators, since they are more reliable compared with sensors like strain gauges. The method proposed in this paper takes advantage of the frequential characteristics of visual information that are reflected as a blurred background scene. With the high-frequency component of the projected image features, a Kalman filter-based observer is implemented as the estimator for the vibration. This implementation is characterized by the considerations of incorporating the slow sensor of the camera in the fast servo loop and compensation of the time delay due to image processing. With the vibration estimator, vibration suppression control relying solely on a wrist camera becomes possible. This scheme is successfully verified by experiments.     

Vision-Based Robotic Motion Control for Non-autonomous Environment

A visual servo control system with SOPC structure is implemented on a retrofitted Mitsubishi Movemaster RV-M2 robotic system. The hardware circuit has the functions of quadrature encoder decoding, limit switch detecting, pulse width modulation (PWM) generating and CMOS image signal capturing. The software embedded in Nios II micro processor has the functions of using UART to communicate with PC, robotic inverse kinematics calculation, robotic motion control schemes, digital image processing and gobang game AI algorithms. The digital hardware circuits are designed by using Verilog language, and programs in Nios II micro processor are coded with C language. An Altera Statrix II EP2S60F672C5Es FPGA chip is adopted as the main CPU of the development board. A CMOS color image sensor with 356 ×292 pixels resolution is selected to catch the environment time-varying change for robotic vision-based servo control. The system performance is evaluated by experimental tests. A gobang game is planned to reveal the visual servo robotic motion control objective in non-autonomous environment. Here, a model-free intelligent self-organizing fuzzy control strategy is employed to design the robotic joint controller. A vision based trajectory planning algorithm is designed to calculate the desired angular positions or trajectory on-line of each robotic joint. The experimental results show that this visual servo control robot has reliable control actions.     

Designing Discrete Predictor-Based Controllers for Networked Control Systems with Time-varying Delays: Application to A Visual Servo Inverted Pendulum System

A discrete predictor-based control method is developed for a class of linear time-invariant networked control systems with a sensor-to-controller time-varying delay and a controller-to-actuator uncertain constant delay, which can be potentially applied to vision-based control systems. The control scheme is composed of a state prediction and a discrete predictor-based controller. The state prediction is used to compensate for the effect of the sensor-to-controller delay, and the system can be stabilized by the discrete predictor-based controller. Moreover, it is shown that the control scheme is also robust with respect to slight message rejections. Finally, the main theoretical results are illustrated by simulation results and experimental results based on a networked visual servo inverted pendulum system.      

Hierarchical intelligent control for robotic motion

This paper presents a new scheme for intelligent control of robotic manipulators. This scheme is a hierarchically integrated approach to neuromorphic and symbolic control of robotic manipulators. This includes an applied neural network for servo control and knowledge-based approximation. The neural network in the servo control level is based on a numerical manipulation, while the knowledge based part is symbolic manipulation. The knowledge base part develops control strategies symbolically for the servo level. The neural network compensates for vagueness in the control strategies, nonlinearities of the system and uncertainties in its environment using neuromorphic control.     

Mutational equations for shapes and vision-based control

Basic idea of vision-based control in robotics is to include the vision system directly in the control servo loop of the robot. When images are binary, this problem corresponds to the control of the evolution of a geometric domain. The present paper proposes mathematical tools derived from shape analysis and optimization to study this problem in a quite general way, i.e., without any regularity assumptions or modelsa priori on the domains that we deal with. Indeed, despite the lackness of a vectorial structure, one can develop a differential calculus in the metric space of all non-empty compact subsets of a given domain ofR ⁿ , and adapt ideas and results of classical differential systems to study and control the evolution of geometric domains. For instance, a shape Lyapunov characterization allows to investigate the asymptotic behavior of these geometric domains using the notion of directional shape derivative. We apply this inR ² to the visual servoing problem using the optical flow equations and some experimental simulations illustrate this approach.     

Speed control strategy of dual flexible servo system considering time-varying parameters for flexible manipulator with an axially translating arm

Many factors lead to fluctuations of output speed in the motor side during the movement processes in a flexible manipulator with an axially translating arm. These factors include the time-varying characteristics of manipulator parameters, external disturbance, and flexibility in the manipulator. The vibration of flexible manipulators is strengthened by fluctuations of output speed, which seriously affects the motion accuracy. Therefore, a variable parameters proportional–integral (PI) control strategy based on disturbance observer is adopted to reduce fluctuations of output speed in a dual flexible servo system. First, the dynamic model of the dual flexible servo system is established by the assumed mode method. Then, the variable parameters PI control strategy is applied to the servo system, and the controller parameters in different conditions are selected by pole placement strategies. Finally, numerical simulations and control experiments of the servo system are carried out. The results show that the control strategy using the variable parameters and disturbance observer could reduce fluctuations of output speed and improve the motion accuracy of flexible manipulators.     

机器人视觉伺服研究进展:视觉系统与控制策略

视觉伺服控制是机器人系统的重要控制手段. 随着机器人应用需求的日益复杂多样,视觉伺服的研究面临着挑战. 视觉伺服系统的设计主要包括视觉系统、控制策略和实现策略三个方面. 文中对视觉伺服中存在的主要问题进行了分析,重点介绍了视觉系统中改善动态性能和处理噪声的主要技术手段,阐述了处理模型不确定性和约束的控制策略的改进方案,总结了提高视觉伺服系统的可实现性和灵活性的实现策略. 最后,基于当前的研究进展对未来的研究方向进行了展望.     

基于视觉的目标定位技术的研究进展

基于视觉的目标定位是目前计算机视觉领域的研究热点。就现阶段基于视觉的定位技术进行了综述,着重介绍了基于单目视觉定位、基于双目视觉定位和基于全景视觉定位3类定位技术的研究现状,并分析了各自的优缺点。最后就基于视觉定位方法的发展趋势做了简要分析,以期为以后基于视觉定位问题的研究提供参考。     

GA-pattern matching-based manipulator control system for real-time visual servoing

_—In robotic applications, tasks of picking and placing are the most fundamental ones. Also, for a robot manipulator, the recognition of its working environment is one of the most important issues to do intelligent tasks, since this aptitude enables it to work in a variable environment. This paper presents a new control strategy for robot manipulators, which utilizes visual information to direct the manipulator in its working space, to pick up an object of known shape, but with arbitrary position and orientation. During the search for an object to be picked up, vision-based control by closed-loop feedback, referred to as visual servoing, is performed to obtain the motion control of the manipulator hand. The system employs a genetic algorithm (GA) and a pattern matching technique to explore the search space and exploit the best solutions by this search technique. The control strategy utilizes the found results of GA-pattern matching in every step of GA evolution to direct the manipulator towards the target object. We named this control strategy step-GA-evnlution. This control method can be applied for manipulator real-time visual servoing and solve its path planning problem in real-time, i.e. in order for the manipulator to adapt the execution of the task by visual information during the process execution. Simulations have been performed, using a two-link planar manipulator and three image models, in order to find which one is the best for real-time visual servoing and the results show the effectiveness of the control method.     

Path Planning for Improved Visibility Using a Probabilistic Road Map

This paper focuses on the challenges of vision-based motion planning for industrial manipulators. Our approach is aimed at planning paths that are within the sensing and actuation limits of industrial hardware and software. Building on recent advances in path planning, our planner augments probabilistic road maps with vision-based constraints. The resulting planner finds collision-free paths that simultaneously avoid occlusions of an image target and keep the target within the field of view of the camera. The planner can be applied to eye-in-hand visual-target-tracking tasks for manipulators that use point-to-point commands with interpolated joint motion.      

机械臂轨迹跟踪控制研究进展

综述了近年来刚性机械臂轨迹跟踪控制研究领域的最新进展.根据应用于机械臂的不同控制算法进行分类,从自适应PID控制、神经网络自适应控制、模糊自适应控制、滑模变结构控制和鲁棒自适应控制5种主要控制方法进行阐述.重点从关节空间出发,论述了各种控制算法在提高机械臂轨迹跟踪性能方面的各自优缺点,并分析了它们之间的相互联系.对机械...     

Adaptive visual servoing for the robot manipulator with extreme learning machine and reinforcement learning

In this study, a novel image-based visual servo (IBVS) controller for robot manipulators is investigated using an optimized extreme learning machine (ELM) algorithm and an offline reinforcement learning (RL) algorithm. First of all, the classical IBVS method and its difficulties in accurately estimating the image interaction matrix and avoiding the singularity of pseudo-inverse are introduced. Subsequently, an IBVS method based on ELM and RL is proposed to solve the problem of the singularity of the pseudo-inverse solution and tune adaptive servo gain, improving the servo efficiency and stability. Specifically, the ELM algorithm optimized by particle swarm optimization (PSO) was used to approximate the pseudo-inverse of the image interaction matrix to reduce the influence of camera calibration errors. Then, the RL algorithm was adopted to tune the adaptive visual servo gain in continuous space and improve the convergence speed. Finally, comparative simulation experiments on a 6-DOF robot manipulator were conducted to verify the effectiveness of the proposed IBVS controller.     

HMM-based state classification of a user with a walking support system using visual PCA features

The improvement of safety and dependability in systems that physically interact with humans requires investigation with respect to the possible states of the user’s motion and an attempt to recognize these states. In this study, we propose a method for real-time visual state classification of a user with a walking support system. The visual features are extracted using principal component analysis and classification is performed by hidden Markov models, both for real-time fall detection (one-class classification) and real-time state recognition (multi-class classification). The algorithms are used in experiments with a passive-type walker robot called “RT Walker” equipped with servo brakes and a depth sensor (Microsoft Kinect). The experiments are performed with 10 subjects, including an experienced physiotherapist who can imitate the walking pattern of the elderly and people with disabilities. The results of the state classification can be used to improve fall-prevention control algorithms for walking support systems. The proposed method can also be used for other vision-based classification applications, which require real-time abnormality detection or state recognition.     

A review on vision-based control of flexible manipulators

In this review, recent developments in the field of flexible robot arm control using visual servoing are reviewed. In comparison to rigid robots, the end-effector position of flexible links cannot be obtained precisely enough with respect to position control using kinematic information and joint variables. To solve the task here the use of a vision sensor (camera) system, visual servoing is proposed to realize the task of control flexible manipulators with improved quality requirements. The paper is organized as follows: the visual servoing architectures will be reviewed for rigid robots first. The advantages, disadvantages, and comparisons between different approaches of visual servoing are carried out. The using of visual servoing to control flexible robot is addressed next. Open problems such as state variables estimation as well as the combination of different sensor properties as well as some application-oriented points related to flexible robot are discussed in detail.     

基于眼到手视觉伺服的移动机器人模型预测控制

针对具有速度约束的移动机器人视觉轨迹跟踪问题,提出了一种基于LOQO内点法的模型预测控制方法;在眼到手框架下,首先建立了移动机器人误差模型,并对该误差模型进行离散化,给出了移动机器人视觉伺服跟踪的代价函数;同时考虑到实际中移动机器人存在速度约束问题,将代价函数的最小化问题转换为带输入约束的模型预测控制问题;然后采用障碍函数法将移动机器人的速度约束转化为等式约束并采用拉格朗日乘子法引入到代价函数中;进而,利用LOQO内点法求解具有速度约束的最小化问题,得到基于视觉的轨迹跟踪控制器;最后,通过仿真验证了所提算法的有效性和优越性.     

基于虚拟现实技术的移动机器人视觉伺服控制系统设计

为了有效确保移动机器人视觉伺服控制效果,提高移动机器人视觉伺服控制精度,设计了基于虚拟现实技术的移动机器人视觉伺服控制系统。通过三维视觉传感器和立体显示器等虚拟环境的I/O设备、位姿传感器、视觉图像处理器以及伺服控制器元件,完成系统硬件设计。从运动学和动力学两个方面,搭建移动机器人数学模型,利用标定的视觉相机,生成移动机器人实时视觉图像,通过图像滤波、畸变校正等步骤,完成图像的预处理。利用视觉图像,构建移动机器人虚拟移动环境。在虚拟现实技术下,通过目标定位、路线生成、碰撞检测、路线调整等步骤,规划移动机器人行动路线,通过控制量的计算,实现视觉伺服控制功能。系统测试结果表明,所设计控制系统的位置控制误差较小,姿态角和移动速度控制误差仅为0.05°和0.12m/s,移动机器人碰撞次数较少,具有较好的移动机器人视觉伺服控制效果,能够有效提高移动机器人视觉伺服控制精度。     

机器人实时视觉伺服系统控制方案的研究

本文针对机器人实时视觉伺服系统,考虑了机器人关节伺服反馈与视觉伺服反馈之间的相互联系,提出了实时控制方案,设计了合理的关节饲服控制器与视觉伺服控制器,以达到改善系统性能的目的。试验结果表明,当采用上述两个控制器时,系统的动静态性能得到了明显的改善。     

Predictive control architecture for real-time image moments based servoing of robot manipulators

This paper presents a real-time architecture for visual servoing of robot manipulators using nonlinear based predictive control. In order to increase the robustness of the control algorithm, image moments were chosen to be the visual features which describe the objects from the image. A visual predictive architecture is designed to solve tasks addressed to robot manipulators with an eye-in-hand configuration. An implementation of the proposed architecture is done so that the capabilities of a 6 d.o.f robot manipulator are extended. The results of different experiments conducted with two types of image moments based controllers (proportional and predictive with reference trajectory) are presented and discussed.     

Markerless human–robot interface for dual robot manipulators using Kinect sensor

Remote teleoperation of robot manipulators is often necessary in unstructured, dynamic, and dangerous environments. However, the existing mechanical and other contacting interfaces require unnatural, or hinder natural, human motions. At present, the contacting interfaces used in teleoperation for multiple robot manipulators often require multiple operators. Previous vision-based approaches have only been used in the remote teleoperation for one robot manipulator as well as require the special quantity of illumination and visual angle that limit the field of application. This paper presents a noncontacting Kinect-based method that allows a human operator to communicate his motions to the dual robot manipulators by performing double hand–arm movements that would naturally carry out an object manipulation task. This paper also proposes an innovative algorithm of over damping to solve the problem of error extracting and dithering due to the noncontact measure. By making full use of the human hand–arm motion, the operator would feel immersive. This human–robot interface allows the flexible implementation of the object manipulation task done in collaboration by dual robots through the double hand–arm motion by one operator.