Intelligent visual servoing with extreme learning machine and fuzzy logic

While visual servoing (VS) provides the ability of motion using vision for robot manipulators, the approaches for a better VS have to deal with three common problems: obtaining the interaction matrix and its pseudoinverse for defined feature points, finding an appropriate gain value for the VS controller and keeping the features in the field of view (FOV) for VS permanency.In this study, a new intelligent image-based visual servoing (IBVS) system for eye-in-hand configured robot manipulators using extreme learning machine (ELM) and fuzzy logic (FL) is proposed to solve these common problems of VS in a single system. As the first stage of the system, the pseudoinverse of the interaction matrix is approximated using trained ELMs which do not need hidden layer tuning. As the second stage, the classical IBVS controller is modified by a differential equation regarding initial velocity continuity and an appropriate gain in each loop is assigned by an FL unit to provide fast convergence within velocity limits. This unit also promotes manipulability of the manipulator to avoid singularities. As the last stage of the proposed system, regions are defined in the image plane to take precautions before feature missing. When a feature comes close to the edge of a restricted region, an FL unit is activated to obtain negative linear velocities in x and y direction which will be added to the instant velocities to drag the features towards the center of the FOV. In addition to these abilities, some VS metrics are redefined analytically to standardize the performance metric definitions of VS. To show the performance of the proposed system, simulation results of the classical and the proposed IBVS system under practical disturbances are presented for visual servoing of a Puma 560 arm. The advantages of singular matrix and joint configuration avoidance, adaptive gain with smooth gain surface, decreased convergence time within velocity limits, initial velocity continuity, FOV keeping with smooth velocity assurance, redefined VS metrics for standardization and robustness against disturbances are proved by variety of simulations. The simulation results also verify that the proposed system utilizing intelligent methods like ELM and FL is capable of dealing with common problems of VS and achieves sufficient results in terms of VS metrics.     

A modified image-based visual servo controller with hybrid camera configuration for robust robotic grasping

In order to develop an autonomous mobile manipulation system that works in an unstructured environment, a modified image-based visual servo (IBVS) controller using hybrid camera configuration is proposed in this paper. In particular, an eye-in-hand web camera is employed to visually track the target object while a stereo camera is used to measure the depth information online. A modified image-based controller is developed to utilize the information from the two cameras. In addition, a rule base is integrated into the visual servo controller to adaptively tune its gain based on the image deviation data so as to improve the response speed of the controller. A physical mobile manipulation system is developed and the developed IBVS controller is implemented. The experimental results obtained using the systems validate the developed approach.     

Image-based visual servoing using improved image moments in 6-DOF robot systems

This paper addresses the challenges of choosing proper image features for planar symmetric shape objects and designing visual servoing controller to enhance the tracking performance in image-based visual servoing (IBVS). Six image moments are chosen as the image features and the analytical image interaction matrix related to the image features are derived. A controller is designed to efficiently increase the robustness of the visual servoing system. Experimental results on a 6-DOF robot visual servoing system are provided to illustrate the effectiveness of the proposed method.     

Visual attention servo control for task-specific robotic applications

This paper proposes a visual attention servo control (VASC) method which uses the Gaussian mixture model (GMM) for task-specific applications of mobile robots. In particular, low dimensional bias feature template is obtained using GMM to get an efficient attention process. An image-based visual servo (IBVS) controller is used to search for a desired object in a scene through an attention system which forms a task-specific state representation of the environment. First, task definition and object representation in semantic memory (SM) are proposed, and bias feature template is obtained using GMM deduction for features from high dimension to low dimension. Second, the features intensity, color, size and orientation are extracted to build the feature set. Mean shift method is used to segment the visual scene into discrete proto-objects. Given a task-specific object, top-down bias attention is evaluated to generate the saliency map by combining with the bottom-up saliency-based attention. Third, a visual attention servo controller is developed to integrate the IBVS controller and the attention system for robotic cognitive control. A rule-based arbitrator is proposed to switch between the episodic memory (EM)-based controller and the IBVS controller depending on whether the robot obtains the desired attention point on the image. Finally, the proposed method is evaluated on task-specific object detection under different conditions and visual attention servo tasks. The obtained results validate the applicability and usefulness of the developed method for robotics.     

Supervised fuzzy reinforcement learning for robot navigation

This paper addresses a new method for combination of supervised learning and reinforcement learning (RL). Applying supervised learning in robot navigation encounters serious challenges such as inconsistent and noisy data, difficulty for gathering training data, and high error in training data. RL capabilities such as training only by one evaluation scalar signal, and high degree of exploration have encouraged researchers to use RL in robot navigation problem. However, RL algorithms are time consuming as well as suffer from high failure rate in the training phase. Here, we propose Supervised Fuzzy Sarsa Learning (SFSL) as a novel idea for utilizing advantages of both supervised and reinforcement learning algorithms. A zero order Takagi–Sugeno fuzzy controller with some candidate actions for each rule is considered as the main module of robot's controller. The aim of training is to find the best action for each fuzzy rule. In the first step, a human supervisor drives an E-puck robot within the environment and the training data are gathered. In the second step as a hard tuning, the training data are used for initializing the value (worth) of each candidate action in the fuzzy rules. Afterwards, the fuzzy Sarsa learning module, as a critic-only based fuzzy reinforcement learner, fine tunes the parameters of conclusion parts of the fuzzy controller online. The proposed algorithm is used for driving E-puck robot in the environment with obstacles. The experiment results show that the proposed approach decreases the learning time and the number of failures; also it improves the quality of the robot's motion in the testing environments.     

Further results on adaptive iterative learning control of robot manipulators

Based on a combination of a PD controller and a switching type two-parameter compensation force, an iterative learning controller with a projection-free adaptive algorithm is presented in this paper for repetitive control of uncertain robot manipulators. The adaptive iterative learning controller is designed without any a priori knowledge of robot parameters under certain properties on the dynamics of robot manipulators with revolute joints only. This new adaptive algorithm uses a combined time-domain and iteration-domain adaptation law allowing to guarantee the boundedness of the tracking error and the control input, in the sense of the infinity norm, as well as the convergence of the tracking error to zero, without any a priori knowledge of robot parameters. Simulation results are provided to illustrate the effectiveness of the learning controller.     

Uncalibrated visual servoing of robots using a depth-independent interaction matrix

This paper presents a new adaptive controller for image-based dynamic control of a robot manipulator using a fixed camera whose intrinsic and extrinsic parameters are not known. To map the visual signals onto the joints of the robot manipulator, this paper proposes a depth-independent interaction matrix, which differs from the traditional interaction matrix in that it does not depend on the depths of the feature points. Using the depth-independent interaction matrix makes the unknown camera parameters appear linearly in the closed-loop dynamics so that a new algorithm is developed to estimate their values on-line. This adaptive algorithm combines the Slotine-Li method with on-line minimization of the errors between the real and estimated projections of the feature points on the image plane. Based on the nonlinear robot dynamics, we prove asymptotic convergence of the image errors to zero by the Lyapunov theory. Experiments have been conducted to verify the performance of the proposed controller. The results demonstrated good convergence of the image errors.     

Reinforcement learning and optimal adaptive control: An overview and implementation examples

This paper provides an overview of the reinforcement learning and optimal adaptive control literature and its application to robotics. Reinforcement learning is bridging the gap between traditional optimal control, adaptive control and bio-inspired learning techniques borrowed from animals. This work is highlighting some of the key techniques presented by well known researchers from the combined areas of reinforcement learning and optimal control theory. At the end, an example of an implementation of a novel model-free Q-learning based discrete optimal adaptive controller for a humanoid robot arm is presented. The controller uses a novel adaptive dynamic programming (ADP) reinforcement learning (RL) approach to develop an optimal policy on-line. The RL joint space tracking controller was implemented for two links (shoulder flexion and elbow flexion joints) of the arm of the humanoid Bristol-Elumotion-Robotic-Torso II (BERT II) torso. The constrained case (joint limits) of the RL scheme was tested for a single link (elbow flexion) of the BERT II arm by modifying the cost function to deal with the extra nonlinearity due to the joint constraints.     

Model Predictive Control for Constrained Image‐Based Visual Servoing in Uncalibrated Environments

This paper presents a novel approach for image‐based visual servoing (IBVS) of a robotic system by considering the constraints in the case when the camera intrinsic and extrinsic parameters are uncalibrated and the position parameters of the features in 3‐D space are unknown. Based on the model predictive control method, the robotic system's input and output constraints, such as visibility constraints and actuators limitations, can be explicitly taken into account. Most of the constrained IBVS controllers use the traditional image Jacobian matrix, the proposed IBVS scheme is developed by using the depth‐independent interaction matrix. The unknown parameters can appear linearly in the prediction model and they can be estimated by the identification algorithm effectively. In addition, the model predictive control determines the optimal control input and updates the estimated parameters together with the prediction model. The proposed approach can simultaneously handle system constraints, unknown camera parameters and depth parameters. Both the visual positioning and tracking tasks can be achieved desired performances. Simulation results based on a 2‐DOF planar robot manipulator for both the eye‐in‐hand and eye‐to‐hand camera configurations are used to demonstrate the effectiveness of the proposed method.     

一种基于Dyna-Q学习的旋翼无人机视觉伺服智能控制方法

基于图像的视觉伺服机器人控制方法通过机器人的视觉获取图像信息,然后形成基于图像信息的闭环反馈来控制机器人的合理运动.经典视觉伺服的伺服增益的选取在大多数条件下是人工赋值的,故存在鲁棒性差、收敛速度慢等问题.针对该问题,提出一种基于Dyna-Q的旋翼无人机视觉伺服智能控制方法调节伺服增益以提高其自适应性.首先,使用基于费尔曼链码的图像特征提取算法提取目标特征点;然后,使用基于图像的视觉伺服形成特征误差的闭环控制;其次,针对旋翼无人机强耦合欠驱动的动力学特性提出一种解耦的视觉伺服控制模型;最后,建立使用Dyna-Q学习调节伺服增益的强化学习模型,通过训练可以使得旋翼无人机自主选择伺服增益.Dyna-Q学习在经典的Q学习的基础上通过建立环境模型来存储经验,环境模型产生的虚拟样本可以作为学习样本来进行值函数的迭代.实验结果表明,所提出的方法相比于传统控制方法PID控制以及经典的基于图像视觉伺服方法具有收敛速度快、稳定性高的优势.     

An approach to learning mobile robot navigation

This paper describes an approach to learning an indoor robot navigation task through trial-and-error. A mobile robot, equipped with visual, ultrasonic and laser sensors, learns to servo to a designated target object. In less than ten minutes of operation time, the robot is able to navigate to a marked target object in an office environment. The central learning mechanism is the explanation-based neural network learning algorithm (EBNN). EBNN initially learns function purely inductively using neural network representations. With increasing experience, EBNN employs domain knowledge to explain and to analyze training data in order to generalize in a more knowledgeable way. Here EBNN is applied in the context of reinforcement learning, which allows the robot to learn control using dynamic programming.     

基于ai深度学习的机器人碰撞预估计控制器设计

机器人运动过程中与外部障碍物之间容易发生碰撞,当碰撞作用力过大时会造成机器零件损坏的问题,为解决这一问题,设计基于ai深度学习的机器人碰撞预估计控制器。建立人机交互电路与串口通信电路,将伺服电机设备、运动控制器、PC感应装置分别接入既定作用区域内,完成预估计控制器的整体应用结构设计。以PyTorch深度学习框架为基础,定义激活函数,再根据预估计参数的实际取值范围,实现对目标机器人对象的精准检测。按照力矩控制条件表达式,确定碰撞行为的表现强度,完成对机器人运动路径的规划,联合相关应用设备,实现基于ai深度学习的机器人碰撞预估计控制器设计。实验结果表明,ai深度学习算法作用下,机器人与障碍物碰撞部位的接触面积不会超过0.25m2,由碰撞行为导致的外部作用力相对较小,不会造成严重的机器零件损坏问题。     

基于模糊神经网络的强化学习及其在机器人导航中的应用

研究基于行为的移动机器人控制方法.将模糊神经网络与强化学习理论相结合,构成模糊强化系统.它既可获取模糊规则的结论部分和模糊隶属度函数参数,也可解决连续状态空间和动作空间的强化学习问题.将残差算法用于神经网络的学习,保证了函数逼近的快速性和收敛性.将该系统的学习结果作为反应式自主机器人的行为控制器,有效地解决了复杂环境中的机器人导航问题.     

基于任务函数方法的机器人视觉伺服特性研究

为实现在统一的理论框架下对机器人视觉伺服基础特性进行细致深入的研究,本文基于任务函数方法,建立了广义的视觉伺服系统模型.在此模型基础之上,重点研究了基于位置的视觉伺服(PBVS)与基于图像的视觉伺服(IBVS)方法在笛卡尔空间和图像空间的动态特性.仿真结果表明,在相同的比较框架结构下,PBVS方法同样对摄像机标定误差具有鲁棒性.二者虽然在动态系统的稳定性、收敛性方面相类似,但是在笛卡尔空间和图像空间的动态性能上却有很大的差别.对于PBvS方法,笛卡尔轨迹可以保证最短路径,但是对应的图像轨迹是不可控的,可能会发生逃离视线的问题;对于IBVS方法,图像空间虽然能保证最短路径,但是由于缺乏笛卡尔空间的直接控制,在处理大范围旋转伺服的情况时,会发生诸如摄像机退化的笛卡尔轨迹偏移现象.     

移动机器人自适应视觉伺服镇定控制

对有单目视觉的移动机器人系统,提出了一种自适应视觉伺服镇定控制算法;在缺乏深度信息传感器并且摄像机外参数未知的情况下,该算法利用视觉反馈实现了移动机器人位置和姿态的渐近稳定.由于机器人坐标系与摄像机坐标系之间的平移外参数(手眼参数)是未知的,本文利用静态特征点的位姿变化特性,建立移动机器人在摄像机坐标系下的运动学模型.然后,利用单应矩阵分解的方法得到了可测的角度误差信号,并结合2维图像误差信号,通过一组坐标变换,得到了系统的开环误差方程.在此基础之上,基于Lyapunov稳定性理论设计了一种自适应镇定控制算法.理论分析、仿真与实验结果均证明了本文所设计的单目视觉控制器在摄像机外参数未知的情况下,可以使移动机器人渐近稳定到期望的位姿.     

基于流形学习的极端学习机及其在人脸识别中的应用

极端学习机以其快速高效和良好的泛化能力在模式识别领域得到了广泛应用,然而现有的ELM及其改进算法并没有充分考虑到数据维数对ELM分类性能和泛化能力的影响,当数据维数过高时包含的冗余属性及噪音点势必降低ELM的泛化能力,针对这一问题本文提出一种基于流形学习的极端学习机,该算法结合维数约减技术有效消除数据冗余属性及噪声对ELM分类性能的影响,为验证所提方法的有效性,实验使用普遍应用的图像数据,实验结果表明本文所提算法能够显著提高ELM的泛化性能。     

无奇异间接迭代学习控制及其在机器人运动模仿中的应用

针对相当广泛的一类非线性系统有限时间轨迹跟踪问题,提出了间接迭代学习方案. 采用最小二乘算法,根据重复跟踪历史辨识非线性系统的线性化模型.利用一个分段学习方案 可保证学习控制总在有效线性近似区域内进行.探讨了如何在学习过程中避免控制奇异问题, 提出了一种高效的参数修正方法,保证输入耦合矩阵的估计行列式不为零.本文将这一控制方 案应用于未知机器人及摄像机模型下的机器人运动模仿中,而不面临任何奇异问题.这是一个 采用摄像机替代传统程序编写的新的机器人编程方法.     

基于自适应动量优化算法的正则化极限学习机

针对极限学习机(extreme learning machine,ELM)隐节点不确定性导致的系统不稳定,以及对大型数据计算负担过重的问题,提出了基于自适应动量优化算法(adaptive and momentum method,AdaMom)的正则化极限学习机.算法主要思想是构造连续可微的目标函数,在梯度下降过程中计算自适应学习率,求自适应学习率与梯度乘积的指数加权平均值,通过迭代得到损失函数最小值对应的隐层输出权重矩阵.实验结果表明,在相同基准数据集的训练中,AdaMom-ELM算法具有非常良好的泛化性能和鲁棒性,提高了计算效率.     

基于改进极限学习机算法的行为识别

重点研究了极限学习机ELM对行为识别检测的效果。针对在线学习和行为分类上存在计算复杂性和时间消耗大的问题,提出了一种新的行为识别学习算法(ELM-Cholesky)。该算法首先引入了基于Cholesky分解求ELM的方法,接着依据在线学习期间核函数矩阵的更新特点,将分块矩阵Cholesky分解算法用于ELM的在线求解,使三角因子矩阵实现在线更新,从而得出一种新的ELM-Cholesky在线学习算法。新算法充分利用了历史训练数据,降低了计算的复杂性,提高了行为识别的准确率。最后,在基准数据库上采用该算法进行了大量实验,实验结果表明了这种在线学习算法的有效性。     

基于鸽群的鲁棒强化学习算法

强化学习是一种人工智能算法,具有计算逻辑清晰、模型易扩展的优点,可以在较少甚至没有先验信息的前提下,通过和环境交互并最大化值函数,调优策略性能,有效地降低物理模型引起的复杂性。基于策略梯度的强化学习算法目前已成功应用于图像智能识别、机器人控制、自动驾驶路径规划等领域。然而强化学习高度依赖采样的特性决定了其训练过程需要大量样本来收敛,且决策的准确性易受到与仿真环境中不匹配的轻微干扰造成严重影响。特别是当强化学习应用于控制领域时,由于无法保证算法的收敛性,难以对其稳定性进行证明,为此,需要对强化学习进行改进。考虑到群体智能算法可通过群体协作解决复杂问题,具有自组织性及稳定性强的特征,利用其对强化学习进行优化求解是一个提高强化学习模型稳定性的有效途径。结合群体智能中的鸽群算法,对基于策略梯度的强化学习进行改进：针对求解策略梯度时存在迭代求解可能无法收敛的问题,提出了基于鸽群的强化学习算法,以最大化未来奖励为目的求解策略梯度,将鸽群算法中的适应性函数和强化学习结合估计策略的优劣,避免求解陷入死循环,提高了强化学习算法的稳定性。在具有非线性关系的两轮倒立摆机器人控制系统上进行仿真验证,实验结果表...