Neural networks for control in an artificial brain of a recognition and tracking system

This paper presents a new information-processing machine which is called the artificial brain (ABrain). It also considers the structure of artificial neural networks constructed in a Ricoh neurocomputer RN-2000 in the ABrain to track given trajectories which are produced in a micro-computer or by a light moved by hand in a recognition and tracking system. This work was presented, in part, at the Second International Symposium on Artificial Life and Robotics, Oita Japan, February 18–20, 1997     

Neurocomputer control in an artificial brain for tracking moving objects

We developed a new control technique for tracking a moving object using a neurocomputer. The control is produced by the RICOH neurocomputer RN-2000, which is able to learn various control laws instantly, in order to track a moving object within a predetermined range of errors. The system for tracking consists of a new information processing system which is a primitive artificial brain (denoted the ABrain). This paper descrbes the new neurocomputer control technique used in the primitive ABrain and presents the results obtained from the tracking experiments. This work was presented, in part, at International Symposium on Artificial Life and Robotics, Oita, Japan, February 18–20, 1996     

一种智能机器鼠跟踪与交互策略

针对一种智能机器鼠对目标生物鼠的跟踪与行为交互问题,利用智能机器鼠的双目立体视觉系统,对目标生物鼠进行跟踪与定位.引入基于Kalman滤波的改进Camshift算法,实现智能机器鼠对目标生物鼠的实时跟踪;设计基于SVM的智能机器鼠认知系统,实现系统对目标生物鼠的行为识别、行为分类和智能交互,从而提出一种基于仿生学的智能机器鼠跟踪与交互策略.实验结果表明,所提出的跟踪与交互策略能够实现智能机器鼠对目标生物鼠的跟踪与智能交互,从而证实了所提策略的可行性和有效性.该研究可以有效地为生物学家、神经科学家和脑科学研究者等提供研究平台和辅助研究工具.     

Intelligent control based on wavelet decomposition and neural network for predicting of human trajectories with a novel vision-based robotic

In this paper, an intelligent novel vision-based robotic tracking model is developed to predict the performance of human trajectories with a novel vision-based robotic tracking system. The developed model is based on wavelet packet decomposition, entropy and neural network. We represent an implementation of a novel vision-based robotic tracking system based on wavelet decomposition and artificial neural (WD-ANN) which can track desired human trajectory pattern in real environments. The input–output data set of the novel vision-based robotic tracking system were first stored and than these data sets were used to predict the robotic tracking based on WD-ANN. In simulations, performance measures were obtained to compare the predicted and human–robot trajectories like actual values for model validation. In statistical analysis, the RMS value is 0.0729 and the R² value is 99.76% for the WD-ANN model. This study shows that the values predicted with the WD-ANN can be used to predict human trajectory by vision-based robotic tracking system quite accurately. All simulations have shown that the proposed method is more effective and controls the systems quite successful.     

Robotic Knee Tracking Control to Mimic the Intact Human Knee Profile Based on Actor-Critic Reinforcement Learning

We address a state-of-the-art reinforcement learning (RL) control approach to automatically configure robotic prosthesis impedance parameters to enable end-to-end, continuous locomotion intended for transfemoral amputee subjects. Specifically, our actor-critic based RL provides tracking control of a robotic knee prosthesis to mimic the intact knee profile. This is a significant advance from our previous RL based automatic tuning of prosthesis control parameters which have centered on regulation control with a designer prescribed robotic knee profile as the target. In addition to presenting the tracking control algorithm based on direct heuristic dynamic programming (dHDP), we provide a control performance guarantee including the case of constrained inputs. We show that our proposed tracking control possesses several important properties, such as weight convergence of the learning networks, Bellman (sub) optimality of the cost-to-go value function and control input, and practical stability of the human-robot system. We further provide a systematic simulation of the proposed tracking control using a realistic human-robot system simulator, the OpenSim, to emulate how the dHDP enables level ground walking, walking on different terrains and at different paces. These results show that our proposed dHDP based tracking control is not only theoretically suitable, but also practically useful.      

Shape Morphing-Based Control of Robotic Visual Servoing

We present an approach for controlling robotic interactions with objects, using synthetic images generated by morphing shapes. In particular, we attempt the problem of positioning an eye-in-hand robotic system with respect to objects in the workspace for grasping and manipulation. In our formulation, the grasp position (and consequently the approach trajectory of the manipulator), varies with each object. The proposed solution to the problem consists of two parts. First, based on a model-based object recognition framework, images of the objects taken at the desired grasp pose are stored in a database. The recognition and identification of the grasp position for an unknown input object (selected from the family of recognizable objects) occurs by morphing its contour to the templates in the database and using the virtual energy spent during the morph as a dissimilarity measure. In the second step, the images synthesized during the morph are used to guide the eye-in-hand system and execute the grasp. The proposed method requires minimal calibration of the system. Furthermore, it conjoins techniques from shape recognition, computer graphics, and vision-based robot control in a unified engineering amework. Potential applications range from recognition and positioning with respect to partially-occluded or deformable objects to planning robotic grasping based on human demonstration.     

Cooperative passers-by tracking with a mobile robot and external cameras

In this paper, we present a cooperative passers-by tracking system between fixed view wall mounted cameras and a mobile robot. The proposed system fuses visual detections from wall mounted cameras and detections from a mobile robot–in a centralized manner–employing a “tracking-by-detection” approach within a Particle Filtering strategy. This tracking information is then used to endow the robot with passers-by avoidance ability to facilitate its navigation in crowds during the execution of a person following mission. The multi-person tracker’s ability to track passers-by near the robot distinctively is demonstrated through qualitative and quantitative off-line experiments. Finally, the designed perceptual modalities are deployed on our robotic platform, controlling its actuators via visual servoing techniques and free space diagrams in the vicinity of the robot, to illustrate the robot’s ability to follow a given target person in human crowded areas.     

Modeling and Adaptive Neural Network Control for a Soft Robotic Arm With Prescribed Motion Constraints

This paper presents a dynamic model and performance constraint control of a line-driven soft robotic arm. The dynamics model of the soft robotic arm is established by combining the screw theory and the Cosserat theory. The unmodeled dynamics of the system are considered, and an adaptive neural network controller is designed using the backstepping method and radial basis function neural network. The stability of the closed-loop system and the boundedness of the tracking error are verified using Lyapunov theory. The simulation results show that our approach is a good solution to the motion constraint problem of the line-driven soft robotic arm.      

The ManyEars open framework

ManyEars is an open framework for microphone array-based audio processing. It consists of a sound source localization, tracking and separation system that can provide an enhanced speaker signal for improved speech and sound recognition in real-world settings. ManyEars software framework is composed of a portable and modular C library, along with a graphical user interface for tuning the parameters and for real-time monitoring. This paper presents the integration of the ManyEars Library with Willow Garage’s Robot Operating System. To facilitate the use of ManyEars on various robotic platforms, the paper also introduces the customized microphone board and sound card distributed as an open hardware solution for implementation of robotic audition systems.     

Reflective workpiece detection and localization for flexible robotic cells

A smart vision system for industrial robotic cells is presented. It can recognize and localize a reflective workpiece, and allows for automatic adjustments of the robot program. The purpose of the study is to improve industrial robots awareness of the environment and to increase adaptability of the manufacturing processes where full control over environment is not achievable. This approach is particularly relevant to small batch robotic production, often suffering from only partial control of the process parameters, such as the order of jobs, workpiece position, or illumination conditions.A distinguishing aspect of the study is detection of workpieces made of diverse materials, including shiny metals. Reflective surfaces are common in the industrial manufacturing, but are rarely considered in the research on object recognition because they hinder many of the object recognition algorithms. The proposed solution has been qualified and tested on a selected benchmark in a realistic workshop environment with artificial light conditions. The training of the object recognition software is an automatic process and can be executed by non-expert industrial users to allow for recognition of different types of objects.     

Input Displacement Neuro-fuzzy Control and Object Recognition by Compliant Multi-fingered Passively Adaptive Robotic Gripper

The requirement for new flexible adaptive grippers is the ability to detect and recognize objects in their environments. It is known that robotic manipulators are highly nonlinear systems, and an accurate mathematical model is difficult to obtain, thus making it difficult make decision strategies using conventional techniques. Here, an adaptive neuro fuzzy inference system (ANFIS) for controlling input displacement and object recognition of a new adaptive compliant gripper is presented. The grasping function of the proposed adaptive multi-fingered gripper relies on the physical contact of the finger with an object. This design of the each finger has embedded sensors as part of its structure. The use of embedded sensors in a robot gripper gives the control system the ability to control input displacement of the gripper and to recognize particular shapes of the grasping objects. Fuzzy based controllers develop a control signal according to grasping object shape which yields on the firing of the rule base. The selection of the proper rule base depending on the situation can be achieved by using an ANFIS strategy, which becomes an integrated method of approach for the control purposes. In the designed ANFIS scheme, neural network techniques are used to select a proper rule base, which is achieved using the back propagation algorithm. The simulation results presented in this paper show the effectiveness of the developed method.     

小波变换的多层多道焊接拐角跟踪点的识别研究

利用小波变换检测信号边缘的优势,对V型坡口焊接激光图像以及多层多道焊激光图像的焊缝识别方法进行了系统的研究,给出了基于小波变换的焊缝识别算法,并提出了多层多道焊拐角跟踪点的识别算法,同时进行了大量的实验研究;分析了多层多道焊接激光图像的特点,在V型坡口识别算法的基础上进行了改进,利用模板匹配技术,实现拐角跟踪点的识别。     

移动机器人多目标彩色视觉跟踪系统

机器人视觉系统利用颜色、形状等信息来识别环境目标，但是难点在于识别的 鲁棒性和实时性的保证．本文采用全自主移动机器人为平台，提出了一种硬件成本低廉的、 基于颜色学习的实时多目标视觉跟踪系统，并提出了一种新颖的目标颜色学习和跟踪算法． 该视觉系统已成功集成在自主移动机器人系统中，非结构环境下的动态目标跟踪实验表明了 系统的实时性和鲁棒性．     

Projection operator-based robust adaptive control of an aerial robot with a manipulator

This paper describes a quadcopter manipulator system, an aerial robot with an extended workspace, its controller design, and experimental validation. The aerial robot is based on a quadcopter with a three degree of freedom robotic arm connected to the base of the vehicle. The work aims to create a stable airborne robot with a robotic arm that can work above and below the airframe, regardless of where the arm is attached. Integrating a robotic arm into an underactuated, unstable system like a quadcopter can enhance the vehicle's functionality while increasing instability. To execute a mission with accuracy and reliability during a real-time task, the system must overcome the inter-coupling effects and external disturbances. This work presents a novel design for a robust adaptive feedback linearization controller with a model reference adaptive controller and hardware implementation of the quadcopter manipulator system with plant uncertainties. The closed-loop stability of the aerial robot and the tracking error convergence with the robust controller is analyzed using Lyapunov stability analysis. The quadcopter manipulator system is custom developed in the lab with an off-the-shelf quadcopter and a 3D-printed robotic arm. The robotic system architecture is implemented using a Jetson Nano companion computer for autonomous onboard flight. Experiments were conducted on quadcopter manipulator system to evaluate the autonomous aerial robot's stability and trajectory tracking with the proposed controller.     

Adaptive control of nonholonomic robotic systems

This paper considers the problem of controlling the motion of nonholonomic robotic systems in the presence of uncertainty regarding the system model and state, and presents a class of adaptive controllers as a solution to this problem. The proposed control strategies provide simple and robust solutions to a number of important nonholonomic system control problems, including stabilization to an equilibrium manifold, motion control to an equilibrium point via trajectory tracking, and stabilization to an equilibrium point. All of the schemes are computationally efficient, are implementable without knowledge of the system dynamic model, and ensure uniform boundedness of all signals and accurate motion control; furthermore, most of the controllers can be implemented without rate measurements. The efficacy of the proposed approach is illustrated through extensive computer simulations with nonholonomic robotic systems arising from explicit constraints on the system kinematics and from symmetries of the system dynamics. © 1998 John Wiley & Sons, Inc.     

Steps in the development of a robotic scrub nurse

This paper describes how a robotic scrub nurse has been developed to assist human scrub nurses (also called delivery scrub nurses) during surgical interventions. The developed system in this work (nicknamed Quirubot) is equipped with a speech recognition module to recognize the requested surgical instrument; Quirubot locates this element on a storage tray using computer vision and pattern recognition and picks the element from the tray placing it on an interchange tray where the human scrub nurse can finally use this tool. Moreover, to implement the manipulation of surgical instruments, an electromagnetic gripper has been adapted to pick and place them. Fanuc LR Mate 200iB robot has been electrically modified to add a new connector devoted to control the electromagnetic gripper. Nowadays Quirubot can identify up to 27 surgical instruments and more than 82 spoken instructions; the advantages of the system have been demonstrated using a group of experienced and intermediate skilled scrub nurses in a open abdominal aortic aneurysm surgery simulation (experiments performed in a robotics lab and an operating room). After this study, the use of a robotic scrub nurse is justified in an nBio operating room to improve the system up to obtaining a more robust version to be used at the Hospital Universitario de Sant Joan d’Alacant.     

机器人化大直径钢管螺旋焊缝检测视觉跟踪系统的研究与开发

本文介绍了一种应用在螺焊管焊缝超声波无损探伤自动线上，跟踪运动焊缝横向偏移的视觉伺服系统。该系统以单ＣＰＵ运行的软件进行实时图象识别与系统控制。文中提出了一种在线建立特征模型的匹配识别策略，使系统能够很好地适应引起焊缝视觉特征变化的工作环境，条件的变化，准确识别不同特征的焊缝，同时具有较好的识别快速性。在实际检测线上，系统表现了良好的跟踪特性。     

Developing a limb repositioning robotic interface for persons with severe physical disabilities

Limb repositioning is necessary for individuals with severe physical disabilities to sustain muscle strength and prevent pressure sores. As robotic technologies become ubiquitous, these tools offer promise to support the repositioning process. However, research has yet to focus on ways in which individuals with severe physical disabilities can control robots for these tasks. This paper presents a study that examines the needs and attitudes of potential users with physical disabilities to control a robotic aid for limb repositioning. Subjects expressed interest in using brain–computer interface (BCI) and speech recognition technologies for purposes of executing robotic tasks. The performance of four subjects controlling arm movements on an avatar through the keyboard, mouse, BCI, and Dragon NaturallySpeaking speech recognition was evaluated. Although BCI and speech technologies may limit physical fatigue, more challenges were faced using BCI and speech conditions compared to the keyboard and mouse. This research promotes accessibility into mainstream robotic technologies and represents the first step in the development of a robotic prototype using a BCI and speech recognition technologies for limb repositioning.     

Precise planar motion measurement of a swimming multi-joint robotic fish一种面向多关节机器鱼平面运动的精确测量方法

This paper presents a method for planar motion measurement of a swimming multi-joint robotic fish. The motion of the robotic fish is captured via image sequences and a proposed tracking scheme is employed to continuously detect and track the robotic fish. The tracking scheme initially acquires a rough scope of the robotic fish and thereafter precisely locates it. Historical motion information is utilized to determine the rough scope, which can speed up the tracking process and avoid possible ambient interference. A combination of adaptive bilateral filtering and k-means clustering is then applied to segment out color markers accurately. The pose of the robotic fish is calculated in accordance with the centers of these markers. Further, we address the problem of time synchronization between the on-board motion control system of the robotic fish and the motion measurement system. To the best of our knowledge, this problem has not been tackled in previous research on robotic fish. With information about both the multi-link structure and motion law of the robotic fish, we convert the problem to a nonlinear optimization problem, which we then solve using the particle swarm optimization (PSO) algorithm. Further, smoothing splines are adopted to fit curves of poses versus time, in order to obtain a continuous motion state and alleviate the impact of noise. Velocity is acquired via a temporal derivative operation. The results of experiments conducted verify the efficacy of the proposed method.