Reinforcement learning-based shared control for walking-aid robot and its experimental verification

A walking-aid robot is an assistive device for enabling safe, stable and efficient locomotion in elderly or disabled individuals. In this paper, we propose a reinforcement learning-based shared control (RLSC) algorithm for intelligent walking-aid robot to address existing control problems in cooperative walking-aid robot system. Firstly, the intelligent walking-aid robot and the human walking intention estimation algorithm are introduced. Due to the limited physical and cognitive capabilities of elderly and disabled people, robot control input assistance is provided to maintain tactile comfort and a sense of stability. Then, considering the robot’s ability to autonomously adapt to different user operation habits and motor abilities, the RLSC algorithm is proposed. By dynamically adjusting user control weight according to different user control efficiencies and walking environments, the robot can improve the user’s degree of comfort when using the device and automatically adapting to user’s behaviour. Finally, the effectiveness of our algorithm is verified by experiments in a specified environment.     

A New Methodology for the Design of Passive Biped Robots: Determining Conditions on the Robot��s Parameters for the Existence of Stable Walking Cycles

Currently, passive robots are designed following a trial and error process in which the existence of a stable walking cycle for a given passive robot??s model is analyzed using Poincaré maps. The standard stability analysis procedure suffers from discretization aliasing, and it is not able to deal with complex passive models. In this paper a methodology that allows finding conditions on the robot??s parameters of a given passive model in order to obtain a stable walking cycle is proposed. The proposed methodology overcomes the aliasing problem that arises when Poincaré sections are discretized. Basically, it implements a search process that allows finding stable subspaces in the parameters?? space (i.e., regions with parameters?? combinations that produce stable walking cycles), by simulating the robot dynamics for different parameters?? combinations. After initial conditions are randomly selected, the robot??s dynamics is modeled step by step, and in the Poincaré section the existence of a walking cycle is verified. The methodology includes the definition of a search algorithm for exploring the parameters?? space, a method for the partition of the space in hypercubes and their efficient management using proper data structures, and the use of so-called design value functions that quantify the feasibility of the resulting parameters. Among the main characteristics of the proposed methodology are being robot independent (it can be used with any passive robot model, regardless of its complexity), and robust (stable subspaces incorporate a stability margin value that deals with differences between the robot??s model and its physical realization). The methodology is validated in the design process of a complex semi-passive robot that includes trunk, knees, and non-punctual feet. The robot also considers the use of actuators, controllers and batteries for its actuation.     

Optimal impedance via model predictive control for robot-aided rehabilitation

This paper proposes an optimal impedance controller for robot-aided rehabilitation of walking, aiming to increase the patient’s activity during the therapy. In an online procedure, the joint torques produced by the patient during the gait is estimated using the generalized momenta-based disturbance observer and the Extended Kalman filter algorithm. At the same time, a model predictive control is performed to obtain the instantaneous optimal stiffness parameters of the robot’s impedance controller, trying to maximize the patient’s active participation by increasing his/her joint torques. In this feasibility study, experiments with a healthy subject, considering a modular lower limb exoskeleton and a set of user’s behaviors, are performed to evaluate the proposed controller. The results show the robot stiffness converges to a value which increases the user’s active participation.     

基于改进鲸鱼优化算法的外骨骼机器人步态检测

针对传统的外骨骼机器人步态检测算法中的信息单一化、准确率低、易陷入局部最优等问题，提出基于改进鲸鱼算法优化的支持向量机（IWOA-SVM）的外骨骼机器人步态检测算法，即在鲸鱼优化算法（WOA）中引入遗传算法（GA）的选择、交叉、变异操作，进而去优化支持向量机（SVM）的惩罚因子与核参数，再使用参数优化后的SVM建立分类模型，从而扩大算法的搜索范围，减小算法陷入局部最优的概率。首先，使用混合传感技术采集步态数据，即通过足底压力传感器和膝关节、髋关节角度传感器采集外骨骼机器人的运动数据，并作为步态检测系统的输入；然后，使用门限法对步态相位进行划分并标记标签；最后，将足底压力信号与髋关节、膝关节角度信号融合作为输入，使用IWOA-SVM算法完成对步态的检测。对6个标准测试函数进行仿真实验，并与GA、粒子群优化（PSO）算法、WOA进行比较，数值实验表明，改进鲸鱼优化算法（IWOA）的鲁棒性、寻优精度、收敛速度均优于其他优化算法。通过分析不同穿戴者的步态检测结果发现，准确率可达98.8%，验证了所提算法在新一代外骨骼机器人中的可行性和实用性，并与基于遗传优化算法的支持向量机（GA-SVM）、基于粒子群优化算法的支持向量机（PSO-SVM）、基于鲸鱼优化算法的支持向量机（WOA-SVM）算法进行比较，结果表明，该算法识别准确率分别提高了5.33%、2.70%、1.44%，能够对外骨骼机器人的步态进行有效检测，进而实现外骨骼机器人的精确控制及稳定行走。     

Physical human-robot interaction estimation based control scheme for a hydraulically actuated exoskeleton designed for power amplification

We proposed a lower extremity exoskeleton for power amplification that perceives intended human motion via humanexoskeleton interaction signals measured by biomedical or mechanical sensors, and estimates human gait trajectories to implement corresponding actions quickly and accurately. In this study, torque sensors mounted on the exoskeleton links are proposed for obtaining physical human-robot interaction (pHRI) torque information directly. A Kalman smoother is adopted for eliminating noise and smoothing the signal data. Simultaneously, the mapping from the pHRI torque to the human gait trajectory is defined. The mapping is derived from the real-time state of the robotic exoskeleton during movement. The walking phase is identified by the threshold approach using ground reaction force. Based on phase identification, the human gait can be estimated by applying the proposed algorithm, and then the gait is regarded as the reference input for the controller. A proportional-integral-derivative control strategy is constructed to drive the robotic exoskeleton to follow the human gait trajectory. Experiments were performed on a human subject who walked on the floor at a natural speed wearing the robotic exoskeleton. Experimental results show the effectiveness of the proposed strategy.     

A New Directional-Intent Recognition Method for Walking Training Using an Omnidirectional Robot

In order to avoid being bedridden, a preemptive walking rehabilitation is essential for people who lose their walking ability because of illness or accidents. In a previous study, we developed an omnidirectional walking training robot (WTR), the effectiveness of which in rehabilitation was validated by clinical testing. In the primary stage of the walking training, the WTR guides the user to follow the predesigned therapy program to conduct the walking training. This study focuses on the later stages of training in which the user plays an active role of determining the training by himself/herself, and the WTR must follow the user’s intent. However, identifying a user’s intent is challenging. In the present study, we address this problem by introducing a directional-intent identification method based on a distance-type fuzzy reasoning algorithm. The effectiveness of the directional identification method is experimentally confirmed.     

Optimization of layout and path planning of surgical robotic system

Positioning a surgical robot for optimal operation in a crowded operating room is a challenging task. In the robotic-assisted surgical procedures, the surgical robot’s end-effector must reach the patient’s anatomical targets because repositioning of the patient or surgical robot requires additional time and labor. This paper proposes an optimization algorithm to determine the best layout of the operating room, combined with kinematics criteria and optical constraints applied to the surgical assistant robot system. A new method is also developed for trajectory of robot’s end-effector for path planning of the robot motion. The average deviations obtained from repeatability tests for surgical robot’s layout optimization were 1.4 and 4.2 mm for x and y coordinates, respectively. The results of this study show that the proposed optimization method successfully solves the placement problem and path planning of surgical robotic system in operating room.

     

Design of an Under-Actuated Exoskeleton System for Walking Assist While Load Carrying

This study proposes an under-actuated wearable exoskeleton system to carry a heavy load. To synchronize that system with a user, a feasible modular-type wearable system and its corresponding sensor systems are proposed. The design process of the modular-type exoskeleton for lower extremities is presented based on the considered requirements. To operate the system with the user, human walking analysis and intention signal acquisition methods for actuating the proposed system are developed. In particular, a sensing data estimation strategy is applied to synchronize the exoskeleton system with a user correctly. Finally, several experiments were performed to evaluate the performance of the proposed exoskeleton system by measuring the electromyography signal of the wearer's muscles while walking on level ground and climbing up stairs with 20- to 40-kg loads, respectively.     

Whole-body motion planning and control of a quadruped robot for challenging terrain

Quadruped robots working in jungles, mountains or factories should be able to move through challenging scenarios. In this paper, we present a control framework for quadruped robots walking over rough terrain. The planner plans the trajectory of the robot's center of gravity by using the normalized energy stability criterion, which ensures that the robot is in the most stable state. A contact detection algorithm based on the probabilistic contact model is presented, which implements event-based state switching of the quadruped robot legs. And an on-line detection of contact force based on generalized momentum is also showed, which improves the accuracy of proprioceptive force estimation. A controller combining whole body control and virtual model control is proposed to achieve precise trajectory tracking and active compliance with environment interaction. Without any knowledge of the environment, the experiments of the quadruped robot SDUQuad-144 climbs over significant obstacles such as 38 cm high steps and 22.5 cm high stairs are designed to verify the feasibility of the proposed method.     

Improving biped walk stability with complementary corrective demonstration

We contribute a method for improving the skill execution performance of a robot by complementing an existing algorithmic solution with corrective human demonstration. We apply the proposed method to the biped walking problem, which is a good example of a complex low level skill due to the complicated dynamics of the walk process in a high dimensional state and action space. We introduce an incremental learning approach to improve the Nao humanoid robot’s stability during walking. First, we identify, extract, and record a complete walk cycle from the motion of the robot as it executes a given walk algorithm as a black box. Second, we apply offline advice operators for improving the stability of the learned open-loop walk cycle. Finally, we present an algorithm to directly modify the recorded walk cycle using real time corrective human demonstration. The demonstrator delivers the corrective feedback using a commercially available wireless game controller without touching the robot. Through the proposed algorithm, the robot learns a closed-loop correction policy for the open-loop walk by mapping the corrective demonstrations to the sensory readings received while walking. Experiment results demonstrate a significant improvement in the walk stability.     

Simultaneous bipedal locomotion based on haptics for teleoperation

ABSTRACT

This paper proposes a novel, simultaneous bipedal locomotion method using haptics for remote operation of biped robots. In general, traditional biped walking methods require very high computational power and advanced controllers to perform the required task. However, in this proposed method, a master exoskeleton attached to the human’s lower body is used to obtain the trajectory and haptic information to generate the trajectory of the slave biped robot in real time. Lateral motion of the center of mass of the biped is constrained in this experiment. Also, it is considered that no communication delay is presented in between the two systems in this experiment, and they are not discussed in this paper. Since a direct motion transmission is used in the proposed method, this method is quite straight forward and a simultaneous walking can be realized at the same time with high performance. Also, it does not require an exact dynamic model of the biped or specific method to plan the trajectory. The gait pattern of the biped is directly determined by that of the human. Also, the operator can feel the remote environment through the exoskeleton robot. Results obtained from the experiments validate the proposed method.     

A novel path planning and control framework for passive resistance therapy with a robot manipulator

In this article, we present a paradigm for safe path generation and control for a robotic manipulator such that it provides programmable passive resistance therapy to patients with deficits in the upper extremities. When the patient applies an interaction force at the robot's end-effector, a dynamic path generator time parameterises any therapist-specified contour in the robot's workspace–thus, the robot mimics the dynamics of a passive impedance whose anisotropy vector can be continuously reconfigured. The proposed algorithm is easily implementable because it is robust to uncertainty in the robot dynamics. Moreover, the proposed strategy also guarantees user safety by maintaining the net flow of energy during the human robot interaction from the user towards the manipulator.     

Real-time motion detection based on SW/HW-codesign for walking rescue robots

In a rescue operation walking robots offer a great deal of flexibility in traversing uneven terrain in an uncontrolled environment. For such a rescue robot, each motion is a potential vital sign and the robot should be sensitive enough to detect such motion, at the same time maintaining high accuracy to avoid false alarms. However, the existing techniques for motion detection have severe limitations in dealing with strong levels of ego-motion on walking robots. This paper proposes an optical flow-based method for the detection of moving objects using a single camera mounted on a hexapod robot. The proposed algorithm estimates and compensates ego-motion to allow for object detection from a continuously moving robot, using a first-order-flow motion model. Our algorithm can deal with strong rotation and translation in 3D, with four degrees of freedom. Two alternative object detection methods using a 2D-histogram based vector clustering and motion-compensated frame differencing, respectively, are examined for the detection of slow- and fast-moving objects. The FPGA implementation with optimized resource utilization using SW/HW codesign can process video frames in real-time at 31 fps. The new algorithm offers a significant improvement in performance over the state-of-the-art, under harsh environment and performs equally well under smooth motion.     

Neuro-fuzzy control of a robotic exoskeleton with EMG signals

We have been developing robotic exoskeletons to assist motion of physically weak persons such as elderly, disabled, and injured persons. The robotic exoskeleton is controlled basically based on the electromyogram (EMG) signals, since the EMG signals of human muscles are important signals to understand how the user intends to move. Even though the EMG signals contain very important information, however, it is not very easy to predict the user's upper-limb motion (elbow and shoulder motion) based on the EMG signals in real-time because of the difficulty in using the EMG signals as the controller input signals. In this paper, we propose a robotic exoskeleton for human upper-limb motion assist, a hierarchical neuro-fuzzy controller for the robotic exoskeleton, and its adaptation method.     

仿人机器人跑步稳定性准则

A humanoid robot has high mobility but possibly risks of tipping over. Until now, one main topic on humanoid robots is to study the walking stability; the issue of the running stability has rarely been investigated. The running is different from the walking, and is more difficult to maintain its dynamic stability. The objective of this paper is to study the stability criterion for humanoid running based on the whole dynamics. First, the cycle and the dynamics of running are analyzed. Then, the stability criterion of humanoid running is presented. Finally, the effectiveness of the proposed stability criterion is illustrated by a dynamic simulation example using a dynamic analysis and design system (DADS).     

A relational hierarchical model for decision-theoretic assistance

Building intelligent assistants has been a long-cherished goal of AI, and many were built and fine-tuned to specific application domains. In recent work, a domain-independent decision-theoretic model of assistance was proposed, where the task is to infer the user??s goal and take actions that minimize the expected cost of the user??s policy. In this paper, we extend this work to domains where the user??s policies have rich relational and hierarchical structure. Our results indicate that relational hierarchies allow succinct encoding of prior knowledge for the assistant, which in turn enables the assistant to start helping the user after a relatively small amount of experience.     

基于改进涡流搜索算法的外骨骼迭代学习控制

为提升康复外骨骼机器人的步态跟踪性能,提出一种基于改进涡流搜索算法的迭代学习控制方法。首先针对传统迭代学习控制抗扰性差和控制信息缺失问题,引入PD控制器、自适应遗忘因子、误差过渡曲线和控制信息搜索等策略,改进迭代学习控制律;其次,基于多种策略对涡流搜索算法进行改进,提出了一种改进涡流搜索算法,改进后的算法可优化迭代学习控制的PD参数;最后进行行走实验,将提出的迭代学习控制方法与现有的同类算法进行仿真和数值比较,并测试了扰动情况下的跟踪性能。实验结果表明,所提方法的误差更小,跟踪性能更强。该算法改进了迭代学习控制的不足,具有较强的抗扰性能,保证了使用时的稳定性。     

一种下肢康复机器人步态的生成与调节方法

针对下肢康复外骨骼机器人康复训练参考步态的标准化设计,对平地行走、上楼梯和下楼梯等不同情境下的步态轨迹的设计问题进行研究,提出了由无线惯性传感器采集人体步态,利用基于关键点的步态生成与调节方法,得到了融合过渡过程的参考步态曲线.利用得到的参考步态曲线指导下肢康复外骨骼式机器人辅助人体在平地行走、上楼梯和下楼梯等情景下进...     

A Multilayer Perceptrons Model for the Stability of a Bipedal Robot

A neural network model is proposed as a means of controlling the dynamical equilibrium of a walking bipedal robot. As a criterion to determine the stability of such a robot in relation with the organization of the sensorimotor system, we have been making use of the ZMP (Zero Momentum Point). Simulations are used to check the convergence of the algorithm. In the generalization phase, it is shown that the neural network has the ability to stabilise the robot for motions which have not previously been learned. An extended model is proposed, which seeks to closely inspect the physiology of the cerebellar cortex.