Design and Implementation of a Multi Sensor Based Brain Computer Interface for a Robotic Wheelchair

In this study, design and implementation of a multi sensor based brain computer interface for disabled and/or elderly people is proposed. Developed system consists of a wheelchair, a high-power motor controller card, a Kinect camera, electromyogram (EMG) and electroencephalogram (EEG) sensors and a computer. The Kinect sensor is installed on the system to provide safe navigation for the system. Depth frames, captured by the Kinect’s infra-red (IR) camera, are processed with a custom image processing algorithm in order to detect obstacles around the wheelchair. A Consumer grade EMG device (Thalmic Labs) was used to obtain eight channels of EMG data. Four different hand movements: Fist, release, waving hand left and right are used for EMG based control of the robotic wheelchair. EMG data is first classified using artificial neural network (ANN), support vector machines and random forest schemes. The class is then decided by a rule-based scheme constructed on the individual outputs of the three classifiers. EEG based control is adopted as an alternative controller for the developed robotic wheelchair. A wireless 14-channels EEG sensor (Emotiv Epoch) is used to acquire real time EEG data. Three different cognitive tasks: Relaxing, math problem solving, text reading are defined for the EEG based control of the system. Subjects were asked to accomplish the relative cognitive task in order to control the wheelchair. During experiments, all subjects were able to control the robotic wheelchair by hand movements and track a pre-determined route with a reasonable accuracy. The results for the EEG based control of the robotic wheelchair are promising though vary depending on user experience.     

Using a New Model of Recurrent Neural Network for Control

This paper shows the results obtained in controlling a mobile robot by means of local recurrent neural networks based on a radial basis function (RBF) type architecture. The model used has a Finite Impulse Response (FIR) filter feeding back each neuron's output to its own input, while using another FIR filter as a synaptic connection. The network parameters (coefficients of both filters) are adjusted by means of the gradient descent technique, thus obtaining the stability conditions of the process. As a practical application the system has been successfully used for controlling a wheelchair, using an architecture made up by a neurocontroller and a neuroidentifier. The role of the latter, connected up in parallel with the wheelchair, is to propagate the control error to the neurocontroller, thus cutting down the control error in each working cycle.     

Neural Control of the Movements of a Wheelchair

This paper studies the problem of controlling the movements of a handicapped person's motorized wheelchair from a practical point of view. The control system implemented has been divided into two levels: the low level, consisting of an electronic system which directly controls the drivers of the chair's motors, with a classic PID (proportional-integral-derivative) control loop. The aim of this level is to ensure that the speeds of each one of the wheels is similar to the input speed of these control boards. The second control level (high level), implemented by means of neural techniques, ensures that the linear and angular speeds of the wheelchair are those indicated by a trajectory generator. A new recurrent model is used as the neural network, for which the stability conditions of the complete control system are obtained and various practical tests are carried out, which show the correct performance of the actual system implemented.     

Implementing spiking neural networks for real-time signal-processing and control applications: a model-validated FPGA approach.

In this paper, we present two versions of a hardware processing architecture for modeling large networks of leaky-integrate-and-fire (LIF) neurons; the second version provides performance enhancing features relative to the first. Both versions of the architecture use fixed-point arithmetic and have been implemented using a single field-programmable gate array (FPGA). They have successfully simulated networks of over 1000 neurons configured using biologically plausible models of mammalian neural systems. The neuroprocessor has been designed to be employed primarily for use on mobile robotic vehicles, allowing bio-inspired neural processing models to be integrated directly into real-world control environments. When a neuroprocessor has been designed to act as part of the closed-loop system of a feedback controller, it is imperative to maintain strict real-time performance at all times, in order to maintain integrity of the control system. This resulted in the reevaluation of some of the architectural features of existing hardware for biologically plausible neural networks (NNs). In addition, we describe a development system for rapidly porting an underlying model (based on floating-point arithmetic) to the fixed-point representation of the FPGA-based neuroprocessor, thereby allowing validation of the hardware architecture. The developmental system environment facilitates the cooperation of computational neuroscientists and engineers working on embodied (robotic) systems with neural controllers, as demonstrated by our own experience on the Whiskerbot project, in which we developed models of the rodent whisker sensory system.     

A Sensorized Garment Controlled Virtual Robotic Wheelchair

This paper presents the design and performance of a body-machine-interface (BoMI) system, where a user controls a robotic 3D virtual wheelchair with the signals derived from his/her shoulder and elbow movements. BoMI promotes the perspective that system users should no longer be operators of the engineering design but should be an embedded part of the functional design. This BoMI system has real-time controllability of robotic devices based on user-specific dynamic body response signatures in high-density 52-channel sensor shirt. The BoMI system not only gives access to the user’s body signals, but also translates these signals from user’s body to the virtual reality device-control space. We have explored the efficiency of this BoMI system in a semi-cylinderic 3D virtual reality system. Experimental studies are conducted to demonstrate, how this transformation of human body signals of multiple degrees of freedom, controls a robotic wheelchair navigation task in a 3D virtual reality environment. We have also presented how machine learning can enhance the interface to adapt towards the degree of freedoms of human body by correcting the errors performed by the user.     

A Semi-Autonomous Reactive Control Architecture

This paper presents a novel intelligent control architecture for semi-autonomous systems. A semi-autonomous system is defined here as that autonomous system (machine) which interacts intelligently with a human user (collaborator) who might command, modify, or override its behavior. This work has been motivated by the need for a control architecture that can interact with human users of different perceptual and cognitive capabilities. A dynamic arbitration layer forms the core of the proposed architecture. Accordingly, the architecture evolves around three main variables: degree of autonomy to reflect the user's capabilities, user's level of confidence in commanding the machine, and strength of conflict between the user's command and the machine's autonomous command. The analogy between this architecture and horseback riding is presented and finally a demonstrative application example of a robotic wheelchair is given.     

Determining maximum payloads for cooperating robots under time-optimal control

This work presents an algorithm which evaluates the dynamic performance limit of a cooperating robotic system using movements planned for minimum time. Minimum-time movements characteristically require that a set of motors in the robot be driven at their maximum torque throughout the motion. These movements are limited by the combination of motor performance, mechanical advantage of the kinematic chain, and the location of the start and goal positions. By increasing the payload for a motion until a minimum-time solution is no longer feasible the payload limit of the system for the associated path is obtained. To illustrate the algorithm a detailed analysis of a robotic arm developed at Odetics Inc. is presented. The analysis includes numerical results for cooperating Odetics robotic arms using their maximum payload under time-optimal control. Furthermore, the maximum payload for the cooperating robotic system to perform the same motion with a 1 sec time constraint is determined.     

Modeling and Adaptive Neural Network Control for a Soft Robotic Arm With Prescribed Motion Constraints

This paper presents a dynamic model and performance constraint control of a line-driven soft robotic arm. The dynamics model of the soft robotic arm is established by combining the screw theory and the Cosserat theory. The unmodeled dynamics of the system are considered, and an adaptive neural network controller is designed using the backstepping method and radial basis function neural network. The stability of the closed-loop system and the boundedness of the tracking error are verified using Lyapunov theory. The simulation results show that our approach is a good solution to the motion constraint problem of the line-driven soft robotic arm.      

基于改进头姿估计方法的机器人轮椅交互控制

针对现有迭代最近点（ICP）头姿估计算法存在迭代次数偏多且易陷于局部最优、而随机森林（RF）头姿估计算法准确性和稳定性不高的问题,提出一种新的头姿估计改进方法,并基于该改进方法构建机器人轮椅实时交互控制接口．首先,分析现有迭代最近点头姿算法与随机森林头姿算法在准确性、实时性及稳定性方面存在的问题,并提出一种新的基于随机森林与迭代最近点算法融合的头姿估计改进方法;其次,为实现头姿估计到机器人轮椅交互控制的无缝连接,建立基于传统机器人轮椅操纵杆的头部姿态运动空间映射;最后,在基于标准头姿数据库分析改进头姿估计方法性能的基础上,构建机器人轮椅实验平台并规划运动轨迹,以进一步验证基于改进头姿估计方法的人机交互接口在机器人轮椅实时控制方面的有效性．实验结果表明,改进后的头姿估计方法较传统迭代最近点算法减少了迭代次数且避免了陷于局部最优,在仅增加少量运算时间的基础上,其准确性和稳定性都优于传统随机森林算法;同时,基于改进头姿估计方法的人机交互接口亦能实时平稳地控制机器人轮椅沿既定的轨迹运动．     

Adaptive motion control law of a robotic wheelchair

In the present work, a dynamic model of a robotic wheelchair is developed considering a lateral deviation of the center of mass. The Lyapunov and input/output stability theories are used to design a novel tracking and positioning adaptive control for the robotic wheelchair. Properties of the dynamic model with respect to its matrices and parameters are shown. A filter is used to obtain a closed loop equation that allows designing the adaptive control law. Then, a projection algorithm is used to improve the adaptive control in the sense of avoiding parameter drift. Experimental results show good performance of the adaptive control.     

Indirect adaptive control of nonlinear dynamic systems using self recurrent wavelet neural networks via adaptive learning rates

This paper proposes an indirect adaptive control method using self recurrent wavelet neural networks (SRWNNs) for dynamic systems. The architecture of the SRWNN is a modified model of the wavelet neural network (WNN). However, unlike the WNN, since a mother wavelet layer of the SRWNN is composed of self-feedback neurons, the SRWNN can store the past information of wavelets. In the proposed control architecture, two SRWNNs are used as both an identifier and a controller. The SRWNN identifier approximates dynamic systems and provides the SRWNN controller with information about the system sensitivity. The gradient-descent method using adaptive learning rates (ALRs) is applied to train all weights of the SRWNN. The ALRs are derived from discrete Lyapunov stability theorem, which are applied to guarantee the convergence of the proposed control system. Finally, we perform some simulations to verify the effectiveness of the proposed control scheme.     

A computational neuromusculoskeletal model of human arm movements

This work proposes a computational neuromusculoskeletal model of human arm movements. The model consists of three components: the supraspinal neural control system, the spinal motor system, and the muscle-tendon actuation system. In the supraspinal neural system model, the cerebellum is regarded as having feedforward control and the cerebrum as feedback control principally based on the feedback-error learning scheme. This computational model proposes that the feedforward control of the cerebellum may not need to be an explicit locus of an inverse dynamic model. This model also includes the modularly organized spinal motor system such that it simplifies controlling redundant muscular actuators. Cerebellar feedforward control and the spinal motor system are assumed to be adaptive. The two motor adaptations seem to synergistically promote motion flexibility and simplify the neural system structure. The neural control system is combined with the Hill-type muscle-tendon model to generate arm movement. The overall model proposes that an approximate inverse dynamic model may implicitly be constructed over the integrated neuromusculoskeletal system, and it is not necessary to be explicitly computed in a specific motor system. To cope with the human neural system, neuromuscular activation dynamics and neural transmission delays are included in the model. A computational simulation study using the model was implemented to verify the feasibility of the model. Center out reaching movements and learning of those movements as well s generations of figure eightlike movements were computationally tested. A plausible motor control scheme of movement is discussed using the model.     

A Neural Tactile Architecture Applied to Real-time Stiffness Estimation for a Large Scale of Robotic Grasping Systems

This paper presents a model for solving the problem of real-time neural estimation of stiffness characteristics for unknown objects. For that, an original neural architecture is proposed for a large scale robotic grasping systems applied for unknown object with unspecified stiffness characteristics. The force acquisition is based on tactile information from force sensors in robotic manipulator. The proposed model has been implemented on a robotic gripper with two parallel fingers and on a one d.o.f. robotic finger with opponent artificial muscles and angular displacements. This self-organized model is inspired of human biological system, and is carried out by means of Topographic Maps and Vector Associative Maps. Experimental results demonstrate the efficiency of this new approach.     

Design and performance of an intelligent predictive controller for a six-degree-of-freedom robot using the Elman network

The aim of this paper was to propose a recurrent neural network-based predictive controller for robotic manipulators. A neural network controller for a six-joint Stanford robotic manipulator was designed using the generalized predictive control (GPC) and the Elman network. The GPC algorithm, which is a class of digital control method, requires long computational time. This is a disadvantage in real-time robot control; therefore, the Elman network controller was designed to reduce processing time by avoiding the highly mathematical and computational complexity of the GPC. The main reason for choosing the Elman network, amongst several neural network algorithms, was that the presence of feedback loops have a profound impact on the learning capability of the network. The designed neural network controller was able to recover quickly because of its significant generalization capability, which allowed it to adapt very rapidly to changes in inputs. The performance of the controller was also shown graphically using simulation software, including the dynamics and kinematics of the robot model.     

Design and testing of a low-cost robotic wheelchair prototype

Many people who are mobility impaired are, for a variety of reasons, incapable of using an ordinary wheelchair. In some instances, a power wheelchair also cannot be used, usually because of the difficulty the person has in controlling it (often due to additional disabilities). This paper describes two low-cost robotic wheelchair prototypes that assist the operator of the chair in avoiding obstacles, going to pre-designated places, and maneuvering through doorways and other narrow or crowded areas. These systems can be interfaced to a variety of input devices, and can give the operator as much or as little moment by moment control of the chair as they wish. This paper describes both systems, the evolution from one system to another, and the lessons learned.     

Integration of top-down and bottom-up visual processing using a recurrent convolutional–deconvolutional neural network for semantic segmentation

Semantic segmentation has a wide array of applications such as scene understanding, autonomous driving, and robot manipulation tasks. While existing segmentation models have achieved good performance using bottom-up deep neural processing, this paper describes a novel deep learning architecture that integrates top-down and bottom-up processing. The resulting model achieves higher accuracy at a relatively low computational cost. In the proposed model, higher-level top-down information is transmitted to the lower layers through recurrent connections in an encoder and a decoder, and the recurrent connection weights are trained using backpropagation. Experiments on several benchmark datasets demonstrate that this use of top-down information improves the mean intersection over union by more than 3% compared with a state-of-the-art bottom-up only network using the CamVid, SUN-RGBD and PASCAL VOC 2012 benchmark datasets. Additionally, the proposed model is successfully applied to a dataset designed for robotic grasping tasks.

     

Look where you're going [robotic wheelchair]

We propose a robotic wheelchair that observes the user and the environment. It can understand the user's intentions from his/her behaviors and the environmental information. It also observes the user when he/she is off the wheelchair, recognizing the user's commands indicated by hand gestures. Experimental results show our approach to be promising. Although the current system uses face direction, for people who find it difficult to move their faces, it can be modified to use the movements of the mouth, eyes, or any other body parts that they can move. Since such movements are generally noisy, the integration of observing the user and the environment will be effective in understanding the real intentions of the user and will be a useful technique for better human interfaces.     

Control architecture for human-robot integration: application to a robotic wheelchair.

Completely autonomous performance of a mobile robot within noncontrolled and dynamic environments is not possible yet due to different reasons including environment uncertainty, sensor/software robustness, limited robotic abilities, etc. But in assistant applications in which a human is always present, she/he can make up for the lack of robot autonomy by helping it when needed. In this paper, the authors propose human-robot integration as a mechanism to augment/improve the robot autonomy in daily scenarios. Through the human-robot-integration concept, the authors take a further step in the typical human-robot relation, since they consider her/him as a constituent part of the human-robot system, which takes full advantage of the sum of their abilities. In order to materialize this human integration into the system, they present a control architecture, called architecture for human-robot integration, which enables her/him from a high decisional level, i.e., deliberating a plan, to a physical low level, i.e., opening a door. The presented control architecture has been implemented to test the human-robot integration on a real robotic application. In particular, several real experiences have been conducted on a robotic wheelchair aimed to provide mobility to elderly people.     

Neural and fuzzy robotic hand control

An efficient first grasp for a wheelchair robotic arm-hand with pressure sensing is determined and presented. The grasp is learned by combining the advantages of neural networks and fuzzy logic into a hybrid control algorithm which learns from its tip and slip control experiences. Neurofuzzy modifications are outlined, and basic steps are demonstrated in preparation for physical implementation. Choice of object approach vector based on fuzzy tip and slip data and an expert supervisor, as well as training of a diagnostic neural tip and slip controller, are the focus of this work.     

On Climbing Winding Stairs in an Open Mode for a New Robotic Wheelchair

This paper presents the mechanical design, locomotion and associated dynamic models of a new robotic wheelchair on climbing winding stairs. The prototype stair-climbing robotic wheelchair is constructed comprising a pair of rotational multi-limbed structures pivotally mounted on opposite sides of a support base so that the robotic wheelchair can ascend and descend stairs; in particular, the capability of climbing winding stairs is addressed. Based on the skid-steering analysis, the dynamic models for climbing winding stairs are developed for the trajectory planning and motion analyses. These models are required to ensure a passenger's safety in such a way that the robotic wheelchair is operated in an open mode. Moreover, an equivalent constraint method is proposed for the prescribed motion of the robotic wheelchair on climbing winding stairs. The results of the simulation and maneuver are reported that show the behavior of the prototype as it climbs winding stairs in a dynamic turning.