Understanding Parkinsonian handwriting through a computational model of basal ganglia

Handwriting in Parkinson's disease (PD) is typically characterized by micrographia, jagged line contour, and unusual fluctuations in pen tip velocity. Although PD handwriting features have been used for diagnostics, they are not based on a signaling model of basal ganglia (BG). In this letter, we present a computational model of handwriting generation that highlights the role of BG. When PD conditions like reduced dopamine and altered dynamics of the subthalamic nucleus and globus pallidus externa subsystems are simulated, the handwriting produced by the model manifested characteristic PD handwriting distortions like micrographia and velocity fluctuations. Our approach to PD modeling is in tune with the perspective that PD is a dynamic disease.     

基于基底神经节机理的行为决策模型

基底神经节是大脑深部一系列的神经功能核团的总称，主要与周围的大脑皮层，丘脑，小脑和脑干相连.基底神经节的主要功能是参与运动调节和随意运动控制，特别是行为的决策确定和动作选择.本文主要介绍基于基底神经节机理的行为决策确定模型，重点综述决策确定和动作选择的理论模型进展.这里首先介绍了基底神经节的解剖结构，即输入核、输出核、多巴胺系统和中继核，进而描述了基底神经节的功能连接(直接通路、间接通路和超直接通路)，然后基于基底神经节的多个皮质-BG回路之间的相互作用，概述了决策模型在运动和行为选择等方面的理论模型，主要有Gurney的选择-控制模型、Humphries的神经元群模型、Frank的认知决策模型、Wang X J的选择决策模型和Rabinovich相空间动力学选择模型等.最后讨论了这些理论决策模型和大脑疾病(帕金森病)的相互关系等.     

LEARNING MOTOR SKILLS BY IMITATION: A BIOLOGICALLY INSPIRED ROBOTIC MODEL

This article presents a biologically inspired model for motor skills imitation. The model is composed of modules whose functinalities are inspired by corresponding brain regions responsible for the control of movement in primates. These modules are high-level abstractions of the spinal cord, the primary and premotor cortexes (M1 and PM), the cerebellum, and the temporal cortex. Each module is modeled at a connectionist level. Neurons in PM respond both to visual observation of movements and to corresponding motor commands produced by the cerebellum. As such, they give an abstract representation of mirror neurons. Learning of new combinations of movements is done in PM and in the cerebellum. Premotor cortexes and cerebellum are modeled by the DRAMA neural architecture which allows learning of times series and of spatio-temporal invariance in multimodal inputs. The model is implemented in a mechanical simulation of two humanoid avatars, the imitator and the imitatee. Three types of sequences learning are presented: (1) learning of repetitive patterns of arm and leg movements; (2) learning of oscillatory movements of shoulders and elbows, using video data of a human demonstration; 3) learning of precise movements of the extremities for grasp and reach.     

Joint detection and clinical score prediction in Parkinson's disease via multi-modal sparse learning

Parkinson's disease (PD) is the world's second most common progressive neurodegenerative disease. This disease is characterized by a combination of various non-motor symptoms (e.g., depression, olfactory, and sleep disturbance) and motor symptoms (e.g., bradykinesia, tremor, rigidity), therefore diagnosis and treatment of PD are usually complex. There are some machine learning techniques that automate PD diagnosis and predict clinical scores. These techniques are promising in assisting assessment of stage of pathology and predicting PD progression. However, existing PD research mainly focuses on single-function model (i.e., only classification or prediction) using one modality, which limits performance. In this work, we propose a novel feature selection framework based on multi-modal neuroimaging data for joint PD detection and clinical score prediction. Specifically, a unique objective function is designed to capture discriminative features which are used to train a support vector regression (SVR) model for predicting clinical score (e.g., sleep scores and olfactory scores), and a support vector classification (SVC) model for class label identification. We evaluate our method using a dataset of 208 subjects, which includes 56 normal controls (NC), 123 PD and 29 scans without evidence of dopamine deficit (SWEDD) via a 10-fold cross-validation strategy. Our experimental results demonstrate that multi-modal data effectively improves the performance in disease status identification and clinical scores prediction as compared to one single modality. Comparative analysis reveals that the proposed method outperforms state-of-art methods.     

A computational neuromusculoskeletal model of human arm movements

This work proposes a computational neuromusculoskeletal model of human arm movements. The model consists of three components: the supraspinal neural control system, the spinal motor system, and the muscle-tendon actuation system. In the supraspinal neural system model, the cerebellum is regarded as having feedforward control and the cerebrum as feedback control principally based on the feedback-error learning scheme. This computational model proposes that the feedforward control of the cerebellum may not need to be an explicit locus of an inverse dynamic model. This model also includes the modularly organized spinal motor system such that it simplifies controlling redundant muscular actuators. Cerebellar feedforward control and the spinal motor system are assumed to be adaptive. The two motor adaptations seem to synergistically promote motion flexibility and simplify the neural system structure. The neural control system is combined with the Hill-type muscle-tendon model to generate arm movement. The overall model proposes that an approximate inverse dynamic model may implicitly be constructed over the integrated neuromusculoskeletal system, and it is not necessary to be explicitly computed in a specific motor system. To cope with the human neural system, neuromuscular activation dynamics and neural transmission delays are included in the model. A computational simulation study using the model was implemented to verify the feasibility of the model. Center out reaching movements and learning of those movements as well s generations of figure eightlike movements were computationally tested. A plausible motor control scheme of movement is discussed using the model.     

Learning to Acquire Whole-Body Humanoid Center of Mass Movements to Achieve Dynamic Tasks

This paper presents a novel approach for acquiring dynamic whole-body movements on humanoid robots focused on learning a control policy for the center of mass (CoM). In our approach, we combine both a model-based CoM controller and a model-free reinforcement learning (RL) method to acquire dynamic whole-body movements in humanoid robots. (i) To cope with high dimensionality, we use a model-based CoM controller as a basic controller that derives joint angular velocities from the desired CoM velocity. The balancing issue can also be considered in the controller. (ii) The RL method is used to acquire a controller that generates the desired CoM velocity based on the current state. To demonstrate the effectiveness of our approach, we apply it to a ball-punching task on a simulated humanoid robot model. The acquired whole-body punching movement was also demonstrated on Fujitsu's Hoap-2 humanoid robot.     

Properties of synergies arising from a theory of optimal motor behavior

We consider the properties of motor components, also known as synergies, arising from a computational theory (in the sense of Marr, 1982) of optimal motor behavior. An actor's goals were formalized as cost functions, and the optimal control signals minimizing the cost functions were calculated. Optimal synergies were derived from these optimal control signals using a variant of nonnegative matrix factorization. This was done using two different simulated two--joint arms--an arm controlled directly by torques applied at the joints and an arm in which forces were applied by muscles--and two types of motor tasks-reaching tasks and via-point tasks. Studies of the motor synergies reveal several interesting findings. First, optimal motor actions can be generated by summing a small number of scaled and time-shifted motor synergies, indicating that optimal movements can be planned in a low-dimensional space by using optimal motor synergies as motor primitives or building blocks. Second, some optimal synergies are task independent--they arise regardless of the task context-whereas other synergies are task dependent--they arise in the context of one task but not in the contexts of other tasks. Biological organisms use a combination of task--independent and task--dependent synergies. Our work suggests that this may be an efficient combination for generating optimal motor actions from motor primitives. Third, optimal motor actions can be rapidly acquired by learning new linear combinations of optimal motor synergies. This result provides further evidence that optimal motor synergies are useful motor primitives. Fourth, synergies with similar properties arise regardless if one uses an arm controlled by torques applied at the joints or an arm controlled by muscles, suggesting that synergies, when considered in "movement space," are more a reflection of task goals and constraints than of fine details of the underlying hardware.     

机械臂抓取任务的脑电分类

针对大脑认知完好无损的患者,却患有重度神经肌肉疾病导致肢体行动受限的问题,为使患者重新获取障碍肢体的自主控制能力,本文提出了一种机械臂抓取任务的脑电分类方法对患者进行障碍肢体运动康复训练.首先使用非侵入式脑电技术对运动想象脑电信号进行采集,通过预处理、特征提取以及多尺度特征融合卷积神经网络进行分类识别;最后利用分类模型得到的标签解码成机械臂能够识别的指令,控制机械臂完成特定任务.实验结果表明:实验选取的15名健康受试者运动想象实验采集的脑电数据具有可行性,平均准确率达到了82%以上;为机械臂抓取任务的脑电分类提供了一种新思路.     

The role of plant properties in arm trajectory formation: a neuralnetwork study

In this paper, we first introduce a neural network model of a planar, six-muscle, redundant arm whose structure and operation principles were inspired by those of the human arm. We developed the model with a motor-learning framework in mind, i.e., with the long-term goal of incorporating it in a parallel distributed learning scheme for the arm controller. We then demonstrate the response of the model to various patterns of activation of the arm muscles in order to study the relative role of control strategies and plant properties in trajectory formation. The results of our simulations emphasize the role of the intrinsic properties of the plant in generating movements with anthropomorphic qualities such as smoothness and unimodal velocity profiles, and demonstrate that the task of an eventual controller for the arm could be simply that of programming the amplitudes and durations of steps of neural input without considering additional motor details. Our findings are relevant to the design of artificial arms and, with some caveats, to the study of the brain strategies in the arm motor system.     

Parkinson's disease: a motor control study using a wrist robot

Deep brain stimulation (DBS) is the most common surgical procedure for patients with Parkinson's disease (PD). DBS has been shown to have a positive effect on PD symptoms; however, its specific effects on motor control are not yet understood. We introduce the novel use of a wrist robot in studying the effects of stimulation on motor performance and learning. We present results from patients performing reaching movements in a null field and in a force field with and without stimulation. We discuss special cases where robotic testing reveals otherwise undiagnosed impairments, and where clinical scores and robot-based scores display opposing trends.     

Adaptation and learning in control of voluntary movement by the central nervous system

In order to control voluntary movements, the central nervous system must solve the following three computational problems at different levels: (1) the determination of a desired trajectory in visual coordinates; (2) the transformation of its coordinates into body coordinates; and (3) the generation of motor command. Concerning these problems, relevant experimental observations obtained in the field of neuroscience are briefly reviewed. On the basis of physiological information and previous models, we propose computational theories and a neural network model which account for these three problems. (1) A minimum torque-change model which predicts a wide range of trajectories in human multi-joint arm movements is proposed as a computational model for trajectory formation. (2) An iterative learning scheme is presented as an algorithm which solves the coordinate transformation and the control problem simultaneously. This algorithm can be regarded as a Newton-like method in function spaces. (3) A neural network model for generation of motor command is proposed. This model contains internal neural models of the motor system and its inverse system. The inverse-dynamics model is acquired by heterosynaptic plasticity using a feedback motor command (torque) as an error signal. The hierarchical arrangement of these neural networks and their global control are discussed. Their applications to robotics are also discussed.     

Validity of the Acti4 software using ActiGraph GT3X+accelerometer for recording of arm and upper body inclination in simulated work tasks

The validity of inclinometer measurements by ActiGraph GT3X+ (AG) accelerometer, when analysed with the Acti4 customised software, was examined by comparison of inclinometer measurements with a reference system (TrakStar) in a protocol with standardised arm movements and simulated working tasks. The sensors were placed at the upper arm (distal to the deltoid insertion) and at the spine (level of T1-T2) on eight participants. Root mean square errors (RMSEs) values of inclination between the two systems were low for the slow- and medium-speed standardised arm movements and in simulated working tasks. Fast arm movements caused the inclination estimated by the AG to deviate from the reference measurements (RMSE values up to ～10°). Furthermore, it was found that AG positioned at the upper arm provided inclination data without bias compared to the reference system. These findings indicate that the AG provides valid estimates of arm and upper body inclination in working participants.     

Reinforcement Learning in Autonomic Computing: A Manifesto and Case Studies

Reinforcement learning is a promising new approach for automatically developing effective policies for real-time self-management. RL can achieve superior performance to traditional methods, while requiring less built-in domain knowledge. Several case studies from real and simulated systems management applications demonstrate RL's promises and challenges. These studies show that standard online RL can learn effective policies in feasible training times. Moreover, a Hybrid RL approach can profit from any knowledge contained in an existing policy by training on the policy's observable behavior, without needing to interface directly to such knowledge     

Sensorimotor Interactions During Locomotion: Principles Derived from Biological Systems

Rhythmic movements in biological systems are produced in part by central circuits called central pattern generators (CPGs). For example, locomotion in vertebrates derives from the spinal CPG with activity initiated by the brain and controlled by sensory feedback. Sensory feedback is traditionally viewed as controlling CPGs cycle by cycle, with the brain commanding movements on a top down basis. We present an alternative view which in sensory feedback alters the properties of the CPG on a fast as well as a slow time scale. The CPG, in turn, provides feedforward filtering of the sensory feedback. This bidirectional interaction is widespread across animals, suggesting it is a common feature of motor systems, and, therefore, might offer a new way to view sensorimotor interactions in all systems including robotic systems. Bidirectional interactions are also apparent between the cerebral cortex and the CPG. The motor cortex doesn't simply command muscle contractions, but rather operates with the CPG to produce adaptively structured movements. To facilitate these adaptive interactions, the motor cortex receives feedback from the CPG that creates a temporal activity pattern mirroring the spinal motor output during locomotion. Thus, the activity of the motor cortical cells is shaped by the spinal pattern generator as they drive motor commands. These common features of CPG structure and function are suggested as offering a new perspective for building robotic systems. CPGs offer a potential for adaptive control, especially when combined with the principles of sensorimotor integration described here.     

Movement generation with circuits of spiking neurons

How can complex movements that take hundreds of milliseconds be generated by stereotypical neural microcircuits consisting of spiking neurons with a much faster dynamics? We show that linear readouts from generic neural microcircuit models can be trained to generate basic arm movements. Such movement generation is independent of the arm model used and the type of feedback that the circuit receives. We demonstrate this by considering two different models of a two-jointed arm, a standard model from robotics and a standard model from biology, that each generates different kinds of feedback. Feedback that arrives with biologically realistic delays of 50 to 280 ms turns out to give rise to the best performance. If a feedback with such desirable delay is not available, the neural microcircuit model also achieves good performance if it uses internally generated estimates of such feedback. Existing methods for movement generation in robotics that take the particular dynamics of sensors and actuators into account (embodiment of motor systems) are taken one step further with this approach, which provides methods for also using the embodiment of motion generation circuitry, that is, the inherent dynamics and spatial structure of neural circuits, for the generation of movement.     

MOSAIC for multiple-reward environments

Reinforcement learning (RL) can provide a basic framework for autonomous robots to learn to control and maximize future cumulative rewards in complex environments. To achieve high performance, RL controllers must consider the complex external dynamics for movements and task (reward function) and optimize control commands. For example, a robot playing tennis and squash needs to cope with the different dynamics of a tennis or squash racket and such dynamic environmental factors as the wind. In addition, this robot has to tailor its tactics simultaneously under the rules of either game. This double complexity of the external dynamics and reward function sometimes becomes more complex when both the multiple dynamics and multiple reward functions switch implicitly, as in the situation of a real (multi-agent) game of tennis where one player cannot observe the intention of her opponents or her partner. The robot must consider its opponent's and its partner's unobservable behavioral goals (reward function). In this article, we address how an RL agent should be designed to handle such double complexity of dynamics and reward. We have previously proposed modular selection and identification for control (MOSAIC) to cope with nonstationary dynamics where appropriate controllers are selected and learned among many candidates based on the error of its paired dynamics predictor: the forward model. Here we extend this framework for RL and propose MOSAIC-MR architecture. It resembles MOSAIC in spirit and selects and learns an appropriate RL controller based on the RL controller's TD error using the errors of the dynamics (the forward model) and the reward predictors. Furthermore, unlike other MOSAIC variants for RL, RL controllers are not a priori paired with the fixed predictors of dynamics and rewards. The simulation results demonstrate that MOSAIC-MR outperforms other counterparts because of this flexible association ability among RL controllers, forward models, and reward predictors.     

Standardized low-load repetitive work: evidence of different motor control strategies between experienced workers and a reference group

The purpose of this laboratory study was to evaluate the possible differences in motor strategies to a new standardized low-load repetitive work task in between healthy experienced workers and a reference group. Work task event duration, i.e. working rhythm, cutting forces, surface electromyographic (EMG) activity from four shoulder muscles, postural activity, and arm and trunk movements in 3D were recorded during low-load repetitive work simulation. The experienced group showed lower EMG activity and frequency contents (P<0.05), more abducted position of the upper arm and forward flexion of the trunk prior to work simulation (P<0.05), and increased arm and trunk range of motion (P<0.05) compared with the reference group. The results highlight that experienced butchers have a different motor strategy compared with a reference group, i.e. more variable form of coordination pattern. Furthermore, the initial implementation of a possible protective motor strategy by experienced workers might be a very important prognostic factor.     

A novel adaptive control for reaching movements of an anthropomorphic arm driven by pneumatic artificial muscles

Pneumatic artificial muscle is widely used since it has the advantage including simple structure, lightweight, force controllable, compliance and so on. In this paper, a two-link anthropomorphic arm is constructed by pneumatic artificial muscles for the upper-limb rehabilitation training. In order to assist impaired upper-limb patients to achieve various training, the anthropomorphic arm should realize flexible human reaching movements. Due to frictions and model uncertainties of the anthropomorphic arm system, an adaptive fuzzy backstepping control is proposed to ensure the stability and the adaptivity during the motion. The control method is proved by Lyapunov asymptotic stability and is verified by numerical simulations. Furthermore, experiments are performed and results demonstrate that the proposed control method is efficient and robust.     

Dynamic motion learning for multi-DOF flexible-joint robots using active–passive motor babbling through deep learning

This paper proposes a learning strategy for robots with flexible joints having multi-degrees of freedom in order to achieve dynamic motion tasks. In spite of there being several potential benefits of flexible-joint robots such as exploitation of intrinsic dynamics and passive adaptation to environmental changes with mechanical compliance, controlling such robots is challenging because of increased complexity of their dynamics. To achieve dynamic movements, we introduce a two-phase learning framework of the body dynamics of the robot using a recurrent neural network motivated by a deep learning strategy. The proposed methodology comprises a pre-training phase with motor babbling and a fine-tuning phase with additional learning of the target tasks. In the pre-training phase, we consider active and passive exploratory motions for efficient acquisition of body dynamics. In the fine-tuning phase, the learned body dynamics are adjusted for specific tasks. We demonstrate the effectiveness of the proposed methodology in achieving dynamic tasks involving constrained movement requiring interactions with the environment on a simulated robot model and an actual PR2 robot both of which have a compliantly actuated seven degree-of-freedom arm. The results illustrate a reduction in the required number of training iterations for task learning and generalization capabilities for untrained situations.     

Models of human movement: Trajectory planning and inverse kinematics studies

The seemingly simple everyday actions of moving limb and body to accomplish a motor task or interact with the environment are incredibly complex. To reach for a target we first need to sense the target’s position with respect to an external coordinate system; we then need to plan a limb trajectory which is executed by issuing an appropriate series of neural commands to the muscles. These, in turn, exert appropriate forces and torques on the joints leading to the desired movement of the arm. Here we review some of the earlier work as well as more recent studies on the control of human movement, focusing on behavioral and modeling studies dealing with task space and joint-space movement planning. At the task level, we describe studies investigating trajectory planning and inverse kinematics problems during point-to-point reaching movements as well as two-dimensional (2D) and three-dimensional (3D) drawing movements. We discuss models dealing with the two-thirds power law, particularly differential geometrical approaches dealing with the relation between path geometry and movement velocity. We also discuss optimization principles such as the minimum-jerk model and the isochrony principle for point-to-point and curved movements.We next deal with joint-space movement planning and generation, discussing the inverse kinematics problem and common solutions to the problems of kinematic redundancy. We address the question of which reference frames are used by the nervous system and review studies examining the employment of kinematic constraints such as Donders’ and Listing’s laws. We also discuss optimization approaches based on Riemannian geometry.One principle of motor coordination during human locomotion emerging from this body of work is the intersegmental law of coordination. However, the nature of the coordinate systems underlying motion planning remains of interest as they are related to the principles governing the control of human arm movements.