Integrating Human Inputs with Autonomous Behaviors on an Intelligent Wheelchair Platform

Nearly five million individuals in the US have limited arm and hand movement, making it difficult or impossible for them to use computers and products with embedded computers, such as wheelchairs, household appliances, office electronic equipment, and robotic aids. Although some current wheelchair systems have embedded computers, they have very little computer control and require precise, low-level control inputs from the user; interfaces are similar to those found in passenger cars. The rider must continuously specify the chair's direction and, in some cases, velocity using a joystick-like device. Unfortunately, many users who could benefit from powered wheelchairs lack these fine motor skills. For instance, those with cerebral palsy might not be able to guide a chair through a narrow opening, such as a doorway, without repeatedly colliding into the sides. These types of physically challenging environments can be frustrating and require a lot of user effort. At the University of Pennsylvania's general robotics, automation, sensing, and perception lab, we developed the smart chair, a smart wheelchair with intelligent controllers that lets people with physical disabilities overcome these difficulties. By outfitting the wheelchair with cameras, a laser range finder, and onboard processing, we give the user an adaptable, intelligent control system. A computer-controlled wheelchair's shared control framework allows users complete control of the chair while ensuring their safety     

Hefestos: an intelligent system applied to ubiquitous accessibility

This article proposes Hefestos, an intelligent system applied to ubiquitous accessibility. This model uses ubiquitous computing concepts to manage accessibility resources for people with disabilities. Among the concepts employed, context awareness, user profiles and trails management can be highlighted. The paper proposes an ontology for accessibility and delineates scenarios of its application in everyday life of people with disabilities. Moreover, the implementation of a smart wheelchair prototype and its application in a practical experiment is described. Ten users with a range of disability degrees tried the system and filled out a survey based on the technology acceptance model. This experiment demonstrated the main functionalities and the acceptance of the system. The results showed 96 % of acceptance regarding perceived easy of use and 98 % in perceived usefulness. These results were encouraging and show the potential for implementing Hefestos in real life situations.     

Development of a future Intelligent Sweet Home for the disabled

This article introduces a new smart house, the Intelligent Sweet Home, developed at the Korea Advanced Institute of Science and Technology (KAIST), Korea, for testing advanced concepts for independent living for elderly and physically handicapped individuals. The work focuses on human-friendly technical solutions for motion/mobility assistance, health monitoring, and advanced human–machine interfaces that provide easy control of both assistive devices and home-installed appliances. The smart house behaves in accordance with the user's commands, the user's intentions, and the user's current health status. Various interfaces based on hand gestures, voice, body movement and posture, and the health monitoring system were studied and tested. This article emphasizes the structure of the system. This work was presented in part at the 11th International Symposium on Artificial Life and Robotics, Oita, Japan, January 23–25, 2006     

Integration of a Rehabilitation Robotic System (KARES II) with Human-Friendly Man-Machine Interaction Units

In this paper, we report some important results of design and evaluation of a wheelchair-based robotic arm system, named as KARES II (KAIST Rehabilitation Engineering Service System II), which is newly developed for the disabled. KARES II is designed in consideration of surveyed necessary tasks for the target users (that is, people with spinal cord injury). At first, we predefined twelve important tasks according to extensive interviews and questionnaires. Next, based on these tasks, all subsystems are designed, simulated and developed. A robotic arm with active compliance and intelligent visual servoing capability is developed by using cable-driven mechanism. Various kinds of human-robot interfaces are developed to provide broad range of services according to the levels of disability. Eye-mouse, shoulder/head interface, EMG signal-based control subsystems are used for this purpose. Besides, we describe the process of integration of our rehabilitation robotic system KARES II, and discuss about user trials. A mobile platform and a wheelchair platform are two main platforms in which various subsystems are installed. For a real-world application of KARES II system, we have performed user trials with six selected potential end-users (with spinal cord injury).     

面向智能家居/智慧生活的服务机器人技术与系统

随着机器人技术的发展以及服务机器人形态的日益丰富,智能家居/智慧生活逐渐成为一种未来的生活方式。文章探索了智能家居和智慧生活的服务机器人技术与系统,在其众多的关键技术中,重点探讨了面向服务机器人的云端融合技术以及高用户体验度的人-机器人交互技术。并以家庭服务机器人为例,实现了基于微信和语音云的人-机器人交互方式,验证了基于云架构的家庭服务机器人体系结构的可行性。文章所提出的服务机器人技术与系统是智能家居、智慧生活的一种实现方式,为未来形形色色的服务机器人技术方案提供了一些思路。     

A robotics wheelchair for crowded public environment

Describes the robotic wheelchair MAid (Mobility Aid for Elderly and Disabled People). MAid's task is to support and transport people with limited motion skills. It is based on a commercial wheelchair that has been equipped with an intelligent control and navigation system. Conversations with disabled and elderly people and with their physicians indicate that the automatic functions desired in a robotic wheelchair do not include following walls or passing doorways, but do include navigation in narrow, cluttered environments and through wide, crowded areas. MAid performs these functions     

Semi-autonomous Navigation of a Robotic Wheelchair

The present work considers the development of a wheelchair for people with special needs, which is capable of navigating semi-autonomously within its workspace. This system is expected to prove useful to people with impaired mobility and limited fine motor control of the upper extremities. Among the implemented behaviors of this robotic system are the avoidance of obstacles, the motion in the middle of the free space and the following of a moving target specified by the user (e.g., a person walking in front of the wheelchair). The wheelchair is equipped with sonars, which are used for distance measurement in preselected critical directions, and with a panoramic camera with a 360 degree field of view, which is used for following a moving target. After suitably processing the color sequence of the panoramic images using the color histogram of the desired target, the orientation of the target with respect to the wheelchair is determined, while its distance is determined by the sonars. The motion control laws developed for the system use the sensory data and take into account the non-holonomic kinematic constraints of the wheelchair, in order to guarantee certain desired features of the closed-loop system, such as stability. Moreover, they are as simplified as possible to minimize implementation requirements. An experimental prototype has been developed at ICS–FORTH, based on a commercially-available wheelchair. The sensors, the computing power and the electronics needed for the implementation of the navigation behaviors and of the user interfaces (touch screen, voice commands) were developed as add-on modules and integrated with the wheelchair.     

Design and Implementation of a Multi Sensor Based Brain Computer Interface for a Robotic Wheelchair

In this study, design and implementation of a multi sensor based brain computer interface for disabled and/or elderly people is proposed. Developed system consists of a wheelchair, a high-power motor controller card, a Kinect camera, electromyogram (EMG) and electroencephalogram (EEG) sensors and a computer. The Kinect sensor is installed on the system to provide safe navigation for the system. Depth frames, captured by the Kinect’s infra-red (IR) camera, are processed with a custom image processing algorithm in order to detect obstacles around the wheelchair. A Consumer grade EMG device (Thalmic Labs) was used to obtain eight channels of EMG data. Four different hand movements: Fist, release, waving hand left and right are used for EMG based control of the robotic wheelchair. EMG data is first classified using artificial neural network (ANN), support vector machines and random forest schemes. The class is then decided by a rule-based scheme constructed on the individual outputs of the three classifiers. EEG based control is adopted as an alternative controller for the developed robotic wheelchair. A wireless 14-channels EEG sensor (Emotiv Epoch) is used to acquire real time EEG data. Three different cognitive tasks: Relaxing, math problem solving, text reading are defined for the EEG based control of the system. Subjects were asked to accomplish the relative cognitive task in order to control the wheelchair. During experiments, all subjects were able to control the robotic wheelchair by hand movements and track a pre-determined route with a reasonable accuracy. The results for the EEG based control of the robotic wheelchair are promising though vary depending on user experience.     

A Sensorized Garment Controlled Virtual Robotic Wheelchair

This paper presents the design and performance of a body-machine-interface (BoMI) system, where a user controls a robotic 3D virtual wheelchair with the signals derived from his/her shoulder and elbow movements. BoMI promotes the perspective that system users should no longer be operators of the engineering design but should be an embedded part of the functional design. This BoMI system has real-time controllability of robotic devices based on user-specific dynamic body response signatures in high-density 52-channel sensor shirt. The BoMI system not only gives access to the user’s body signals, but also translates these signals from user’s body to the virtual reality device-control space. We have explored the efficiency of this BoMI system in a semi-cylinderic 3D virtual reality system. Experimental studies are conducted to demonstrate, how this transformation of human body signals of multiple degrees of freedom, controls a robotic wheelchair navigation task in a 3D virtual reality environment. We have also presented how machine learning can enhance the interface to adapt towards the degree of freedoms of human body by correcting the errors performed by the user.     

Look where you're going [robotic wheelchair]

We propose a robotic wheelchair that observes the user and the environment. It can understand the user's intentions from his/her behaviors and the environmental information. It also observes the user when he/she is off the wheelchair, recognizing the user's commands indicated by hand gestures. Experimental results show our approach to be promising. Although the current system uses face direction, for people who find it difficult to move their faces, it can be modified to use the movements of the mouth, eyes, or any other body parts that they can move. Since such movements are generally noisy, the integration of observing the user and the environment will be effective in understanding the real intentions of the user and will be a useful technique for better human interfaces.     

DOGeye: Controlling your home with eye interaction

Nowadays home automation, with its increased availability, reliability and with its ever reducing costs is gaining momentum and is starting to become a viable solution for enabling people with disabilities to autonomously interact with their homes and to better communicate with other people. However, especially for people with severe mobility impairments, there is still a lack of tools and interfaces for effective control and interaction with home automation systems, and general-purpose solutions are seldom applicable due to the complexity, asynchronicity, time dependent behavior, and safety concerns typical of the home environment. This paper focuses on user-environment interfaces based on the eye tracking technology, which often is the only viable interaction modality for users as such. We propose an eye-based interface tackling the specific requirements of smart environments, already outlined in a public Recommendation issued by the COGAIN European Network of Excellence. The proposed interface has been implemented as a software prototype based on the ETU universal driver, thus being potentially able to run on a variety of eye trackers, and it is compatible with a wide set of smart home technologies, handled by the Domotic OSGi Gateway. A first interface evaluation, with user testing sessions, has been carried and results show that the interface is quite effective and usable without discomfort by people with almost regular eye movement control.     

Adaptable navigational assistance for intelligent wheelchairs by means of an implicit personalized user model

Elderly and disabled people can experience considerable difficulties when driving a powered wheelchair, especially if they do not possess the fine steering capacities that are required to perform certain manoeuvres, like avoiding obstacles or docking at tables. In order to help these people, several “intelligent” wheelchairs have been developed in the past, meaning that a powered wheelchair was endowed with the abilities to provide navigational assistance to its user. In such a scenario, control over the wheelchair is shared between the user and the intelligent assistance. The problem of the existing intelligent wheelchairs is, however, that the rules for assistance are hard-coded and by consequence not adaptable to the personal handicap and needs of a specific user. Therefore, this paper proposes a new, more user-centered approach to shared wheelchair control. The presented framework executes two tasks: it continuously estimates the user’s intention and it determines whether the user needs assistance to achieve that intention. An implicit user model is introduced and incorporated in the framework, in order to make the execution of both tasks adaptable to a specific user. This paper presents the proposed framework, along with experimental results in simulation and on a real wheelchair.     

基于SPCE061A的智能轮椅控制系统硬件设计

随着人口老龄化问题越来越严重以及人民生活水平的提高,无论是高龄老年人还是先天或后天肢残者对轮椅等辅助步行工具的需求将日益迫切。在脑电设备的发送和接收控制端加入了基于SPCE061技术的语音识别系统,当大脑发出操作意图时,该信号经过处理触发语音播报系统,而受控设备上安装的语音接收端经过语音辨识,发出正确的驱动信息并进行语音播报,从而控制轮椅的电机系统,如果驾驶者从听觉上感知到信息错误,可及时调整大脑意识,从而实现脑电识别和语音识别对智能轮椅的双保险控制,为有肢体障碍的残疾人提供一种新的控制方式。     

Moving towards inclusive design guidelines for socially and ethically aware HCI

Most people acknowledge that personal computers have enormously enhanced the autonomy and communication capacity of people with special needs. The key factor for accessibility to these opportunities is the adequate design of the user interface which, consequently, has a high impact on the social lives of users with disabilities.The design of universally accessible interfaces has a positive effect over the socialisation of people with disabilities. People with sensory disabilities can profit from computers as a way of personal direct and remote communication. Personal computers can also assist people with severe motor impairments to manipulate their environment and to enhance their mobility by means of, for example, smart wheelchairs. In this way they can become more socially active and productive. Accessible interfaces have become so indispensable for personal autonomy and social inclusion that in several countries special legislation protects people from ‘digital exclusion’.To apply this legislation, inexperienced HCI designers can experience difficulties. They would greatly benefit from inclusive design guidelines in order to be able to implement the ‘design for all’ philosophy. In addition, they need clear criteria to avoid negative social and ethical impact on users. This paper analyses the benefits of the use of inclusive design guidelines in order to facilitate a universal design focus so that social exclusion is avoided. In addition, the need for ethical and social guidelines in order to avoid undesirable side effects for users is discussed. Finally, some preliminary examples of socially and ethically aware guidelines are proposed.     

Brain-coupled interaction for semi-autonomous navigation of an assistive robot

This paper presents a novel semi-autonomous navigation strategy designed for low throughput interfaces. A mobile robot (e.g. intelligent wheelchair) proposes the most probable action, as analyzed from the environment, to a human user who can either accept or reject the proposition. In the case of refusal, the robot will propose another action, until both entities agree on what needs to be done.In an unknown environment, the robotic system first extracts features so as to recognize places of interest where a human–robot interaction should take place (e.g. crossings). Based on the local topology, relevant actions are then proposed, the user providing answers by means of a button or a brain–computer interface (BCI). Our navigation strategy is successfully tested both in simulation and with a real robot, and a feasibility study for the use of a BCI confirms the potential of such an interface.     

基于智能Agent的Web信息检索系统

设计和实现了一个基于智能Agent的面向领域的Web信息检索系统，讨论了该系统的体系结构和各模块设计思想。经实验验证，该系统具有检索精度商、检索方式多样、用户界面友好等特点。     

A Wheelchair Steered through Voice Commands and Assisted by a Reactive Fuzzy-Logic Controller

This paper describes new results with a Reactive Shared-Control system that enables a semi-autonomous navigation of a wheelchair in unknown and dynamic environments. The purpose of the reactive shared controller is to assist wheelchair users providing an easier and safer navigation. It is designed as a fuzzy-logic controller and follows a behaviour-based architecture. The implemented behaviours are three: intelligent obstacle avoidance, collision detection and contour following. Intelligent obstacle avoidance blends user commands, from voice or joystick, with an obstacle avoidance behaviour. Therefore, the user and the vehicle share the control of the wheelchair. The reactive shared control was tested on the RobChair powered wheelchair prototype [6] equipped with a set of ranging sensors. Experimental results are presented demonstrating the effectiveness of the controller.     

Reinforcement learning based compliance control of a robotic walk assist device

ABSTRACT

Millions of people around the globe have to deal with walking disability. Robotic walk assist devices can help people with walking disabilities, especially those with weak legs. However, safety, cost, efficiency and user friendliness are some of the key challenges. For robotic walk assist devices, light weight structure and energy efficient design as well as optimal control are vitally important. In addition, compliance control can help to improve the safety of such devices as well as contribute to their user friendliness. In this paper, an optimal adaptive compliance control is proposed for a Robotic walk assist device. The suggested scheme is based on bio-inspired reinforcement learning. It is completely dynamic-model-free scheme and employs joint position and velocity feedback as well as sensed joint torque (applied by user during walk) for compliance control. The efficiency of the controller is tested in simulation on a robotic walk assisting device model.     

基于物联网技术的智能家居系统的研究与探讨

文章介绍了智能家居的架构和物联网技术,简述了智能家居系统所要研究的内容,分析了智能家居系统的组成、采用的技术和通信手段等。从系统可靠性、实用性、安全性等出发,提出了一套基于ZigBee和移动终端技术的智能家居系统的设计方案,并针对此方案进行了比较全面的分析、研究和探讨,以期对智能家居市场的发展起到一定的引导和推动作用。     

Mobility assistance system for an electric wheelchair using annotated maps

An electric wheelchair is very useful for physically challenged or elderly people. A electric wheelchair can be equipped with a computer and sensors such as cameras and LIDARs to make it a robotic wheelchair. This opens the possibility to reduce the burden and increase the comfort of the users with use of robotics technology. In this paper, we begin with a dynamic shared control and propose to use a map with annotations to increase the user’s comfort while keeping safety. We provide a system with localization, which can relate the current position of the wheelchair to the annotations. Both quantitative evaluation and subjective survey demonstrate the effectiveness of the proposed method.