Development and Control of Multi‐Degree‐of‐Freedom Mobile Robot for Acquisition of Road Environmental Modes

The acquisition of information about the environment around a mobile robot is important for purposes such as controlling the robot from a remote location and in situations such as that when the robot is running autonomously. In some research, audiovisual information has been used. However, the acquisition of information about force sensation, which is included in environmental information, has not been well investigated. The mobile‐hapto, which is a remote control system with force information, has been proposed, but the robot used for the system can acquire only the horizontal components of forces. For this reason, in this research a three‐wheeled mobile robot consisting of seven actuators was developed and a control system was created for it. The robot can acquire information about horizontal and vertical forces without using force sensors. By using this robot, detailed information on the forces in the environment can be acquired and the operability of the robot and its ability to adjust to the environment are expected to improve.     

Design of a haptic data visualization system for people with visual impairments.

Data visualization is a technique used to explore real or simulated data by representing it in a form more suitable for comprehension. This form is usually visual since vision provides a means to perceive large quantities of spatial information quickly. However, people who are blind or visually impaired must rely on other senses to accomplish this perception. Haptic interface technology makes digital information tangible, which can provide an additional medium for data exploration and analysis. Unfortunately, the amount of information that can be perceived through a haptic interface is considerably less than that which can be perceived through vision, so a haptic environment must be enhanced to aid the comprehension of the display. This enhancement includes speech output and the addition of object properties such as friction and texture. Textures are generated which can be modified according to a characteristic or property of the object to which it is applied. For example, textures can be used as an analog to color in graphical displays to highlight variations in data. Taking all of these factors into account, methods for representing various forms of data are presented here with the goal of providing a haptic visualization system without the need for a visual component. The data forms considered include one-, two-, and three-dimensional (1-D, 2-D, and 3-D) data which can be rendered using points, lines, surfaces, or vector fields similar to traditional graphical displays. The end result is a system for the haptic display of these common data sets which is accessible for people with visual impairments.     

Non‐grounded haptic display by controlling wind direction

In recent years, a non‐grounded haptic display that is unconstrained from the ground and the human body has been studied intensively. Route navigation for the visually impaired and escape during an emergency have been considered as applications of such devices because the device should be free in space. However, it is difficult to display the actual force other than a pseudo force because the reaction force received by a user should be supported elsewhere. A wind blower is a possible solution that can display the actual force in midair because the air reacts with the wind pressure without grounding. Here, we propose a non‐grounded haptic display that uses wind pressure. The display changes the force directions because the device should generate translational and rotational forces to navigate forward and change directions. The device should also display the force independent of the device posture. The designed device has two fans, each of which is mounted on two‐axis gimbals at the end of the grip. The device can display translational and rotational forces by orienting the two fans to the same and opposite directions, respectively. We developed a prototype device and conducted a human subject study to investigate the performance of the device in terms of direction discrimination and completion time with different directions and force patterns. The results showed that both translational and rotational forces could be displayed and that rotational forces were perceived more clearly. © 2016 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.     

Enhancement of steady‐state visual evoked potential‐based brain–computer interface systems via a steady‐state motion visual stimulus modality

Steady‐state visual evoked potential (SSVEP)‐based brain–computer interface (BCI) systems are among the most accurate assistive devices for patients with severe disabilities. However, existing visual stimulation patterns lead to eye fatigue, which affects the system performance. Therefore, in this study, we propose two improvements to SSVEP‐based BCI systems. First, we propose a novel visual stimulator that incorporates a visual motion stimulus for the steady‐state visual stimulus to reduce eye fatigue while maintaining the advantages associated with SSVEPs. We also propose two corresponding feature extraction algorithms, i.e. SSVEP detection and visual attention detection, to capture the phenomena of steady‐state motion visual stimulus responses. The accuracy of the test was ∼83.6%. Second, we propose a novel hybrid BCI‐SSVEP system and a motion visual stimulus hybrid BCI system to enhance the SSVEP‐based BCI system during a state of eye fatigue. Participants used the SSVEP system until reaching a fatigued state and then began using a hybrid motion visual stimulus. The accuracy of the proposed system was ∼85.6%. The proposed improvements can be incorporated into practical BCI systems for wheelchair control. © 2017 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.     

Pruning a decision tree for selecting computer-related assistive devices for people with disabilities

Many disabled individuals lack extensive knowledge about assistive technology, which could help them use computers. In 1997, Denis Anson developed a decision tree of 49 evaluative questions designed to evaluate the functional capabilities of the disabled user and choose an appropriate combination of assistive devices, from a selection of 26, that enable the individual to use a computer. In general, occupational therapists guide the disabled users through this process. They often have to go over repetitive questions in order to find an appropriate device. A disabled user may require an alphanumeric entry device, a pointing device, an output device, a performance enhancement device, or some combination of these. Therefore, the current research eliminates redundant questions and divides Anson's decision tree into multiple independent subtrees to meet the actual demand of computer users with disabilities. The modified decision tree was tested by six disabled users to prove it can determine a complete set of assistive devices with a smaller number of evaluative questions. The means to insert new categories of computer-related assistive devices was included to ensure the decision tree can be expanded and updated. The current decision tree can help the disabled users and assistive technology practitioners to find appropriate computer-related assistive devices that meet with clients' individual needs in an efficient manner.     

Toward Okinawa‐Style Robot‐Embedded Systems

For the Okinawan economy to grow independently, it is essential to create new businesses; this will require the cultivation of technologies that will form the basis for those businesses, as well as the training of technicians and researchers. To promote the creation of new businesses and innovation that are “Unique to Okinawa,” the Institute of Electrical Engineers of Japan set up a cooperative research committee to develop Okinawa‐style robot‐embedded systems. We proposed the concept of Okinawa‐style robot‐embedded systems. To realize this concept, we organized technical meetings to promote research. Now our research focuses on developing playback‐type navigation robots for use in land and air, and also on developing an underwater robot for conserving marine life in the waters around Okinawa. It is necessary to conduct proper measurements, observations, and sampling in water. To successfully perform these tasks, we needed to develop an underwater robot. We have made efforts in developing hardware, software, and applications for embedded techniques. We suggest that coarsely integrated operand scanning (CIOS) and bilateral filters (BFs) be implemented in hardware. We also developed devices to improve quality of life and to support disabled children through software.     

Multi‐command SSAEP‐based BCI system with training sessions for SSVEP during an eye fatigue state

This paper proposes a steady‐state auditory stimulus modality and a detection algorithm to replace steady‐state visual evoked potential (SSVEP )‐based brain–computer interface (BCI ) systems during visual fatigue periods. The optimal speaker position for the steady‐state auditory evoked potential (SSAEP )‐based BCI system and possible electrode positions are investigated. Using the proposed system, an accuracy of 85% for two commands was achieved based on the T3–T5 and T4–T6 electrode positions using only one speaker. SSAEP is a promising BCI modality for mitigating the problem of eye fatigue that often occurs during the use of SSVEP ‐based BCI systems. However, SSAEP ‐based BCI systems suffer from low accuracy. To increase accuracy, we propose a new enhanced SSAEP training method. The training process was enhanced by instructing users to control their attention levels while simultaneously detecting an auditory stimulus frequency. Furthermore, we propose a corresponding single‐frequency, multi‐command BCI paradigm for the proposed training method. With the proposed paradigm, four commands can be detected using only one auditory stimulus frequency. The proposed training system yielded an accuracy of ∼81% compared to 66% for sessions performed without the proposed training method. © 2017 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.     

Position and Rotation Estimation for Mobile Robots in Outside of Recording Path Using Ego‐Motion

In this research, estimating the position and rotation of a mobile robot outside of a recording path is realized by applying ego‐motion to view‐based navigation. The ego‐motion is calculated based on the differences in 3D positions of SURF feature points between recording and current images obtained by a Kinect sensor. In conventional view‐based navigation, it is difficult to plan another path when people and objects are found in the recording path. By using the authors’ proposed estimation method, it is possible to realize flexible path planning in actual environments that include people and objects. Based on the results of experiments performed in actual indoor environments, the authors evaluated measurement accuracy for the robot's position and rotation estimated under their method, and confirmed the viability of their method for actual environments including people and objects.     

A navigation system for increasing the autonomy and the security of powered wheelchairs.

Assistive technology is an emerging area where some robotic devices can be used to strengthen the residual abilities of individuals with motor disabilities or to substitute their missing function thus helping them to gain a level of independence at least in the activities of daily living. This paper presents the design of a navigation system and its integration with a commercial powered wheelchair. The navigation system provides the commercial wheelchair with a set of functions which increase the autonomy of elderly and people with motor disabilities. In general, a robot device must be adapted to assistive applications in such a way as to be easily managed by the user. Users, especially young ones, prefer to directly control the robotic device and this aspect of usability has to be managed without affecting the security and efficiency of the navigation module. These aspects have been considered as specifications for the navigation module of powered wheelchairs. Different autonomy levels of the navigation module and proper user interfaces have been developed. Two autonomy levels have been designed. Simple collision avoidance is also implemented in order to stop the mobile base when an obstacle is detected. The preliminary technical tests performed on the navigation system have shown satisfactory results in terms of security and response time. A modular solution for the navigation module was considered in order to simplify the adaptation of the module to different powered wheelchairs.     

Object manipulation improvements due to single session training outweigh the differences among stimulation sites during vibrotactile feedback

Most hand prostheses do not provide intentional haptic feedback about movement performance; thus users must rely almost completely on visual feedback. This paper focuses on understanding the effects of learning and different stimulation sites when vibrotactile stimulation is used as the intentional haptic feedback. Eighteen unimpaired individuals participated in this study with a robotic interface to manipulate a virtual object with visual and vibrotactile feedback at four body sites (finger, arm, neck, and foot) presented in a random order. All participants showed improvements in object manipulation performance with the addition of vibrotactile feedback. Specifically, performance showed a strong learning effect across time, with learning transferring across different sites of vibrotactile stimulation. The effects of learning over the experiment overshadowed the effects of different stimulation sites. The addition of a cognitive task slowed participants and increased the subjective difficulty. User preference ratings showed no difference in their preference among vibrotactile stimulation sites. These findings indicate that the stimulation site may not be as critical as ensuring adequate training with vibrotactile feedback during object manipulation. Future research to identify improvements in vibrotactile-based feedback parameters with amputees is warranted.     

Guiding a mobile robot with cellular neural networks

We show how cellular neural networks (CNNs) are capable of providing the necessary signal processing needed for visual navigation of an autonomous mobile robot. In this way, even complex feature detection and object recognition can be obtained in real time by analogue hardware, making fully autonomous real‐time operation feasible. An autonomous robot was first simulated and then implemented by simulating the CNN with a DSP. The robot is capable of navigating in a maze following lines painted on the floor. Images are processed entirely by a CNN‐based algorithm, and navigation is controlled by a fuzzy‐rule‐based algorithm. Copyright © 2002 John Wiley & Sons, Ltd.     

Control of a 6DOF Mobile Manipulator with Object Detection and Tracking Using Stereo Vision

A supportive mobile robot for assisting the elderly is an emerging requirement mainly in countries like Japan where population ageing become relevant in near future. Falls related injuries are considered as a critical issue when taking into account the physical health of older people. A personal assistive robot with the capability of picking up and carrying objects for long/short distances can be used to overcome or lessen this problem. Here, we design and introduce a 3D dynamic simulation of such an assistive robot to perform pick and place of objects through visual recognition. The robot consists of two major components; a robotic arm or manipulator to do the pick and place, and an omnidirectional wheeled robotic platform to support mobility. Both components are designed and operated according to their kinematics and dynamics and the controllers are integrated for the combined performance. The objective was to improve the ac-curacy of the robot at a considerably high speed. Designed mobile manipulator has been successfully tested and sim-ulated with a stereo vision system to perform object recognition and tracking in a virtual environment resembling aroom of an elderly care. The tracking accuracy of the mobile manipulator at an average speed of 0.5m/s is 90% and is well suited for the proposed application.     

机器人轮椅用户行为与自主导航动态共享控制方法

机器人轮椅人机交互研究表明长时间采用单一模式交互容易导致用户操作意图误判且稳定性会变差,完全自主模式亦会因用户缺乏控制体验而产生沮丧情绪。针对当前基于人机协作的机器人轮椅交互存在模式间硬切换与缺乏对环境变化的动态调整能力的问题,本文以手势交互控制轮椅为基础,提出一种基于用户行为与自主导航相结合的机器人轮椅动态共享控制方法。首先基于Leap Motion传感器追踪用户掌心坐标,生成用户手势速度指令;其次,基于RPLIDAR A1激光雷达传感器并结合自主导航算法,生成自主导航控制指令;最后基于距离、疲劳及误差等多种约束条件实时更新人-机控制指令权重,实现对机器人轮椅的动态共享控制。实验结果表明,本文所提动态共享控制方法能根据轮椅运行环境与用户操作性能动态调整不同模式间的角色分配,避免了不同模式间的直接硬切换,具有较好的用户体验。     

Virtual reality-based orthopedic telerehabilitation.

Rehabilitation interventions in remote areas are problematic because of distance and available resources. Orthopedic impairments acquired by individuals in remote areas can then lead to permanent disabilities/loss of function because of lack of appropriate rehabilitation. A system being developed by Rutgers and Stanford Universities provides therapy at the patient's home, with remote monitoring and periodic re-assessment. This telerehabilitation system uses virtual reality and haptic interfaces, and a pair of networked PCs. It is intended for rehabilitation of patients with hand, elbow, knee and ankle impairments. Data from the first patient treated with the telerehabilitation system is encouraging.     

An audio- and speech-based interface for computer-controlled scientific instruments.

Laboratory instruments are intrinsic to research and work in a wide array of scientific fields. They are used for the control of devices, data storage, and data analysis. The control of instruments is increasingly changing from independent on-instrument controls to multiple instrument integrate software control. Unfortunately, the graphical representation of controls and data makes it difficult for an individual with a visual impairment to independently operate laboratory instruments. Alternative interfaces have been previously developed for these individuals but have often proved limited in scope and accuracy, or otherwise expensive. The resulting inaccessibility to affordable and accurate scientific instrumentation, unfortunately, discourages many individuals with a visual impairment from entering scientific fields of research or work. This paper introduces an alternative interface method developed for LabVIEW, National Instruments' instrumentation software package. The method is specifically designed for individuals with visual impairments, and uses alternative navigation techniques as well as audio feedback. The developed user interface uses simple keyboard inputs to traverse through a hierarchical tree-based menu system. Speech and audio tones are used to alert the user to system settings and errors, as well as a help mechanism and data analysis tool. At this time, alternative interfaces have been developed for the following basic laboratory instruments: an oscilloscope and function/arbitrary waveform generator. The interface methodology, however, can be extended to include any scientific instrument that can be controlled by LabVIEW.     

Multi‐target detection and grasping control for humanoid robot NAO

Graspirng objects is an important capability for humanoid robots. Due to complexity of environmental and diversity of objects, it is difficult for the robot to accurately recognize and grasp multiple objects. In response to this problem, we propose a robotic grasping method that uses the deep learning method You Only Look Once v3 for multi‐target detection and the auxiliary signs to obtain target location. The method can control the movement of the robot and plan the grasping trajectory based on visual feedback information. It is verified by experiments that this method can make the humanoid robot NAO grasp the object effectively, and the success rate of grasping can reach 80% in the experimental environment.     

Stochastic stability of a neural‐net robot controller subject to signal‐dependent noise in the learning rule

We consider a neural network‐based controller for a rigid serial link manipulator with uncertain plant parameters. We assume that the training signal to the network is corrupted by signal‐dependent noise. A radial basis function network is utilized in the feedforward control to approximate the unknown inverse dynamics. The weights are adaptively adjusted according to a gradient descent plus a regulation term (Narendra's e‐modification). We prove a theorem that extends the Yoshizawa D‐boundedness results to the stochastic setting. As in the deterministic setting, this result is particularly useful for neural network robot control when there exists bounded torque disturbances and neural net approximation errors over a known compact set. Using this result, we establish bounds on the feedback gains and learning rate parameters that guarantee the origin of the closed‐loop system is semi‐globally, uniformly bounded in expected value. Copyright © 2009 John Wiley & Sons, Ltd.     

Teaching sampled‐data H∞ control theory using arm robot experiment system

This paper explains how to use an arm robot experiment system to teach sampled‐data H_∞ control theory. A design procedure is presented for a digital tracking control system for a continuous plant with structured uncertainties; the target is the positioning control of an arm robot. To guarantee the robust stability of the closed‐loop system and provide the desired closed‐loop performance, the design problem is first formulated as a sampled‐data H_∞ control problem, and is then transformed into an equivalent discrete‐time H_∞ control problem. Finally, linear matrix inequalities are used to obtain a reduced‐order output‐feedback controller and a static state‐feedback controller. In a course, the design procedure is explained and practice is provided through simulations and experiments. © 2011 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.     

A self‐calibrated delay‐locked loop with low static phase error

In conventional delay‐locked loop circuits, the charge and discharge of the charge pump result in mismatched current reflecting the size of the static phase error. The static phase error between feedback clock and reference clock is likely to be within tens or hundreds of picoseconds (ps). We thus propose an approach using digital calibration methods to reduce the charge pump current mismatch by means of the setup time of the D‐type flip flop. The setup time of D‐type flip flop is determined and duplicated to detect the phase error between the reference clock and feedback clock. It results in a very small static phase error between the reference clock and feedback clock. This paper used a 0.18 µm CMOS process design, with a reference frequency of 700 ~ 900 MHz. The active area is 0.031 mm², and the phase error after correction is less than 5 ps. Copyright © 2015 John Wiley & Sons, Ltd.     

An optimized collaborative filtering method to construct spatial‐temporal behavior pattern‐based user interest model

Mobile social network (MSN ) has properties of real time, mobility, and social relationship, and provides more real‐time, more dimensional, and more heterogeneous metadata. The traditional user interest models represent mobile users' interests unsatisfactorily because of these inherent characteristics. This paper aims to combine the characteristics of mobile social networks appropriately to build a user interest model effectively for representing user interests. The constructed model is supposed to enable users to filter uninterested information, to provide personalized mobile service, to solve information overload, and to gain a good user experience. In order to realize these, first, we explore various issues of the traditional user interest model, and analyze the numerous problems in using traditional models to represent mobile user interests. Then a user spatial‐temporal behavior pattern in conjunction with mobile personalization attributes and context information is proposed, which is followed by a proposal of a user interest model to represent user interests. Second, we introduce a hybrid collaborative filtering method based on users and subjects to calculate user interests and to build the user interest model, considering the user interest extensive degree, the subject popularity, as well as the user influence. Finally, a model construction algorithm is proposed based on the user's spatial‐temporal behavior pattern. Experimental results show that the proposed model can represent user interests effectively under the mobile social network. Furthermore, it is verified that the adaptability and accuracy of the construction algorithm are significantly improved by considering the user interest extensive degree, the subject popularity, and the user influence. © 2016 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.