Brain-computer interfaces for 1-D and 2-D cursor control: designs using volitional control of the EEG spectrum or steady-state visual evoked potentials.

We have developed and tested two electroencephalogram (EEG)-based brain-computer interfaces (BCI) for users to control a cursor on a computer display. Our system uses an adaptive algorithm, based on kernel partial least squares classification (KPLS), to associate patterns in multichannel EEG frequency spectra with cursor controls. Our first BCI, Target Practice, is a system for one-dimensional device control, in which participants use biofeedback to learn voluntary control of their EEG spectra. Target Practice uses a KPLS classifier to map power spectra of 62-electrode EEG signals to rightward or leftward position of a moving cursor on a computer display. Three subjects learned to control motion of a cursor on a video display in multiple blocks of 60 trials over periods of up to six weeks. The best subject's average skill in correct selection of the cursor direction grew from 58% to 88% after 13 training sessions. Target Practice also implements online control of two artifact sources: 1) removal of ocular artifact by linear subtraction of wavelet-smoothed vertical and horizontal electrooculograms (EOG) signals, 2) control of muscle artifact by inhibition of BCI training during periods of relatively high power in the 40-64 Hz band. The second BCI, Think Pointer, is a system for two-dimensional cursor control. Steady-state visual evoked potentials (SSVEP) are triggered by four flickering checkerboard stimuli located in narrow strips at each edge of the display. The user attends to one of the four beacons to initiate motion in the desired direction. The SSVEP signals are recorded from 12 electrodes located over the occipital region. A KPLS classifier is individually calibrated to map multichannel frequency bands of the SSVEP signals to right-left or up-down motion of a cursor on a computer display. The display stops moving when the user attends to a central fixation point. As for Target Practice, Think Pointer also implements wavelet-based online removal of ocular artifact; however, in Think Pointer muscle artifact is controlled via adaptive normalization of the SSVEP. Training of the classifier requires about 3 min. We have tested our system in real-time operation in three human subjects. Across subjects and sessions, control accuracy ranged from 80% to 100% correct with lags of 1-5 s for movement initiation and turning. We have also developed a realistic demonstration of our system for control of a moving map display (http://ti.arc.nasa.gov/).     

The Wadsworth BCI Research and Development Program: at home with BCI.

The ultimate goal of brain-computer interface (BCI) technology is to provide communication and control capacities to people with severe motor disabilities. BCI research at the Wadsworth Center focuses primarily on noninvasive, electroencephalography (EEG)-based BCI methods. We have shown that people, including those with severe motor disabilities, can learn to use sensorimotor rhythms (SMRs) to move a cursor rapidly and accurately in one or two dimensions. We have also improved P300-based BCI operation. We are now translating this laboratory-proven BCI technology into a system that can be used by severely disabled people in their homes with minimal ongoing technical oversight. To accomplish this, we have: improved our general-purpose BCI software (BCI2000); improved online adaptation and feature translation for SMR-based BCI operation; improved the accuracy and bandwidth of P300-based BCI operation; reduced the complexity of system hardware and software and begun to evaluate home system use in appropriate users. These developments have resulted in prototype systems for every day use in people's homes.     

The Berlin Brain-Computer Interface: EEG-based communication without subject training.

The Berlin Brain-Computer Interface (BBCI) project develops a noninvasive BCI system whose key features are 1) the use of well-established motor competences as control paradigms, 2) high-dimensional features from 128-channel electroencephalogram (EEG), and 3) advanced machine learning techniques. As reported earlier, our experiments demonstrate that very high information transfer rates can be achieved using the readiness potential (RP) when predicting the laterality of upcoming left- versus right-hand movements in healthy subjects. A more recent study showed that the RP similarily accompanies phantom movements in arm amputees, but the signal strength decreases with longer loss of the limb. In a complementary approach, oscillatory features are used to discriminate imagined movements (left hand versus right hand versus foot). In a recent feedback study with six healthy subjects with no or very little experience with BCI control, three subjects achieved an information transfer rate above 35 bits per minute (bpm), and further two subjects above 24 and 15 bpm, while one subject could not achieve any BCI control. These results are encouraging for an EEG-based BCI system in untrained subjects that is independent of peripheral nervous system activity and does not rely on evoked potentials even when compared to results with very well-trained subjects operating other BCI systems.     

An enhanced time-frequency-spatial approach for motor imagery classification.

Human motor imagery (MI) tasks evoke electroencephalogram (EEG) signal changes. The features of these changes appear as subject-specific temporal traces of EEG rhythmic components at specific channels located over the scalp. Accurate classification of MI tasks based upon EEG may lead to a noninvasive brain-computer interface (BCI) to decode and convey intention of human subjects. We have previously proposed two novel methods on time-frequency feature extraction, expression and classification for high-density EEG recordings (Wang and He 2004; Wang, Deng, and He, 2004). In the present study, we refined the above time-frequency-spatial approach and applied it to a one-dimensional "cursor control" BCI experiment with online feedback. Through offline analysis of the collected data, we evaluated the capability of the present refined method in comparison with the original time-frequency-spatial methods. The enhanced performance in terms of classification accuracy was found for the proposed approach, with a mean accuracy rate of 91.1% for two subjects studied.     

Electrocorticography-based brain computer interface--the Seattle experience.

Electrocorticography (ECoG) has been demonstrated to be an effective modality as a platform for brain-computer interfaces (BCIs). Through our experience with ten subjects, we further demonstrate evidence to support the power and flexibility of this signal for BCI usage. In a subset of four patients, closed-loop BCI experiments were attempted with the patient receiving online feedback that consisted of one-dimensional cursor movement controlled by ECoG features that had shown correlation with various real and imagined motor and speech tasks. All four achieved control, with final target accuracies between 73%-100%. We assess the methods for achieving control and the manner in which enhancing online control can be accomplished by rescreening during online tasks. Additionally, we assess the relevant issues of the current experimental paradigm in light of their clinical constraints.     

Conversion of EEG activity into cursor movement by a brain-computer interface (BCI)

The Wadsworth electroencephalogram (EEG)-based brain-computer interface (BCI) uses amplitude in mu or beta frequency bands over sensorimotor cortex to control cursor movement. Trained users can move the cursor in one or two dimensions. The primary goal of this research is to provide a new communication and control option for people with severe motor disabilities. Currently, cursor movements in each dimension are determined 10 times/s by an empirically derived linear function of one or two EEG features (i.e., spectral bands from different electrode locations). This study used offline analysis of data collected during system operation to explore methods for improving the accuracy of cursor movement. The data were gathered while users selected among three possible targets by controlling vertical [i.e., one-dimensional (1-D)] cursor movement. The three methods analyzed differ in the dimensionality of the cursor movement [1-D versus two-dimensional (2-D)] and in the type of the underlying function (linear versus nonlinear). We addressed two questions: Which method is best for classification (i.e., to determine from the EEG which target the user wants to hit)? How does the number of EEG features affect the performance of each method? All methods reached their optimal performance with 10-20 features. In offline simulation, the 2-D linear method and the 1-D nonlinear method improved performance significantly over the 1-D linear method. The 1-D linear method did not do so. These offline results suggest that the 1-D nonlinear or the 2-D linear cursor function will improve online operation of the BCI system.     

Training locked-in patients: a challenge for the use of brain-computer interfaces

Training severely paralyzed patients to use a brain-computer interface (BCI) for communication poses a number of issues and problems. Over the past six years, we have trained 11 patients to self-regulate their slow cortical brain potentials and to use this skill to move a cursor on a computer screen. This paper describes our experiences with this patient group including the problems of accepting and rejecting patients, communicating and interacting with patients, how training may be affected by social, familial, and institutional circumstances, and the importance of motivation and available reinforcers.     

Direct control of a computer from the human central nervous system.

We describe an invasive alternative to externally applied brain-computer interface (BCI) devices. This system requires implantation of a special electrode into the outer layers of the human neocortex. The recorded signals are transmitted to a nearby receiver and processed to drive a cursor on a computer monitor in front of the patient. Our present patient has learned to control the cursor for the production of synthetic speech and typing.     

Multi‐command SSAEP‐based BCI system with training sessions for SSVEP during an eye fatigue state

This paper proposes a steady‐state auditory stimulus modality and a detection algorithm to replace steady‐state visual evoked potential (SSVEP )‐based brain–computer interface (BCI ) systems during visual fatigue periods. The optimal speaker position for the steady‐state auditory evoked potential (SSAEP )‐based BCI system and possible electrode positions are investigated. Using the proposed system, an accuracy of 85% for two commands was achieved based on the T3–T5 and T4–T6 electrode positions using only one speaker. SSAEP is a promising BCI modality for mitigating the problem of eye fatigue that often occurs during the use of SSVEP ‐based BCI systems. However, SSAEP ‐based BCI systems suffer from low accuracy. To increase accuracy, we propose a new enhanced SSAEP training method. The training process was enhanced by instructing users to control their attention levels while simultaneously detecting an auditory stimulus frequency. Furthermore, we propose a corresponding single‐frequency, multi‐command BCI paradigm for the proposed training method. With the proposed paradigm, four commands can be detected using only one auditory stimulus frequency. The proposed training system yielded an accuracy of ∼81% compared to 66% for sessions performed without the proposed training method. © 2017 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.     

Brain-computer interface technology: a review of the first international meeting.

Over the past decade, many laboratories have begun to explore brain-computer interface (BCI) technology as a radically new communication option for those with neuromuscular impairments that prevent them from using conventional augmentative communication methods. BCI's provide these users with communication channels that do not depend on peripheral nerves and muscles. This article summarizes the first international meeting devoted to BCI research and development. Current BCI's use electroencephalographic (EEG) activity recorded at the scalp or single-unit activity recorded from within cortex to control cursor movement, select letters or icons, or operate a neuroprosthesis. The central element in each BCI is a translation algorithm that converts electrophysiological input from the user into output that controls external devices. BCI operation depends on effective interaction between two adaptive controllers, the user who encodes his or her commands in the electrophysiological input provided to the BCI, and the BCI which recognizes the commands contained in the input and expresses them in device control. Current BCI's have maximum information transfer rates of 5-25 b/min. Achievement of greater speed and accuracy depends on improvements in signal processing, translation algorithms, and user training. These improvements depend on increased interdisciplinary cooperation between neuroscientists, engineers, computer programmers, psychologists, and rehabilitation specialists, and on adoption and widespread application of objective methods for evaluating alternative methods. The practical use of BCI technology depends on the development of appropriate applications, identification of appropriate user groups, and careful attention to the needs and desires of individual users. BCI research and development will also benefit from greater emphasis on peer-reviewed publications, and from adoption of standard venues for presentations and discussion.     

Boosting bit rates and error detection for the classification of fast-paced motor commands based on single-trial EEG analysis

Brain-computer interfaces (BCIs) involve two coupled adapting systems-the human subject and the computer. In developing our BCI, our goal was to minimize the need for subject training and to impose the major learning load on the computer. To this end, we use behavioral paradigms that exploit single-trial EEG potentials preceding voluntary finger movements. Here, we report recent results on the basic physiology of such premovement event-related potentials (ERP). 1) We predict the laterality of imminent left- versus right-hand finger movements in a natural keyboard typing condition and demonstrate that a single-trial classification based on the lateralized Bereitschaftspotential (BP) achieves good accuracies even at a pace as fast as 2 taps/s. Results for four out of eight subjects reached a peak information transfer rate of more than 15 b/min; the four other subjects reached 6-10 b/min. 2) We detect cerebral error potentials from single false-response trials in a forced-choice task, reflecting the subject's recognition of an erroneous response. Based on a specifically tailored classification procedure that limits the rate of false positives at, e.g., 2%, the algorithm manages to detect 85% of error trials in seven out of eight subjects. Thus, concatenating a primary single-trial BP-paradigm involving finger classification feedback with such secondary error detection could serve as an efficient online confirmation/correction tool for improvement of bit rates in a future BCI setting. As the present variant of the Berlin BCI is designed to achieve fast classifications in normally behaving subjects, it opens a new perspective for assistance of action control in time-critical behavioral contexts; the potential transfer to paralyzed patients will require further study.     

Brain-computer interface research at the Wadsworth Center.

Studies at the Wadsworth Center over the past 14 years have shown that people with or without motor disabilities can learn to control the amplitude of mu or beta rhythms in electroencephalographic (EEG) activity recorded from the scalp over sensorimotor cortex and can use that control to move a cursor on a computer screen in one or two dimensions. This EEG-based brain-computer interface (BCI) could provide a new augmentative communication technology for those who are totally paralyzed or have other severe motor impairments. Present research focuses on improving the speed and accuracy of BCI communication.     

How many people are able to operate an EEG-based brain-computer interface (BCI)?

Ninety-nine healthy people participated in a brain-computer interface (BCI) field study conducted at an exposition held in Graz, Austria. Each subject spent 20-30 min on a two-session BCI investigation. The first session consisted of 40 trials conducted without feedback. Then, a subject-specific classifier was set up to provide the subject with feedback, and the second session - 40 trials in which the subject had to control a horizontal bar on a computer screen - was conducted. Subjects were instructed to imagine a right-hand movement or a foot movement after a cue stimulus depending on the direction of an arrow. Bipolar electrodes were mounted over the right-hand representation area and over the foot representation area. Classification results achieved with 1) an adaptive autoregressive model (39 subjects) and 2) band power estimation (60 subjects) are presented. Roughly 93% of the subjects were able to achieve classification accuracy above 60% after two sessions of training.     

A minimal set of electrodes for motor imagery BCI to control an assistive device in chronic stroke subjects: a multi-session study

The brain-computer interface (BCI) system has been developed to assist people with motor disability. To make the system more user-friendly, it is a challenge to reduce the electrode preparation time and have a good reliability. This study aims to find a minimal set of electrodes for an individual stroke subject for motor imagery to control an assistive device using functional electrical stimulation for 20 sessions with accuracy higher than 90%. The characteristics of this minimal electrode set were evaluated with two popular algorithms: Fisher's criterion and support-vector machine recursive feature elimination (SVM-RFE). The number of calibration sessions for channel selection required for robust control of these 20 sessions was also investigated. Five chronic stroke patients were recruited for the study. Our results suggested that the number of calibration sessions for channel selection did not have a significant effect on the classification accuracy. A performance index devised in this study showed that one training day with 12 electrodes using the SVM-RFE method achieved the best balance between the number of electrodes and accuracy in the 20-session data. Generally, 8-36 channels were required to maintain accuracy higher than 90% in 20 BCI training sessions for chronic stroke patients.     

ECoG factors underlying multimodal control of a brain-computer interface.

Most current brain-computer interface (BCI) systems for humans use electroencephalographic activity recorded from the scalp, and may be limited in many ways. Electrocorticography (ECoG) is believed to be a minimally-invasive alternative to electroencephalogram (EEG) for BCI systems, yielding superior signal characteristics that could allow rapid user training and faster communication rates. In addition, our preliminary results suggest that brain regions other than the sensorimotor cortex, such as auditory cortex, may be trained to control a BCI system using similar methods as those used to train motor regions of the brain. This could prove to be vital for users who have neurological disease, head trauma, or other conditions precluding the use of sensorimotor cortex for BCI control.     

The BCI competition. III: Validating alternative approaches to actual BCI problems.

A brain-computer interface (BCI) is a system that allows its users to control external devices with brain activity. Although the proof-of-concept was given decades ago, the reliable translation of user intent into device control commands is still a major challenge. Success requires the effective interaction of two adaptive controllers: the user's brain, which produces brain activity that encodes intent, and the BCI system, which translates that activity into device control commands. In order to facilitate this interaction, many laboratories are exploring a variety of signal analysis techniques to improve the adaptation of the BCI system to the user. In the literature, many machine learning and pattern classification algorithms have been reported to give impressive results when applied to BCI data in offline analyses. However, it is more difficult to evaluate their relative value for actual online use. BCI data competitions have been organized to provide objective formal evaluations of alternative methods. Prompted by the great interest in the first two BCI Competitions, we organized the third BCI Competition to address several of the most difficult and important analysis problems in BCI research. The paper describes the data sets that were provided to the competitors and gives an overview of the results.     

Peripheral electrical stimulation triggered by self-paced detection of motor intention enhances motor evoked potentials

This paper proposes the development and experimental tests of a self-paced asynchronous brain-computer interfacing (BCI) system that detects movement related cortical potentials (MRCPs) produced during motor imagination of ankle dorsiflexion and triggers peripheral electrical stimulations timed with the occurrence of MRCPs to induce corticospinal plasticity. MRCPs were detected online from EEG signals in eight healthy subjects with a true positive rate (TPR) of 67.15 ± 7.87% and false positive rate (FPR) of 22.05 ±9.07%. The excitability of the cortical projection to the target muscle (tibialis anterior) was assessed before and after the intervention through motor evoked potentials (MEP) using transcranial magnetic stimulation (TMS). The peak of the evoked potential significantly (P=0.02) increased after the BCI intervention by 53 ± 43% (relative to preintervention measure), although the spinal excitability (tested by stretch reflexes) did not change. These results demonstrate for the first time that it is possible to alter the corticospinal projections to the tibialis anterior muscle by using an asynchronous BCI system based on online motor imagination that triggered peripheral stimulation. This type of repetitive proprioceptive feedback training based on self-generated brain signal decoding may be a requirement for purposeful skill acquisition in intact humans and in the rehabilitation of persons with brain damage.     

Electroencephalography (EEG)-based brain-computer interface (BCI): a 2-D virtual wheelchair control based on event-related desynchronization/synchronization and state control

This study aims to propose an effective and practical paradigm for a brain-computer interface (BCI)-based 2-D virtual wheelchair control. The paradigm was based on the multi-class discrimination of spatiotemporally distinguishable phenomenon of event-related desynchronization/synchronization (ERD/ERS) in electroencephalogram signals associated with motor execution/imagery of right/left hand movement. Comparing with traditional method using ERD only, where bilateral ERDs appear during left/right hand mental tasks, the 2-D control exhibited high accuracy within a short time, as incorporating ERS into the paradigm hypothetically enhanced the spatiotemoral feature contrast of ERS versus ERD. We also expected users to experience ease of control by including a noncontrol state. In this study, the control command was sent discretely whereas the virtual wheelchair was moving continuously. We tested five healthy subjects in a single visit with two sessions, i.e., motor execution and motor imagery. Each session included a 20 min calibration and two sets of games that were less than 30 min. Average target hit rate was as high as 98.4% with motor imagery. Every subject achieved 100% hit rate in the second set of wheelchair control games. The average time to hit a target 10 m away was about 59 s, with 39 s for the best set. The superior control performance in subjects without intensive BCI training suggested a practical wheelchair control paradigm for BCI users.     

Classification of motor imagery tasks for brain-computer interface applications by means of two equivalent dipoles analysis.

We have developed a novel approach using source analysis for classifying motor imagery tasks. Two-equivalent-dipoles analysis was proposed to aid classification of motor imagery tasks for brain-computer interface (BCI) applications. By solving the electroencephalography (EEG) inverse problem of single trial data, it is found that the source analysis approach can aid classification of motor imagination of left- or right-hand movement without training. In four human subjects, an averaged accuracy of classification of 80% was achieved. The present study suggests the merits and feasibility of applying EEG inverse solutions to BCI applications from noninvasive EEG recordings.     

Implementation of a telemonitoring system for the control of an EEG-based brain-computer interface

By the use of a brain-computer interface (BCI), it is possible for completely paralyzed patients, who have lost their ability to speak, to have a new possibility to communicate with their environment. The training with such a BCI system can be performed at the patient's home, if there is a responsible person present who is familiar with the system. This person has to adjust different parameters and to adapt the training individually to each patient. Since this function is usually taken over by the developers of the system, the number of patients who can be included in regular BCI training is restricted due to geographical distances. This paper describes the implementation of a telemonitoring system, which makes it possible for the developer to control and supervise the BCI training from his or her own place of work. First experiences with a patient living far away from the developer's lab are reported.