结合表面肌电与稳态视觉诱发电位的混合脑机接口关键技术研究

脑-机接口技术旨在大脑与外部环境之间建立一种全新的不依赖于外周神经和肌肉的交流与控制通道。基于稳态视觉诱发电位的脑-机接口是目前信息传输率最高的无创脑 机接口范式，但是仍低于传统的交互方式。提出一种结合表面肌电与稳态视觉诱发电位的混合脑 机接口，以进一步提高系统的信息传输率。通过不同频率的高频稳态视觉诱发电位结合sEMG编码，实现二者混合脑 机接口系统。利用典型相关分析方法对SSVEP信号进行频率识别，sEMG的检测则采用频域分析方法。来自8名健康受试者的离线结果表明该系统能够获得8428% 的平均准确率，平均信息传输率为7263 bits/min。这些结果为结合表面肌电与稳态视觉诱发电位的混合脑 机接口研究奠定了基础。     

基于Resinv-Unet的图像特征点检测方法

针对传统特征点检测算法需人为制定检测机制和基于深度学习的特征点检测网络泛化能力不强的问题,引入灰度不变量和残差结构,设计并实现具备像素级特征点检测能力的残差不变量神经网络(residual-invariant neural network, Resinv-Unet)。采用自标注的方式,在真实场景图像数据集的基础上构建用于训练神经网络的数据集。实验结果表明,Resinv-Unet相较于现有的特征点检测算法和特征点检测网络,在真实场景图像上具有更强的泛化能力和鲁棒性,在平均精确度、精确度和召回率上均取得更好的性能指标,其中,平均精确度达到0.715 5、精确度达到0.776 2、召回率达到0.713 7。     

Visual spatial attention tracking using high-density SSVEP data for independent brain-computer communication.

The steady-state visual evoked potential (SSVEP) has been employed successfully in brain-computer interface (BCI) research, but its use in a design entirely independent of eye movement has until recently not been reported. This paper presents strong evidence suggesting that the SSVEP can be used as an electrophysiological correlate of visual spatial attention that may be harnessed on its own or in conjunction with other correlates to achieve control in an independent BCI. In this study, 64-channel electroencephalography data were recorded from subjects who covertly attended to one of two bilateral flicker stimuli with superimposed letter sequences. Offline classification of left/right spatial attention was attempted by extracting SSVEPs at optimal channels selected for each subject on the basis of the scalp distribution of SSVEP magnitudes. This yielded an average accuracy of approximately 71% across ten subjects (highest 86%) comparable across two separate cases in which flicker frequencies were set within and outside the alpha range respectively. Further, combining SSVEP features with attention-dependent parieto-occipital alpha band modulations resulted in an average accuracy of 79% (highest 87%).     

Brain-computer interfaces for 1-D and 2-D cursor control: designs using volitional control of the EEG spectrum or steady-state visual evoked potentials.

We have developed and tested two electroencephalogram (EEG)-based brain-computer interfaces (BCI) for users to control a cursor on a computer display. Our system uses an adaptive algorithm, based on kernel partial least squares classification (KPLS), to associate patterns in multichannel EEG frequency spectra with cursor controls. Our first BCI, Target Practice, is a system for one-dimensional device control, in which participants use biofeedback to learn voluntary control of their EEG spectra. Target Practice uses a KPLS classifier to map power spectra of 62-electrode EEG signals to rightward or leftward position of a moving cursor on a computer display. Three subjects learned to control motion of a cursor on a video display in multiple blocks of 60 trials over periods of up to six weeks. The best subject's average skill in correct selection of the cursor direction grew from 58% to 88% after 13 training sessions. Target Practice also implements online control of two artifact sources: 1) removal of ocular artifact by linear subtraction of wavelet-smoothed vertical and horizontal electrooculograms (EOG) signals, 2) control of muscle artifact by inhibition of BCI training during periods of relatively high power in the 40-64 Hz band. The second BCI, Think Pointer, is a system for two-dimensional cursor control. Steady-state visual evoked potentials (SSVEP) are triggered by four flickering checkerboard stimuli located in narrow strips at each edge of the display. The user attends to one of the four beacons to initiate motion in the desired direction. The SSVEP signals are recorded from 12 electrodes located over the occipital region. A KPLS classifier is individually calibrated to map multichannel frequency bands of the SSVEP signals to right-left or up-down motion of a cursor on a computer display. The display stops moving when the user attends to a central fixation point. As for Target Practice, Think Pointer also implements wavelet-based online removal of ocular artifact; however, in Think Pointer muscle artifact is controlled via adaptive normalization of the SSVEP. Training of the classifier requires about 3 min. We have tested our system in real-time operation in three human subjects. Across subjects and sessions, control accuracy ranged from 80% to 100% correct with lags of 1-5 s for movement initiation and turning. We have also developed a realistic demonstration of our system for control of a moving map display (http://ti.arc.nasa.gov/).     

Enhancement of steady‐state visual evoked potential‐based brain–computer interface systems via a steady‐state motion visual stimulus modality

Steady‐state visual evoked potential (SSVEP)‐based brain–computer interface (BCI) systems are among the most accurate assistive devices for patients with severe disabilities. However, existing visual stimulation patterns lead to eye fatigue, which affects the system performance. Therefore, in this study, we propose two improvements to SSVEP‐based BCI systems. First, we propose a novel visual stimulator that incorporates a visual motion stimulus for the steady‐state visual stimulus to reduce eye fatigue while maintaining the advantages associated with SSVEPs. We also propose two corresponding feature extraction algorithms, i.e. SSVEP detection and visual attention detection, to capture the phenomena of steady‐state motion visual stimulus responses. The accuracy of the test was ∼83.6%. Second, we propose a novel hybrid BCI‐SSVEP system and a motion visual stimulus hybrid BCI system to enhance the SSVEP‐based BCI system during a state of eye fatigue. Participants used the SSVEP system until reaching a fatigued state and then began using a hybrid motion visual stimulus. The accuracy of the proposed system was ∼85.6%. The proposed improvements can be incorporated into practical BCI systems for wheelchair control. © 2017 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.     

基于Hilbert Huang变换的SSVEP相位提取

稳态视觉诱发电位(steady-state visual evoked potential,SSVEP)被广泛应用于脑-机接口和大脑的认知研究,相位信息是其重要的特征指标之一.针对快速傅里叶变换在SSVEP相位提取中受不确定性原理约束的特点,提出了一种基于Hilbert-Huang变换的SSVEP相位提取方法.该方法通过经验模态分解将脑电信号分解为一系列固有模态函数(intrinsic mode functions,IMF),并通过分析各模态函数瞬时频率的均值判断该IMF分量是否属于噪声.若为噪声则将其从原始信号中滤除,再对滤波后的各IMF分量进行Hilbert变换,并与基准信号做运算即可求得SSVEP相位.实验结果表明,与快速傅里叶法相比该方法可在去除噪声分量的同时提取SSVEP的相位信息,且具有较高的准确率、精度和自适应性.     

Multi‐command SSAEP‐based BCI system with training sessions for SSVEP during an eye fatigue state

This paper proposes a steady‐state auditory stimulus modality and a detection algorithm to replace steady‐state visual evoked potential (SSVEP )‐based brain–computer interface (BCI ) systems during visual fatigue periods. The optimal speaker position for the steady‐state auditory evoked potential (SSAEP )‐based BCI system and possible electrode positions are investigated. Using the proposed system, an accuracy of 85% for two commands was achieved based on the T3–T5 and T4–T6 electrode positions using only one speaker. SSAEP is a promising BCI modality for mitigating the problem of eye fatigue that often occurs during the use of SSVEP ‐based BCI systems. However, SSAEP ‐based BCI systems suffer from low accuracy. To increase accuracy, we propose a new enhanced SSAEP training method. The training process was enhanced by instructing users to control their attention levels while simultaneously detecting an auditory stimulus frequency. Furthermore, we propose a corresponding single‐frequency, multi‐command BCI paradigm for the proposed training method. With the proposed paradigm, four commands can be detected using only one auditory stimulus frequency. The proposed training system yielded an accuracy of ∼81% compared to 66% for sessions performed without the proposed training method. © 2017 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.     

Dry and noncontact EEG sensors for mobile brain-computer interfaces

Dry and noncontact electroencephalographic (EEG) electrodes, which do not require gel or even direct scalp coupling, have been considered as an enabler of practical, real-world, brain-computer interface (BCI) platforms. This study compares wet electrodes to dry and through hair, noncontact electrodes within a steady state visual evoked potential (SSVEP) BCI paradigm. The construction of a dry contact electrode, featuring fingered contact posts and active buffering circuitry is presented. Additionally, the development of a new, noncontact, capacitive electrode that utilizes a custom integrated, high-impedance analog front-end is introduced. Offline tests on 10 subjects characterize the signal quality from the different electrodes and demonstrate that acquisition of small amplitude, SSVEP signals is possible, even through hair using the new integrated noncontact sensor. Online BCI experiments demonstrate that the information transfer rate (ITR) with the dry electrodes is comparable to that of wet electrodes, completely without the need for gel or other conductive media. In addition, data from the noncontact electrode, operating on the top of hair, show a maximum ITR in excess of 19 bits/min at 100% accuracy (versus 29.2 bits/min for wet electrodes and 34.4 bits/min for dry electrodes), a level that has never been demonstrated before. The results of these experiments show that both dry and noncontact electrodes, with further development, may become a viable tool for both future mobile BCI and general EEG applications.     

听觉刺激对稳态视觉诱发电位影响研究

基于稳态视觉诱发电位(steady state visual evoked potential,SSVEP)和听觉脑机接口技术目前已成为研究重点.未来的挑战是研究基于视觉和听觉联合统一系统框架的脑机接口技术.多感觉在不同脑区间存在的跨膜整合以及视听双刺激的交互作用,给该技术研究带来较大困难,故研究听觉刺激对枕区SSVEP影响很有意义.在闪光刺激频率为12 Hz,占空比分别为5％、20％、30％、40％、50％、60％、70％、80％、95％条件下,分别加入500、1 000、1 500 Hz的正弦纯音、响度为50 dB的听觉刺激,研究听觉对SSVEP的变化规律.结果表明,对同一受试者,视听双刺激条件下SSVEP随占空比变化依然呈现“窗口”效应,听觉刺激对SSVEP影响起增强或抑制作用.此外听觉刺激对SSVEP影响出现的占空比“窗口”的位置、数量以及对SSVEP增强或抑制作用的程度也因人而已.结果为更好研究视听相互作用机理及其在脑机接口技术应用提供有意义的实验依据.     

基于稳态视觉诱发电位的相频特性分类算法研究

目前基于稳态视觉诱发电位（SSVEP）的脑-机接口在人机协作中受到广泛关注,现有面向SSVEP信号的相位与频率信息的深度学习分类方法,仍存在由于信息利用不充分导致的SSVEP信号分类效果较差等问题。而目前已出现多种分类算法用于解决上述问题。本文基于迁移学习思想提出一种用于SSVEP信号分类的深度神经网络模型,将快速傅里叶变换后的复向量作为输入,对各个导联的实、虚部向量进行卷积,学习对应的相频特性。该模型分为两部分：第一部分利用所有被试者之间的统计共性获得相位和频率信息的全局相频特征模块;第二部分利用训练好的全局相频特征模块对局部相频特征模块进行初始化,通过局部相频特征模块的进一步强化学习对训练参数进行微调,以减少每个被试者之间的个体差异。在公开数据集BETA上进行测试,在时窗长度为1.5 s时,平均准确率和平均信息传输率分别为89.98%和71.80 bit/min。实验结果表明,与其他方法相比,本文的分类算法模型取得了较为不错的分类效果,所设计的全局、局部相频特征模块能够改善个体差异因素对分类结果的影响,为深入挖掘、利用SSVEP信号中的相位和频率信息提供了全新思路。     

Steady-state somatosensory evoked potentials: suitable brain signals for brain-computer interfaces?

One of the main issues in designing a brain-computer interface (BCI) is to find brain patterns, which could easily be detected. One of these pattern is the steady-state evoked potential (SSEP). SSEPs induced through the visual sense have already been used for brain-computer communication. In this work, a BCI system is introduced based on steady-state somatosensory evoked potentials (SSSEPs). Transducers have been used for the stimulation of both index fingers using tactile stimulation in the "resonance"-like frequency range of the somatosensory system. Four subjects participated in the experiments and were trained to modulate induced SSSEPs. Two of them learned to modify the patterns in order to set up a BCI with an accuracy of between 70% and 80%. Results presented in this work give evidence that it is possible to set up a BCI which is based on SSSEPs.     

Electroencephalography (EEG)-based brain-computer interface (BCI): a 2-D virtual wheelchair control based on event-related desynchronization/synchronization and state control

This study aims to propose an effective and practical paradigm for a brain-computer interface (BCI)-based 2-D virtual wheelchair control. The paradigm was based on the multi-class discrimination of spatiotemporally distinguishable phenomenon of event-related desynchronization/synchronization (ERD/ERS) in electroencephalogram signals associated with motor execution/imagery of right/left hand movement. Comparing with traditional method using ERD only, where bilateral ERDs appear during left/right hand mental tasks, the 2-D control exhibited high accuracy within a short time, as incorporating ERS into the paradigm hypothetically enhanced the spatiotemoral feature contrast of ERS versus ERD. We also expected users to experience ease of control by including a noncontrol state. In this study, the control command was sent discretely whereas the virtual wheelchair was moving continuously. We tested five healthy subjects in a single visit with two sessions, i.e., motor execution and motor imagery. Each session included a 20 min calibration and two sets of games that were less than 30 min. Average target hit rate was as high as 98.4% with motor imagery. Every subject achieved 100% hit rate in the second set of wheelchair control games. The average time to hit a target 10 m away was about 59 s, with 39 s for the best set. The superior control performance in subjects without intensive BCI training suggested a practical wheelchair control paradigm for BCI users.     

基于SSVEP的脑-机接口自动车系统研究

阐述了视觉诱发电位用于脑-机接口的原理,系统采用单片机设计视觉刺激器,同时在LABVIEW平台上,利用希尔伯特黄变换实时提取诱发电位向量,产生脑机接口控制信号,并用于自动车控制系统,从而控制小车的前后左右运动.通过大量实验验证,设计的基于稳态视觉诱发电位的脑-机接口自动车控制系统,发送控制命令正确率高于83％,发送一个...     

Peripheral electrical stimulation triggered by self-paced detection of motor intention enhances motor evoked potentials

This paper proposes the development and experimental tests of a self-paced asynchronous brain-computer interfacing (BCI) system that detects movement related cortical potentials (MRCPs) produced during motor imagination of ankle dorsiflexion and triggers peripheral electrical stimulations timed with the occurrence of MRCPs to induce corticospinal plasticity. MRCPs were detected online from EEG signals in eight healthy subjects with a true positive rate (TPR) of 67.15 ± 7.87% and false positive rate (FPR) of 22.05 ±9.07%. The excitability of the cortical projection to the target muscle (tibialis anterior) was assessed before and after the intervention through motor evoked potentials (MEP) using transcranial magnetic stimulation (TMS). The peak of the evoked potential significantly (P=0.02) increased after the BCI intervention by 53 ± 43% (relative to preintervention measure), although the spinal excitability (tested by stretch reflexes) did not change. These results demonstrate for the first time that it is possible to alter the corticospinal projections to the tibialis anterior muscle by using an asynchronous BCI system based on online motor imagination that triggered peripheral stimulation. This type of repetitive proprioceptive feedback training based on self-generated brain signal decoding may be a requirement for purposeful skill acquisition in intact humans and in the rehabilitation of persons with brain damage.     

Classifying EEG and ECoG signals without subject training for fast BCI implementation: comparison of nonparalyzed and completely paralyzed subjects.

We summarize results from a series of related studies that aim to develop a motor-imagery-based brain-computer interface using a single recording session of electroencephalogram (EEG) or electrocorticogram (ECoG) signals for each subject. We apply the same experimental and analytical methods to 11 nonparalysed subjects (eight EEG, three ECoG), and to five paralyzed subjects (four EEG, one ECoG) who had been unable to communicate for some time. While it was relatively easy to obtain classifiable signals quickly from most of the nonparalyzed subjects, it proved impossible to classify the signals obtained from the paralyzed patients by the same methods. This highlights the fact that though certain BCI paradigms may work well with healthy subjects, this does not necessarily indicate success with the target user group. We outline possible reasons for this failure to transfer.     

基于 P300 与 ErrP 决策融合的脑-机接口目标检测方法

针对脑-机接口(BCI)技术在目标检测中的应用仍然存在检测准确率受限的问题,提出基于事件相关电位(ERP)中的 P300 与错误相关电位(ErrP)决策融合的新型编解码方法。 BCI 系统编码方面通过目标图像和视觉反馈分别诱发 P300 与 ErrP 特征,解码方面采用单独 P300 特征、单独 ErrP 特征、P300 与 ErrP 特征层融合、P300 与 ErrP 决策层融合这 4 种方案进行目标检 测。 10 名健康受试者 4 种方案进行目标检测的平均结果显示,使用 P300 与 ErrP 决策层融合的平衡正确率最高,达到 80. 03%± 5. 20%,相比单独使用 P300 特征的方法提升了 4. 38%,相比单独使用 ErrP 特征的方法提升了 11. 29%,验证了混合 BCI 技术在 目标检测任务中的可行性。     

Information transfer rate in a five-classes brain-computerinterface

The information transfer rate, given in bits per trial, is used as an evaluation measurement in a brain-computer interface (BCI). Three subjects performed four motor-imagery (left hand, right hand, foot, and tongue) and one mental-calculation task. Classification of the EEG patterns is based on band power estimates and hidden Markov models. We propose a method that combines the EEG patterns based on separability into subsets of two, three, four, and five mental tasks. The information transfer rates of the BCI systems comprised of these subsets are reported. The achieved information transfer rates vary from 0.42 to 0.81 bits per trial and reveal that the upper limit of different mental tasks for a BCI system is three. In each subject, different combinations of three tasks resulted in the best performance     

Parallel man-machine training in development of EEG-based cursor control.

A new parallel man-machine training approach to brain-computer interface (BCI) succeeded through a unique application of machine learning methods. The BCI system could train users to control an animated cursor on the computer screen by voluntary electroencephalogram (EEG) modulation. Our BCI system requires only two to four electrodes, and has a relatively short training time for both the user and the machine. Moving the cursor in one dimension, our subjects were able to hit 100% of randomly selected targets, while in two dimensions, accuracies of approximately 63% and 76% was achieved with our two subjects.     

The Berlin Brain-Computer Interface: EEG-based communication without subject training.

The Berlin Brain-Computer Interface (BBCI) project develops a noninvasive BCI system whose key features are 1) the use of well-established motor competences as control paradigms, 2) high-dimensional features from 128-channel electroencephalogram (EEG), and 3) advanced machine learning techniques. As reported earlier, our experiments demonstrate that very high information transfer rates can be achieved using the readiness potential (RP) when predicting the laterality of upcoming left- versus right-hand movements in healthy subjects. A more recent study showed that the RP similarily accompanies phantom movements in arm amputees, but the signal strength decreases with longer loss of the limb. In a complementary approach, oscillatory features are used to discriminate imagined movements (left hand versus right hand versus foot). In a recent feedback study with six healthy subjects with no or very little experience with BCI control, three subjects achieved an information transfer rate above 35 bits per minute (bpm), and further two subjects above 24 and 15 bpm, while one subject could not achieve any BCI control. These results are encouraging for an EEG-based BCI system in untrained subjects that is independent of peripheral nervous system activity and does not rely on evoked potentials even when compared to results with very well-trained subjects operating other BCI systems.     

Electrocorticography-based brain computer interface--the Seattle experience.

Electrocorticography (ECoG) has been demonstrated to be an effective modality as a platform for brain-computer interfaces (BCIs). Through our experience with ten subjects, we further demonstrate evidence to support the power and flexibility of this signal for BCI usage. In a subset of four patients, closed-loop BCI experiments were attempted with the patient receiving online feedback that consisted of one-dimensional cursor movement controlled by ECoG features that had shown correlation with various real and imagined motor and speech tasks. All four achieved control, with final target accuracies between 73%-100%. We assess the methods for achieving control and the manner in which enhancing online control can be accomplished by rescreening during online tasks. Additionally, we assess the relevant issues of the current experimental paradigm in light of their clinical constraints.