Decoding visual brain states from fMRI using an ensemble of classifiers

Decoding perceptual or cognitive states based on brain activity measured using functional magnetic resonance imaging (fMRI) can be achieved using machine learning algorithms to train classifiers of specific stimuli. However, the high dimensionality and intrinsically low signal to noise ratio (SNR) of fMRI data poses great challenges to such techniques. The problem is aggravated in the case of multiple subject experiments because of the high inter-subject variability in brain function. To address these difficulties, the majority of current approaches uses a single classifier. Since, in many cases, different stimuli activate different brain areas, it makes sense to use a set of classifiers each specialized in a different stimulus. Therefore, we propose in this paper using an ensemble of classifiers for decoding fMRI data. Each classifier in the ensemble has a favorite class or stimulus and uses an optimized feature set for that particular stimulus. The output for each individual stimulus is therefore obtained from the corresponding classifier and the final classification is achieved by simply selecting the best score. The method was applied to three empirical fMRI datasets from multiple subjects performing visual tasks with four classes of stimuli. Ensembles of GNB and k-NN base classifiers were tested. The ensemble of classifiers systematically outperformed a single classifier for the two most challenging datasets. In the remaining dataset, a ceiling effect was observed which probably precluded a clear distinction between the two classification approaches. Our results may be explained by the fact that different visual stimuli elicit specific patterns of brain activation and indicate that an ensemble of classifiers provides an advantageous alternative to commonly used single classifiers, particularly when decoding stimuli associated with specific brain areas.     

fMRI的视觉神经信息编解码方法综述

视觉神经信息编解码旨在利用功能磁共振成像(functional magnetic resonance imaging,fMRI)等神经影像数据研究视觉刺激与大脑神经活动之间的关系。编码研究可以对神经活动模式进行建模和预测,有助于脑科学与类脑智能的发展;解码研究可以对人的视知觉状态进行解译,能够促进脑机接口领域的发展。因此,基于fMRI的视觉神经信息编解码方法研究具有重要的科学意义和工程价值。本文在总结基于fMRI的视觉神经信息编解码关键技术与研究进展的基础上,分析现有视觉神经信息编解码方法的局限。在视觉神经信息编码方面,详细介绍了基于群体感受野估计方法的发展过程;在视觉神经信息解码方面,首先,按照任务类型将其划分为语义分类、图像辨识和图像重建3个部分,并深入阐述了每个部分的代表性研究工作和所用的方法。特别地,在图像重建部分着重介绍了基于深度生成模型(主要包括变分自编码器和生成对抗网络)的简单图像、人脸图像和复杂自然图像的重建技术。其次,统计整理了该领域常用的10个开源数据集,并对数据集的样本规模、被试个数、刺激类型、研究用途及下载地址进行了详细归纳。最后,详细介绍了视觉神经信息编解码模...     

On the relation between encoding and decoding of neuronal spikes

Neural coding is a field of study that concerns how sensory information is represented in the brain by networks of neurons. The link between external stimulus and neural response can be studied from two parallel points of view. The first, neural encoding, refers to the mapping from stimulus to response. It focuses primarily on understanding how neurons respond to a wide variety of stimuli and constructing models that accurately describe the stimulus-response relationship. Neural decoding refers to the reverse mapping, from response to stimulus, where the challenge is to reconstruct a stimulus from the spikes it evokes. Since neuronal response is stochastic, a one-to-one mapping of stimuli into neural responses does not exist, causing a mismatch between the two viewpoints of neural coding. Here we use these two perspectives to investigate the question of what rate coding is, in the simple setting of a single stationary stimulus parameter and a single stationary spike train represented by a renewal process. We show that when rate codes are defined in terms of encoding, that is, the stimulus parameter is mapped onto the mean firing rate, the rate decoder given by spike counts or the sample mean does not always efficiently decode the rate codes, but it can improve efficiency in reading certain rate codes when correlations within a spike train are taken into account.     

Question-Answering Pair Matching Based on Question Classification and Ensemble Sentence Embedding

Question-answering (QA) models find answers to a given question. The necessity of automatically finding answers is increasing because it is very important and challenging from the large-scale QA data sets. In this paper, we deal with the QA pair matching approach in QA models, which finds the most relevant question and its recommended answer for a given question. Existing studies for the approach performed on the entire dataset or datasets within a category that the question writer manually specifies. In contrast, we aim to automatically find the category to which the question belongs by employing the text classification model and to find the answer corresponding to the question within the category. Due to the text classification model, we can effectively reduce the search space for finding the answers to a given question. Therefore, the proposed model improves the accuracy of the QA matching model and significantly reduces the model inference time. Furthermore, to improve the performance of finding similar sentences in each category, we present an ensemble embedding model for sentences, improving the performance compared to the individual embedding models. Using real-world QA data sets, we evaluate the performance of the proposed QA matching model. As a result, the accuracy of our final ensemble embedding model based on the text classification model is 81.18%, which outperforms the existing models by 9.81%∼14.16% point. Moreover, in terms of the model inference speed, our model is faster than the existing models by 2.61∼5.07 times due to the effective reduction of search spaces by the text classification model.     

Decoding a temporal population code

Encoding of sensory events in internal states of the brain requires that this information can be decoded by other neural structures. The encoding of sensory events can involve both the spatial organization of neuronal activity and its temporal dynamics. Here we investigate the issue of decoding in the context of a recently proposed encoding scheme: the temporal population code. In this code, the geometric properties of visual stimuli become encoded into the temporal response characteristics of the summed activities of a population of cortical neurons. For its decoding, we evaluate a model based on the structure and dynamics of cortical microcircuits that is proposed for computations on continuous temporal streams: the liquid state machine. Employing the original proposal of the decoding network results in a moderate performance. Our analysis shows that the temporal mixing of subsequent stimuli results in a joint representation that compromises their classification. To overcome this problem, we investigate a number of initialization strategies. Whereas we observe that a deterministically initialized network results in the best performance, we find that in case the network is never reset, that is, it continuously processes the sequence of stimuli, the classification performance is greatly hampered by the mixing of information from past and present stimuli. We conclude that this problem of the mixing of temporally segregated information is not specific to this particular decoding model but relates to a general problem that any circuit that processes continuous streams of temporal information needs to solve. Furthermore, as both the encoding and decoding components of our network have been independently proposed as models of the cerebral cortex, our results suggest that the brain could solve the problem of temporal mixing by applying reset signals at stimulus onset, leading to a temporal segmentation of a continuous input stream.     

基于依存关系的问句理解与问句分类

问句理解是问答系统的首要过程,问句分类是问句理解的主要组成部分,它在问答系统中具有非常重要的作用,因为问句类型有助于在文档中定位和抽取答案。问句分类的目标是基于预期的答案类型,准确地分类问句。本文提出依存关系规则与统计方法相结合,实现了基于依存关系的中文问句理解与问句分类机制。实验表明：支持向量机结合依存关系的特征抽取方法,获得了较高问句分类正确率。     

基于MHSA和句法关系增强的机器阅读理解方法研究

机器阅读理解 (Machine reading comprehension, MRC)是自然语言处理领域中一项重要研究任务, 其目标是通过机器理解给定的阅读材料和问题, 最终实现自动答题. 目前联合观点类问题解答和答案依据挖掘的多任务联合学习研究在机器阅读理解应用中受到广泛关注, 它可以同时给出问题答案和支撑答案的相关证据, 然而现有观点类问题的答题方法在答案线索识别上表现还不是太好, 已有答案依据挖掘方法仍不能较好捕获段落中词语之间的依存关系. 基于此, 引入多头自注意力(Multi-head self-attention, MHSA)进一步挖掘阅读材料中观点类问题的文字线索, 改进了观点类问题的自动解答方法; 将句法关系融入到图构建过程中, 提出了基于关联要素关系图的多跳推理方法, 实现了答案支撑句挖掘; 通过联合优化两个子任务, 构建了基于多任务联合学习的阅读理解模型. 在2020中国“法研杯”司法人工智能挑战赛(China AI Law Challenge 2020, CAIL2020)和HotpotQA数据集上的实验结果表明, 本文提出的方法比已有基线模型的效果更好.     

A study on using data clustering for feature extraction to improve the quality of classification

There is a certain belief among data science researchers and enthusiasts alike that clustering can be used to improve classification quality. Insofar as this belief is fairly uncontroversial, it is also very general and therefore produces a lot of confusion around the subject. There are many ways of using clustering in classification and it obviously cannot always improve the quality of predictions, so a question arises, in which scenarios exactly does it help? Since we were unable to find a rigorous study addressing this question, in this paper, we try to shed some light on the concept of using clustering for classification. To do so, we first put forward a framework for incorporating clustering as a method of feature extraction for classification. The framework is generic w.r.t. similarity measures, clustering algorithms, classifiers, and datasets and serves as a platform to answer ten essential questions regarding the studied subject. Each answer is formulated based on a separate experiment on 16 publicly available datasets, followed by an appropriate statistical analysis. After performing the experiments and analyzing the results separately, we discuss them from a global perspective and form general conclusions regarding using clustering as feature extraction for classification.

     

Question answering using statistical language modelling

In this paper we present a statistical approach to question answering (QA). Our motivation is to build robust systems for many languages without the need for highly tuned linguistic modules. Consequently, word tokens and web data are used extensively but neither explicit linguistic knowledge nor annotated data is incorporated. A mathematical model for answer retrieval and answer classification is derived. Experiments are conducted by searching for answers in the AQUAINT corpus, as well as in web data. The redundancy inherent in web data outperforms retrieval from a fixed corpus, where there are typically relatively few answer occurrences for any given question. We participated with an implementation of this framework in the TREC 2006 QA evaluations, where we ranked 9th among 27 participants on the factoid task.     

On decoding the responses of a population of neurons from short time windows.

The effectiveness of various stimulus identification (decoding) procedures for extracting the information carried by the responses of a population of neurons to a set of repeatedly presented stimuli is studied analytically, in the limit of short time windows. It is shown that in this limit, the entire information content of the responses can sometimes be decoded, and when this is not the case, the lost information is quantified. In particular, the mutual information extracted by taking into account only the most likely stimulus in each trial turns out to be, if not equal, much closer to the true value than that calculated from all the probabilities that each of the possible stimuli in the set was the actual one. The relation between the mutual information extracted by decoding and the percentage of correct stimulus decodings is also derived analytically in the same limit, showing that the metric content index can be estimated reliably from a few cells recorded from brief periods. Computer simulations as well as the activity of real neurons recorded in the primate hippocampus serve to confirm these results and illustrate the utility and limitations of the approach.     

基于句法结构特征分析及分类技术的答案提取算法

由于中文自然语言处理的特点和困难以及相应的语言处理基础资源的相对缺乏,使得国外一些成熟技术和研究成果不能直接应用到中文问答系统中.为此,针对中文事实型问答系统,提出一种新的基于句法结构特征分析及分类技术的答案提取算法,该方法将答案提取问题看成是候选答案的分类问题,即将候选答案分类为正确和错误两类.首先,该方法根据与问题类型所对应的候选答案的类型信息,从文本片断中提取出候选答案及其在句子中的简单特征和句法结构特征;然后利用这些特征训练分类器;最后用训练得到的分类器判别候选答案是否为正确答案.针对中文事实性问题,该方法与目前典型的基于模式匹配的中文答案提取算法相比,准确率提升6.2%,MRR提升9.7%.     

A web based collaborative testing environment

This paper presents a computer supported collaborative testing system built upon the Siette web-based assessment environment. The application poses the same set of questions to a group of students. Each student in the group should answer the same question twice. An initial response is given individually, without knowing the answers of others. Then the system provides some tools to show the other partners' responses, to support distance collaboration. Finally a second individual answer is requested. In this way assessment and collaboration activities are interlaced. At the end of a collaborative testing session, each student will have two scores: the initial score and the final score. Three sets of experiments have been carried out: (1) a set of experiments designed to evaluate and fine tune the application, improve usability, and to collect users' feelings and opinions about the system; (2) a second set of experiments to analyze the impact of collaboration in test results, comparing individual and group performance, and analyzing the factors that correlate to those results; and (3) a set of experiments designed to measure individual short-term learning directly related to the collaborative testing activity. We study whether the use of the system is associated with actual learning, and whether this learning is directly related to collaboration between students. Our studies confirm previous results and provide the following evidence (1) the performance increase is directly related to the access to other partners' answers; (2) a student tends to reach a common answer in most cases; and (3) the consensus is highly correlated with the correct response. Moreover, we have found evidence indicating that most of the students really do learn from collaborative testing. High-performing students improve by self-reflection, regardless the composition of the group, but low-performing students need to be in a group with higher-performing students in order to improve.     

Soundness verification for conceptual workflow nets with data: Early detection of errors with the most precision possible

A conceptual workflow model specifies the control flow of a workflow together with abstract data information. This model is later on refined by adding specific data information, resulting in an executable workflow which is then run on an information system. It is desirable that correctness properties of the conceptual workflow are transferable to its refinements. In this paper, we present classical workflow nets extended with data operations as a conceptual workflow model. For these nets, we develop a novel technique to verify soundness. An executable workflow is sound if from every reachable state it is always possible to terminate properly. Our technique allows us to analyze a conceptual workflow and to conclude whether there exists at least one sound refinement of it, and whether any refinement of a conceptual workflow model is sound. The positive answer to the first question in combination with the negative answer to the second question means that sound and unsound refinements for the conceptual workflow in question are possible.     

基于知识库的开放领域问答系统

问答系统能够理解用户问题,并直接返回答案。现有问答系统大多是面向领域的,仅能回答特定领域的问题。文中提出了基于大规模知识库的开放领域问答系统实现方法。该系统首先采用自定义词典分词和CRF模型相结合的方法识别问句中的主体;其次,采用模糊匹配方法将问句中的主体与知识库中实体建立链接;然后,通过相似度计算以及规则匹配等多种方法识别问句中的谓词并与知识库实体的属性建立关联;最后,进行实体消歧和答案获取。该系统平均F-Measure值为0.695 6,表明所提方法在基于知识库的开放领域问答上具有可行性。     

多特征层次化答案质量评价方法研究

社交媒体中的问答对可以为自动问答系统提供答案,但有些答案的质量不高,因此答案质量评价方法具有研究价值。已有的评价方法没有考虑问题类别特征,对不同类型的问题采用统一的评价方法。因此提出了一个层次分类模型。首先分析问题类型;然后提取文本、非文本、语言翻译性、答案中的链接数4类特征,依据特征分类影响力随问题类型不同而不同这一客观现象,采用逻辑回归算法对各类型问题的答案质量进行评价,取得了较好的实验效果;最后分析了影响各类问题答案质量的主要特征。     

QUESTION ANSWERING USING QUESTION CLASSIFICATION AND DOCUMENT TAGGING

Question answering (QA) is a relatively new area of research. We took the approach of designing a question answering system that is based on question classification and document tagging. Question classification extracts useful information from the question about how to answer the question. Document tagging extracts useful information from the documents, which are used to find the answer to the question. We used different available systems to tag the documents. Our system classifies the questions using manually developed rules. An evaluation of the system is performed using Text REtrieval Conference (TREC) data.     

基于改进贝叶斯模型的问题分类

随着计算机及互联网络技术的发展,开放域问答系统越来越受到人们的关注,因为它能够给用户提供相对简洁、准确的结果。开放域问答系统通常包括问题分类、问题扩展、搜索引擎、答案抽取和答案选择五个主要部分。问题分类在问答系统中起着很重要的作用,它的准确性直接影响到最终抽取的答案的准确性。 本文在对已有的贝叶斯分类方法进行分析的基础上,对该方法进行了改进。为了验证该方法的效果,构造了问题的训练集和测试集。从实验结果可以看出,该方法在实际应用中获得了较好的效果。     

基于多标签策略的中文知识图谱问答系统研究

现有多数中文知识图谱问答（CKBQA）系统侧重于回答单个三元组查询的简单问题,而不能有效解决涉及多个实体和关系的复杂问题。提出一种基于多标签策略进行答案搜索的CKBQA系统,该系统主要包括问题处理和答案搜索2个部分。在问题处理部分,结合预训练语言模型构建新的模型框架,对问题进行实体提及识别、实体链接和关系抽取处理,通过设置3种分类标签将问题划分为简单问题、链式问题和多实体问题。在答案搜索部分,对上述3种分类问题分别给出不同的解决方法。实验结果表明,该系统在CCKS2019-CKBQA评测数据验证集上的平均F1值可达66.76%。     

Answering questions with an <Emphasis Type="Italic">n</Emphasis>-gram based passage retrieval engine

In this paper, we present a Question Answering system based on redundancy and a Passage Retrieval method that is specifically oriented to Question Answering. We suppose that in a large enough document collection the answer to a given question may appear in several different forms. Therefore, it is possible to find one or more sentences that contain the answer and that also include tokens from the original question. The Passage Retrieval engine is almost language-independent since it is based on n-gram structures. Question classification and answer extraction modules are based on shallow patterns.     

基于问题分析的旅游咨询系统

针对咨询系统缺乏对问句的语义分析,提出在汉语框架语义知识库的基础上,利用语义Web语言,对旅游中有关交通的问句进行语义分析,并利用旅游本体知识库对答案进行抽取和处理。给出一种问题分类的新方法,结合传统分类与本体分类的方法,以及汉语框架语义知识库,提高了问题识别的效率。