FragDPI: a novel drug-protein interaction prediction model based on fragment understanding and unified coding

Prediction of drug-protein binding is critical for virtual drug screening. Many deep learning methods have been proposed to predict the drug-protein binding based on protein sequences and drug representation sequences. However, most existing methods extract features from protein and drug sequences separately. As a result, they can not learn the features characterizing the drug-protein interactions. In addition, the existing methods encode the protein (drug) sequence usually based on the assumption that each amino acid (atom) has the same contribution to the binding, ignoring different impacts of different amino acids (atoms) on the binding. However, the event of drug-protein binding usually occurs between conserved residue fragments in the protein sequence and atom fragments of the drug molecule. Therefore, a more comprehensive encoding strategy is required to extract information from the conserved fragments.In this paper, we propose a novel model, named FragDPI, to predict the drug-protein binding affinity. Unlike other methods, we encode the sequences based on the conserved fragments and encode the protein and drug into a unified vector. Moreover, we adopt a novel two-step training strategy to train FragDPI. The pre-training step is to learn the interactions between different fragments using unsupervised learning. The fine-tuning step is for predicting the binding affinities using supervised learning. The experiment results have illustrated the superiority of FragDPI.     

BERT for Conversational Question Answering Systems Using Semantic Similarity Estimation

Most of the questions from users lack the context needed to thoroughly understand the problem at hand, thus making the questions impossible to answer. Semantic Similarity Estimation is based on relating user’s questions to the context from previous Conversational Search Systems (CSS) to provide answers without requesting the user's context. It imposes constraints on the time needed to produce an answer for the user. The proposed model enables the use of contextual data associated with previous Conversational Searches (CS). While receiving a question in a new conversational search, the model determines the question that refers to more past CS. The model then infers past contextual data related to the given question and predicts an answer based on the context inferred without engaging in multi-turn interactions or requesting additional data from the user for context. This model shows the ability to use the limited information in user queries for best context inferences based on Closed-Domain-based CS and Bidirectional Encoder Representations from Transformers for textual representations.     

融合深层语义和显式特征的中文句子对相似性判别方法

中文句子对相似性计算任务旨在利用模型对两个句子的相似性进行判别,在文本挖掘领域有广泛的应用。考虑到现有机器学习方法不能同时兼顾句子对的深层语义特征和显式特征的问题,该文提出融合深层语义和显式特征的中文句子对相似性判别方法。采用BERT和全连接网络来获取深层语义向量,再拼接显式特征构造新的特征向量,最后通过分类器完成句子对的相似性判别。实验结果表明,该方法在3个公开的中文句子对相似性评测数据集上的性能均优于基线方法。     

基于BERT深度学习模型的地质资料目录自动分类研究

整理地质资料目录并进行分类统计需要大量专业性工作,为将应用现代自动化分类技术引入此项工作,在对比了传统机器学习建模和神经网络建模技术的基础上,本文提出了一种基于BERT深度学习模型的地质资料目录自动分类的方法,以此提高地质资料目录分类的准确率。实验结果表明,BERT分类技术的准确率为99%,超过了其他测试模型,且具有较好的易用性和扩展性,证明BERT微调分类技术能够有效的应用于地质材料目录的自动化分类。     

Monitoring and Early Warning of New Cyber-Telecom Crime Platform Based on BERT Migration Learning

The network is a major platform for implementing new cyber-telecom crimes.Therefore,it is important to carry out monitoring and early warning research on new cyber-telecom crime platforms,which will lay the foundation for the establishment of prevention and control systems to protect citizens’property.However,the deep-learning methods applied in the monitoring and early warning of new cyber-telecom crime platforms have some apparent drawbacks.For instance,the methods suffer from data-distribution differences and tremendous manual efforts spent on data labeling.Therefore,a monitoring and early warning method for new cyber-telecom crime platforms based on the BERT migration learning model is proposed.This method first identifies the text data and their tags,and then performs migration training based on a pre-training model.Finally,the method uses the fine-tuned model to predict and classify new cyber-telecom crimes.Experimental analysis on the crime data collected by public security organizations shows that higher classification accuracy can be achieved using the proposed method,compared with the deep-learning method.     

端到端对话系统意图语义槽联合识别研究综述

目前基于深度学习的端到端对话系统因具有泛化能力强、训练参数少、性能好等优势，在学术界和工业界成为了研究热点。意图识别和语义槽填充的结果对于对话系统的性能至关重要。介绍了端到端任务型对话系统意图和语义槽联合识别的主流方法，对注意力机制、Transformer模型在捕获长期依赖关系方面的效果同循环神经网络、长短时记忆网络进行对比，并分析了因其并行处理导致无法对文本词序位置信息完整捕获的局限；阐述了胶囊网络相较于卷积神经网络在捕获小概率语义信息保证特征完整性方面的优势；重点介绍了基于BERT（Bidirectional Encoder Representations from Transformers）模型的联合识别方法，不仅能够并行处理而且可以解决一词多义的问题，是目前性能最好的方法。最后对未来研究的发展方向进行讨论和分析。     

基于BERT的常见作物病害问答系统问句分类

问句分类作为问答系统的关键模块，也是制约问答系统检索效率的关键性因素。针对农业问答系统中用户问句语义信息复杂、差异大的问题，为了满足用户快速、准确地获取常见作物病害问句的分类结果的需求，构建了基于BERT的常见作物病害问答系统的问句分类模型。首先，对问句数据集进行预处理；然后，分别构建双向长短期记忆(Bi-LSTM)自注意力网络分类模型、Transformer分类模型和基于BERT的微调分类模型，并利用三种模型提取问句的信息，进行问句分类模型的训练；最后，对基于BERT的微调分类模型进行测试，同时探究数据集规模对分类结果的影响。实验结果表明，基于BERT的微调常见作物病害问句分类模型的分类准确率、精确率、召回率、精确率和召回率的加权调和平均值分别高于双向长短期记忆自注意力网络模型和Transformer分类模型2～5个百分点，在常见作物病害问句数据集（CCDQD）上能获得最高准确率92.46%，精确率92.59%，召回率91.26%，精确率和召回率的加权调和平均值91.92%。基于BERT的微调分类模型具有结构简单、训练参数少、训练速度快等特点，并能够高效地对常见作物病害问句准确分类，可以作为常见作物病害问答系统的问句分类模型。     

基于Qsys的高速信号误码测试系统设计

随着高速数据传输广泛应用,在设计相关产品时需要测试其误码率。本文以FPGA平台,Qsys片上系统为手段,设计了一款以光纤为接口的高速信号误码测试系统。能实现4通道,速率为5Gbps的伪随机码误码测试。     

基于BERT的不完全数据情感分类

不完全数据,如社交平台的互动信息、互联网电影资料库中的影评内容,广泛存在于现实生活中。而现有情感分类模型大多建立在完整的数据集上,没有考虑不完整数据对分类性能的影响。针对上述问题提出基于BERT的栈式降噪神经网络模型,用于面向不完全数据的情感分类。该模型由栈式降噪自编码器（SDAE）和BERT两部分组成。首先将经词嵌入处理的不完全数据输入到SDAE中进行去噪训练,以提取深层特征来重构缺失词和错误词的特征表示;接着将所得输出传入BERT预训练模型中进行精化以进一步改进词的特征向量表示。在两个常用的情感数据集上的实验结果表明,所提方法在不完全数据情感分类中的F1值和准确率分别提高了约6%和5%,验证了所提模型的有效性。     

融合BERT的多层次语义协同模型情感分析研究

由于基于变换器的双向编码器表征技术（Bidirectional Encoder Representations from Transformers,BERT）的提出,改变了传统神经网络解决句子级文本情感分析问题的方法。目前的深度学习模型BERT本身学习模式为无监督学习,其需要依赖后续任务补全推理和决策环节,故存在缺乏目标领域知识的问题。提出一种多层协同卷积神经网络模型（Multi-level Convolutional Neural Network,MCNN）,该模型能学习到不同层次的情感特征来补充领域知识,并且使用BERT预训练模型提供词向量,通过BERT学习能力的动态调整将句子真实的情感倾向嵌入模型,最后将不同层次模型输出的特征信息同双向长短期记忆网络输出信息进行特征融合后计算出最终的文本情感性向。实验结果表明即使在不同语种的语料中,该模型对比传统神经网络和近期提出的基于BERT深度学习的模型,情感极性分类的能力有明显提升。