多分类器区分性组合在二次解码中的应用

提出利用基于隐马尔可夫模型的谱特征模型、基于高斯混合模型的声调分类器以及基于多层感知器的音素分类器模型的组合来提高语音识别中二次解码中的识别率。在模型组合中,使用上下文相关的模型权重加权模型得分,并使用区分性训练来优化上下文相关权重来进一步改进识别结果。对人工选取各种上下文相关权重集合进行了性能评估,连续语音识别实验表明,使用局部分类器进行二次解码能够明显降低系统误识率。在模型组合中,使用当前音节类型及左上下文相结合的模型权重集合能够最大程度降低系统误识率。实验表明该方法得到的识别结果优于基于谱特征与基频特征和音素后验概率特征合并得到特征组合的识别系统。     

汉语语音识别中的区分性声调建模方法

提出从特征提取参数、模型参数对隐马尔可夫声调模型进行区分型训练,来提高声调识别率;提出模型相关的权重对谱特征模型和声调模型的概率进行加权,并根据最小音子错误区分性目标函数对权重进行训练,来提高声调模型加入连续语音识别时的性能。声调识别实验表明区分性的声调模型训练以及特征提取方法显著提高了声调识别率。区分性模型权重训练能够在声调模型加入之后进一步连续语音识别系统的识别率。     

基于轻量级梯度提升机优化的工业互联网入侵检测方法

入侵检测作为一种积极主动的安全防护技术,对于确保工业互联网安全起着至关重要的作用。为了满足工业互联网高准确率和高实时性的入侵检测需求,提出基于轻量级梯度提升机优化的工业互联网入侵检测方法。针对工业互联网业务数据中难分类样本导致检测准确率低的问题,改进轻量级梯度提升机原有的损失函数为焦点损失函数,该损失函数可自适应动态调节不同类别数据样本的损失值和权重,支持模型在训练过程中降低易分类样本的权重,进而提高难分类样本的检测准确率;针对轻量级梯度提升机参数较多并且对模型的检测准确率、检测时间和拟合程度等影响较大的问题,利用果蝇优化算法选择模型的最优参数组合;在密西西比州立大学提供的天然气管道数据集上得到模型的最优参数组合并进行验证,并在储水罐数据集上进一步验证所提模型的有效性。实验结果表明,采用所提方法改进的模型在天然气管道数据集上的检测准确率较对比模型最少提高了3.14%,检测时间较对比模型中的随机森林和支持向量机分别降低了0.35 s和19.53 s,较决策树和极端梯度提升机分别增加了0.06 s和0.02 s,同时在储水罐数据集上取得了良好的检测结果。因此证明所提方法可以很好地识别工业互...     

基于代表性数据的决策树集成*

为了获得更好的决策树集成效果,在理论分析的基础上从数据的角度提出了一种基于代表性数据的决策树集成方法。该方法使用围绕中心点的划分(PAM)算法从原始训练集中提取出代表性训练集,由该代表性训练集来训练出多个决策树分类器,并由此建立决策树集成模型。该方法能选取尽可能少的代表性数据来训练出尽可能好的决策树集成模型。实验结果表明,该方法使用更少的代表性数据能获得比Bagging和Boosting还要高的决策树集成精度。     

基于遗传算法支持向量机的网络入侵预测

在预测网络安全问题的研究中,针对网络入侵检测优化问题,为了改变传统入侵检测算法存在训练精度高,预测精度相当低的过拟合难题,提出一种基于遗传算法的支持向量机。支持向量机首先利用遗传算法搜索最优的支持向量机参数,然后用得到的最优参数来训练,利用训练得到的最优算法模型对测试集进行建模预测。并利用支持向量机对KDD 1999 CUP数据集进行了仿真。实验结果表明,方法在降低训练时间的同时有着很好的检测率,优于经典的神经网络算法,方法提高了预测效率。     

基于关联记忆网络的中文细粒度命名实体识别

细粒度命名实体识别是对文本中的实体进行定位,并将其分类至预定义的细粒度类别中.目前,中文细粒度命名实体识别仅使用预训练语言模型对句子中的字符进行上下文编码,并没有考虑到类别的标签信息具有区分实体类别的能力.由于预测句子不带有实体标签,使用关联记忆网络来捕获训练集句子的实体标签信息,并将标签信息融入预测句子的字符表示中.该方法将训练集中带实体标签的句子作为记忆单元,利用预训练语言模型获取原句子和记忆单元句子的上下文表示,再通过注意力机制将记忆单元句子的标签信息与原句子的表示结合,从而提升识别效果.在CLUENER 2020中文细粒度命名实体识别任务上,该方法对比基线方法获得了提升.     

一种在矩阵空间中识别文本蕴涵的动态交互网络

针对文本蕴涵问题提出一种动态交互网络（dynamic interactive network,DIN）进行识别。不同于已有交互模型,DIN将两句词向量投射到二维矩阵空间中进行交互,然后利用输出矩阵为同时处理上下文信息和控制信息流动的GRU编码器生成动态权重。前者通过更高阶形式的信息交互挖掘深层逻辑片段,后者通过改变交互信息与上下文信息的结合模式帮助编码器有效区分两者的重要性差异。模型在SNLI测试集上获得了88.0%的识别准确度,超过已有的最佳模型,且使用的训练参数仅为它的一半。     

一种基于社会化网络的上下文相关推荐模型

后搜索引擎时代如何构建推荐系统来为用户提供准确的信息成为研究者关注的问题,而有效地利用社会化网络中的上下文信息已然是解决这一问题的一把钥匙。在本文中,我们提出了一种新的推荐模型,通过上下文相关,对社会化网络信息进行再加工,不同于一般上下文信息处理方法,我们通过随机决策树对传统"用户-项目"评价矩阵中评价相似的上下文信息进行区分,然后通过矩阵分解来预测用户的缺失偏好,同时,为了更加充分地利用社会化网络信息,我们将偏好相似用户可能的推荐进行量化,以影响矩阵分解目标函数的结果,以此达到精确推荐的目的,最终以此作为系统推荐。通过实验表明,本文提出的上下文相关推荐模型可获得更高的用户满意度。     

基于自适应池化的神经网络的服装图像识别

针对传统池化方式不能提取有效特征值的问题,提出根据池化域的尺寸、池化域内的元素值和网络的训练轮数调整池化结果的自适应池化方法,该算法依据插值原理与最大值池化模型构建函数,以特定函数值作为池化结果,然后利用交叉验证进行模型对比实验。同时提出了小样本调优法以解决目前依靠经验值在全部数据集上验证选取超参数效率较低的问题。在原始数据集上,按照分层抽样的规则抽取小样本,并基于小样本数据集对已编码的超参数组合循环训练并测试,通过对识别率最高的组合解码确定最优超参数。选用DeepFashion数据库进行相关实验,结果显示自适应池化模型的识别率达到83%左右,与最大值池化模型相比提高约2.5%。通过小样本选定超参数,并与随机组合超参数在原始数据集上进行对比实验,结果显示小样本调优法选择的超参数在经验值范围内最优,识别结果为86.98%,与随机组合超参数的平均识别率相比提高了约41.4%。自适应池化方法可以扩展到其他的神经网络中,小样本调优法对高效选取神经网络的超参数提供了依据。     

模型决策树:一种决策树加速算法

决策树算法采用递归方法构建,训练效率较低,过度分类的决策树可能产生过拟合现象.因此,文中提出模型决策树算法.首先在训练数据集上采用基尼指数递归生成一棵不完全决策树,然后使用一个简单分类模型对其中的非纯伪叶结点(非叶结点且结点包含的样本不属于同一类)进行分类,生成最终的决策树.相比原始的决策树算法,这样产生的模型决策树能在算法精度不损失或损失很小的情况下,提高决策树的训练效率.在标准数据集上的实验表明,文中提出的模型决策树在速度上明显优于决策树算法,具备一定的抗过拟合能力.     

基于区分性准则的Bottleneck特征及其在LVCSR中的应用

基于深层神经网络中间层的Bottleneck(BN)特征由于可以采用传统的混合高斯模型-隐马尔可夫建模(Gaussian mixture model-hidden Markov model, GMM-HMM),在大规 模连续语音识别中获得了广泛的应用。为了提取区分性的BN特征,本文提出在使用传统的BN特征训练好GMM-HMM模型之后,利用最小音素错误率（Minimum phone error, MPE）准则来优化BN网络参数以及GMM-HMM模型参数。该算法相对于其他区分性训练算法而言,采用的是全部数据作为一个大的数据包,而 不是小的包方式来训练深度神经网络,从而可以大大加快训练速度。实验结果表明,优化后的BN特征提取网络比传统方法能获得9%的相对词错误率下降。     

Building A Highly Accurate Mandarin Speech Recognizer With Language-Independent Technologies and Language-Dependent Modules

We describe a system for highly accurate large-vocabulary Mandarin speech recognition. The prevailing hidden Markov model based technologies are essentially language independent and constitute the backbone of our system. These include minimum-phone-error discriminative training and maximum-likelihood linear regression adaptation, among others. Additionally, careful considerations are taken into account for Mandarin-specific issues including lexical word segmentation, tone modeling, phone set design, and automatic acoustic segmentation. Our system comprises two sets of acoustic models for the purposes of cross adaptation. The systems are designed to be complementary in terms of errors but with similar overall accuracy by using different phone sets and different combinations of discriminative learning. The outputs of the two subsystems are then rescored by an adapted n-gram language model. Final confusion network combination yielded 9.1% character error rate on the DARPA GALE 2007 official evaluation, the best Mandarin recognition system in that year.     

Functional Trees

In the context of classification problems, algorithms that generate multivariate trees are able to explore multiple representation languages by using decision tests based on a combination of attributes. In the regression setting, model trees algorithms explore multiple representation languages but using linear models at leaf nodes. In this work we study the effects of using combinations of attributes at decision nodes, leaf nodes, or both nodes and leaves in regression and classification tree learning. In order to study the use of functional nodes at different places and for different types of modeling, we introduce a simple unifying framework for multivariate tree learning. This framework combines a univariate decision tree with a linear function by means of constructive induction. Decision trees derived from the framework are able to use decision nodes with multivariate tests, and leaf nodes that make predictions using linear functions. Multivariate decision nodes are built when growing the tree, while functional leaves are built when pruning the tree. We experimentally evaluate a univariate tree, a multivariate tree using linear combinations at inner and leaf nodes, and two simplified versions restricting linear combinations to inner nodes and leaves. The experimental evaluation shows that all functional trees variants exhibit similar performance, with advantages in different datasets. In this study there is a marginal advantage of the full model. These results lead us to study the role of functional leaves and nodes. We use the bias-variance decomposition of the error, cluster analysis, and learning curves as tools for analysis. We observe that in the datasets under study and for classification and regression, the use of multivariate decision nodes has more impact in the bias component of the error, while the use of multivariate decision leaves has more impact in the variance component.     

Tree Height and Diameter Extraction with 3D Reconstruction in a Forest based on TLS

In view of the low accuracy of Tree Height(TH) and Diameter at Breast Height(DBH) estimation,as well as the difficulty of individual tree modeling in dense forest,a method to extract forest structure parameters(TH and DBH) and reconstruct a Three-Dimensional(3D) model of forest in subtropical environment based on TLS point cloud data is proposed.The first step is to apply a multi-scale method to extract the ground points for the generation of Digital Elevation Model(DEM).Secondly,using similarity of principal direction between neighboring points and distribution density of points,trunk and other plant organs are separated.Next the trunk points are processed to automatically estimate the tree position and DBH by iterative least squares cylinder fitting;the tree height is automatically estimated by using the octree segmentation.Finally,by combining with the technology of individual tree modeling,a plot-scale 3D forest scene has been reconstructed by planting individual tree model on the terrain model iteratively.The results showed that the correlation coefficient of DBH is R²=0.996,and the average relative error was 2.09%,RMSE was 0.66 cm;the correlation coefficient of tree height is R²=0.972,and the average relative error was 2.16% with RMSE of 0.92 m.The plot-scale reconstructed 3D model of the forest can express the true shape of forest.     

基于模型混淆度的模型组合算法研究

传统的声学模型训练算法如最大似然估计(Maximum likelihood estimation, MLE), 在训练时只考虑了模型自身而没有考虑模型之间的相互影响. 为了进一步提升模型的识别效果, 区分性训练算法被提出. 本文在最小音素错误(Minimum phone error, MPE)区分性训练算法的基础上提出一种基于模型间混淆程度进行模型组合的算法: 针对单混合分量模型, 依据模型间混淆程度对MLE和MPE的模型进行加权组合; 针对多混合分量模型, 提出一种模型选择的算法来获取新的模型参数. 实验表明, 与MPE算法相比, 对单分量的情况, 该算法可以使系统的误识率相对降低4%左右; 对于多分量的情况, 该算法可以使系统的误识率相对降低3%左右.     

Approximate Test Risk Bound Minimization Through Soft Margin Estimation

Inspired by the great success of margin-based classifiers, there is a trend to incorporate the margin concept into hidden Markov modeling for speech recognition. Several attempts based on margin maximization were proposed recently. In this paper, a new discriminative learning framework, called soft margin estimation (SME), is proposed for estimating the parameters of continuous-density hidden Markov models. The proposed method makes direct use of the successful ideas of soft margin in support vector machines to improve generalization capability and decision feedback learning in minimum classification error training to enhance model separation in classifier design. SME is illustrated from a perspective of statistical learning theory. By including a margin in formulating the SME objective function, SME is capable of directly minimizing an approximate test risk bound. Frame selection, utterance selection, and discriminative separation are unified into a single objective function that can be optimized using the generalized probabilistic descent algorithm. Tested on the TIDIGITS connected digit recognition task, the proposed SME approach achieves a string accuracy of 99.43%. On the 5 k-word Wall Street Journal task, SME obtains relative word error rate reductions of about 10% over our best baseline results in different experimental configurations. We believe this is the first attempt to show the effectiveness of margin-based acoustic modeling for large vocabulary continuous speech recognition in a hidden Markov model framework. Further improvements are expected because the approximate test risk bound minimization principle offers a flexible and rigorous framework to facilitate incorporation of new margin-based optimization criteria into hidden Markov model training.     

Space-Time Tree Ensemble for Action Recognition and Localization

Human actions are, inherently, structured patterns of body movements. We explore ensembles of hierarchical spatio-temporal trees, discovered directly from training data, to model these structures for action recognition and spatial localization. Discovery of frequent and discriminative tree structures is challenging due to the exponential search space, particularly if one allows partial matching. We address this by first building a concise action word vocabulary via discriminative clustering of the hierarchical space-time segments, which is a two-level video representation that captures both static and non-static relevant space-time segments of the video. Using this vocabulary we then utilize tree mining with subsequent tree clustering and ranking to select a compact set of discriminative tree patterns. Our experiments show that these tree patterns, alone, or in combination with shorter patterns (action words and pairwise patterns) achieve promising performance on three challenging datasets: UCF Sports, HighFive and Hollywood3D. Moreover, we perform cross-dataset validation, using trees learned on HighFive to recognize the same actions in Hollywood3D, and using trees learned on UCF-Sports to recognize and localize the similar actions in JHMDB. The results demonstrate the potential for cross-dataset generalization of the trees our approach discovers.     

Maximum confidence hidden markov modeling for face recognition

This paper presents a hybrid framework of feature extraction and hidden Markov modeling(HMM) for two-dimensional pattern recognition. Importantly, we explore a new discriminative training criterion to assure model compactness and discriminability. This criterion is derived from hypothesis test theory via maximizing the confidence of accepting the hypothesis that observations are from target HMM states rather than competing HMM states. Accordingly, we develop the maximum confidence hidden Markov modeling (MC-HMM) for face recognition. Under this framework, we merge a transformation matrix to extract discriminative facial features. The closed-form solutions to continuous-density HMM parameters are formulated. Attractively, the hybrid MC-HMM parameters are estimated under the same criterion and converged through the expectation-maximization procedure. From the experiments on FERET and GTFD facial databases, we find that the proposed method obtains robust segmentation in presence of different facial expressions, orientations, etc. In comparison with maximum likelihood and minimum classification error HMMs, the proposed MC-HMM achieves higher recognition accuracies with lower feature dimensions.