Deep recurrent neural networks with word embeddings for Urdu named entity recognition

Named entity recognition (NER) continues to be an important task in natural language processing because it is featured as a subtask and/or subproblem in information extraction and machine translation. In Urdu language processing, it is a very difficult task. This paper proposes various deep recurrent neural network (DRNN) learning models with word embedding. Experimental results demonstrate that they improve upon current state‐of‐the‐art NER approaches for Urdu. The DRRN models evaluated include forward and bidirectional extensions of the long short‐term memory and back propagation through time approaches. The proposed models consider both language‐dependent features, such as part‐of‐speech tags, and language‐independent features, such as the “context windows” of words. The effectiveness of the DRNN models with word embedding for NER in Urdu is demonstrated using three datasets. The results reveal that the proposed approach significantly outperforms previous conditional random field and artificial neural network approaches. The best f‐measure values achieved on the three benchmark datasets using the proposed deep learning approaches are 81.1%, 79.94%, and 63.21%, respectively.     

Simple and effective neural coreference resolution for Korean language

We propose an end-to-end neural coreference resolution for the Korean language that uses an attention mechanism to point to the same entity. Because Korean is a head-final language, we focused on a method that uses a pointer network based on the head. The key idea is to consider all nouns in the document as candidates based on the head-final characteristics of the Korean language and learn distributions over the referenced entity positions for each noun. Given the recent success of applications using bidirectional encoder representation from transformer (BERT) in natural language-processing tasks, we employed BERT in the proposed model to create word representations based on contextual information. The experimental results indicated that the proposed model achieved state-of-the-art performance in Korean language coreference resolution.     

A brief survey on adaptive video streaming quality assessment

Quality of experience (QoE) assessment for adaptive video streaming plays a significant role in advanced network management systems. It is especially challenging in case of dynamic adaptive streaming schemes over HTTP (DASH) which has increasingly complex characteristics including additional playback issues. In this paper, we provide a brief overview of adaptive video streaming quality assessment. Upon our review of related works, we analyze and compare different variations of objective QoE assessment models with or without using machine learning techniques for adaptive video streaming. Through the performance analysis, we observe that hybrid models perform better than both quality-of-service (QoS) driven QoE approaches and signal fidelity measurement. Moreover, the machine learning-based model slightly outperforms the model without using machine learning for the same setting. In addition, we find that existing video streaming QoE assessment models still have limited performance, which makes it difficult to be applied in practical communication systems. Therefore, based on the success of deep learned feature representations for traditional video quality prediction, we also apply the off-the-shelf deep convolutional neural network (DCNN) to evaluate the perceptual quality of streaming videos, where the spatio-temporal properties of streaming videos are taken into consideration. Experiments demonstrate its superiority, which sheds light on the future development of specifically designed deep learning frameworks for adaptive video streaming quality assessment. We believe this survey can serve as a guideline for QoE assessment of adaptive video streaming.     

Korean Coreference Resolution with Guided Mention Pair Model Using Deep Learning

The general method of machine learning has encountered disadvantages in terms of the significant amount of time and effort required for feature extraction and engineering in natural language processing. However, in recent years, these disadvantages have been solved using deep learning. In this paper, we propose a mention pair (MP) model using deep learning, and a system that combines both rule‐based and deep learning‐based systems using a guided MP as a coreference resolution, which is an information extraction technique. Our experiment results confirm that the proposed deep‐learning based coreference resolution system achieves a better level of performance than rule‐ and statistics‐based systems applied separately.     

基于深度学习的手语识别综述

手语识别涉及计算机视觉、模式识别、人机交互等领域,具有重要的研究意义与应用价值。深度学习技术的蓬勃发展为更加精准、实时的手语识别带来了新的机遇。该文综述了近年来基于深度学习的手语识别技术,从孤立词与连续语句两个分支展开详细的算法阐述与分析。孤立词识别技术划分为基于卷积神经网络(CNN)、3维卷积神经网络(3D-CNN)和循环神经网络(RNN) 3种架构的方法;连续语句识别所用模型复杂度更高,通常需要辅助某种长时时序建模算法,按其主体结构分为双向长短时记忆网络模型、3维卷积网络模型和混合模型。归纳总结了目前国内外常用手语数据集,探讨了手语识别技术的研究挑战与发展趋势,高精度前提下的鲁棒性和实用化仍有待于推进。     

A survey on deploying mobile deep learning applications: A systemic and technical perspective

With the rapid development of mobile devices and deep learning, mobile smart applications using deep learning technology have sprung up. It satisfies multiple needs of users, network operators and service providers, and rapidly becomes a main research focus. In recent years, deep learning has achieved tremendous success in image processing, natural language processing, language analysis and other research fields. Despite the task performance has been greatly improved, the resources required to run these models have increased significantly. This poses a major challenge for deploying such applications on resource-restricted mobile devices. Mobile intelligence needs faster mobile processors, more storage space, smaller but more accurate models, and even the assistance of other network nodes. To help the readers establish a global concept of the entire research direction concisely, we classify the latest works in this field into two categories, which are local optimization on mobile devices and distributed optimization based on the computational position of machine learning tasks. We also list a few typical scenarios to make readers realize the importance and indispensability of mobile deep learning applications. Finally, we conjecture what the future may hold for deploying deep learning applications on mobile devices research, which may help to stimulate new ideas.     

Effective short text classification via the fusion of hybrid features for IoT social data

Nowadays short texts can be widely found in various social data in relation to the 5G-enabled Internet of Things (IoT). Short text classification is a challenging task due to its sparsity and the lack of context. Previous studies mainly tackle these problems by enhancing the semantic information or the statistical information individually. However, the improvement achieved by a single type of information is limited, while fusing various information may help to improve the classification accuracy more effectively. To fuse various information for short text classification, this article proposes a feature fusion method that integrates the statistical feature and the comprehensive semantic feature together by using the weighting mechanism and deep learning models. In the proposed method, we apply Bidirectional Encoder Representations from Transformers (BERT) to generate word vectors on the sentence level automatically, and then obtain the statistical feature, the local semantic feature and the overall semantic feature using Term Frequency-Inverse Document Frequency (TF-IDF) weighting approach, Convolutional Neural Network (CNN) and Bidirectional Gate Recurrent Unit (BiGRU). Then, the fusion feature is accordingly obtained for classification. Experiments are conducted on five popular short text classification datasets and a 5G-enabled IoT social dataset and the results show that our proposed method effectively improves the classification performance.     

基于Web的无指导译文消歧词模型与N-gram模型及对比研究

该文提出了基于Web的无指导译文消歧的词模型及N-gram模型方法,并在尽可能相同的条件下进行了比较。两种方法均利用搜索引擎统计不同搜索片段在Web上的Page Count作为主要消歧信息。词模型定义了汉语词汇与英语词汇之间的双语词汇Web相关度,根据汉语上下文词汇与英语译文之间的相关度进行消歧;N-gram模型首先假设不同语义下的多义词N-gram序列行为模式不同,从而可对多义词不同语义类下词汇在实例中的N-gram序列进行统计与分析以进行消歧。两个模型的性能均超过了在国际语义评测SemEval2007的task#5上可比较的最好无指导系统。对这两个模型进行试验对比可发现N-gram模型性能优于词模型,也表明组合两类模型的结果有进一步提升消歧性能的潜力。     

基于异构数据联合训练的中文分词法

中文分词技术作为中文信息处理中的关键基础技术之一,基于深度学习模型的中文分词法受到广泛关注。然而,深度学习模型需要大规模数据训练才能获得良好的性能,而当前中文分词语料数据相对缺乏且标准不一。文中提出了一种简单有效的异构数据处理方法,对不同语料数据加上两个人工设定的标识符,使用处理过的数据应用于双向长短期记忆网络结合条件随机场(Bi-LSTM-CRF)的中文分词模型的联合训练。实验结果表明,基于异构数据联合训练的Bi-LSTM-CRF模型比单一数据训练的模型具有更好的分词性能。     

Surrogate Model via Artificial Intelligence Method for Accelerating Screening Materials and Performance Prediction

Predicting the performance of mechanical properties is an important and current issue in the field of engineering and materials science, but traditional experiments and modeling calculations often consume large amounts of time and resources. Therefore, it is imperative to use appropriate methods to accelerate the process of material selection and design. The artificial intelligence method, particularly deep learning models, has been verified as an effective and efficient method for handling computer vision and neural language problems. In this paper, a deep learning surrogate model (DLS) is proposed for predicting the mechanical performance of materials, that is, the maximum stress value under complex working conditions. The DLS can reproduce the finite element analysis model results with 98.79% accuracy. The results show that deep learning has great potential. This research also provides a new approach for material screening in practical engineering.     

基于Transformer的机器阅读理解对抗数据生成

机器阅读理解任务是衡量模型对于文本信息理解程度的一种重要方式,一直以来备受关注。近年来,很多学者在这一任务上提出了自己的模型,并取得了相当不错的成绩,其中一部分甚至已经超越了人工回答的准确率。然而,这些模型是否真正地、深入地理解了文本语义,还是仅依靠浅层的词语相似度和答案类型来进行简单的搜索?为了进一步评价阅读理解模型对于文章语义的理解程度,本文提出了一种基于Transformer结构的对抗数据生成方法,并对主流阅读理解模型进行了检测。     

全媒体内容质量评价研究综述

在全媒体时代,媒体内容的表现形式逐渐丰富,开始成为影响信息传播的一个重要因素。内容质量评价仍停留在“流量思维”阶段,难以客观评价内容质量,亟需发展以用户为中心的全媒体内容质量评价方法。本文主要概述近十年来国内外公开发表的不同媒介的评价模型,回顾了图像、视频、音频、文本四类的客观质量评价在全媒体数据中的研究工作及相应的应用,主要介绍基于传统方法和基于深度学习方法两大方向中一些影响力较大的方法,每类方法有分成有参考和无参考的方法,对此总结了各方法特点,对一些具有代表性的方法进行了实验对比分析。最后对四种媒介内容质量评价领域仍面临的问题进行了总结并展望未来可能的发展方向。      

基于BERT和LightGBM的文本关键词提取方法

传统的文本关键词提取方法忽略了上下文语义信息,不能解决一词多义问题,提取效果并不理想。基于LDA和BERT模型,文中提出LDA-BERT-LightG BM(LB-LightG BM)模型。该方法选择LDA主题模型获得每个评论的主题及其词分布,根据阈值筛选出候选关键词,将筛选出来的词和原评论文本拼接在一起输入到BERT模型中,进行词向量训练,得到包含文本主题词向量,从而将文本关键词提取问题通过LightG BM算法转化为二分类问题。通过实验对比了textrank算法、LDA算法、LightG BM算法及文中提出的LB-LightG BM模型对文本关键词提取的准确率P、召回率R以及F1。结果表明,当Top N取3～6时,F1的平均值比最优方法提升3.5%,该方法的抽取效果整体上优于实验中所选取的对比方法,能够更准确地发现文本关键词。     

基于监督对比学习正则化的高分辨率SAR图像建筑物提取方法

近年来,高分辨合成孔径雷达(SAR)图像的智能解译技术在城市规划、变化监测等方面得到了广泛应用.不同于光学图像,SAR图像的获取方式、图像中目标的几何结构等因素制约了现有深度学习方法对SAR图像地物目标的解译效果.该文针对高分辨SAR图像城市区域建筑物提取,提出了基于监督对比学习的正则化方法,其主要思想是增强同一类别像...     

一种RNN-T与BERT相结合的端到端语音识别模型

端到端语音识别模型由于结构简单且容易训练,已成为目前最流行的语音识别模型。然而端到端语音识别模型通常需要大量的语音-文本对进行训练,才能取得较好的识别性能。而在实际应用中收集大量配对数据既费力又昂贵,因此其无法在实际应用中被广泛使用。本文提出一种将RNN-T(Recurrent Neural Network Transducer,RNN-T)模型与BERT(Bidirectional Encoder Representations from Transformers,BERT)模型进行结合的方法来解决上述问题,其通过用BERT模型替换RNN-T中的预测网络部分,并对整个网络进行微调,从而使RNN-T模型能有效利用BERT模型中的语言学知识,进而提高模型的识别性能。在中文普通话数据集AISHELL-1上的实验结果表明,采用所提出的方法训练后的模型与基线模型相比能获得更好的识别结果。     

融合TextRank算法的中文短文本相似度计算

传统的VSM向量空间模型忽略了文本语义,构建的文本特征矩阵具有稀疏性。基于深度学习词向量技术,文中提出一种融合改进TextRank算法的相似度计算方法。该方法利用词向量嵌入的技术来构建文本向量空间,使得构建的向量空间模型具有了语义相关性,同时采用改进的TextRank算法提取文本关键字,增强了文本特征的表达并消除了大量冗余信息,降低了文本特征矩阵的稀疏性,使文本相似度的计算更加高效。不同模型的仿真实验结果表明,融合改进的TextRank算法与Bert词向量技术的方法具有更好的文本相似度计算性能。     

Intelligent detection of vulnerable functions in software through neural embedding-based code analysis

Software vulnerability is a fundamental problem in cybersecurity, which poses severe threats to the secure operation of devices and systems. In this paper, we propose a new vulnerability detection framework of employing advanced neural embedding. For example, CodeBERT is a large-scale pre-trained embedding model for natural language and programming language. It achieves state-of-the-art performance on various natural language processing and code analysis tasks, demonstrating improved generalization ability compared with conventional models. The proposed framework encapsulates CodeBERT as a code representation generator and combines it with transfer learning to conduct cross-project vulnerability detection. Considering the problem of lacking code embedding models on C source code, we extract the knowledge from C source code to fine-tune the pre-trained embedding model, so as to better facilitate the detection of function-level vulnerabilities in C open-source projects. To address the severe data imbalance issue in real-world scenarios, we introduce code argumentation idea and use a large number of synthetic vulnerability data to further improve the robustness of the detection method. Experimental results show that the proposed vulnerability detection framework achieves better performance than existing methods.     

Deep Learning Based Semi-Blind Tracking for Aging Wireless Communication Channels

The time-varying nature of the wireless propagation channel under high user mobilities, termed as channel aging, is a major performance impediment in many communication systems. In this paper, we discuss deep learning models for semi-blind channel estimation in a single input single output wireless communication system under channel aging. In our proposed scheme, we first use pilot based training to obtain initial channel estimates. Following this, we treat the detected symbols as pilots and perform further channel estimation using an Encoder-Decoder LSTM network for constant and sliding window schemes. To show the effectiveness of our method, we show the training capabilities of our models and the BER vs SNR graphs for multiple simulations. We discuss integrating these Encoder-Decoder LSTM models with deep learning enabled symbol detection techniques like the DetNet to further improve spectral efficiency. The Encoder-Decoder LSTM network gives us a low BER, with the moving window scheme outperforming the constant window scheme.