Retraining Deep Neural Network with Unlabeled Data Collected in Embedded Devices

Because of computational complexity, the deep neural network (DNN) in embedded devices is usually trained on high-performance computers or graphic processing units (GPUs), and only the inference phase is implemented in embedded devices. Data processed by embedded devices, such as smartphones and wearables, are usually personalized, so the DNN model trained on public data sets may have poor accuracy when inferring the personalized data. As a result, retraining DNN with personalized data collected locally in embedded devices is necessary. Nevertheless, retraining needs labeled data sets, while the data collected locally are unlabeled, then how to retrain DNN with unlabeled data is a problem to be solved. This paper proves the necessity of retraining DNN model with personalized data collected in embedded devices after trained with public data sets. It also proposes a label generation method by which a fake label is generated for each unlabeled training case according to users’ feedback, thus retraining can be performed with unlabeled data collected in embedded devices. The experimental results show that our fake label generation method has both good training effects and wide applicability. The advanced neural networks can be trained with unlabeled data from embedded devices and the individualized accuracy of the DNN model can be gradually improved along with personal using.     

Acoustic Event Detection in Multichannel Audio Using Gated Recurrent Neural Networks with High‐Resolution Spectral Features

Recently, deep recurrent neural networks have achieved great success in various machine learning tasks, and have also been applied for sound event detection. The detection of temporally overlapping sound events in realistic environments is much more challenging than in monophonic detection problems. In this paper, we present an approach to improve the accuracy of polyphonic sound event detection in multichannel audio based on gated recurrent neural networks in combination with auditory spectral features. In the proposed method, human hearing perception‐based spatial and spectral‐domain noise‐reduced harmonic features are extracted from multichannel audio and used as high‐resolution spectral inputs to train gated recurrent neural networks. This provides a fast and stable convergence rate compared to long short‐term memory recurrent neural networks. Our evaluation reveals that the proposed method outperforms the conventional approaches.     

红外目标检测网络改进半监督迁移学习方法

针对红外数据集规模小,标记样本少的特点,提出了一种红外目标检测网络的半监督迁移学习方法,主要用于提高目标检测网络在小样本红外数据集上的训练效率和泛化能力,提高深度学习模型在训练样本较少的红外目标检测等场景当中的适应性。文中首先阐述了在标注样本较少时无标注样本对提高模型泛化能力、抑制过拟合方面的作用。然后提出了红外目标检测网络的半监督迁移学习流程:在大量的RGB图像数据集中训练预训练模型,后使用少量的有标注红外图像和无标注红外图像对网络进行半监督学习调优。另外,文中提出了一种特征相似度加权的伪监督损失函数,使用同一批次样本的预测结果相互作为标注,以充分利用无标注图像内相似目标的特征分布信息;为降低半监督训练的计算量,在伪监督损失函数的计算中,各目标仅将其特征向量邻域范围内的预测目标作为伪标注。实验结果表明,文中方法所训练的目标检测网络的测试准确率高于监督迁移学习所获得的网络,其在Faster R-CNN上实现了1.1%的提升,而在YOLO-v3上实现了4.8%的显著提升,验证了所提出方法的有效性。     

基于数据扩维的SAR目标识别性能提升技术

卷积神经网络的出现使得深度学习在视觉领域取得了巨大的成功,并逐渐延伸到合成孔径雷达(SAR)图像识别领域。然而,SAR图像样本量不足,难以支撑卷积神经网络的训练需求,并且SAR图像包含大量相干斑噪声及不确定性,网络结构的设计较为困难。所以,深度学习在SAR图像识别领域的应用受到阻碍。针对上述问题,文中提出一种基于数据扩维的SAR目标识别性能提升方法,通过对原始SAR 图像进行相关预处理操作并把处理后图像与原始图像结合,从而将一维的原始数据扩充成多维数据来作为训练样本。该扩维方法不仅间接扩充了样本量来支撑网络训练,同时也在网络训练前加入了“主动学习冶影响,所以无需针对SAR图像特性来构建复杂卷积网络,而采用成熟、简单的网络进行训练就可以达到理想的测试精度。最后,使用MSTAR 数据对该方法进行了性能验证,实验结果显示了所提方法的有效性。     

基于聚类核函数的最小二乘支持向量机高光谱图像半监督分类

针对大规模的高光谱数据分类,为了利用未标签样本所含信息,来提升分类器性能,提出了一种半监督分类算法。该算法根据聚类假设,即属于同一类地物的样本点在聚类中被分为同一类的可能性较大的原则来改进核函数,采用基于光谱角度量的K均值聚类算法对样本集进行聚类,根据多次聚类的结果,构造包袋核函数,然后利用加法和乘法运算将包袋核函数和RBF核函数组合成新的核函数,从而把未标签样本信息融入分类器。而且采用最小二乘支持向量机,将标准支持向量机的二次规划问题转换为求解线性方程组的问题。高光谱实测数据实验表明了本文方法的优越性。      

Neural-network-based segmentation of multi-modal medical images: a comparative and prospective study

This work presents an investigation of the potential of artificial neural networks for classification of registered magnetic resonance and X-ray computer tomography images of the human brain. First, topological and learning parameters are established experimentally. Second, the learning and generalization properties of the neural networks are compared to those of a classical maximum likelihood classifier and the superiority of the neural network approach is demonstrated when small training sets are utilized. Third, the generalization properties of the neural networks are utilized to develop an adaptive learning scheme able to overcome interslice intensity variations typical of MR images. This approach permits the segmentation of image volumes based on training sets selected on a single slice. Finally, the segmentation results obtained both with the artificial neural network and the maximum likelihood classifiers are compared to contours drawn manually.     

深度神经网络在风电功率预测中的应用研究

风电功率预测对于电网建设具有重要意义。文中提出一种基于深度神经网络的风电功率预测方法,该方法充分考虑了影响风电功率的若干因素,如风速、风向、空气密度和季节,通过深度神经网络训练、学习给出最佳的功率预测值。通过深度神经网络学习的特征能够更为有效地反映实际情况,因此提高了风电功率预测的稳健性。基于内蒙古某风电厂的实测数据进行了验证实验,结果表明了提出方法的有效性。     

An improved noise loss correction algorithm for learning from noisy labels

Despite excellent performance in image classification researches, the training of the deep neural networks (DNN) needs a large set of clean data with accurate annotations. The collection of a dataset is easy, but annotating the collected data is difficult on the contrary. There are many image data on the websites, which contain inaccurate annotations, but trainings on these datasets may make networks easier to over-fit noisy data and cause performance degradation. In this work, we propose an improved joint optimization framework for noise correction, which uses the Combination of Mix-up entropy and Kullback-Leibler entropy (CMKL) as the loss function. The new loss function can achieve better fine-tuning results after updating all label annotations. The experimental results on publicly available CIFAR-10 dataset and Clothing1M dataset show superior performance of our approach compared with other state-of-the-art methods.     

基于改进池化层的弱标记声音事件检测

针对DCASE2017挑战赛任务4提供的大规模弱标记声音事件检测数据集,搭建了基于梅尔滤波器特征(Fbank)、卷积神经网络(CNN)以及循环神经网络(RNN)的多类别声音事件检测系统,分析了attention和linear softmax两种已有的常用池化层在神经网络反向传播中的部分推演过程,并在linear softmax池化层的基础上进行改进,提出了一种“指数可学习的幂函数softmax”池化层。实验结果表明,相比于DCASE竞赛中获得第一名的模型,应用“指数可学习的幂函 softmax”池化层的检测系统,将段级别的声音事件预测的F1值从0.556提高到0.652,帧级别预测的F1值从0.518提高到0.583,帧级别预测的error rate (ER) 从0.730降低到0.667。      

Framework for TCAD augmented machine learning on multi-I-V characteristics using convolutional neural network and multiprocessing

In a world where data is increasingly important for making breakthroughs,microelectronics is a field where data is sparse and hard to acquire.Only a few entities have the infrastructure that is required to automate the fabrication and testing of semiconductor devices.This infrastructure is crucial for generating sufficient data for the use of new information technolo-gies.This situation generates a cleavage between most of the researchers and the industry.To address this issue,this paper will introduce a widely applicable approach for creating custom datasets using simulation tools and parallel computing.The multi-I-V curves that we obtained were processed simultaneously using convolutional neural networks,which gave us the abil-ity to predict a full set of device characteristics with a single inference.We prove the potential of this approach through two con-crete examples of useful deep learning models that were trained using the generated data.We believe that this work can act as a bridge between the state-of-the-art of data-driven methods and more classical semiconductor research,such as device en-gineering,yield engineering or process monitoring.Moreover,this research gives the opportunity to anybody to start experi-menting with deep neural networks and machine learning in the field of microelectronics,without the need for expensive experi-mentation infrastructure.     

采用深度卷积神经网络方法的毫米波图像目标检测

通过收集大量的毫米波图像并建立相应的人体数据集进行检测,提出基于Faster R-CNN深度学习的方法检测隐藏于人体上的危险物品。该方法将区域建议网络和VGG19训练卷积神经网络模型相结合,构建了面向毫米波图像目标检测的深度卷积神经网络。为了提高毫米波图像的处理能力,采用Caffe深度学习框架在图形处理单元上进行训练和测试。实验结果证明了基于Faster R-CNN深度卷积神经网络的目标检测方法能有效检测毫米波图像中的危险物品,并且目标检测的平均准确率约94%,检测速度约为6 frame/s,对毫米波安检系统的智能化发展有着极其重要的参考价值。     

Hybrid identification of nuclear power plant transients with artificial neural networks

Proper and rapid identification of malfunctions (transients) is of premier importance for the safe operation of nuclear power plants. Feedforward neural networks trained with the backpropagation (BP) algorithm are frequently applied to model simulated nuclear power plant malfunctions. The correct identification of unlabeled transients-or transients of the "don't-know" type have proven to be especially challenging. A novel hybrid neural network methodology is presented which also correctly classifies the unlabeled transients. From this analysis the importance for properly accommodating practical aspects such as the drift of electronics elements of a simulator, the digitization of simulated and actual plant signals, and the accumulating errors during numerical integration became obvious. Beside the feedforward neural networks trained with the BP algorithm, many other types of networks and codes were used for finding the best (sensitive and robust) algorithms. Various neural network based models were successfully applied to identify labeled and unlabeled malfunctions of the Hungarian Paks nuclear power plant simulator. The BP and probabilistic methods have been proven as the most robust against the misleading recognition of unlabeled malfunctions.     

深度卷积神经网络图像识别模型对抗鲁棒性技术综述

近年来,以卷积神经网络为代表的深度识别模型取得重要突破,不断刷新光学和SAR图像场景分类、目标检测、语义分割与变化检测等多项任务性能水平.然而深度识别模型以统计学习为主要特征,依赖大规模高质量训练数据,只能提供有限的可靠性能保证.深度卷积神经网络图像识别模型很容易被视觉不可感知的微小对抗扰动欺骗,给其在医疗、安防、自动...     

基于复述增广的医疗领域机器翻译

医疗机器翻译对于跨境医疗、医疗文献翻译等应用具有重要价值.汉英神经机器翻译依靠深度学习强大的建模能力和大规模双语平行数据取得了长足的进步.神经机器翻译通常依赖于大规模的平行句对训练翻译模型.目前,汉英翻译数据主要以新闻、政策等领域数据为主,缺少医疗领域的数据,导致医疗领域的汉英机器翻译效果不佳.针对医疗垂直领域机器翻译...     

Fault diagnosis of active magnetic bearings

Active magnetic bearings (AMBs) are intrinsically unstable systems and require feedback control to ensure stable operation. Further, sensors, actuators, and the rotor need to operate under normal conditions, and a fault detection and diagnostics system is necessary to ensure a safe and reliable operation. Accordingly, several studies have developed methods to detect failures associated with the rotor or the electrical system (i.e., AMB). However, prior identification of the dynamic system parameters or the magnetic forces is usually desired, which can be impractical for real machines. To overcome this problem, this study proposes a failure detection method based on a mathematical model and the correlation between the measured states related to the rotor and the control. Artificial neural networks are used to map the states that cannot be measured, and faults are determined by comparing the output correlations of neural networks. Faults in the AMB/rotor system are identified considering various rotor unbalance configurations (mechanical failures) and failures in the position sensor gain and in the magnetic actuator current (electrical failures). Various fault configurations were explored for each case cited. A comparison of the theoretical and experimental results showed good agreement, which demonstrates the adequacy of the method in detecting mechanical and electrical failures in industrial machines.     

基于神经网络的模糊呼叫接纳控制器

本文提出了一种新的基于神经网络的模糊逻辑技术实现呼叫接纲控制（ＣＡＣ）的方法,与传统的神经网络方法相比,该方法利用了两种技术的优点,能合理地选择初始参数,同时具有自学习能力,模拟结果表明,我们提出的方法能够利用较少的训练数据获得较高的精确怀,减少了训练时间,体现了传统才神经网络方法不具备的优越性。     

用于视网膜血管分割的半监督深度学习框架北大核心CSCD

针对目前视网膜血管分割任务中伪标签质量参差不齐,获得高质量的伪标签需要经过筛选的问题,本文提出了一种新的用于视网膜血管分割的半监督深度学习框架。该框架采用分而治之的思想来处理数据,针对有标签数据,采用传统的深度学习方法;针对无标签数据,采用Mean teacher模型,通过对比同一输入的不同形态输出,让模型学习无标签数据之间的共同特征,避免了采用伪标签技术带来的筛选过程。本文将U型网络(u-neural networks,U-Net)、Dense-Net和Ladder-Net 3个基准网络放入该框架,在DRIVE和CHASEDB1数据集上进行训练测试,均取得了较好的分割效果,表明本文框架具有提高网络区分不同阈值像素的能力。     

基于高效用神经网络的文本分类方法

现有的基于深度学习的文本分类方法没有考虑文本特征的重要性和特征之间的关联关系,影响了分类的准确率.针对此问题,本文提出一种基于高效用神经网络（High Utility Neural Networks,HUNN）的文本分类模型,可以有效地表示文本特征的重要性及其关联关系.利用高效用项集挖掘（Mining High Utility Itemsets,MHUI）算法获取数据集中各个特征的重要性以及共现频率.其中,共现频率在一定程度上反映了特征之间的关联关系.将MHUI作为HUNN的挖掘层,用于挖掘每个类别数据中重要性和关联性强的文本特征.然后将这些特征作为神经网络的输入,再经过卷积层进一步提炼类别表达能力更强的高层次文本特征,从而提高模型分类的准确率.通过在6个公开的基准数据集上进行实验分析,提出的算法优于卷积神经网络（Convolutional Neural Networks,CNN）,循环神经网络（Recurrent Neural Networks,RNN）,循环卷积神经网络（Recurrent Convolutional Neural Networks,RCNN）,快速文本分类（Fast Text Classifier,FAST）,分层注意力网络（Hierarchical Attention Networks,HAN）等5个基准算法.     

基于卷积神经网络的图像分类算法综述

随着大数据的到来以及计算能力的提高,深度学习(Deep Learning, DL)席卷全球。传统的图像分类方法难以处理庞大的图像数据以及无法满足人们对图像分类精度和速度上的要求,基于卷积神经网络(Convolutional Neural Network, CNN)的图像分类方法冲破了传统图像分类方法的瓶颈,成为目前图像分类的主流算法,如何有效利用卷积神经网络来进行图像分类成为国内外计算机视觉领域研究的热点。本文在对卷积神经网络进行系统的研究并且深入研究卷积神经网络在图像处理中的应用后,给出了基于卷积神经网络的图像分类所采用的主流结构模型、优缺点、时间/空间复杂度、模型训练过程中可能遇到的问题和相应的解决方案,与此同时也对基于深度学习的图像分类拓展模型的生成式对抗网络和胶囊网络进行介绍;然后通过仿真实验验证了在图像分类精度上,基于卷积神经网络的图像分类方法优于传统图像分类方法,同时综合比较了目前较为流行的卷积神经网络模型之间的性能差异并进一步验证了各种模型的优缺点;最后对于过拟合问题、数据集构建方法、生成式对抗网络及胶囊网络性能进行相关实验及分析。      

Monitoring and diagnosis of rolling element bearings usingartificial neural networks

Vibration monitoring of components in manufacturing plants involves the collection of vibration data from plant components and detailed analysis to detect features that reflect the operational state of the machinery. The analysis leads to the identification of potential failures and their causes and makes it possible to perform efficient preventive maintenance. Work on the design of a vibration monitoring methodology for rolling element bearings (REB) based on neural network technology is presented. This technology provides an attractive complement to traditional vibration analysis because of the potential of neural networks to operate in real-time mode and to handle data that may be distorted or noisy. The significance of this work relies on the fact that REB failures are responsible for a large fraction of the malfunctions in manufacturing equipment. The technique enhances traditional vibration analysis and provides a means of automating the monitoring and diagnosis of vibrating equipment