基于在线半监督学习的故障诊断方法研究

针对机械故障诊断中准确、完备的故障训练样本获取困难,而现有分类方法难以有效地发掘大量未标记故障样本中蕴含的有用信息,提出了一种基于在线半监督学习的故障诊断方法.该方法基于Tri-training算法将在线贯序极限学习机从监督学习模式扩展到半监督学习模式,利用少量不精确的标记样本构建初始分类器,并从大量未标记样本中在线扩充标记样本,对分类器进行增量式更新以提高其泛化性能.半监督基准数据试验结果表明,训练样本总数相同但标记样本数与未标记样本数比例不同时,所提算法得到的分类准确率相当且训练时间相差小于1.2倍.以柴油机8种工况的故障模式为对象进行试验验证,结果表明标记故障样本较少时,未标记故障样本的加入可使故障分类准确率提高5％～8％.     

MII: A Novel Text Classification Model Combining Deep Active Learning with BERT

Active learning has been widely utilized to reduce the labeling cost of supervised learning. By selecting specific instances to train the model, the performance of the model was improved within limited steps. However, rare work paid attention to the effectiveness of active learning on it. In this paper, we proposed a deep active learning model with bidirectional encoder representations from transformers (BERT) for text classification. BERT takes advantage of the self-attention mechanism to integrate contextual information, which is beneficial to accelerate the convergence of training. As for the process of active learning, we design an instance selection strategy based on posterior probabilities Margin, Intra-correlation and Inter-correlation (MII). Selected instances are characterized by small margin, low intra-cohesion and high inter-cohesion. We conduct extensive experiments and analytics with our methods. The effect of learner is compared while the effect of sampling strategy and text classification is assessed from three real datasets. The results show that our method outperforms the baselines in terms of accuracy.     

多波束声呐海底底质半监督学习分类方法

多波束测深声呐的反向散射数据中包含海底表层的声学信息,可以用来进行海底表层底质分类。但实际中通过物理采样获得大范围的底质类型的标签信息所需成本过高,制约了传统监督分类算法的性能。针对实际应用中只拥有大量无标签数据和少量有标签数据的情况,文章提出了基于自动编码器预训练以及伪标签自训练的半监督学习底质分类算法。利用2018年和2019年两次同一海域实验采集的多波束测深声呐反向散射数据,对所提算法进行了验证。数据处理结果表明,相比仅利用有标签数据的监督分类算法,提出的半监督学习分类算法保证分类准确率的同时所需的有标签数据更少。自动编码器预训练的半监督学习分类方法在有标签样本数量极少时的准确率仍高于75%。     

Tibetan Sentiment Classification Method Based on Semi-Supervised Recursive Autoencoders

We apply the semi-supervised recursive autoencoders (RAE) model for the sentiment classification task of Tibetan short text, and we obtain a better classification effect. The input of the semi-supervised RAE model is the word vector. We crawled a large amount of Tibetan text from the Internet, got Tibetan word vectors by using Word2vec, and verified its validity through simple experiments. The values of parameter α and word vector dimension are important to the model effect. The experiment results indicate that when α is 0.3 and the word vector dimension is 60, the model works best. Our experiment also shows the effectiveness of the semi-supervised RAE model for Tibetan sentiment classification task and suggests the validity of the Tibetan word vectors we trained.     

用于水声目标识别的自适应遗传样本选择算法

针对训练样本集中含有噪声样本、冗余样本以及无关样本,导致分类系统分类性能下降、不稳定的水声目标识别问题,提出了一种新的自适应遗传样本选择算法(Adaptive Genetic Instance Selection Algorithm, AGISA)。算法先随机生成初始种群,接着利用设计的遗传算子(跨代选择、自适应交叉和简化最近邻变异)指导种群进化,每代中对分类贡献大且选择样本数目少的个体适应度值高。提取了实测3类水声目标的多域特征,进行样本选择和分类识别仿真实验,结果表明：AGISA可以选出有效样本子集,在样本维数下降约73%的情况下,支持向量机分类器的正确分类率能提高约2.5%;并且AGISA具有较好的收敛性、稳定性,所得优化样本子集具有较好泛化能力且能明显减少分类的时间。     

Fuzzy Based Latent Dirichlet Allocation for Intrusion Detection in Cloud Using ML

The growth of cloud in modern technology is drastic by provisioning services to various industries where data security is considered to be common issue that influences the intrusion detection system (IDS). IDS are considered as an essential factor to fulfill security requirements. Recently, there are diverse Machine Learning (ML) approaches that are used for modeling effectual IDS. Most IDS are based on ML techniques and categorized as supervised and unsupervised. However, IDS with supervised learning is based on labeled data. This is considered as a common drawback and it fails to identify the attack patterns. Similarly, unsupervised learning fails to provide satisfactory outcomes. Therefore, this work concentrates on semi-supervised learning model known as Fuzzy based semi-supervised approach through Latent Dirichlet Allocation (F-LDA) for intrusion detection in cloud system. This helps to resolve the aforementioned challenges. Initially, LDA gives better generalization ability for training the labeled data. Similarly, to handle the unlabelled data, Fuzzy model has been adopted for analyzing the dataset. Here, pre-processing has been carried out to eliminate data redundancy over network dataset. In order to validate the efficiency of F-LDA towards ID, this model is tested under NSL-KDD cup dataset is a common traffic dataset. Simulation is done in MATLAB environment and gives better accuracy while comparing with benchmark standard dataset. The proposed F-LDA gives better accuracy and promising outcomes than the prevailing approaches.     

A classification tool for N-way array based on SIMCA methodology

In the literature there are only few papers concerned with classification methods for multi-way arrays. The most common procedure, by far, is to unfold the multi-way data array into an ordinary matrix and then to apply the traditional multivariate tools for classification. As opposed to unfolding the data several possibilities exist for building classification models more directly based on the multi-way structure of the data. As an example, multi-way partial least squares discriminant analysis has been used as a supervised classification method, another alternative that has been investigated is to perform classification using Fisher's LDA or SIMCA on the score matrix from e.g. a PARAFAC or a Tucker model. Despite a few attempts of applying such multi-way classification approaches, no-one has looked into how such models are best built and implemented.In this work, the SIMCA method is extended to three-way arrays. Included in this work is also actual code that will work on general multi-way arrays rather than just three-way arrays. In analogy with two-way SIMCA, a decomposition model is separately built for the multi-way data for each class, using multi-way decomposition method such as PARAFAC or Tucker3. In the choice of the best class dimensionality, i.e. number of latent factors, both the results of cross-validation but mainly the sensitivity/specificity values are evaluated. In order to estimate the class limits for each class model, orthogonal and score distances are considered, and different statistics are implemented and tested to set confidence limits for these two parameters. Classification performance using different definitions of class boundaries and classification rules, including the use of cross-validated residuals and scores is compared.The proposed N-SIMCA methodology and code, besides simulated data sets of varying dimensionality, has been tested on two case studies, concerning food authentication tasks for typical food products.     

Paragraph Vector Representation Based on Word to Vector and CNN Learning

Document processing in natural language includes retrieval, sentiment analysis, theme extraction, etc. Classical methods for handling these tasks are based on models of probability, semantics and networks for machine learning. The probability model is loss of semantic information in essential, and it influences the processing accuracy. Machine learning approaches include supervised, unsupervised, and semi-supervised approaches, labeled corpora is necessary for semantics model and supervised learning. The method for achieving a reliably labeled corpus is done manually, it is costly and time-consuming because people have to read each document and annotate the label of each document. Recently, the continuous CBOW model is efficient for learning high-quality distributed vector representations, and it can capture a large number of precise syntactic and semantic word relationships, this model can be easily extended to learn paragraph vector, but it is not precise. Towards these problems, this paper is devoted to developing a new model for learning paragraph vector, we combine the CBOW model and CNNs to establish a new deep learning model. Experimental results show that paragraph vector generated by the new model is better than the paragraph vector generated by CBOW model in semantic relativeness and accuracy.     

Sentiment Classification Based on Piecewise Pooling Convolutional Neural Network

Recently, the effectiveness of neural networks, especially convolutional neural networks, has been validated in the field of natural language processing, in which, sentiment classification for online reviews is an important and challenging task. Existing convolutional neural networks extract important features of sentences without local features or the feature sequence. Thus, these models do not perform well, especially for transition sentences. To this end, we propose a Piecewise Pooling Convolutional Neural Network (PPCNN) for sentiment classification. Firstly, with a sentence presented by word vectors, convolution operation is introduced to obtain the convolution feature map vectors. Secondly, these vectors are segmented according to the positions of transition words in sentences. Thirdly, the most significant feature of each local segment is extracted using max pooling mechanism, and then the different aspects of features can be extracted. Specifically, the relative sequence of these features is preserved. Finally, after processed by the dropout algorithm, the softmax classifier is trained for sentiment classification. Experimental results show that the proposed method PPCNN is effective and superior to other baseline methods, especially for datasets with transition sentences.     

A hybrid SVM and supervised learning approach to fuzzy min‐max hyperbox classifiers

Abstract

In this paper, a fuzzy min‐max hyperbox classifier is designed to solve M‐class classification problems using a hybrid SVM and supervised learning approach. In order to solve a classification problem, a set of training patterns is gathered from a considered classification problem. However, the training set may include several noisy patterns. In order to delete the noisy patterns from the training set, the support vector machine is applied to find the noisy patterns so that the remaining training patterns can describe the behavior of the considered classification system well. Subsequently, a supervised learning method is proposed to generate fuzzy min‐max hyperboxes for the remaining training patterns so that the generated fuzzy min‐max hyperbox classifier has good generalization performance. Finally, the Iris data set is considered to demonstrate the good performance of the proposed approach for solving this classification problem.     

A supervised dimensionality reduction method-based sparse representation for face recognition

The sparse representation-based classification (SRC) method is a powerful tool to present high-dimensionality data and its superiority in many fields, especially in face recognition application has been proved. With sparsity appropriately harnessed, the SRC can solve face classification problems caused by varying expression, illumination as well as occlusion and disguise. However, face images as high-dimensionality data are usually noisy and the dimensionality is always larger than the number of training sample in real-world applications, which bring a disadvantage for the performance of SRC. Therefore, it is beneficial to perform dimensionality reduction (DR) before utilizing the SRC method. But most prevalent DR methods have no direct connection to SRC. In this paper, we proposed a supervised DR algorithm which suits SRC well and improves the discriminating ability in the low-dimensionality space. The proposed method utilizes the fisher discriminant criterion and low-dimensionality reconstructive restriction to extract the discriminating structure of data. The extensive experiments on public face databases verified the effectiveness of the supervised DR with the model of sparse representation.     

Progress in Neural NLP: Modeling,Learning, and Reasoning

Natural language processing (NLP) is a subfield of artificial intelligence that focuses on enabling computers to understand and process human languages. In the last five years, we have witnessed the rapid development of NLP in tasks such as machine translation, question-answering, and machine reading comprehension based on deep learning and an enormous volume of annotated and unannotated data. In this paper, we will review the latest progress in the neural network-based NLP framework (neural NLP) from three perspectives: modeling, learning, and reasoning. In the modeling section, we will describe several fundamental neural network-based modeling paradigms, such as word embedding, sentence embedding, and sequence-to-sequence modeling, which are widely used in modern NLP engines. In the learning section, we will introduce widely used learning methods for NLP models, including supervised, semi-supervised, and unsupervised learning; multitask learning; transfer learning; and active learning. We view reasoning as a new and exciting direction for neural NLP, but it has yet to be well addressed. In the reasoning section, we will review reasoning mechanisms, including the knowledge, existing non-neural inference methods, and new neural inference methods. We emphasize the importance of reasoning in this paper because it is important for building interpretable and knowledge-driven neural NLP models to handle complex tasks. At the end of this paper, we will briefly outline our thoughts on the future directions of neural NLP.     

声呐渗流测量数据降噪的分类模型研究

渗漏造成的一系列安全隐患己严重威胁到地下隐蔽工程的建设与正常运行.为了研发新的渗流测量手段与技术方法,减少控制渗漏事故的发生,提出了一种基于梯度提升树的声呐渗流检测结果分类模型.模型利用ReliefF算法选取贡献权重大的特征作为训练数据集,利用属性标注的数据集训练出区分水库渗流、井孔渗流与噪声的梯度提升树模型.实验结果...     

Error modeling for surrogates of dynamical systems using machine learning

A machine learning–based framework for modeling the error introduced by surrogate models of parameterized dynamical systems is proposed. The framework entails the use of high‐dimensional regression techniques (eg, random forests, and LASSO) to map a large set of inexpensively computed “error indicators” (ie, features) produced by the surrogate model at a given time instance to a prediction of the surrogate‐model error in a quantity of interest (QoI). This eliminates the need for the user to hand‐select a small number of informative features. The methodology requires a training set of parameter instances at which the time‐dependent surrogate‐model error is computed by simulating both the high‐fidelity and surrogate models. Using these training data, the method first determines regression‐model locality (via classification or clustering) and subsequently constructs a “local” regression model to predict the time‐instantaneous error within each identified region of feature space. We consider 2 uses for the resulting error model: (1) as a correction to the surrogate‐model QoI prediction at each time instance and (2) as a way to statistically model arbitrary functions of the time‐dependent surrogate‐model error (eg, time‐integrated errors). We apply the proposed framework to model errors in reduced‐order models of nonlinear oil‐water subsurface flow simulations, with time‐varying well‐control (bottom‐hole pressure) parameters. The reduced‐order models used in this work entail application of trajectory piecewise linearization in conjunction with proper orthogonal decomposition. When the first use of the method is considered, numerical experiments demonstrate consistent improvement in accuracy in the time‐instantaneous QoI prediction relative to the original surrogate model, across a large number of test cases. When the second use is considered, results show that the proposed method provides accurate statistical predictions of the time‐ and well‐averaged errors.     

Investigation on the Chinese Text Sentiment Analysis Based on Convolutional Neural Networks in Deep Learning

Nowadays, the amount of wed data is increasing at a rapid speed, which presents a serious challenge to the web monitoring. Text sentiment analysis, an important research topic in the area of natural language processing, is a crucial task in the web monitoring area. The accuracy of traditional text sentiment analysis methods might be degraded in dealing with mass data. Deep learning is a hot research topic of the artificial intelligence in the recent years. By now, several research groups have studied the sentiment analysis of English texts using deep learning methods. In contrary, relatively few works have so far considered the Chinese text sentiment analysis toward this direction. In this paper, a method for analyzing the Chinese text sentiment is proposed based on the convolutional neural network (CNN) in deep learning in order to improve the analysis accuracy. The feature values of the CNN after the training process are nonuniformly distributed. In order to overcome this problem, a method for normalizing the feature values is proposed. Moreover, the dimensions of the text features are optimized through simulations. Finally, a method for updating the learning rate in the training process of the CNN is presented in order to achieve better performances. Experiment results on the typical datasets indicate that the accuracy of the proposed method can be improved compared with that of the traditional supervised machine learning methods, e.g., the support vector machine method.     

基于谱聚类下采样失衡数据下SVM故障检测

在故障诊断领域中,对传统支持向量机（SVM）算法在数据失衡情况下无法有效实现故障检测的不足,提出一种基于谱聚类下采样失衡数据下SVM故障检测算法。该算法在核空间中对多数类进行谱聚类,然后选择具有代表意义的信息点,最终实现样本均衡。将该算法应用在轴承故障检测领域,并同其他算法进行比较,试验结果表明本文建议的算法在失衡数据情况下较其他算法具有较强的故障检测性能。     

A new method to deal with fuzzy classification problems by tuning membership functions for fuzzy classification systems

Abstract

This paper presents a new method to construct and tune membership functions and generate fuzzy classification rules from training instances for handling the Iris data classification problem. First, we find two attributes of the Iris data from the training instances that are suitable to serve as classification criteria. Then, we construct and tune the membership functions of these two attributes and generate fuzzy classification rules from the training instances. The proposed method generates the same number of fuzzy classification rules as the number of species of the training instances. It generates fewer fuzzy classification rules and can get a higher average classification accuracy rate than the existing methods.     

Analyzing Cross-domain Transportation Big Data of New York City with Semi-supervised and Active Learning

The majority of big data analytics applied to transportation datasets suffer from being too domain-specific, that is, they draw conclusions for a dataset based on analytics on the same dataset. This makes models trained from one domain (e.g. taxi data) applies badly to a different domain (e.g. Uber data). To achieve accurate analyses on a new domain, substantial amounts of data must be available, which limits practical applications. To remedy this, we propose to use semi-supervised and active learning of big data to accomplish the domain adaptation task: Selectively choosing a small amount of datapoints from a new domain while achieving comparable performances to using all the datapoints. We choose the New York City (NYC) transportation data of taxi and Uber as our dataset, simulating different domains with 90% as the source data domain for training and the remaining 10% as the target data domain for evaluation. We propose semi-supervised and active learning strategies and apply it to the source domain for selecting datapoints. Experimental results show that our adaptation achieves a comparable performance of using all datapoints while using only a fraction of them, substantially reducing the amount of data required. Our approach has two major advantages: It can make accurate analytics and predictions when big datasets are not available, and even if big datasets are available, our approach chooses the most informative datapoints out of the dataset, making the process much more efficient without having to process huge amounts of data.     

Enhancing the Prediction of User Satisfaction with Metaverse Service Through Machine Learning

Metaverse is one of the main technologies in the daily lives of several people, such as education, tour systems, and mobile application services. Particularly, the number of users of mobile metaverse applications is increasing owing to the merit of accessibility everywhere. To provide an improved service, it is important to analyze online reviews that contain user satisfaction. Several previous studies have utilized traditional methods, such as the structural equation model (SEM) and technology acceptance method (TAM) for exploring user satisfaction, using limited survey data. These methods may not be appropriate for analyzing the users of mobile applications. To overcome this limitation, several researchers perform user experience analysis through online reviews and star ratings. However, some online reviews occasionally have inconsistencies between the star rating and the sentiment of the text. This variation disturbs the performance of machine learning. To alleviate the inconsistencies, Valence Aware Dictionary and sEntiment Reasoner (VADER), which is a sentiment classifier based on lexicon, is introduced. The current study aims to build a more accurate sentiment classifier based on machine learning with VADER. In this study, five sentiment classifiers are used, such as Naïve Bayes, K-Nearest Neighbors (KNN), Logistic Regression, Light Gradient Boosting Machine (LightGBM), and Categorical boosting algorithm (Catboost) with three embedding methods (Bag-of-Words (BoW), Term Frequency-Inverse Document Frequency (TF-IDF), Word2Vec). The results show that classifiers that apply VADER outperform those that do not apply VADER, excluding one classifier (Logistic Regression with Word2Vec). Moreover, LightGBM with TF-IDF has the highest accuracy 88.68% among other models.     

Threshold Filtering Semi-Supervised Learning Method for SAR Target Recognition

The semi-supervised deep learning technology driven by a small part of labeled data and a large amount of unlabeled data has achieved excellent performance in the field of image processing. However, the existing semi-supervised learning techniques are all carried out under the assumption that the labeled data and the unlabeled data are in the same distribution, and its performance is mainly due to the two being in the same distribution state. When there is out-of-class data in unlabeled data, its performance will be affected. In practical applications, it is difficult to ensure that unlabeled data does not contain out-of-category data, especially in the field of Synthetic Aperture Radar (SAR) image recognition. In order to solve the problem that the unlabeled data contains out-of-class data which affects the performance of the model, this paper proposes a semi-supervised learning method of threshold filtering. In the training process, through the two selections of data by the model, unlabeled data outside the category is filtered out to optimize the performance of the model. Experiments were conducted on the Moving and Stationary Target Acquisition and Recognition (MSTAR) dataset, and compared with existing several state-of-the-art semi-supervised classification approaches, the superiority of our method was confirmed, especially when the unlabeled data contained a large amount of out-of-category data.