面向小样本数据的机器学习方法研究综述

小样本学习是面向小样本数据的机器学习,旨在利用较少的有监督样本数据去构建能够解决实际问题的机器学习模型。小样本学习能够解决传统机器学习方法在样本数据不充分时性能严重下降的问题,可以为新型小样本任务实现低成本和快速的模型部署,缩小人类智能与人工智能之间的距离,对推动发展通用型人工智能具有重要意义。从小样本学习的概念、基础模型和实际应用入手,系统梳理当前小样本学习的相关工作,将小样本学习方法分类为基于模型微调、基于数据增强、基于度量学习和基于元学习,并具体阐述这4大类方法的核心思想、基本模型、细分领域和最新研究进展,以及每一类方法在科学研究或实际应用中存在的问题,总结目前小样本学习研究的常用数据集和评价指标,整理基于部分典型小样本学习方法在Omniglot和Mini-ImageNet数据集上的实验结果。最后对各种小样本学习方法及其优缺点进行总结,分别从数据层面、理论研究和应用研究3个方面对小样本学习的未来研究方向进行展望。     

Ant colony and particle swarm optimization for financial classification problems

Financial decisions are often based on classification models which are used to assign a set of observations into predefined groups. Such models ought to be as accurate as possible. One important step towards the development of accurate financial classification models involves the selection of the appropriate independent variables (features) which are relevant for the problem at hand. This is known as the feature selection problem in the machine learning/data mining field. In financial decisions, feature selection is often based on the subjective judgment of the experts. Nevertheless, automated feature selection algorithms could be of great help to the decision-makers providing the means to explore efficiently the solution space. This study uses two nature-inspired methods, namely ant colony optimization and particle swarm optimization, for this problem. The modelling context is developed and the performance of the methods is tested in two financial classification tasks, involving credit risk assessment and audit qualifications.     

基于强化学习的大时延过程控制策略研究

具有长时延的过程控制被公认为是较难的系统过程控制.模型预测控制(MPC)是一种适用于大时延过程的新的过程控制方法.相比于PID等传统的控制方法,MPC基于模型对未来状态的预测进行决策,能够兼顾及时反馈与长期规划.但MPC对于过程的预测步数依然是有限的.强化学习作为机器学习的重要部分,原则上能够预测策略在无限长时间内的收...     

融合极端学习机的判别性分析字典学习算法

研究表明,端学习机和判别性字典学习算法在图像分类领域极具有高效和准确的优势。然而,这两种方法也具有各自的缺点,极端学习机对噪声的鲁棒性较差,判别性字典学习算法在分类过程中耗时较长。为统一这种互补性以提高分类性能,文中提出了一种融合极端学习机的判别性分析字典学习模型。该模型利用迭代优化算法学习最优的判别性分析字典和极端学习机分类器。为验证所提算法的有效性,利用人脸数据集进行分类。实验结果表明,与目前较为流行的字典学习算法和极端学习机相比,所提算法在分类过程中具有更好的效果。     

基于模拟退火算法的改进极限学习机

传统的极限学习机作为一种有监督的学习模型,任意对隐藏层神经元的输入权值和偏置进行赋值,通过计算隐藏层神经元的输出权值完成学习过程.针对传统的极限学习机在数据分析预测研究中存在预测精度不足的问题,提出一种基于模拟退火算法改进的极限学习机.首先,利用传统的极限学习机对训练集进行学习,得到隐藏层神经元的输出权值,选取预测结果评价标准.然后利用模拟退火算法,将传统的极限学习机隐藏层输入权值和偏置视为初始解,预测结果评价标准视为目标函数,通过模拟退火的降温过程,找到最优解即学习过程中预测误差最小的极限学习机的隐藏层神经元输入权值和偏置,最后通过传统的极限学习机计算得到隐藏层输出权值.实验选取鸢尾花分类数据和波士顿房价预测数据进行分析.实验发现与传统的极限学习机相比,基于模拟退火改进的极限学习机在分类和回归性能上都更优.     

Machine learning methods for sub-pixel land-cover classification in the spatially heterogeneous region of Flanders (Belgium): a multi-criteria comparison

Until now, few research has addressed the use of machine learning methods for classification at the sub-pixel level. To close this knowledge gap, in this article, six machine learning methods were compared for the specific task of sub-pixel land-cover extraction in the spatially heterogeneous region of Flanders (Belgium). In addition to the classification accuracy at the pixel and the municipality level, three evaluation criteria reflecting the methods’ ease-of-use were added to the comparison: the time needed for training, the number of meta-parameters, and the minimum training set size. Robustness to changing training data was also included as the sixth evaluation criterion. Based on their scores for these six criteria, the machine learning methods were ranked according to three multi-criteria ranking scenarios. These ranking scenarios correspond to different decision-making scenarios that differ in their weighting of the criteria. In general, no overall winner could be designated: no method performs best for all evaluation scenarios. However, when both time available for preprocessing and the magnitude of the training data set are unconstrained, Support Vector Machines (SVMs) clearly outperform the other methods.     

Argument Based Machine Learning Applied to Law

In this paper we discuss the application of a new machine learning approach – Argument Based Machine Learning – to the legal domain. An experiment using a dataset which has also been used in previous experiments with other learning techniques is described, and comparison with previous experiments made. We also tested this method for its robustness to noise in learning data. Argumentation based machine learning is particularly suited to the legal domain as it makes use of the justifications of decisions which are available. Importantly, where a large number of decided cases are available, it provides a way of identifying which need to be considered. Using this technique, only decisions which will have an influence on the rules being learned are examined.     

Learning strategies for task delegation in norm-governed environments

How do I choose whom to delegate a task to? This is an important question for an autonomous agent collaborating with others to solve a problem. Were similar proposals accepted from similar agents in similar circumstances? What arguments were most convincing? What are the costs incurred in putting certain arguments forward? Can I exploit domain knowledge to improve the outcome of delegation decisions? In this paper, we present an agent decision-making mechanism where models of other agents are refined through evidence from past dialogues and domain knowledge, and where these models are used to guide future delegation decisions. Our approach combines ontological reasoning, argumentation and machine learning in a novel way, which exploits decision theory for guiding argumentation strategies. Using our approach, intelligent agents can autonomously reason about the restrictions (e.g., policies/norms) that others are operating with, and make informed decisions about whom to delegate a task to. In a set of experiments, we demonstrate the utility of this novel combination of techniques. Our empirical evaluation shows that decision-theory, machine learning and ontology reasoning techniques can significantly improve dialogical outcomes.     

面向神经机器翻译的集成学习方法分析

集成学习是一种联合多个学习器进行协同决策的机器学习方法,应用在机器翻译任务的推断过程中可以有效整合多个模型预测的概率分布,达到提升翻译系统准确性的目的。虽然该方法的有效性已在机器翻译评测中得到了广泛验证,但关于子模型的选择与融合的策略仍鲜有研究。该文主要针对机器翻译任务中的参数平均与模型融合两种集成学习方法进行大量的实验,分别从模型与数据层面、多样性与模型数量层面对集成学习的策略进行了深入探索。实验结果表明在WMT中英新闻任务上,所提模型相比Transformer单模型有3.19个BLEU值的提升。     

不同相似度测量方式的模糊C均值聚类分析

聚类技术是机器学习、模式识别及数据挖掘等领域中的一个重要研究内容。采用不同相似度测量方式,应用标准模糊C均值聚类算法在UCI的三个知名数据集上完成聚类实验,从正确率和运行效率两个方面对比分析其性能,为聚类分析研究提供了有益的参考。     

Machine learning for the New York City power grid

Power companies can benefit from the use of knowledge discovery methods and statistical machine learning for preventive maintenance. We introduce a general process for transforming historical electrical grid data into models that aim to predict the risk of failures for components and systems. These models can be used directly by power companies to assist with prioritization of maintenance and repair work. Specialized versions of this process are used to produce 1) feeder failure rankings, 2) cable, joint, terminator, and transformer rankings, 3) feeder Mean Time Between Failure (MTBF) estimates, and 4) manhole events vulnerability rankings. The process in its most general form can handle diverse, noisy, sources that are historical (static), semi-real-time, or realtime, incorporates state-of-the-art machine learning algorithms for prioritization (supervised ranking or MTBF), and includes an evaluation of results via cross-validation and blind test. Above and beyond the ranked lists and MTBF estimates are business management interfaces that allow the prediction capability to be integrated directly into corporate planning and decision support; such interfaces rely on several important properties of our general modeling approach: that machine learning features are meaningful to domain experts, that the processing of data is transparent, and that prediction results are accurate enough to support sound decision making. We discuss the challenges in working with historical electrical grid data that were not designed for predictive purposes. The “rawness” of these data contrasts with the accuracy of the statistical models that can be obtained from the process; these models are sufficiently accurate to assist in maintaining New York City’s electrical grid.     

Software effort estimation accuracy prediction of machine learning techniques: A systematic performance evaluation

Software effort estimation accuracy is a key factor in effective planning, controlling, and delivering a successful software project within budget and schedule. The overestimation and underestimation both are the key challenges for future software development, henceforth there is a continuous need for accuracy in software effort estimation. The researchers and practitioners are striving to identify which machine learning estimation technique gives more accurate results based on evaluation measures, datasets and other relevant attributes. The authors of related research are generally not aware of previously published results of machine learning effort estimation techniques. The main aim of this study is to assist the researchers to know which machine learning technique yields the promising effort estimation accuracy prediction in software development. In this article, the performance of the machine learning ensemble and solo techniques are investigated on publicly and non-publicly domain datasets based on the two most commonly used accuracy evaluation metrics. We used the systematic literature review methodology proposed by Kitchenham and Charters. This includes searching for the most relevant papers, applying quality assessment (QA) criteria, extracting data, and drawing results. We have evaluated a state-of-the-art accuracy performance of 35 selected studies (17 ensemble, 18 solo) using mean magnitude of relative error and PRED (25) as a set of reliable accuracy metrics for performance evaluation of accuracy among two techniques to report the research questions stated in this study. We found that machine learning techniques are the most frequently implemented in the construction of ensemble effort estimation (EEE) techniques. The results of this study revealed that the EEE techniques usually yield a promising estimation accuracy than the solo techniques.     

基于卷积神经网络的焊缝表面缺陷检测方法

针对工业激光焊接中,采用传统方法进行焊缝质量检测效率低下的问题,提出了一种基于卷积神经网络的工业钢板表面焊缝缺陷检测方法;首先基于卷积神经网络,搭建了一个多分类模型框架,并分析了各层中所用到的函数及相关参数;然后基于工业数控机床和工业相机进行了焊缝数据采集,并对这些数据进行了分类、增强、扩增等前期预处理;最后基于数控机器轴,采用滑动窗口检测的形式采集实际待测图像,并通过实验对比了传统的机器学习算法在该类图像数据中的性能评估;经实验证实,通过卷积神经网络训练得到的多分类模型,焊缝缺陷检测精度能达到97%以上,且每张待测图像的测试时间均在300 ms左右,远超机器学习算法,在准确性和实时性上均能达到实际工业要求。     

The effect of principal component analysis on machine learning accuracy with high-dimensional spectral data

This paper presents the results of an investigation into the use of machine learning methods for the identification of narcotics from Raman spectra. The classification of spectral data and other high-dimensional data, such as images, gene-expression data and spectral data, poses an interesting challenge to machine learning, as the presence of high numbers of redundant or highly correlated attributes can seriously degrade classification accuracy. This paper investigates the use of principal component analysis (PCA) to reduce high-dimensional spectral data and to improve the predictive performance of some well-known machine learning methods. Experiments are carried out on a high-dimensional spectral dataset. These experiments employ the NIPALS (Non-Linear Iterative Partial Least Squares) PCA method, a method that has been used in the field of chemometrics for spectral classification, and is a more efficient alternative than the widely used eigenvector decomposition approach. The experiments show that the use of this PCA method can improve the performance of machine learning in the classification of high-dimensional data.     

Sales forecasting system based on Gray extreme learning machine with Taguchi method in retail industry

Due to the strong competition that exists today, most retailers are in a continuous effort for increasing profits and reducing their cost. An accurate sales forecasting system is an efficient way to achieve the aforementioned goals and lead to improve the customers’ satisfaction, reduce destruction of products, increase sales revenue and make production plan efficiently. In this study, the Gray extreme learning machine (GELM) integrates Gray relation analysis and extreme learning machine with Taguchi method to support purchasing decisions. GRA can sieve out the more influential factors from raw data and transforms them as the input data in a novel neural network such as ELM. The proposed system evaluated the real sales data in the retail industry. The experimental results demonstrate that our proposed system outperform several sales forecasting methods which are based on back-propagation neural networks such as BPN and MFLN models.     

基于概率模型检查的树模型公平性验证方法

如今,越来越多的社会决策借助机器学习模型给出,包括法律决策、财政决策等等.对于这些决策,算法的公平性是极为重要的.事实上,在这些环境中引入机器学习的目的之一,就是为了规避或减少人类在决策过程中存在的偏见.然而,数据集常常包含敏感特征,或可能存在历史性偏差,会使得机器学习算法产生带有偏见的模型.由于特征选择对基于树的模型具有重要性,它们容易受到敏感属性的影响.提出一种基于概率模型检查的方法,以形式化验证决策树和树集成模型的公平性.将公平性问题转换为概率验证问题,为算法模型构建PCSP#模型,并使用PAT模型检查工具求解,以不同定义的公平性度量衡量模型公平性.基于该方法开发了FairVerify工具,并在多个基于不同数据集和复合敏感属性的分类器上验证了不同的公平性度量,展现了较好的性能.与现有的基于分布的验证器相比,该方法具有更高的可扩展性和鲁棒性.     

基于机器学习的军用软件过时淘汰评估方法研究

为了对军用软件进行科学系统的过时淘汰评估,提出基于机器学习的软件过时淘汰评估模型。首先使用机器学习预处理与缩放技术处理相关的特征数据,然后基于主成分分析模型进行特征提取和降维,消除特征数据中的噪音值并选择重要的军用软件过时淘汰特征数据,使用由粒子群优化算法改进的支持向量机模型进行分类和评估建模,并使用混淆矩阵的精度评估模型,最后通过案例验证模型有效性、适用性和科学性。     

Detection of Mesangial hypercellularity of MEST-C score in immunoglobulin A-nephropathy using deep convolutional neural network

Immunoglobulin A (IgA)-nephropathy (IgAN) is one of the major reasons for renal failure. It provides vital clues to estimate the stage and the proliferation rate of end-stage kidney disease. IgA stage can be estimated with the help of MEST-C score. The manual estimation of MEST-C score from whole slide kidney images is a very tedious and difficult task. This study uses some Convolutional neural networks (CNNs) related models to detect mesangial hypercellularity (M score) in MEST-C. CNN learns the features directly from image data without the requirement of analytical data. CNN is trained efficiently when image data size is large enough for a particular class. In the case of smaller data size, transfer learning can be used efficiently in which CNN is pre-trained on some general images and then on subject images. Since the data set size is small, time spent in collecting large data set is saved. The training time of transfer learning is also reduced because the model is already pre-trained. This research work aims at the detection of mesangial hypercellularity from biopsy images with small data size by utilizing the transfer learning. The dataset used in this research work consists of 138 individual glomerulus (× 20 magnification digital biopsy) images of IgA patients received from All India Institute of Medical Science, Delhi. Here, machine learning (k-nearest neighbour (KNN) and support vector machine (SVM)) classifiers are compared to transfer learning CNN methods. The deep extracted image features are used by machine learning classifiers. The different evaluation parameters have been used for comparing the predictions of basic classifiers to the deep learning model. The research work concludes that the transfer learning deep CNN method can improve the detection of mesangial hypercellularity as compare to KNN, SVM methods when using the small data set. This model could help the pathologists to understand the stages of kidney failure.

     

An extensive evaluation of seven machine learning methods for rainfall prediction in weather derivatives

Regression problems provide some of the most challenging research opportunities in the area of machine learning, and more broadly intelligent systems, where the predictions of some target variables are critical to a specific application. Rainfall is a prime example, as it exhibits unique characteristics of high volatility and chaotic patterns that do not exist in other time series data. This work’s main impact is to show the benefit machine learning algorithms, and more broadly intelligent systems have over the current state-of-the-art techniques for rainfall prediction within rainfall derivatives. We apply and compare the predictive performance of the current state-of-the-art (Markov chain extended with rainfall prediction) and six other popular machine learning algorithms, namely: Genetic Programming, Support Vector Regression, Radial Basis Neural Networks, M5 Rules, M5 Model trees, and k-Nearest Neighbours. To assist in the extensive evaluation, we run tests using the rainfall time series across data sets for 42 cities, with very diverse climatic features. This thorough examination shows that the machine learning methods are able to outperform the current state-of-the-art. Another contribution of this work is to detect correlations between different climates and predictive accuracy. Thus, these results show the positive effect that machine learning-based intelligent systems have for predicting rainfall based on predictive accuracy and with minimal correlations existing across climates.     

A performance evaluation framework for association mining in spatial data

The evaluation of the process of mining associations is an important and challenging problem in database systems and especially those that store critical data and are used for making critical decisions. Within the context of spatial databases we present an evaluation framework in which we use probability distributions to model spatial regions, and Bayesian networks to model the joint probability distribution and the structural relationships among spatial and non-spatial predicates. We demonstrate the applicability of the proposed framework by evaluating representatives from two well-known approaches that are used for learning associations, i.e., dependency analysis (using statistical tests of independence) and Bayesian methods. By controlling the parameters of the framework we provide extensive comparative results of the performance of the two approaches. We obtain measures of recovery of known associations as a function of the number of samples used, the strength, number and type of associations in the model, the number of spatial predicates associated with a particular non-spatial predicate, the prior probabilities of spatial predicates, the conditional probabilities of the non-spatial predicates, the image registration error, and the parameters that control the sensitivity of the methods. In addition to performance we investigate the processing efficiency of the two approaches.