Pool-based active learning in approximate linear regression

The goal of pool-based active learning is to choose the best input points to gather output values from a ‘pool’ of input samples. We develop two pool-based active learning criteria for linear regression. The first criterion allows us to obtain a closed-form solution so it is computationally very efficient. However, this solution is not necessarily optimal in the single-trial generalization error analysis. The second criterion can give a better solution, but it does not have a closed-form solution and therefore some additional search strategy is needed. To cope with this problem, we propose a practical procedure which enables us to efficiently search for a better solution around the optimal solution of the first method. Simulations with toy and benchmark datasets show that the proposed active learning method compares favorably with other active learning methods as well as the baseline passive learning scheme. Furthermore, the usefulness of the proposed active learning method is also demonstrated in wafer alignment in semiconductor exposure apparatus.     

Adaptive sampling design for multi-task learning of Gaussian processes in manufacturing

Approximation models (or surrogate models) have been widely used in engineering problems to mitigate the cost of running expensive experiments or simulations. Gaussian processes (GPs) are a popular tool used to construct these models due to their flexibility and computational tractability. The accuracy of these models is a strong function of the density and locations of the sampled points in the parametric space used for training. Previously, multi-task learning (MTL) has been used to learn similar-but-not-identical tasks together, thus increasing the effective density of training points. Also, several adaptive sampling strategies have been developed to identify regions of interest for intelligent sampling in single-task learning of GPs. While both these methods have addressed the density and location constraint separately, sampling design approaches for MTL are lacking. In this paper, we formulate an adaptive sampling strategy for MTL of GPs, thereby further improving data efficiency and modeling performance in GP. To this end, we develop variance measures for an MTL framework to effectively identify optimal sampling locations while learning multiple tasks simultaneously. We demonstrate the effectiveness of the proposed method using a case study on a real-world engine surface dataset. We observe that the proposed method leverages both MTL and intelligent sampling to significantly outperform state-of-the-art methods which use either approach separately. The developed sampling design strategy is readily applicable to many problems in various fields.     

A stopping criterion for active learning

Active learning (AL) is a framework that attempts to reduce the cost of annotating training material for statistical learning methods. While a lot of papers have been presented on applying AL to natural language processing tasks reporting impressive savings, little work has been done on defining a stopping criterion. In this work, we present a stopping criterion for active learning based on the way instances are selected during uncertainty-based sampling and verify its applicability in a variety of settings. The statistical learning models used in our study are support vector machines (SVMs), maximum entropy models and Bayesian logistic regression and the tasks performed are text classification, named entity recognition and shallow parsing. In addition, we present a method for multiclass mutually exclusive SVM active learning.     

Active learning for logistic regression: an evaluation

Which active learning methods can we expect to yield good performance in learning binary and multi-category logistic regression classifiers? Addressing this question is a natural first step in providing robust solutions for active learning across a wide variety of exponential models including maximum entropy, generalized linear, log-linear, and conditional random field models. For the logistic regression model we re-derive the variance reduction method known in experimental design circles as ‘A-optimality.’ We then run comparisons against different variations of the most widely used heuristic schemes: query by committee and uncertainty sampling, to discover which methods work best for different classes of problems and why. We find that among the strategies tested, the experimental design methods are most likely to match or beat a random sample baseline. The heuristic alternatives produced mixed results, with an uncertainty sampling variant called margin sampling and a derivative method called QBB-MM providing the most promising performance at very low computational cost. Computational running times of the experimental design methods were a bottleneck to the evaluations. Meanwhile, evaluation of the heuristic methods lead to an accumulation of negative results. We explore alternative evaluation design parameters to test whether these negative results are merely an artifact of settings where experimental design methods can be applied. The results demonstrate a need for improved active learning methods that will provide reliable performance at a reasonable computational cost.     

Skyblocking for entity resolution

In this paper, we introduce a novel framework for entity resolution blocking, called skyblocking, which aims to learn scheme skylines. In this skyblocking framework, each blocking scheme is mapped as a point to a multi-dimensional scheme space where each blocking measure represents one dimension. A scheme skyline contains blocking schemes that are not dominated by any other blocking schemes in the scheme space. To efficiently learn scheme skylines, two challenges exist: one is the class imbalance problem and the other is the search space problem. We tackle these two challenges by developing an active sampling strategy and a scheme extension strategy. Based on these two strategies, we develop three scheme skyline learning algorithms for efficiently learning scheme skylines under a given number of blocking measures and within a label budget limit. We experimentally verify that our algorithms outperform the baseline approaches in all of the following aspects: label efficiency, blocking quality and learning efficiency, over five real-world datasets.     

Onvergence and application of online active sampling using orthogonal pillar vectors

The analysis of convergence and its application is shown for the Active Sampling-at-the-Boundary method applied to multidimensional space using orthogonal pillar vectors. Active learning method facilitates identifying an optimal decision boundary for pattern classification in machine learning. The result of this method is compared with the standard active learning method that uses random sampling on the decision boundary hyperplane. The comparison is done through simulation and application to the real-world data from the UCI benchmark data set. The boundary is modeled as a nonseparable linear decision hyperplane in multidimensional space with a stochastic oracle.     

Two-stage online inference model for traffic pattern analysis and anomaly detection

In this paper, we propose a method for modeling trajectory patterns with both regional and velocity observations through the probabilistic topic model. By embedding Gaussian models into the discrete topic model framework, our method uses continuous velocity as well as regional observations unlike existing approaches. In addition, the proposed framework combined with Hidden Markov Model can cover the temporal transition of the scene state, which is useful in checking a violation of the rule that some conflict topics (e.g. two cross-traffic patterns) should not occur at the same time. To achieve online learning even with the complexity of the proposed model, we suggest a novel learning scheme instead of collapsed Gibbs sampling. The proposed two-stage greedy learning scheme is not only efficient at reducing the search space but also accurate in a way that the accuracy of online learning becomes not worse than that of the batch learning. To validate the performance of our method, experiments were conducted on various datasets. Experimental results show that our model explains satisfactorily the trajectory patterns with respect to scene understanding, anomaly detection, and prediction.     

高速动车组多模型切换主动容错预测控制

高速动车组持续高速运行,对控制系统的可靠性和抗干扰能力提出了更高要求.结合高速动车组非线性动力学特点和系统运行数据,应用减法聚类和模式分类算法建立高速动车组多模型集;为适应对象和扰动特性的变化建立高速动车组自适应模型;采用基于累计误差最小的切换策略在线选择最优控制模型,据此设计主动容错预测控制算法来实现高速动车组安全高效运行.最后,仿真实例验证了该方法的有效性.     

Active learning for object-based image classification using predefined training objects

Object-based image analysis (OBIA) is a new remote-sensing-based image processing technology that has become popular in recent years. In spite of its remarkable advantages, the segmentation results that it generates feature a large number of mixed objects owing to the limitations of OBIA segmentation technology. The mixed objects directly influence the acquisition of training samples and the labelling of objects and thus affect the stability of classification performance. In light of this issue, this article evaluates the influence of classification uncertainty on classification performance and proposes a sampling strategy based on active learning. This sampling strategy is novel in two ways: (1) information entropy is used to evaluate the classification uncertainty of segmented objects; all segmented objects are classified as having zero or non-zero entropies, and the latter are arranged in terms of decreasing entropy. (2) Based on an evaluation of the influence of classification uncertainty on classification performance, an active learning technology is developed. A certain proportion of zero-entropy objects is acquired via random sampling used as seed training samples for active learning, non-zero-entropy objects are used as a candidate set for active learning, and the entropy query-by-bagging (EQB) algorithm is used to conduct active learning to acquire optimal training samples. In this study, three groups of high-resolution images were tested. The test results show that zero-entropy and non-zero-entropy objects are indispensable to the classifier, where the optimal range of the ratio of combination of the two is between 0.2 and 0.6. Moreover, the proposed sampling strategy can effectively improve the stability and accuracy of classification.     

Sample-based software defect prediction with active and semi-supervised learning

Software defect prediction can help us better understand and control software quality. Current defect prediction techniques are mainly based on a sufficient amount of historical project data. However, historical data is often not available for new projects and for many organizations. In this case, effective defect prediction is difficult to achieve. To address this problem, we propose sample-based methods for software defect prediction. For a large software system, we can select and test a small percentage of modules, and then build a defect prediction model to predict defect-proneness of the rest of the modules. In this paper, we describe three methods for selecting a sample: random sampling with conventional machine learners, random sampling with a semi-supervised learner and active sampling with active semi-supervised learner. To facilitate the active sampling, we propose a novel active semi-supervised learning method ACoForest which is able to sample the modules that are most helpful for learning a good prediction model. Our experiments on PROMISE datasets show that the proposed methods are effective and have potential to be applied to industrial practice.     

主动贝叶斯网络分类器

在机器学习中，主动学习具有很长的研究历史。给出了主动贝叶斯分类模型，并讨论了主动学习中几种常用的抽样策略。提出了基于最大最小熵的主动学习方法和基于不确定抽样与最小分类损失相结合的主动学习策略，给出了增量地分类测试实例和修正分类参数的方法。人工和实际的数据实验结果表明，提出的主动学习方法在少量带有类别标注训练样本的情况下获得了较好的分类精度和召回率。     

考虑学习/遗忘特性的软件项目调度多目标优化方法

人员作为软件项目调度过程中的核心资源,其学习遗忘特性是无法忽视的.借鉴已有学习和遗忘模型,构建学习/遗忘效应与人员技能水平之间的动态关联模型,进而给出考虑人员学习/遗忘效应的软件项目调度多目标优化模型.针对该模型,采用新型调度方案编码方式和不可行解修复方法,给出基于改进NSGA-II的软件项目调度多目标优化方法.面向具有不同项目规模的算例仿真实验表明,考虑人员的学习能力有利于改善调度方案性能,而遗忘效应则会使调度方案的项目总工期和成本增加.因此,在软件项目调度问题中,考虑人员的学习和遗忘效应是十分必要的.     

基于主动学习的本体概念关系判断

该文依据关系判断任务特点将主动学习应用到本体概念关系的辅助判断中,对边缘采样、熵采样、最不确信采样等主动学习查询生成策略进行了比较研究。在此基础上,从实际应用角度出发,讨论了在三种不同样本初始情况下主动学习技术的应用。对于初始样本正反例充足的情况,采用基于熵采样和边缘采样产生查询;对于初始样本仅有正例的情况,依据样本相似度主动的学习策略生成候选反例;对于缺乏初始样本的情况,使用概念在样本间距离等统计信息,同时生成候选正例和候选反例。从而,实现了在概念关系判定过程中对用户反馈信息的有效利用。     

Using explanations for determining carcinogenecity in chemical compounds

The goal of predictive toxicology is the automatic construction of carcinogenecity models. Most common artificial intelligence techniques used to construct these models are inductive learning methods. In a previous work we presented an approach that uses lazy learning methods for solving the problem of predicting carcinogenecity. Lazy learning methods solve new problems based on their similarity to already solved problems. Nevertheless, a weakness of these kind of methods is that sometimes the result is not completely understandable by the user. In this paper we propose an explanation scheme for a concrete lazy learning method. This scheme is particularly interesting to justify the predictions about the carcinogenesis of chemical compounds. In addition we propose that these explanations could be used to build a partial domain knowledge. In our particular case, we use the explanations for building general knowledge about carcinogenesis.     

主动学习中一种基于委员会的误分类采样算法

主动学习通过主动选择要学习的样例进行标注,从而有效地降低学习算法的样本复杂度。针对当前主动学习算法普遍采用的平分版本空间策略,本文提出过半缩减版本空间的策略,这种策略避免了平分版本空间策略所要求的较强假设。基于过半缩减版本空间的策略,本文实现了一种选取具有最大可能性被误分类的样例作为训练样例的启发式主动动学习算法（CBMPMS）。该算法计算版本空间中随机抽取的假设组成的委员会和当前学习器对样例预测的类概率差异的熵,以此作为选择样例的标准。针对UCI数据集的实验表明,该算法能够在大多数数据集上取得比相关研究更好的性能。     

Bayesian decision theory for support vector machines: Imbalance measurement and feature optimization

Classification approaches usually present the poor generalization performance with an apparent class imbalance problem. Surely, a measures of the quality of the possible models reflected the remaining uncertainty in the class imbalance on learning. The purpose of our learning method is to lead an attractive pragmatic expansion scheme of the Bayesian approach to assess how well it is aligned with the class imbalance problem. Thus, we propose a method with a model assessment of the interplay between various classification decisions using probability, corresponding decision costs, and quadratic program of optimal margin classifier called: Bayesian Support Vector Machines (BSVMs) learning strategy. In the framework, we did modify in the objects and conditions of primal problem to reproduce an appropriate learning rule for an observation sample. The experiments on several existing data sets showed that BSVMs may appropriately capture the true relationship between the inputs and outputs by experimental evidence.     

A multi-constraint failure-pursuing sampling method for reliability-based design optimization using adaptive Kriging

Using surrogate models to substitute the computationally expensive limit state functions is a promising way to decrease the cost of implementing reliability-based design optimization (RBDO). To train the models efficiently, the active learning strategies have been intensively studied. However, the existing learning strategies either do not individually build the models according to importance measurement or do not completely relate to the reliability analysis results. Consequently, some points that are useless to refine the limit state functions or far away from the RBDO solutions are generated. This paper proposes a multi-constraint failure-pursuing sampling method to maximize the reward of adding new training points. A simultaneous learning strategy is employed to sequentially update the Kriging models with the points selected in the current approximate safe region. Moreover, the sensitive Kriging model as well as the sensitive sample point are identified based on the failure-pursuing scheme. A new point that is highly potential to improve the accuracy of reliability analysis and optimization can then be generated near the sensitive sample point and used to update the sensitive model. Besides, numerical examples and engineering application are used to validate the performance of the proposed method.

     

基于近邻熵的主动学习算法

在主动学习中,采用近邻熵(NeighborhoodEntropy)作为样例的挑选标准,熵值最大的样例体现基于近邻分类规则,最无法确定该样例的类标。而标注不确定性高的样例可用尽量少的样例获得较高的分类性能。文中提出一种基于近邻熵的主动学习算法。该算法首先计算未标注样例的近邻样例类别熵,然后挑选熵值最大样例的进行标注。实验表明,基于近邻熵挑选样例进行标注,较基于最大距离(MaximalDistance)挑选和随机样例挑选可获得更高的分类性能。     

A self-training approach to cost sensitive uncertainty sampling

Uncertainty sampling is an effective method for performing active learning that is computationally efficient compared to other active learning methods such as loss-reduction methods. However, unlike loss-reduction methods, uncertainty sampling cannot minimize total misclassification costs when errors incur different costs. This paper introduces a method for performing cost-sensitive uncertainty sampling that makes use of self-training. We show that, even when misclassification costs are equal, this self-training approach results in faster reduction of loss as a function of number of points labeled and more reliable posterior probability estimates as compared to standard uncertainty sampling. We also show why other more naive methods of modifying uncertainty sampling to minimize total misclassification costs will not always work well.     

Active Sampling for Class Probability Estimation and Ranking

In many cost-sensitive environments class probability estimates are used by decision makers to evaluate the expected utility from a set of alternatives. Supervised learning can be used to build class probability estimates; however, it often is very costly to obtain training data with class labels. Active learning acquires data incrementally, at each phase identifying especially useful additional data for labeling, and can be used to economize on examples needed for learning. We outline the critical features of an active learner and present a sampling-based active learning method for estimating class probabilities and class-based rankings. BOOTSTRAP-LV identifies particularly informative new data for learning based on the variance in probability estimates, and uses weighted sampling to account for a potential example's informative value for the rest of the input space. We show empirically that the method reduces the number of data items that must be obtained and labeled, across a wide variety of domains. We investigate the contribution of the components of the algorithm and show that each provides valuable information to help identify informative examples. We also compare BOOTSTRAP-LV with UNCERTAINTY SAMPLING, an existing active learning method designed to maximize classification accuracy. The results show that BOOTSTRAP-LV uses fewer examples to exhibit a certain estimation accuracy and provide insights to the behavior of the algorithms. Finally, we experiment with another new active sampling algorithm drawing from both UNCERTAINTY SAMPLING and BOOTSTRAP-LV and show that it is significantly more competitive with BOOTSTRAP-LV compared to UNCERTAINTY SAMPLING. The analysis suggests more general implications for improving existing active sampling algorithms for classification.