DRAL: a tool for discovering relevant e-activities for learners

Web-based educational systems routinely collect vast quantities of data on students’ e-activity generating log files that offer researchers unique opportunities to apply data mining techniques and discover interesting information to improve the learning process. This paper proposes a friendly and intuitive tool called DRAL to detect the most relevant e-activities that a student needs to pass a course based on features extracted from logged data in an education web-based system. The method uses a more flexible representation of the available information based on multiple instance learning to prevent the appearance of a great number of missing values and is based on a multi-objective grammar guided genetic programming algorithm which obtains simple and clear classification rules which are markedly useful to identify the number, type and time of e-activities more relevant so that a student has a high probability to pass a course. To validate this approach, our proposal is compared with the most traditional proposals in multiple instance learning over the years. Experimental results demonstrate that the approach proposed successfully improves the accuracy of previous models by finding a balance between specificity and sensitivity values.     

视觉弱监督学习研究进展

视觉理解,如物体检测、语义和实例分割以及动作识别等,在人机交互和自动驾驶等领域中有着广泛的应用并发挥着至关重要的作用。近年来,基于全监督学习的深度视觉理解网络取得了显著的性能提升。然而,物体检测、语义和实例分割以及视频动作识别等任务的数据标注往往需要耗费大量的人力和时间成本,已成为限制其广泛应用的一个关键因素。弱监督学习作为一种降低数据标注成本的有效方式,有望对缓解这一问题提供可行的解决方案,因而获得了较多的关注。围绕视觉弱监督学习,本文将以物体检测、语义和实例分割以及动作识别为例综述国内外研究进展,并对其发展方向和应用前景加以讨论分析。在简单回顾通用弱监督学习模型,如多示例学习(multiple instance learning,MIL)和期望—最大化(expectation-maximization,EM)算法的基础上,针对物体检测和定位,从多示例学习、类注意力图机制等方面分别进行总结,并重点回顾了自训练和监督形式转换等方法;针对语义分割任务,根据不同粒度的弱监督形式,如边界框标注、图像级类别标注、线标注或点标注等,对语义分割研究进展进行总结分析,并主要回顾了基于图像级别类别标注和边界框标注的弱监督实例分割方法;针对视频动作识别,从电影脚本、动作序列、视频级类别标签和单帧标签等弱监督形式,对弱监督视频动作识别的模型与算法进行回顾,并讨论了各种弱监督形式在实际应用中的可行性。在此基础上,进一步讨论视觉弱监督学习面临的挑战和发展趋势,旨在为相关研究提供参考。     

Deep and fast: Deep learning hashing with semi-supervised graph construction

Learning-based hashing methods are becoming the mainstream for approximate scalable multimedia retrieval. They consist of two main components: hash codes learning for training data and hash functions learning for new data points. Tremendous efforts have been devoted to designing novel methods for these two components, i.e., supervised and unsupervised methods for learning hash codes, and different models for inferring hashing functions. However, there is little work integrating supervised and unsupervised hash codes learning into a single framework. Moreover, the hash function learning component is usually based on hand-crafted visual features extracted from the training images. The performance of a content-based image retrieval system crucially depends on the feature representation and such hand-crafted visual features may degrade the accuracy of the hash functions. In this paper, we propose a semi-supervised deep learning hashing (DLH) method for fast multimedia retrieval. More specifically, in the first component, we utilize both visual and label information to learn an optimal similarity graph that can more precisely encode the relationship among training data, and then generate the hash codes based on the graph. In the second stage, we apply a deep convolutional network to simultaneously learn a good multimedia representation and a set of hash functions. Extensive experiments on five popular datasets demonstrate the superiority of our DLH over both supervised and unsupervised hashing methods.     

Supervised machine learning in multimodal learning analytics for estimating success in project‐based learning

Multimodal learning analytics provides researchers new tools and techniques to capture different types of data from complex learning activities in dynamic learning environments. This paper investigates the use of diverse sensors, including computer vision, user‐generated content, and data from the learning objects (physical computing components), to record high‐fidelity synchronised multimodal recordings of small groups of learners interacting. We processed and extracted different aspects of the students' interactions to answer the following question: Which features of student group work are good predictors of team success in open‐ended tasks with physical computing? To answer this question, we have explored different supervised machine learning approaches (traditional and deep learning techniques) to analyse the data coming from multiple sources. The results illustrate that state‐of‐the‐art computational techniques can be used to generate insights into the "black box" of learning in students' project‐based activities. The features identified from the analysis show that distance between learners' hands and faces is a strong predictor of students' artefact quality, which can indicate the value of student collaboration. Our research shows that new and promising approaches such as neural networks, and more traditional regression approaches can both be used to classify multimodal learning analytics data, and both have advantages and disadvantages depending on the research questions and contexts being investigated. The work presented here is a significant contribution towards developing techniques to automatically identify the key aspects of students success in project‐based learning environments, and to ultimately help teachers provide appropriate and timely support to students in these fundamental aspects.     

Applying semi-supervised learning in hierarchical multi-label classification

In classification problems with hierarchical structures of labels, the target function must assign labels that are hierarchically organized and it can be used either for single-label (one label per instance) or multi-label classification problems (more than one label per instance). In parallel to these developments, the idea of semi-supervised learning has emerged as a solution to the problems found in a standard supervised learning procedure (used in most classification algorithms). It combines labelled and unlabelled data during the training phase. Some semi-supervised methods have been proposed for single-label classification methods. However, very little effort has been done in the context of multi-label hierarchical classification. Therefore, this paper proposes a new method for supervised hierarchical multi-label classification, called HMC-RAkEL. Additionally, we propose the use of semi-supervised learning, self-training, in hierarchical multi-label classification, leading to three new methods, called HMC-SSBR, HMC-SSLP and HMC-SSRAkEL. In order to validate the feasibility of these methods, an empirical analysis will be conducted, comparing the proposed methods with their corresponding supervised versions. The main aim of this analysis is to observe whether the semi-supervised methods proposed in this paper have similar performance of the corresponding supervised versions.     

基于弱标签的多示例迁移学习方法

作为监督学习的一种变体,多示例学习(MIL)试图从包中的示例中学习分类器。在多示例学习中,标签与包相关联,而不是与单个示例相关联。包的标签是已知的,示例的标签是未知的。MIL可以解决标记模糊问题,但要解决带有弱标签的问题并不容易。对于弱标签问题,包和示例的标签都是未知的,但它们是潜在的变量。现在有多个标签和示例,可以通过对不同标签进行加权来近似估计包和示例的标签。提出了一种新的基于迁移学习的多示例学习框架来解决弱标签的问题。首先构造了一个基于多示例方法的迁移学习模型,该模型可以将知识从源任务迁移到目标任务中,从而将弱标签问题转换为多示例学习问题。在此基础上,提出了一种求解多示例迁移学习模型的迭代框架。实验结果表明,该方法优于现有多示例学习方法。     

迭代拉普拉斯半监督学习本体算法

本体相似度计算和本体映射是知识表示和信息处理的核心研究内容。利用迭代拉普拉斯半监督学习方法将本体图中每个顶点映射成一个实数,通过比较顶点对应实数间的差值得到本体相似度计算算法和本体映射策略。通过两个实验表明,该方法对特定的应用领域是有效的。     

多通道Haar-like特征多示例学习目标跟踪

目的 提出一种基于多通道Haar-like特征的多示例学习目标跟踪算法,克服了多示例跟踪算法在处理彩色视频时利用信息少和弱特征不能更换的缺点。方法 首先,针对原始多示例学习跟踪算法对彩色视频帧采用单通道信息或将其简单转化为灰度图像进行跟踪会丢失部分特征信息的缺点,提出在RGB三通道上生成位置、大小和通道完全随机的Haar-like特征来更好地表示目标。其次,针对多示例学习跟踪算法中Haar-like弱特征不能更换,难以反映目标自身和外界条件变化的特点,提出在弱分类器选择过程中,用随机生成的新Haar-like特征实时替换部分判别力最弱的Haar-like特征,从而在目标模型中引入新的信息,以适应目标外观的动态变化。结果 对8个具有挑战性的彩色视频序列的实验结果表明,与原始多示例学习跟踪算法、加权多示例学习跟踪算法、基于分布场的跟踪算法相比,提出的方法不仅获得了最小的平均中心误差,而且平均跟踪准确率比上述3种算法分别高52.85%,34.75%和5.71%,在4种算法中获得最优性能。结论 通过将Haar-like特征从RGB三通道随机生成,并将判别力最弱的部分Haar-like弱特征实时更换,显著提升了原始多示例学习跟踪算法对彩色视频的跟踪效果,扩展了其应用前景。     

一种针对弱标记文档的分类方法

多标记学习不同于传统的监督学习,它是为了解决客观世界中多义性对象的建模问题而提出的一种学习框架。在该框架下,一个实例可以同时隶属于多个标记。已有的多标记学习算法大多假设每个样本的标记集合都是完整的,但有时某些实例对应的标记会出现缺失。为了应对这一问题,本文提出一种针对弱标记文档的分类方法,该方法基于标记之间不同的相关性和相似实例具有相似标记的假设,构造一个最优化问题,以尽可能地补全缺失的标记。实验结果表明,该方法可以有效地提升学习系统的泛化性能。      

Multi-instance multi-label learning

In this paper, we propose the MIML (Multi-Instance Multi-Label learning) framework where an example is described by multiple instances and associated with multiple class labels. Compared to traditional learning frameworks, the MIML framework is more convenient and natural for representing complicated objects which have multiple semantic meanings. To learn from MIML examples, we propose the MimlBoost and MimlSvm algorithms based on a simple degeneration strategy, and experiments show that solving problems involving complicated objects with multiple semantic meanings in the MIML framework can lead to good performance. Considering that the degeneration process may lose information, we propose the D-MimlSvm algorithm which tackles MIML problems directly in a regularization framework. Moreover, we show that even when we do not have access to the real objects and thus cannot capture more information from real objects by using the MIML representation, MIML is still useful. We propose the InsDif and SubCod algorithms. InsDif works by transforming single-instances into the MIML representation for learning, while SubCod works by transforming single-label examples into the MIML representation for learning. Experiments show that in some tasks they are able to achieve better performance than learning the single-instances or single-label examples directly.     

A lazy learning approach for building classification models

In this paper, we propose a lazy learning strategy for building classification learning models. Instead of learning the models with the whole training data set before observing the new instance, a selection of patterns is made depending on the new query received and a classification model is learnt with those selected patterns. The selection of patterns is not homogeneous, in the sense that the number of selected patterns depends on the position of the query instance in the input space. That selection is made using a weighting function to give more importance to the training patterns that are more similar to the query instance. Our intention is to provide a lazy learning mechanism suited to any machine learning classification algorithm. For this reason, we study two different methods to avoid fixing any parameter. Experimental results show that classification rates of traditional machine learning algorithms based on trees, rules, or functions can be improved when they are learnt with the lazy learning approach proposed. © 2011 Wiley Periodicals, Inc.     

监督学习模型指导的函数级编译优化参数选择方法研究

基于机器学习的迭代编译方法可以在对新程序进行迭代编译时,有效预测新程序的最佳优化参数组合。现有方法在模型训练过程中存在优化参数组合搜索效率较低、程序特征表示不恰当、预测精度不高的问题。因此,基于机器学习的迭代编译方法是当前迭代编译领域内的一个研究热点,其研究挑战在于学习算法选择、优化参数搜索以及程序特征表示等问题。基于监督学习技术,提出了一种程序优化参数预测方法。该方法首先通过约束多目标粒子群算法对优化参数空间进行搜索,找到样本函数的最佳优化参数;然后,通过动静结合的程序特征表示技术,对函数特征进行抽取;最后,通过由函数特征和优化参数形成的样本构建监督学习模型,对新程序的优化参数进行预测。分别采用k近邻法和softmax回归建立统计模型,实验结果表明,新方法在NPB测试集和大型科学计算程序上实现了较好的预测性能。     

Multiple instance learning with bag dissimilarities

Multiple instance learning (MIL) is concerned with learning from sets (bags) of objects (instances), where the individual instance labels are ambiguous. In this setting, supervised learning cannot be applied directly. Often, specialized MIL methods learn by making additional assumptions about the relationship of the bag labels and instance labels. Such assumptions may fit a particular dataset, but do not generalize to the whole range of MIL problems. Other MIL methods shift the focus of assumptions from the labels to the overall (dis)similarity of bags, and therefore learn from bags directly. We propose to represent each bag by a vector of its dissimilarities to other bags in the training set, and treat these dissimilarities as a feature representation. We show several alternatives to define a dissimilarity between bags and discuss which definitions are more suitable for particular MIL problems. The experimental results show that the proposed approach is computationally inexpensive, yet very competitive with state-of-the-art algorithms on a wide range of MIL datasets.     

ReliefF-MI: An extension of ReliefF to multiple instance learning

In machine learning the so-called curse of dimensionality, pertinent to many classification algorithms, denotes the drastic increase in computational complexity and classification error with data having a great number of dimensions. In this context, feature selection techniques try to reduce dimensionality finding a new more compact representation of instances selecting the most informative features and removing redundant, irrelevant, and/or noisy features. In this paper, we propose a filter-based feature selection method for working in the multiple-instance learning scenario called ReliefF-MI; it is based on the principles of the well-known ReliefF algorithm. Different extensions are designed and implemented and their performance checked in multiple instance learning. ReliefF-MI is applied as a pre-processing step that is completely independent from the multi-instance classifier learning process and therefore is more efficient and generic than wrapper approaches proposed in this area. Experimental results on five benchmark real-world data sets and 17 classification algorithms confirm the utility and efficiency of this method, both statistically and from the point of view of execution time.     

Efficient monte carlo methods for multi-dimensional learning with classifier chains

Multi-dimensional classification (MDC) is the supervised learning problem where an instance is associated with multiple classes, rather than with a single class, as in traditional classification problems. Since these classes are often strongly correlated, modeling the dependencies between them allows MDC methods to improve their performance – at the expense of an increased computational cost. In this paper we focus on the classifier chains (CC) approach for modeling dependencies, one of the most popular and highest-performing methods for multi-label classification (MLC), a particular case of MDC which involves only binary classes (i.e., labels). The original CC algorithm makes a greedy approximation, and is fast but tends to propagate errors along the chain. Here we present novel Monte Carlo schemes, both for finding a good chain sequence and performing efficient inference. Our algorithms remain tractable for high-dimensional data sets and obtain the best predictive performance across several real data sets.     

MIMLRBF: RBF neural networks for multi-instance multi-label learning

In multi-instance multi-label learning (MIML), each example is not only represented by multiple instances but also associated with multiple class labels. Several learning frameworks, such as the traditional supervised learning, can be regarded as degenerated versions of MIML. Therefore, an intuitive way to solve MIML problem is to identify its equivalence in its degenerated versions. However, this identification process would make useful information encoded in training examples get lost and thus impair the learning algorithm's performance. In this paper, RBF neural networks are adapted to learn from MIML examples. Connections between instances and labels are directly exploited in the process of first layer clustering and second layer optimization. The proposed method demonstrates superior performance on two real-world MIML tasks.     

Multiple kernel learning using single stage function approximation for binary classification problems

In this paper, the multiple kernel learning (MKL) is formulated as a supervised classification problem. We dealt with binary classification data and hence the data modelling problem involves the computation of two decision boundaries of which one related with that of kernel learning and the other with that of input data. In our approach, they are found with the aid of a single cost function by constructing a global reproducing kernel Hilbert space (RKHS) as the direct sum of the RKHSs corresponding to the decision boundaries of kernel learning and input data and searching that function from the global RKHS, which can be represented as the direct sum of the decision boundaries under consideration. In our experimental analysis, the proposed model had shown superior performance in comparison with that of existing two stage function approximation formulation of MKL, where the decision functions of kernel learning and input data are found separately using two different cost functions. This is due to the fact that single stage representation helps the knowledge transfer between the computation procedures for finding the decision boundaries of kernel learning and input data, which inturn boosts the generalisation capacity of the model.     

一种主动学习式P2P流识别方法

P2P流的识别对于网络的维护与运营都具有重要意义,基于机器学习的流识别技术是目前研究的热点和难点内容,但目前仍然存在着建立分类模型需要大量适用的训练数据、训练数据的标记需要依赖领域专家以及因此而导致的工作量及难度过大和实用性不强等问题,而当前的研究工作很少涉及到这些问题的解决办法。针对这一问题,采用主动学习技术提取少量高质量的训练样本进行建模,并结合SVM分类算法提出了一种基于锦标赛选择的样本筛选方法。实验结果表明,其相对于已有的流识别方法,能够在仅依赖少量高质量训练样本的前提下,保证较高召回率及较低误报率,更适用于现实网络环境。     

基于双重低秩分解的不完整多视图子空间学习

多视图数据在现实世界中应用广泛,各种视角和不同的传感器有助于更好的数据表示,然而,来自不同视图的数据具有较大的差异,尤其当多视图数据不完整时,可能导致训练效果较差甚至失败。为了解决该问题,本文提出了一个基于双重低秩分解的不完整多视图子空间学习算法。所提算法通过两方面来解决不完整多视图问题:一方面,基于双重低秩分解子空间框架,引入潜在因子来挖掘多视图数据中缺失的信息;另一方面,通过预先学习的多视图数据低维特征获得更好的鲁棒性,并以有监督的方式来指导双重低秩分解。实验结果证明,所提算法较之前的多视图子空间学习算法有明显优势;即使对于不完整的多视图数据,该算法也具有良好的分类性能。     

A novel semi-supervised learning framework with simultaneous text representing

Text representation has received extensive attention in text mining tasks. There are various text representation models. Among them, vector space model is the most commonly used one. For vector space model, the core technique is term weighting. To date, a great deal of different term-weighting methods have been proposed, which can be divided into supervised group and unsupervised group. However, it is not advisable to use these two groups of methods directly in semi-supervised applications. In semi-supervised applications, the majority of the supervised term-weighting methods are not applicable as the label information is insufficient; meanwhile, the unsupervised term-weighting methods cannot make use of the provided category labels. Thus, a semi-supervised learning framework for iteratively revising the text representation by an EM-like strategy is proposed in this paper. Furthermore, a new supervised term-weighting method t f.sd f is proposed. T f.sd f has the ability to emphasize the importance of terms that are unevenly distributed among all the classes and weaken the importance of terms that are uniformly distributed. Experimental results on real text data show that the proposed semi-supervised learning framework with the aid of t f.sd f performs well. Also, t f.sd f is shown to be efficient for supervised learning.