Software design patterns classification and selection using text categorization approach

ContextNumerous software design patterns have been introduced and cataloged either as a canonical or a variant solution to solve a design problem. The existing automatic techniques for design pattern(s) selection aid novice software developers to select the more appropriate design pattern(s) from the list of applicable patterns to solve a design problem in the designing phase of software development life cycle.GoalHowever, the existing automatic techniques are limited to the semi-formal specification, multi-class problem, an adequate sample size to make precise learning and individual classifier training in order to determine a candidate design pattern class and suggest more appropriate pattern(s).MethodTo address these issues, we exploit a text categorization based approach via Fuzzy c-means (unsupervised learning technique) that targets to present a systematic way to group the similar design patterns and suggest the appropriate design pattern(s) to developers related to the specification of a given design problem. We also propose an evaluation model to assess the effectiveness of the proposed approach in the context of several real design problems and design pattern collections. Subsequently, we also propose a new feature selection method Ensemble-IG to overcome the multi-class problem and improve the classification performance of the proposed approach.ResultsThe promising experimental results suggest the applicability of the proposed approach in the domain of classification and selection of appropriate design patterns. Subsequently, we also observed the significant improvement in learning precision of the proposed approach through Ensemble-IG.ConclusionThe proposed approach has four advantages as compared to previous work. First, the semi-formal specification of design patterns is not required as a prerequisite; second, the ground reality of class label assignment is not mandatory; third, lack of classifier’s training for each design pattern class and fourth, an adequate sample size is not required to make precise learning.     

Subspace learning for unsupervised feature selection via matrix factorization

Dimensionality reduction is an important and challenging task in machine learning and data mining. Feature selection and feature extraction are two commonly used techniques for decreasing dimensionality of the data and increasing efficiency of learning algorithms. Specifically, feature selection realized in the absence of class labels, namely unsupervised feature selection, is challenging and interesting. In this paper, we propose a new unsupervised feature selection criterion developed from the viewpoint of subspace learning, which is treated as a matrix factorization problem. The advantages of this work are four-fold. First, dwelling on the technique of matrix factorization, a unified framework is established for feature selection, feature extraction and clustering. Second, an iterative update algorithm is provided via matrix factorization, which is an efficient technique to deal with high-dimensional data. Third, an effective method for feature selection with numeric data is put forward, instead of drawing support from the discretization process. Fourth, this new criterion provides a sound foundation for embedding kernel tricks into feature selection. With this regard, an algorithm based on kernel methods is also proposed. The algorithms are compared with four state-of-the-art feature selection methods using six publicly available datasets. Experimental results demonstrate that in terms of clustering results, the proposed two algorithms come with better performance than the others for almost all datasets we experimented with here.     

Statistical pattern recognition: a review

The primary goal of pattern recognition is supervised or unsupervised classification. Among the various frameworks in which pattern recognition has been traditionally formulated, the statistical approach has been most intensively studied and used in practice. More recently, neural network techniques and methods imported from statistical learning theory have been receiving increasing attention. The design of a recognition system requires careful attention to the following issues: definition of pattern classes, sensing environment, pattern representation, feature extraction and selection, cluster analysis, classifier design and learning, selection of training and test samples, and performance evaluation. In spite of almost 50 years of research and development in this field, the general problem of recognizing complex patterns with arbitrary orientation, location, and scale remains unsolved. New and emerging applications, such as data mining, web searching, retrieval of multimedia data, face recognition, and cursive handwriting recognition, require robust and efficient pattern recognition techniques. The objective of this review paper is to summarize and compare some of the well-known methods used in various stages of a pattern recognition system and identify research topics and applications which are at the forefront of this exciting and challenging field     

Supervised and unsupervised relevance sampling in handcrafted and deep learning features obtained from image collections

Image collections are currently widely available and are being generated in a fast pace due to mobile and accessible equipment. In principle, that is a good scenario taking into account the design of successful visual pattern recognition systems. However, in particular for classification tasks, one may need to choose which examples are more relevant in order to build a training set that well represents the data, since they often require representative and sufficient observations to be accurate. In this paper we investigated three methods for selecting relevant examples from image collections based on learning models from small portions of the available data. We considered supervised methods that need labels to allow selection, and an unsupervised method that is agnostic to labels. The image datasets studied were described using both handcrafted and deep learning features. A general purpose algorithm is proposed which uses learning methods as subroutines. We show that our relevance selection algorithm outperforms random selection, in particular when using unlabelled data in an unsupervised approach, significantly reducing the size of the training set with little decrease in the test accuracy.     

一种基于局部流形结构的无监督特征学习方法(英文)

Unsupervised feature selection is fundamental in statistical pattern recognition, and has drawn persistent attention in the past several decades. Recently, much work has shown that feature selection can be formulated as nonlinear dimensionality reduction with discrete constraints. This line of research emphasizes utilizing the manifold learning techniques, where feature selection and learning can be studied based on the manifold assumption in data distribution. Many existing feature selection methods such as Laplacian score, SPEC(spectrum decomposition of graph Laplacian), TR(trace ratio) criterion, MSFS(multi-cluster feature selection) and EVSC(eigenvalue sensitive criterion) apply the basic properties of graph Laplacian, and select the optimal feature subsets which best preserve the manifold structure defined on the graph Laplacian. In this paper, we propose a new feature selection perspective from locally linear embedding(LLE), which is another popular manifold learning method. The main difficulty of using LLE for feature selection is that its optimization involves quadratic programming and eigenvalue decomposition, both of which are continuous procedures and different from discrete feature selection. We prove that the LLE objective can be decomposed with respect to data dimensionalities in the subset selection problem, which also facilitates constructing better coordinates from data using the principal component analysis(PCA) technique. Based on these results, we propose a novel unsupervised feature selection algorithm,called locally linear selection(LLS), to select a feature subset representing the underlying data manifold. The local relationship among samples is computed from the LLE formulation, which is then used to estimate the contribution of each individual feature to the underlying manifold structure. These contributions, represented as LLS scores, are ranked and selected as the candidate solution to feature selection. We further develop a locally linear rotation-selection(LLRS) algorithm which extends LLS to identify the optimal coordinate subset from a new space. Experimental results on real-world datasets show that our method can be more effective than Laplacian eigenmap based feature selection methods.     

Self-representation and PCA embedding for unsupervised feature selection

Feature selection is an important preprocessing step for dealing with high dimensional data. In this paper, we propose a novel unsupervised feature selection method by embedding a subspace learning regularization (i.e., principal component analysis (PCA)) into the sparse feature selection framework. Specifically, we select informative features via the sparse learning framework and consider preserving the principal components (i.e., the maximal variance) of the data at the same time, such that improving the interpretable ability of the feature selection model. Furthermore, we propose an effective optimization algorithm to solve the proposed objective function which can achieve stable optimal result with fast convergence. By comparing with five state-of-the-art unsupervised feature selection methods on six benchmark and real-world datasets, our proposed method achieved the best result in terms of classification performance.     

Supervised Training Using an Unsupervised Approach to Active Learning

Active learning algorithms allow neural networks to dynamically take part in the selection of the most informative training patterns. This paper introduces a new approach to active learning, which combines an unsupervised clustering of training data with a pattern selection approach based on sensitivity analysis. Training data is clustered into groups of similar patterns based on Euclidean distance, and the most informative pattern from each cluster is selected for training using the sensitivity analysis incremental learning algorithm in (Engelbrecht and Cloete, 1999). Experimental results show that the clustering approach improves on standard active learning as presented in (Engelbrecht and Cloete, 1999).     

Unsupervised feature selection based on kernel fisher discriminant analysis and regression learning

In this paper, we propose a new feature selection method called kernel fisher discriminant analysis and regression learning based algorithm for unsupervised feature selection. The existing feature selection methods are based on either manifold learning or discriminative techniques, each of which has some shortcomings. Although some studies show the advantages of two-steps method benefiting from both manifold learning and discriminative techniques, a joint formulation has been shown to be more efficient. To do so, we construct a global discriminant objective term of a clustering framework based on the kernel method. We add another term of regression learning into the objective function, which can impose the optimization to select a low-dimensional representation of the original dataset. We use L_2,1-norm of the features to impose a sparse structure upon features, which can result in more discriminative features. We propose an algorithm to solve the optimization problem introduced in this paper. We further discuss convergence, parameter sensitivity, computational complexity, as well as the clustering and classification accuracy of the proposed algorithm. In order to demonstrate the effectiveness of the proposed algorithm, we perform a set of experiments with different available datasets. The results obtained by the proposed algorithm are compared against the state-of-the-art algorithms. These results show that our method outperforms the existing state-of-the-art methods in many cases on different datasets, but the improved performance comes with the cost of increased time complexity.

     

基于自适应稀疏结构学习的神经精神疾病特征选择方法

在计算机辅助诊断神经精神疾病研究中,需要专业人士为样本进行诊断级的语义标注,耗费大量时间和精力,因此,以无监督的方式开展神经精神疾病辅助诊断研究具有重要意义.文中提出基于自适应稀疏结构学习的无监督特征选择方法,用于精神分裂症和阿兹海默症辅助诊断.在统一框架下同时学习稀疏表示和数据流形结构,并在该框架中采用一般化范数对稀疏学习的重构误差进行建模,不断迭代更新数据集的流形结构,解决传统特征选择方法存在的鲁棒性不足问题.在精神分裂症和阿兹海默症两个公共数据集上的实验表明文中方法在神经精神疾病分类中的有效性     

Semi-supervised change detection using modified self-organizing feature map neural network

In the present article, semi-supervised learning is integrated with an unsupervised context-sensitive change detection technique based on modified self-organizing feature map (MSOFM) network. In the proposed methodology, training of the MSOFM network is initially performed using only a few labeled patterns. Thereafter, the membership values, in both the classes, for each unlabeled pattern are determined using the concept of fuzzy set theory. The soft class label for each of the unlabeled patterns is then estimated using the membership values of its K nearest neighbors. Here, training of the network using the unlabeled patterns along with a few labeled patterns is carried out iteratively. A heuristic method has been suggested to select some patterns from the unlabeled ones for training. To check the effectiveness of the proposed methodology, experiments are conducted on three multi-temporal and multi-spectral data sets. Performance of the proposed work is compared with that of two unsupervised techniques, a supervised technique and two semi-supervised techniques. Results are also statistically validated using paired t-test. The proposed method produced promising results.     

A Hybrid Training Algorithm for Feedforward Neural Networks

Feedforward neural networks (FNN) have been proposed to solve complex problems in pattern recognition, classification and function approximation. Despite the general success of learning methods for FNN, such as the backpropagation (BP) algorithm, second-order algorithms, long learning time for convergence remains a problem to be overcome. In this paper, we propose a new hybrid algorithm for a FNN that combines unsupervised training for the hidden neurons (Kohonen algorithm) and supervised training for the output neurons (gradient descent method). Simulation results show the effectiveness of the proposed algorithm compared with other well-known learning methods.     

基于多对多生成对抗网络的非对称跨域迁移行人再识别

无监督跨域迁移学习是行人再识别中一个非常重要的任务. 给定一个有标注的源域和一个没有标注的目标域, 无监督跨域迁移的关键点在于尽可能地把源域的知识迁移到目标域. 然而, 目前的跨域迁移方法忽略了域内各视角分布的差异性, 导致迁移效果不好. 针对这个缺陷, 本文提出了一个基于多视角的非对称跨域迁移学习的新问题. 为了实现这种非对称跨域迁移, 提出了一种基于多对多生成对抗网络(Many-to-many generative adversarial network, M2M-GAN)的迁移方法. 该方法嵌入了指定的源域视角标记和目标域视角标记作为引导信息, 并增加了视角分类器用于鉴别不同的视角分布, 从而使模型能自动针对不同的源域视角和目标域视角组合采取不同的迁移方式. 在行人再识别基准数据集Market1501、DukeMTMC-reID和MSMT17上, 实验验证了本文的方法能有效提升迁移效果, 达到更高的无监督跨域行人再识别准确率.     

基于稀疏图表示的特征选择方法研究

特征选择旨在降低待处理数据的维度,剔除冗余特征,是机器学习领域的关键问题之一。现有的半监督特征选择方法一般借助图模型提取数据集的聚类结构,但其所提取的聚类结构缺乏清晰的边界,影响了特征选择的效果。为此,提出一种基于稀疏图表示的半监督特征选择方法,构建了聚类结构和特征选择的联合学习模型,采用l__1范数约束图模型以得到清晰的聚类结构,并引入l_2,1范数以避免噪声的干扰并提高特征选择的准确度。为了验证本方法的有效性,选择了目前流行的几种特征方法进行对比分析,实验结果表明了本方法的有效性。     

IDR: A participatory methodology for interdisciplinary design in technology enhanced learning

One of the important themes that emerged from the CAL’07 conference was the failure of technology to bring about the expected disruptive effect to learning and teaching. We identify one of the causes as an inherent weakness in prevalent development methodologies. While the problem of designing technology for learning is irreducibly multi-dimensional, design processes often lack true interdisciplinarity. To address this problem we present IDR, a participatory methodology for interdisciplinary techno-pedagogical design, drawing on the design patterns tradition [Alexander, C., Silverstein, M., & Ishikawa, S. (1977). A pattern language: Towns, buildings, construction (Center for environmental structure series). New York, NY: Oxford University Press] and the design research paradigm [DiSessa, A. A., & Cobb, P. (2004) Ontological innovation and the role of theory in design experiments. Journal of the Learning Sciences, 13(1), 77–103]. We discuss the iterative development and use of our methodology by a pan-European project team of educational researchers, software developers and teachers. We reflect on our experiences of the participatory nature of pattern design and discuss how, as a distributed team, we developed a set of over 120 design patterns, created using our freely available open source web toolkit. Furthermore, we detail how our methodology is applicable to the wider community through a workshop model, which has been run and iteratively refined at five major international conferences, involving over 200 participants.     

A kernel learning framework for domain adaptation learning

Domain adaptation learning(DAL) methods have shown promising results by utilizing labeled samples from the source(or auxiliary) domain(s) to learn a robust classifier for the target domain which has a few or even no labeled samples.However,there exist several key issues which need to be addressed in the state-of-theart DAL methods such as sufficient and effective distribution discrepancy metric learning,effective kernel space learning,and multiple source domains transfer learning,etc.Aiming at the mentioned-above issues,in this paper,we propose a unified kernel learning framework for domain adaptation learning and its effective extension based on multiple kernel learning(MKL) schema,regularized by the proposed new minimum distribution distance metric criterion which minimizes both the distribution mean discrepancy and the distribution scatter discrepancy between source and target domains,into which many existing kernel methods(like support vector machine(SVM),v-SVM,and least-square SVM) can be readily incorporated.Our framework,referred to as kernel learning for domain adaptation learning(KLDAL),simultaneously learns an optimal kernel space and a robust classifier by minimizing both the structural risk functional and the distribution discrepancy between different domains.Moreover,we extend the framework KLDAL to multiple kernel learning framework referred to as MKLDAL.Under the KLDAL or MKLDAL framework,we also propose three effective formulations called KLDAL-SVM or MKLDAL-SVM with respect to SVM and its variant μ-KLDALSVM or μ-MKLDALSVM with respect to v-SVM,and KLDAL-LSSVM or MKLDAL-LSSVM with respect to the least-square SVM,respectively.Comprehensive experiments on real-world data sets verify the outperformed or comparable effectiveness of the proposed frameworks.     

A quantitative approach for evaluating the quality of design patterns

In recent years, the influence of design patterns on software quality has attracted an increasing attention in the area of software engineering, as design patterns encapsulate valuable knowledge to resolve design problems, and more importantly to improve design quality. As the paradigm continues to increase in popularity, a systematic and objective approach to verify the design of a pattern is increasingly important. The intent session in a design pattern indicates the problem the design pattern wants to resolve, and the solution session describes the structural model for the problem. When the problem in the intent is a quality problem, the structure model should provide a solution to improve the relevant quality. In this work we provide an approach, based on object-oriented quality model, to validate if a design pattern is well-designed, i.e., it answers the question of the proposed structural model really resolves the quality problems described in the intent. We propose a validation approach to help pattern developers check if a design pattern is well-designed. In addition, a quantitative method is proposed to measure the effectiveness of the quality improvement of a design pattern that pattern users can determine which design patterns are applicable to meet their functional and quality requirements.     

基于支持对挖掘的主动学习行人再识别

基于监督学习的行人再识别方法需要大量人工标注的数据,对于实际应用并不适用。为了降低大规模行人再识别的标注成本,提出了一种基于支持对挖掘主动学习(support pair active learning, SPAL)的行人再识别方法。具体地,建立了一种无监督主动学习框架,在该框架中设计了一种双重不确定性选择策略迭代地挖掘支持样本对并提供给标注者标注;其次引入了一种约束聚类算法,将有标签的支持样本对的关系传播到其他无标签的样本中;最后提出了一种由无监督对比损失和监督支持样本对损失组成的混合学习策略来学习具有判别性的特征表示。在大规模行人再识别数据集MSMT17上,该方法相比于当前最先进的方法,标注成本降低了64.0%,同时mAP和rank1分别提升了11.0%和14.9%。大量实验结果表明,该方法有效地降低了标注成本并且优于目前最先进的无监督主动学习行人再识别方法。     

多模态引导的局部特征选择小样本学习方法

深度学习模型取得了令人瞩目的成绩,但其训练依赖于大量的标注样本,在标注样本匮乏的场景下模型表现不尽人意.针对这一问题,近年来以研究如何从少量样本快速学习的小样本学习被提了出来,方法主要采用元学习方式对模型进行训练,取得了不错的学习效果.但现有方法:1)通常仅基于样本的视觉特征来识别新类别,信息源较为单一; 2)元学习的使用使得模型从大量相似的小样本任务中学习通用的、可迁移的知识,不可避免地导致模型特征空间趋于一般化,存在样本特征表达不充分、不准确的问题.为解决上述问题,将预训练技术和多模态学习技术引入小样本学习过程,提出基于多模态引导的局部特征选择小样本学习方法.所提方法首先在包含大量样本的已知类别上进行模型预训练,旨在提升模型的特征表达能力;而后在元学习阶段,方法利用元学习对模型进行进一步优化,旨在提升模型的迁移能力或对小样本环境的适应能力,所提方法同时基于样本的视觉特征和文本特征进行局部特征选择来提升样本特征的表达能力,以避免元学习过程中模型特征表达能力的大幅下降;最后所提方法利用选择后的样本特征进行小样本学习.在MiniImageNet、CIFAR-FS和FC-100这3个基准数...     

Treelet kernel incorporating cyclic,stereo and inter pattern information in chemoinformatics

Chemoinformatics is a research field concerned with the study of physical or biological molecular properties through computer science?s research fields such as machine learning and graph theory. From this point of view, graph kernels provide a nice framework which allows to naturally combine machine learning and graph theory techniques. Graph kernels based on bags of patterns have proven their efficiency on several problems both in terms of accuracy and computational time. Treelet kernel is a graph kernel based on a bag of small subtrees. We propose in this paper several extensions of this kernel devoted to chemoinformatics problems. These extensions aim to weight each pattern according to its influence, to include the comparison of non-isomorphic patterns, to include stereo information and finally to explicitly encode cyclic information into kernel computation.     

Learning good prototypes for classification using filtering and abstraction of instances

We propose a framework for learning good prototypes, called prototype generation and filtering (PGF), by integrating the strength of instance-filtering and instance-abstraction techniques using two different integration methods. The two integration methods differ in the filtering granularity as well as the degree of coupling of the techniques. In order to characterize the behavior of the effect of integration, we categorize instance-filtering techniques into three kinds, namely, (1) removing border instances, (2) retaining border instance, (3) retaining center instances. The effect of using different kinds of filtering in different variants of our PGF framework are investigated. We have conducted experiments on 35 real-world benchmark data sets. We found that our PGF framework maintains or achieves better classification accuracy and gains a significant improvement in data reduction compared with pure filtering and pure abstraction techniques as well as KNN and C4.5.