基于分块非负矩阵分解人脸识别增量学习*

非负矩阵分解(NMF)算法可以提取图像的局部特征,然而NMF算法有两个主要缺点：a）当矩阵维数较大时,NMF算法非常耗时;b）当增加新的训练样本或类别时,NMF算法必须进行重复学习。为克服NMF算法这些缺点,提出了一种新的分块NMF算法(BNMF)。特别地,该方法还可用于增量学习。通过在FERET和CMU PIE人脸数据库上进行实验,结果表明该算法均优于NMF和PCA算法。     

Nonlinear nonnegative matrix factorization based on Mercer kernel construction

Generalizations ofnonnegative matrix factorization (NMF) in kernel feature space, such as projected gradient kernel NMF (PGKNMF) and polynomial Kernel NMF (PNMF), have been developed for face and facial expression recognition recently. However, these existing kernel NMF approaches cannot guarantee the nonnegativity of bases in kernel feature space and thus are essentially semi-NMF methods. In this paper, we show that nonlinear semi-NMF cannot extract the localized components which offer important information in object recognition. Therefore, nonlinear NMF rather than semi-NMF is needed to be developed for extracting localized component as well as learning the nonlinear structure. In order to address the nonlinear problem of NMF and the semi-nonnegative problem of the existing kernel NMF methods, we develop the nonlinear NMF based on a self-constructed Mercer kernel which preserves the nonnegative constraints on both bases and coefficients in kernel feature space. Experimental results in face and expressing recognition show that the proposed approach outperforms the existing state-of-the-art kernel methods, such as KPCA, GDA, PNMF and PGKNMF.     

Manifold-respecting discriminant nonnegative matrix factorization

Nonnegative matrix factorization (NMF) is an unsupervised learning method for low-rank approximation of nonnegative data, where the target matrix is approximated by a product of two nonnegative factor matrices. Two important ingredients are missing in the standard NMF methods: (1) discriminant analysis with label information; (2) geometric structure (manifold) in the data. Most of the existing variants of NMF incorporate one of these ingredients into the factorization. In this paper, we present a variation of NMF which is equipped with both these ingredients, such that the data manifold is respected and label information is incorporated into the NMF. To this end, we regularize NMF by intra-class and inter-class k-nearest neighbor (k-NN) graphs, leading to NMF-kNN, where we minimize the approximation error while contracting intra-class neighborhoods and expanding inter-class neighborhoods in the decomposition. We develop simple multiplicative updates for NMF-kNN and present monotonic convergence results. Experiments on several benchmark face and document datasets confirm the useful behavior of our proposed method in the task of feature extraction.     

Evaluation of distance metrics for recognition based on non-negative matrix factorization

Non-negative matrix factorization (NMF) is an unsupervised algorithm that presents the ability of learning “parts” from visual data. The goal of this technique is to find basis functions such that training examples can be faithfully reconstructed using appropriate combinations of the discovered basis functions. Bases are restricted to non-negative values, and original data is represented by additive combinations of the basis vectors. The space defined by NMF basis lacks of a suitable metric. The aim of this paper is to explore different distance metrics for NMF in the context of statistical classification of objects, and to compare these results to those obtained with a related algorithm: principal component analysis (PCA). We evaluate Earth mover’s distance as a relevant metric that takes into account the positive definition of the NMF bases, and it presents the best recognition rates when the dimensionality of data is correctly estimated. We also show that NMF outperforms PCA-based representation when visual data can be partially occluded.     

Robust embedded projective nonnegative matrix factorization for image analysis and feature extraction

Nonnegative matrix factorization (NMF) is an unsupervised learning method for decomposing high-dimensional nonnegative data matrices and extracting basic and intrinsic features. Since image data are described and stored as nonnegative matrices, the mining and analysis process usually involves the use of various NMF strategies. NMF methods have well-known applications in face recognition, image reconstruction, handwritten digit recognition, image denoising and feature extraction. Recently, several projective NMF (P-NMF) methods based on positively constrained projections have been proposed and were found to perform better than the standard NMF approach in some aspects. However, some drawbacks still affect the existing NMF and P-NMF algorithms; these include dense factors, slow convergence, learning poor local features, and low reconstruction accuracy. The aim of this paper is to design algorithms that address the aforementioned issues. In particular, we propose two embedded P-NMF algorithms: the first method combines the alternating least squares (ALS) algorithm with the P-NMF update rules of the Frobenius norm and the second one embeds ALS with the P-NMF update rule of the Kullback–Leibler divergence. To assess the performances of the proposed methods, we conducted various experiments on four well-known data sets of faces. The experimental results reveal that the proposed algorithms outperform other related methods by providing very sparse factors and extracting better localized features. In addition, the empirical studies show that the new methods provide highly orthogonal factors that possess small entropy values.     

非负矩阵分解算法及其在生物信息学中的应用研究

非负矩阵分解是近年来快速发展的一类机器学习算法,能够实现对高维数据的维度规约及局部特征提取,在诸多生物信息问题的分析与处理中得到了广泛应用,并衍生出一系列实用算法。本文系统分析了非负矩阵分解的数学理论基础及其特有的局部表达属性,综述了标准非负矩阵分解与各种衍生算法的发展历程及算法初始化与参数选取方法的研究进展,并从序列特征分析、表达模式与功能模块识别、生物医学文献挖掘等几个方面总结了非负矩阵分解算法在生物信息学领域的应用成果。最后,指出了非负矩阵分解算法研究及其应用于生物信息处理所面临的问题,分析和预测了可能的发展方向。     

基于图正则化的半监督非负矩阵分解

提出了一种基于图正则化的半监督非负矩阵分解算法（GSNMF）,克服了非负矩阵分解（NMF）、约束非负矩阵分解（CNMF）和图正则化非负矩阵分解（GNMF）方法忽略样本数据的局部几何结构或标签信息不足的缺陷,且NMF、CNMF和GNMF均为GSNMF的特例。也从理论上证明了GSNMF算法的收敛性。该算法对样本数据进行低维非负分解时,在图框架下既保持数据的几何结构,又利用已知样本的标签信息,在进行半监督学习时,同类样本能更好地聚集而类间距离尽可能大。在人脸数据库ORL、FERET和手写体数据库USPS上的仿真结果表明,相对于NMF及其一些改进算法,GSNMF均具有更高的聚类精度。     

Max-margin non-negative matrix factorization with flexible spatial constraints based on factor analysis

Non-negative matrix factorization (NMF) is a popular feature encoding method for image understanding due to its non-negative properties in representation, but the learnt basis images are not always local due to the lack of explicit constraints in its objective. Various algebraic or geometric local constraints are hence proposed to shape the behaviour of the original NMF. Such constraints are usually rigid in the sense that they have to be specified beforehand instead of learning from the data. In this paper, we propose a flexible spatial constraint method for NMF learning based on factor analysis. Particularly, to learn the local spatial structure of the images, we apply a series of transformations such as orthogonal rotation and thresholding to the factor loading matrix obtained through factor analysis. Then we map the transformed loading matrix into a Laplacian matrix and incorporate this into a max-margin non-negative matrix factorization framework as a penalty term, aiming to learn a representation space which is non-negative, discriminative and localstructure- preserving. We verify the feasibility and effectiveness of the proposed method on several real world datasets with encouraging results.     

Personalized mobile English vocabulary learning system based on item response theory and learning memory cycle

Since learning English is very popular in non-English speaking countries, developing modern assisted-learning tools that support effective English learning is a critical issue in the English-language education field. Learning English involves memorization and practice of a large number of vocabulary words and numerous grammatical structures. Vocabulary learning is a principal issue for English learning because vocabulary comprises the basic building blocks of English sentences. Therefore, many studies have attempted to improve the efficiency and performance when learning English vocabulary. With the accelerated growth in wireless and mobile technologies, mobile learning using mobile devices such as PDAs, tablet PCs, and cell phones has gradually become considered effective because it inherits all the advantages of e-learning and overcomes limitations of learning time and space that limit web-based learning systems. Therefore, this study presents a personalized mobile English vocabulary learning system based on Item Response Theory and learning memory cycle, which recommends appropriate English vocabulary for learning according to individual learner vocabulary ability and memory cycle. The proposed system has been successfully implemented on personal digital assistant (PDA) for personalized English vocabulary learning. The experimental results indicated that the proposed system could obviously promote the learning performances and interests of learners due to effective and flexible learning mode for English vocabulary learning.     

Fast vocabulary acquisition in an NMF-based self-learning vocal user interface

In command-and-control applications, a vocal user interface (VUI) is useful for handsfree control of various devices, especially for people with a physical disability. The spoken utterances are usually restricted to a predefined list of phrases or to a restricted grammar, and the acoustic models work well for normal speech. While some state-of-the-art methods allow for user adaptation of the predefined acoustic models and lexicons, we pursue a fully adaptive VUI by learning both vocabulary and acoustics directly from interaction examples. A learning curve usually has a steep rise in the beginning and an asymptotic ceiling at the end. To limit tutoring time and to guarantee good performance in the long run, the word learning rate of the VUI should be fast and the learning curve should level off at a high accuracy. In order to deal with these performance indicators, we propose a multi-level VUI architecture and we investigate the effectiveness of alternative processing schemes. In the low-level layer, we explore the use of MIDA features (Mutual Information Discrimination Analysis) against conventional MFCC features. In the mid-level layer, we enhance the acoustic representation by means of phone posteriorgrams and clustering procedures. In the high-level layer, we use the NMF (Non-negative Matrix Factorization) procedure which has been demonstrated to be an effective approach for word learning. We evaluate and discuss the performance and the feasibility of our approach in a realistic experimental setting of the VUI-user learning context.     

Snapshot ensembles of non-negative matrix factorization for stability of topic modeling

Recently many topic models such as Latent Dirichlet Allocation (LDA) and Non-negative Matrix Factorization (NMF) have made important progress towards generating high-level knowledge from a large corpus. However, these algorithms based on random initialization generate different results on the same corpus using the same parameters, denoted as instability problem. For solving this problem, ensembles of NMF are known to be much more stable and accurate than individual NMFs. However, training multiple NMFs for ensembling is computationally expensive. In this paper, we propose a novel scheme to obtain the seemingly contradictory goal of ensembling multiple NMFs without any additional training cost. We train a single NMF algorithm with the cyclical learning rate schedule, which can converge to several local minima along its optimization path. We save the results to the ensemble when the model converges, and then restart the optimization with a large learning rate that can help escape the current local minimum. Based on experiments performed on text corpora using a number of measures to assess, our method can reduce instability at no additional training cost, while simultaneously yields more accurate topic models than traditional single methods and ensemble methods.     

A ubiquitous English vocabulary learning system: Evidence of active/passive attitudes vs. usefulness/ease-of-use

English vocabulary learning and ubiquitous learning have separately received considerable attention in recent years. However, research on English vocabulary learning in ubiquitous learning contexts has been less studied. In this study, we develop a ubiquitous English vocabulary learning (UEVL) system to assist students in experiencing a systematic vocabulary learning process in which ubiquitous technology is used to develop the system, and video clips are used as the material. Afterward, the technology acceptance model and partial least squares approach are used to explore students’ perspectives on the UEVL system. The results indicate that (1) both the system characteristics and the material characteristics of the UEVL system positively and significantly influence the perspectives of all students on the system; (2) the active students are interested in perceived usefulness; (3) the passive students are interested in perceived ease of use.     

基于图正则化的受限非负矩阵分解算法及在图像表示中的应用

非负矩阵分解(NMF)是一种非常有效的图像表示方法,已被广泛应用到模式识别领域.针对NMF算法是无监督学习算法,无法同时考虑样本类别信息和固有几何结构信息的缺点,提出一种基于图正则化的受限非负矩阵分解(GRCNMF)的算法.该算法利用硬约束保持样本的类别信息,增强算法的鉴别能力,同时还利用近邻图来保持样本间固有的几何结构.通过在COIL20和ORL图像库中的聚类实验结果表明GRCNMF优于其它几种算法,说明GRCNMF的有效性.     

Internet cognitive failure and fatigue relevant to learners' self‐regulation and learning progress in English vocabulary with a calibration scheme

To determine the factors of learning effectiveness in English vocabulary learning when using a calibration scheme, this study developed a freshman English mobile device application (for iPhone 4) for students with low levels of English proficiency to practise vocabulary in the beginning of their Freshman English course. Data were collected and validated from 243 subjects for confirmatory factor analysis and structural equation modeling. The findings revealed that Internet cognitive failure (i.e., trait cognitive disability) was positively correlated to Internet cognitive fatigue (i.e., state cognitive disability). Both types of Internet cognitive disability were negatively correlated to self‐regulation in English vocabulary learning (SREVL). SREVL was positively correlated to the degree of learning improvement. The findings implied that the use of a calibration design for mobile English vocabulary learning could benefit students with low levels of Internet cognitive disability but high levels of SREVL.     

Non-negative Matrix Factorization with Symmetric Manifold Regularization

Non-negative matrix factorization (NMF) is becoming an important tool for information retrieval and pattern recognition. However, in the applications of image decomposition, it is not enough to discover the intrinsic geometrical structure of the observation samples by only considering the similarity of different images. In this paper, symmetric manifold regularized objective functions are proposed to develop NMF based learning algorithms (called SMNMF), which explore both the global and local features of the manifold structures for image clustering and at the same time improve the convergence of the graph regularized NMF algorithms. For different initializations, simulations are utilized to confirm the theoretical results obtained in the convergence analysis of the new algorithms. Experimental results on COIL20, ORL, and JAFFE data sets demonstrate the clustering effectiveness of the proposed algorithms by comparing with the state-of-the-art algorithms.

     

Extracting non-negative basis images using pixel dispersion penalty

Non-negativity matrix factorization (NMF) and its variants have been explored in the last decade and are still attractive due to its ability of extracting non-negative basis images. However, most existing NMF based methods are not ready for encoding higher-order data information. One reason is that they do not directly/explicitly model structured data information during learning, and therefore the extracted basis images may not completely describe the “parts” in an image [1] very well. In order to solve this problem, the structured sparse NMF has been recently proposed in order to learn structured basis images. It however depends on some special prior knowledge, i.e. one needs to exhaustively define a set of structured patterns in advance. In this paper, we wish to perform structured sparsity learning as automatically as possible. To that end, we propose a pixel dispersion penalty (PDP), which effectively describes the spatial dispersion of pixels in an image without using any manually predefined structured patterns as constraints. In PDP, we consider each part-based feature pattern of an image as a cluster of non-zero pixels; that is the non-zero pixels of a local pattern should be spatially close to each other. Furthermore, by incorporating the proposed PDP, we develop a spatial non-negative matrix factorization (Spatial NMF) and a spatial non-negative component analysis (Spatial NCA). In Spatial NCA, the non-negativity constraint is only imposed on basis images and such constraint on coefficients is released, so both subtractive and additive combinations of non-negative basis images are allowed for reconstructing any images. Extensive experiments are conducted to validate the effectiveness of the proposed pixel dispersion penalty. We also experimentally show that Spatial NCA is more flexible for extracting non-negative basis images and obtains better and more stable performance.     

基于图正则化和稀疏约束的半监督非负矩阵分解

非负矩阵分解是在矩阵非负约束下的分解算法。为了提高识别率,提出了一种基于稀疏约束和图正则化的半监督非负矩阵分解方法。该方法对样本数据进行低维非负分解时,既保持数据的几何结构,又利用已知样本的标签信息进行半监督学习,而且对基矩阵施加稀疏性约束,最后将它们整合于单个目标函数中。构造了一个有效的更新算法,并且在理论上证明了该算法的收敛性。在多个人脸数据库上的仿真结果表明,相对于NMF、GNMF、CNMF等算法,GCNMFS具有更好的聚类精度和稀疏性。     

Manifold ranking graph regularization non-negative matrix factorization with global and local structures

Pattern Analysis and Applications - Non-negative matrix factorization (NMF) is a recently popularized technique for learning parts-based, linear representations of non-negative data. Although the...     

结构正则化多视图非负矩阵分解

现存非负矩阵分解（non-negative matrix factorization,NMF）研究多考虑单一视图分解数据,忽略了数据信息的全面性。此外,NMF限制其获取数据的内在几何结构。针对以上问题,提出一个结构正则化多视图非负矩阵分解算法（structure regularized multi-view nonnegative matrix factorization,SRMNMF）。首先,通过主成分分析来对数据进行全局结构的判别式学习;其次,利用流形学习来捕获数据的局部结构;然后,通过利用多视图数据的多样性和差异性来学习表征。模型提升了算法聚类的整体性能,更加有效地挖掘数据的结构信息。此外,采用高效的交替迭代算法优化目标函数得到最优的因子矩阵。在六个数据集上与现存的代表性方法比较,所提出的SRMNMF的准确率、NMI和Purity分别最大提高4.4%、6.1%和4.05%。     

Six seconds to visualize the word: improving EFL learners' vocabulary through VVVs

This paper moves away from reminiscent mechanical repetition and drills, which were in vogue when teaching vocabulary before the rise of technology. With the support of technology, innovative methodologies that are more effective and enjoyable can be implemented into vocabulary teaching. In this particular context, there seems to be a lack of technology integration in vocabulary teaching because of teachers being untrained and/or not provided with the necessary technology. The aim of this study was to foster vocabulary development through the implementation of Vine vocabulary videos in English vocabulary learning. An embedded mixed methods design was employed to collect necessary data for analysis. The results of the post‐test revealed that the practice of Vine vocabulary videos was effective and improved participants' vocabulary. The content analysis of the semi‐structured interviews carried out with participants indicated that they had enjoyed the whole process and found it very motivating and effective. This study claims that adopting smartphones into a vocabulary course will enable English as a foreign language learners to expand and consolidate their vocabulary learning outside the classroom, engage them in a collaborative learning environment, practice and use the language being learnt and share their knowledge and experiences with their peers.