SmartVisionApp: A framework for computer vision applications on mobile devices

In this paper a novel framework for the development of computer vision applications that exploit sensors available in mobile devices is presented. The framework is organized as a client–server application that combines mobile devices, network technologies and computer vision algorithms with the aim of performing object recognition starting from photos captured by a phone camera. The client module on the mobile device manages the image acquisition and the query formulation tasks, while the recognition module on the server executes the search on an existing database and sends back relevant information to the client. To show the effectiveness of the proposed solution, the implementation of two possible plug-ins for specific problems is described: landmark recognition and fashion shopping. Experiments on four different landmark datasets and one self-collected dataset of fashion accessories show that the system is efficient and robust in the presence of objects with different characteristics.     

An efficient approach for scene categorization based on discriminative codebook learning in bag-of-words framework

This paper proposes an efficient technique for learning a discriminative codebook for scene categorization. A state-of-the-art approach for scene categorization is the Bag-of-Words (BoW) framework, where codebook generation plays an important role in determining the performance of the system. Traditionally, the codebook generation methods adopted in the BoW techniques are designed to minimize the quantization error, rather than optimize the classification accuracy. In view of this, this paper tries to address the issue by careful design of the codewords such that the resulting image histograms for each category will retain strong discriminating power, while the online categorization of the testing image is as efficient as in the baseline BoW. The codewords are refined iteratively to improve their discriminative power offline. The proposed method is validated on UIUC Scene-15 dataset and NTU Scene-25 dataset and it is shown to outperform other state-of-the-art codebook generation methods in scene categorization.     

A novel approach for fast codebook re-quantization

In this paper, we present a fast codebook re-quantization algorithm (FCRA) using codewords of a codebook being re-quantized as the training vectors to generate the re-quantized codebook. Our method is different from the available approach, which uses the original training set to generate a re-quantized codebook. Compared to the traditional approach, our method can reduce the computing time dramatically, since the number of codewords of a codebook being re-quantized is usually much smaller than the number of original training vectors. Our method first classifies codewords of a re-quantized codebook into static and active groups. This approach uses the information of codeword displacements between successive partitions to reject impossible candidates in the partition process of codebook re-quantization. By implementing a fast search algorithm used for vector quantization encoding (MFAUPI) in the partition step of FCRA, the computational complexity of codebook re-quantization can be further reduced significantly. Using MFAUPI, the computing time of FCRA can be reduced by a factor of 1.55–3.78. Compared with the available approach OARC (optimization algorithm for re-quantization codebook), our proposed method can reduce the codebook re-quantization time by a factor of about 8005 using a training set of six real images. This reduction factor is increased when the re-quantized codebook size and/or training set size are increased. It is noted that our proposed algorithm can generate the same re-quantized codebook as that produced by the OARC.     

Extreme learning machine-based device displacement free activity recognition model

Activity recognition based on mobile device is an important aspect in developing human-centric pervasive applications like gaming, industrial maintenance and health monitoring. However, the data distribution of accelerometer is heavily affected by varying device locations and orientations, which will degrade the performance of recognition model. To solve this problem, we propose a fast, robust and device displacement free activity recognition model in this paper, which integrates principal component analysis (PCA) and extreme learning machine (ELM) to realize location-adaptive activity recognition. On the one hand, PCA is employed to reduce the dimensionality of feature space and extract robust features for recognition. On the other hand, in the online phase ELM is applied to classify the activity and adapt the recognition model to new device locations based on high confident recognition results in real time. Experimental results show that, with robust features and fast adaptation capability, the proposed model can adapt the classifier to new device locations quickly and obtain good recognition performance.     

基于流形学习的极端学习机及其在人脸识别中的应用

极端学习机以其快速高效和良好的泛化能力在模式识别领域得到了广泛应用,然而现有的ELM及其改进算法并没有充分考虑到数据维数对ELM分类性能和泛化能力的影响,当数据维数过高时包含的冗余属性及噪音点势必降低ELM的泛化能力,针对这一问题本文提出一种基于流形学习的极端学习机,该算法结合维数约减技术有效消除数据冗余属性及噪声对ELM分类性能的影响,为验证所提方法的有效性,实验使用普遍应用的图像数据,实验结果表明本文所提算法能够显著提高ELM的泛化性能。     

极限学习机综述

极限学习机是一种单隐层前向网络的训练算法,主要特点是训练速度极快,而且可以达到很高的泛化性能。回顾了极限学习机的发展历程,分析了极限学习机的数学模型,详细介绍了极限学习机的各种改进算法,并列举了极限学习机在识别、预测和医学诊断领域的应用。最后总结预测了极限学习机的改进方向。     

矢量量化中码书旋转压缩的研究

普通码书中的码字之间在不同的方向上具有很大的相关性,存在大量的数据冗余。提出了将码书中的码字旋转压缩的理论。该理论是将各个码字按四个方向垂直旋转后进行相似性检查。如果旋转后的码字其中一个方向上与前面的码字存在相似,则将该码字删除,从而达到压缩的目的。编码时将压缩后的码书旋转恢复后进行编码,从而大幅降低了需要存储的码字数量。同时给出了一种将现有1 024阶16维码书旋转压缩成256阶16维的方法,并对该方法得到的码书性能进行了仿真验证。实验结果表明使用压缩后的码书在硬件实现时与普通的矢量量化码书相比减少了75%的存储空间和输入带宽,而PSNR平均只降低0.28 dB。     

An extreme learning machine approach for speaker recognition

Over the last two decades, automatic speaker recognition has been an interesting and challenging problem to speech researchers. It can be classified into two different categories, speaker identification and speaker verification. In this paper, a new classifier, extreme learning machine, is examined on the text-independent speaker verification task and compared with SVM classifier. Extreme learning machine (ELM) classifiers have been proposed for generalized single hidden layer feedforward networks with a wide variety of hidden nodes. They are extremely fast in learning and perform well on many artificial and real regression and classification applications. The database used to evaluate the ELM and SVM classifiers is ELSDSR corpus, and the Mel-frequency Cepstral Coefficients were extracted and used as the input to the classifiers. Empirical studies have shown that ELM classifiers and its variants could perform better than SVM classifiers on the dataset provided with less training time.     

Face recognition based on extreme learning machine

Extreme learning machine (ELM) is an efficient learning algorithm for generalized single hidden layer feedforward networks (SLFNs), which performs well in both regression and classification applications. It has recently been shown that from the optimization point of view ELM and support vector machine (SVM) are equivalent but ELM has less stringent optimization constraints. Due to the mild optimization constraints ELM can be easy of implementation and usually obtains better generalization performance. In this paper we study the performance of the one-against-all (OAA) and one-against-one (OAO) ELM for classification in multi-label face recognition applications. The performance is verified through four benchmarking face image data sets.     

Landmark-assisted location and tracking in outdoor mobile network

Modern mobile devices integrating sensors, like accelerometers and cameras, are paving the way to the definition of high-quality and accurate geolocation solutions based on the informations acquired by these sensors, and data collected and managed by GSM/3G networks. In this paper, we present a technique that provides geolocation and mobility prediction of mobile devices, mixing the location information acquired with the GSM/3G infrastructure and the results of a landmark matching achieved thanks to the camera integrated on the mobile devices. Our geolocation approach is based on an advanced Time-Forwarding algorithm and on database correlation technique over Received Signal Strength Indication (RSSI) data, and integrates information produced by a landmark recognition infrastructure, to enhance algorithm performances in those areas with poor signal and low accurate geolocation. Performances of the algorithm are evaluated on real data from a complex urban environment.     

Leukocyte image segmentation by visual attention and extreme learning machine

This paper presents a fast and simple framework for leukocyte image segmentation by learning with extreme learning machine (ELM) and sampling via simulating visual system. In sampling stage, visual attention and the effect of microsaccades in fixation are simulated. The high gradient pixels in fixation regions are sampled to group training set. We designed an automatic sampling process for leukocyte image according to the staining knowledge of blood smears. In learning stage, ELM classifier is trained online to simulate visual neuron system and then extracts pixels of object from image. The ELM-based segmentation is fully automatic by the proposed framework, which could find efficient samples actively, train the classification model in real time and almost no parameter adjusted. Experimental results demonstrated the new method could extract entire leukocyte from complex scenes, has equivalent performance compared to the SVM-based method and exceeds the marker-controlled watershed algorithm.     

Constrained-storage multistage vector quantization based on genetic algorithms

Multistage vector quantization (MSVQ) and their variants have been recently proposed. Before MSVQ is designed, the user must artificially determine the number of codewords in each VQ stage. However, the users usually have no idea regarding the number of codewords in each VQ stage, and thus doubt whether the resulting MSVQ is optimal. This paper proposes the genetic design (GD) algorithm to design the MSVQ. The GD algorithm can automatically find the number of codewords to optimize each VQ stage according to the rate–distortion performance. Thus, the MSVQ based on the GD algorithm, namely MSVQ(GD), is proposed here. Furthermore, using a sharing codebook (SC) can further reduce the storage size of MSVQ. Combining numerous similar codewords in the VQ stages of MSVQ produces the codewords of the sharing codebook. This paper proposes the genetic merge (GM) algorithm to design the SC of MSVQ. Therefore, the constrained-storage MSVQ using a SC, namely CSMSVQ, is proposed and outperforms other MSVQs in the experiments presented here.     

矢量量化的初始码书算法

矢量量化的初始码书设计是很重要的,影响或决定着其后码书形成算法的迭代次数和最终的码书质量。针对原有的初始码书算法在性能上随机性强与信源匹配程度不高的问题,提出一种对于训练矢量实施基于分量的和值排序,然后做分离平均的初始码书形成算法。算法使用了矢量的特征量,脱离了对于图像结构因数的依赖,能产生鲁棒性较好的初始码书。实验证明了该方法的有效性,与LBG算法结合可进一步提高码书质量。     

Extreme learning machine and its applications

Recently, a novel learning algorithm for single-hidden-layer feedforward neural networks (SLFNs) named extreme learning machine (ELM) was proposed by Huang et al. The essence of ELM is that the learning parameters of hidden nodes, including input weights and biases, are randomly assigned and need not be tuned while the output weights can be analytically determined by the simple generalized inverse operation. The only parameter needed to be defined is the number of hidden nodes. Compared with other traditional learning algorithms for SLFNs, ELM provides extremely faster learning speed, better generalization performance and with least human intervention. This paper firstly introduces a brief review of ELM, describing the principle and algorithm of ELM. Then, we put emphasis on the improved methods or the typical variants of ELM, especially on incremental ELM, pruning ELM, error-minimized ELM, two-stage ELM, online sequential ELM, evolutionary ELM, voting-based ELM, ordinal ELM, fully complex ELM, and symmetric ELM. Next, the paper summarized the applications of ELM on classification, regression, function approximation, pattern recognition, forecasting and diagnosis, and so on. In the last, the paper discussed several open issues of ELM, which may be worthy of exploring in the future.     

Image-based recognition framework for robotic weed control systems

In this paper, we introduce a novel and efficient image-based weed recognition system for the weed control problem of Broad-leaved Dock (Rumex obtusifolius L.). Our proposed weed recognition system is developed using a framework, that allows the examination of the affects for various image resolutions in detection and recognition accuracy. Moreover, it includes state-of-the-art object/image categorization processes such as feature detection and extraction, codebook learning, feature encoding, image representation and classification. The efficiency of those processes have been improved and optimized by introducing methodologies, techniques and system parameters specially tailored for the goal of weed recognition. Through an exhaustive optimization process, which is presented as our experimental evaluation, we conclude to a weed recognition system that uses an image input resolution of 200 ×150, SURF features over dense feature extraction, an optimized Gaussian Mixture Model based codebook combined with Fisher encoding, using a two level image representation. The resulting image representation vectors are classified using a linear classifier. This system is experimentally shown to yield state-of-the-art recognition accuracy of 89.09% in the examined dataset. Our proposed system is also experimentally shown to comply with the specifications of the examined applications since it provides low false-positive results of 4.38%. As a result, the proposed framework can be efficiently used in weed control robots for precision farming applications.     

Realtime training on mobile devices for face recognition applications

Due to the increases in processing power and storage capacity of mobile devices over the years, an incorporation of realtime face recognition to mobile devices is no longer unattainable. However, the possibility of the realtime learning of a large number of samples within mobile devices must be established. In this paper, we attempt to establish this possibility by presenting a realtime training algorithm in mobile devices for face recognition related applications. This is differentiated from those traditional algorithms which focused on realtime classification. In order to solve the challenging realtime issue in mobile devices, we extract local face features using some local random bases and then a sequential neural network is trained incrementally with these features. We demonstrate the effectiveness of the proposed algorithm and the feasibility of its application in mobile devices through empirical experiments. Our results show that the proposed algorithm significantly outperforms several popular face recognition methods with a dramatic reduction in computational speed. Moreover, only the proposed method shows the ability to train additional samples incrementally in realtime without memory failure and accuracy degradation using a recent mobile phone model.     

一种改进的邻域保持嵌入算法

邻域保持嵌入(NPE)是一种新颖的子空间学习算法,在降维的同时保持了样本集原有的局部邻域流形结构。为了进一步增强NPE在人脸识别和语音识别中的识别功能,提出了一种改进的邻域保持嵌入算法(RNPE)。在NPE的基础上通过引入类间权值矩阵,使得类间离散度最大,类内离散度最小,增加了样本类间散布约束。最后利用极端学习机(ELM)分类器进行分类,在Yale人脸库、Umist人脸库、Isolet语音库上的实验结果表明,RNPE算法的识别率明显高于NPE算法、LMMDE算法以及RAF-GE算法。     

改进鱼群算法优化的ELM在乳腺肿瘤辅助诊断中的应用研究

针对传统极限学习机的输入权值矩阵和隐含层偏差是随机给定进而可能会导致在乳腺肿瘤的辅助诊断应用研究中存在精度明显不足的情况,提出用改进鱼群算法优化ELM方法。在完成对乳腺肿瘤有效的辅助诊断的过程中,本研究工作充分利用ELM能快速地完成训练过程且具有很好的泛化能力的特点,并结合用改进鱼群算法对ELM的隐含层偏差进行优化,构造出了乳腺肿瘤与从乳腺肿瘤样本数据中提取的10个特征向量之间的非线性映射关系。将本文提出的乳腺肿瘤识别方法的仿真结果与AFSA-ELM方法、ELM方法、LVQ方法、BP方法的仿真结果分别从识别准确率、假阴性率、学习速度三个方面做对比分析,仿真结果表明,本文所提方法对乳腺肿瘤诊断具有较高的分类识别准确率、假阴性率以及较快的学习速率。     

基于改进极限学习机算法的行为识别

重点研究了极限学习机ELM对行为识别检测的效果。针对在线学习和行为分类上存在计算复杂性和时间消耗大的问题,提出了一种新的行为识别学习算法(ELM-Cholesky)。该算法首先引入了基于Cholesky分解求ELM的方法,接着依据在线学习期间核函数矩阵的更新特点,将分块矩阵Cholesky分解算法用于ELM的在线求解,使三角因子矩阵实现在线更新,从而得出一种新的ELM-Cholesky在线学习算法。新算法充分利用了历史训练数据,降低了计算的复杂性,提高了行为识别的准确率。最后,在基准数据库上采用该算法进行了大量实验,实验结果表明了这种在线学习算法的有效性。