Semi-supervised low-rank representation for image classification

Low-rank representation (LRR) is a useful tool for seeking the lowest rank representation among all the coefficient matrices that represent the images as linear combinations of the basis in the given dictionary. However, it is an unsupervised method and has poor applicability and performance in real scenarios because of the lack of image information. In this paper, based on LRR, we propose a novel semi-supervised approach, called label constrained sparse low-rank representation (LCSLRR), which incorporates the label information as an additional hard constraint. Specifically, this paper develops an optimization process in which the improvement of the discriminating power of the low-rank decomposition is presented explicitly by adding the label information constraint. We construct LCSLRR-graph to represent data structures for semi-supervised learning and provide the weights of edges in the graph by seeking a low-rank and sparse matrix. We conduct extensive experiments on publicly available databases to verify the effectiveness of our novel algorithm in comparison to the state-of-the-art approaches through a set of evaluations.     

Semantic-gap-oriented active learning for multilabel image annotation

User interaction is an effective way to handle the semantic gap problem in image annotation. To minimize user effort in the interactions, many active learning methods were proposed. These methods treat the semantic concepts individually or correlatively. However, they still neglect the key motivation of user feedback: to tackle the semantic gap. The size of the semantic gap of each concept is an important factor that affects the performance of user feedback. User should pay more efforts to the concepts with large semantic gaps, and vice versa. In this paper, we propose a semantic-gap-oriented active learning method, which incorporates the semantic gap measure into the information-minimization-based sample selection strategy. The basic learning model used in the active learning framework is an extended multilabel version of the sparse-graph-based semisupervised learning method that incorporates the semantic correlation. Extensive experiments conducted on two benchmark image data sets demonstrated the importance of bringing the semantic gap measure into the active learning process.     

A Multi-Directional Search technique for image annotation propagation

Image annotation has attracted lots of attention due to its importance in image understanding and search areas. In this paper, we propose a novel Multi-Directional Search framework for semi-automatic annotation propagation. In this system, the user interacts with the system to provide example images and the corresponding annotations during the annotation propagation process. In each iteration, the example images are clustered and the corresponding annotations are propagated separately to each cluster: images in the local neighborhood are annotated. Furthermore, some of those images are returned to the user for further annotation. As the user marks more images, the annotation process goes into multiple directions in the feature space. The query movements can be treated as multiple path navigation. Each path could be further split based on the user’s input. In this manner, the system provides accurate annotation assistance to the user - images with the same semantic meaning but different visual characteristics can be handled effectively. From comprehensive experiments on Corel and U. of Washington image databases, the proposed technique shows accuracy and efficiency on annotating image databases.     

Fusing semantic aspects for image annotation and retrieval

In this paper, we present an approach based on probabilistic latent semantic analysis (PLSA) to achieve the task of automatic image annotation and retrieval. In order to model training data precisely, each image is represented as a bag of visual words. Then a probabilistic framework is designed to capture semantic aspects from visual and textual modalities, respectively. Furthermore, an adaptive asymmetric learning algorithm is proposed to fuse these aspects. For each image document, the aspect distributions of different modalities are fused by multiplying different weights, which are determined by the visual representations of images. Consequently, the probabilistic framework can predict semantic annotation precisely for unseen images because it associates visual and textual modalities properly. We compare our approach with several state-of-the-art approaches on a standard Corel dataset. The experimental results show that our approach performs more effectively and accurately.     

Structural image retrieval using automatic image annotation and region based inverted file

Image retrieval has lagged far behind text retrieval despite more than two decades of intensive research effort. Most of the research on image retrieval in the last two decades are on content based image retrieval or image retrieval based on low level features. Recent research in this area focuses on semantic image retrieval using automatic image annotation. Most semantic image retrieval techniques in literature, however, treat an image as a bag of features/words while ignore the structural or spatial information in the image. In this paper, we propose a structural image retrieval method based on automatic image annotation and region based inverted file. In the proposed system, regions in an image are treated the same way as keywords in a structural text document, semantic concepts are learnt from image data to label image regions as keywords and weight is assigned to each keyword according to spatial position and relationship. As the result, images are indexed and retrieved in the same way as structural document retrieval. Specifically, images are broken down to regions which are represented using colour, texture and shape features. Region features are then quantized to create visual dictionaries which are similar to monolingual dictionaries like English or Chinese dictionaries. In the next step, a semantic dictionary similar to a bilingual dictionary like the English–Chinese dictionary is learnt to mapping image regions to semantic concepts. Finally, images are then indexed and retrieved using a novel region based inverted file data structure. Results show the proposed method has significant advantage over the widely used Bayesian annotation models.     

嵌入式图像采集系统及软件

NetSightⅡ是功能完善的嵌入式图像采集处理系统,能快速、简便地构成生产线上的机器视觉系统。     

Context-aware mobile image annotation for media search and sharing

In recent years, rapid advances in media technology including acquisition, processing and distribution have led to proliferation of many mobile applications. Amongst them, one of the emerging applications is mobile-based image annotation that uses camera phones to capture images with system-suggested tags before uploading them to the media sharing portals. This procedure can offer information to mobile users and also facilitate the retrieval and sharing of the image for Web users. However, context information that can be acquired from mobile devices is underutilized in many existing mobile image annotation systems. In this paper, we propose a new mobile image annotation system that utilizes content analysis, context analysis and their integration to annotate images acquired from mobile devices. Specifically, three types of context, location, user interaction and Web, are considered in the tagging processes. An image dataset of Nanyang Technological University (NTU) campus has been constructed, and a prototype mobile image tag suggestion system has been developed. The experimental results show that the proposed system performs well in both effectiveness and efficiency on NTU dataset, and shows good potential in domain-specific mobile image annotation for image sharing.     

Cooperative sparse representation in two opposite directions for semi-supervised image annotation

Recent studies have shown that sparse representation (SR) can deal well with many computer vision problems, and its kernel version has powerful classification capability. In this paper, we address the application of a cooperative SR in semi-supervised image annotation which can increase the amount of labeled images for further use in training image classifiers. Given a set of labeled (training) images and a set of unlabeled (test) images, the usual SR method, which we call forward SR, is used to represent each unlabeled image with several labeled ones, and then to annotate the unlabeled image according to the annotations of these labeled ones. However, to the best of our knowledge, the SR method in an opposite direction, that we call backward SR to represent each labeled image with several unlabeled images and then to annotate any unlabeled image according to the annotations of the labeled images which the unlabeled image is selected by the backward SR to represent, has not been addressed so far. In this paper, we explore how much the backward SR can contribute to image annotation, and be complementary to the forward SR. The co-training, which has been proved to be a semi-supervised method improving each other only if two classifiers are relatively independent, is then adopted to testify this complementary nature between two SRs in opposite directions. Finally, the co-training of two SRs in kernel space builds a cooperative kernel sparse representation (Co-KSR) method for image annotation. Experimental results and analyses show that two KSRs in opposite directions are complementary, and Co-KSR improves considerably over either of them with an image annotation performance better than other state-of-the-art semi-supervised classifiers such as transductive support vector machine, local and global consistency, and Gaussian fields and harmonic functions. Comparative experiments with a nonsparse solution are also performed to show that the sparsity plays an important role in the cooperation of image representations in two opposite directions. This paper extends the application of SR in image annotation and retrieval.     

Evaluating the performance in automatic image annotation: Example case by adaptive fusion of global image features

In this work we consider two traditional metrics for evaluating performance in automatic image annotation, the normalised score (NS) and the precision/recall (PR) statistics, particularly in connection with a de facto standard 5000 Corel image benchmark annotation task. We also motivate and describe another performance measure, de-symmetrised termwise mutual information (DTMI), as a principled compromise between the two traditional extremes. In addition to discussing the measures theoretically, we correlate them experimentally for a family of annotation system configurations derived from the PicSOM image content analysis framework. Looking at the obtained performance figures, we notice that such kind of a system, based on adaptive fusion of numerous global image features, clearly outperforms the considered methods in literature.     

Fast semantic diffusion for large-scale context-based image and video annotation

Exploring context information for visual recognition has recently received significant research attention. This paper proposes a novel and highly efficient approach, which is named semantic diffusion, to utilize semantic context for large-scale image and video annotation. Starting from the initial annotation of a large number of semantic concepts (categories), obtained by either machine learning or manual tagging, the proposed approach refines the results using a graph diffusion technique, which recovers the consistency and smoothness of the annotations over a semantic graph. Different from the existing graph-based learning methods that model relations among data samples, the semantic graph captures context by treating the concepts as nodes and the concept affinities as the weights of edges. In particular, our approach is capable of simultaneously improving annotation accuracy and adapting the concept affinities to new test data. The adaptation provides a means to handle domain change between training and test data, which often occurs in practice. Extensive experiments are conducted to improve concept annotation results using Flickr images and TV program videos. Results show consistent and significant performance gain (10 +% on both image and video data sets). Source codes of the proposed algorithms are available online.     

Automatic image annotation based on Gaussian mixture model considering cross-modal correlations

Automatic image annotation has been an active topic of research in the field of computer vision and pattern recognition for decades. In this paper, we present a new method for automatic image annotation based on Gaussian mixture model (GMM) considering cross-modal correlations. To be specific, we first employ GMM fitted by the rival penalized expectation-maximization (RPEM) algorithm to estimate the posterior probabilities of each annotation keyword. Next, a label similarity graph is constructed by a weighted linear combination of label similarity and visual similarity by seamlessly integrating the information from both image low level visual features and high level semantic concepts together, which can effectively avoid the phenomenon that different images with the same candidate annotations would obtain the same refinement results. Followed by the rank-two relaxation heuristics over the built label similarity graph is applied to further mine the correlation of the candidate annotations so as to capture the refining annotation results, which plays a crucial role in the semantic based image retrieval. The main contributions of this work can be summarized as follows: (1) Exploiting GMM that is trained by the RPEM algorithm to capture the initial semantic annotations of images. (2) The label similarity graph is constructed by a weighted linear combination of label similarity and visual similarity of images associated with the corresponding labels. (3) Refining the candidate set of annotations generated by the GMM through solving the max-bisection based on the rank-two relaxation algorithm over the weighted label graph. Compared to the current competitive model SGMM-RW, we can achieve significant improvements of 4% and 5% in precision, 6% and 9% in recall on the Corel5k and Mirflickr25k, respectively.     

Web image concept annotation with better understanding of tags and visual features

This paper focuses on improving the semi-manual method for web image concept annotation. By sufficiently studying the characteristics of tag and visual feature, we propose the Grouping-Based-Precision & Recall-Aided (GBPRA) feature selection strategy for concept annotation. Specifically, for visual features, we construct a more robust middle level feature by concatenating the k-NN results for each type of visual feature. For tag, we construct a concept-tag co-occurrence matrix, based on which the probability of an image belonging to certain concept can be calculated. By understanding the tags’ quality and groupings’ semantic depth, we propose a grouping based feature selection method; by studying the tags’ distribution, we adopt Precision and Recall as a complementary indicator for feature selection. In this way, the advantages of both tags and visual features are boosted. Experimental results show our method can achieve very high Average Precision, which greatly facilitates the annotation of large-scale web image dataset.     

Fuzzy VA-Files for multi-label image annotation based on visual content of regions

In this work, we propose an efficient image annotation approach based on visual content of regions. We assume that regions can be described using low-level features as well as high-level ones. Indeed, given a labeled dataset, we adopt a probabilistic semantic model to capture relationships between low-level features and semantic clusters of regions. Moreover, since most previous works on image annotation do not deal with the curse of dimensionality, we solve this problem by introducing a fuzzy version of the Vector Approximation Files (VA-Files). Indeed, the main contribution of this work resides in the association of the generative model with fuzzy VA-Files, which offer an accurate multi-dimensional indexing, to estimate relationships between low-level features and semantic concepts. In fact, the proposed approach reduces the computation complexity while optimizing the annotation quality. Preliminary experiments highlight that the suggested approach outperforms other state-of-the-art approaches.     

基于FPGA的背光图像校正系统设计

文章结合了OTSU算法和多尺度Retinex算法,在FPGA平台上设计完成了一个实时的背光图像处理系统。Retinex算法是一种广泛使用的图像增强的算法,但是受限于其复杂的运算在硬件上却没有得到很好推广。文章利用查表法和流水线的方法解决多尺度Retinex算法中的复杂运算,使其适合在FPGA上运行。Retinex算法处理的图像会存在局部增强过度颜色失真的问题,针对这一问题文章采用OTSU算法将图像分割,然后对亮部和暗部分别处理,最后再对处理后的结果融合。     

Image distance metric learning based on neighborhood sets for automatic image annotation

Since there is semantic gap between low-level visual features and high-level image semantic, the performance of many existing content-based image annotation algorithms is not satisfactory. In order to bridge the gap and improve the image annotation performance, a novel automatic image annotation (AIA) approach using neighborhood set (NS) based on image distance metric learning (IDML) algorithm is proposed in this paper. According to IDML, we can easily obtain the neighborhood set of each image since obtained image distance can effectively measure the distance between images for AIA task. By introducing NS, the proposed AIA approach can predict all possible labels of the image without caption. The experimental results confirm that the introduction of NS based on IDML can improve the efficiency of AIA approaches and achieve better annotation performance than the existing AIA approaches.     

基于智能车中摄像头的图像采集的研究

摄像头作为视觉信息采集工具被广泛运用到各种智能系统中。文中以全国大学生"飞思卡尔"杯智能车竞赛为背景。首先,通过选择CCD传感器进行路径识别,提取CCD的视频同步信号,控制单片机的A/D进行采集。由于复合视频信号中含有大量无效信息,采用外围芯片将视频信号分离,然后处理。对采集回来的视频数据进行二值化分割,去噪声处理,然后对信号进行滤波,提取黑线的中心位置,获取路径参数信息。一帧图像处理结束,根据图像的前视距离最近的黑线中心位置的偏移量和黑线斜率判断当前赛道信息,并结合预测算法,控制舵机的转向。使得小车能够保证稳定性的前提下,高速行驶。     

Graph regularized low-rank feature mapping for multi-label learning with application to image annotation

Automatic image annotation has emerged as a hot research topic in the last two decades due to its application in social images organization. Most studies treat image annotation as a typical multi-label classification problem, where the shortcoming of this approach lies in that in order to a learn reliable model for label prediction, it requires sufficient number of training images with accurate annotations. Being aware of this, we develop a novel graph regularized low-rank feature mapping for image annotation under semi-supervised multi-label learning framework. Specifically, the proposed method concatenate the prediction models for different tags into a matrix, and introduces the matrix trace norm to capture the correlations among different labels and control the model complexity. In addition, by using graph Laplacian regularization as a smooth operator, the proposed approach can explicitly take into account the local geometric structure on both labeled and unlabeled images. Moreover, considering the tags of labeled images tend to be missing or noisy, we introduce a supplementary ideal label matrix to automatically fill in the missing tags as well as correct noisy tags for given training images. Extensive experiments conducted on five different multi-label image datasets demonstrate the effectiveness of the proposed approach.     

面阵CCD自动检测系统中图像等距采集的智能实现

文章叙述了用面阵ＣＣＤ作为图像传感器在自动检测系统中的等距图像采集的工作原理，软、硬件实现方法。结果表明，系统能有效地沿运动方向定位，减少数据冗余。