Improved scene classification using efficient low-level features and semantic cues

Prior research in scene classification has focused on mapping a set of classic low-level vision features to semantically meaningful categories using a classifier engine. In this paper, we propose improving the established paradigm by using a simplified low-level feature set to predict multiple semantic scene attributes that are integrated probabilistically to obtain a final indoor/outdoor scene classification. An initial indoor/outdoor prediction is obtained by classifying computationally efficient, low-dimensional color and wavelet texture features using support vector machines. Similar low-level features can also be used to explicitly predict the presence of semantic features including grass and sky. The semantic scene attributes are then integrated using a Bayesian network designed for improved indoor/outdoor scene classification.     

Semantic image segmentation using low-level features and contextual cues

Semantic image segmentation aims to partition an image into non-overlapping regions and assign a pre-defined object class label to each region. In this paper, a semantic method combining low-level features and high-level contextual cues is proposed to segment natural scene images. The proposed method first takes the gist representation of an image as its global feature. The image is then over-segmented into many super-pixels and histogram representations of these super-pixels are used as local features. In addition, co-occurrence and spatial layout relations among object classes are exploited as contextual cues. Finally the features and cues are integrated into the inference framework based on conditional random field by defining specific potential terms and introducing weighting functions. The proposed method has been compared with state-of-the-art methods on the MSRC database, and the experimental results show its effectiveness.     

Detecting image orientation based on low-level visual content

Accurately and automatically detecting image orientation is of great importance in intelligent image processing. In this paper, we present automatic image orientation detection algorithms based on both the luminance (structural) and chrominance (color) low-level content features. The statistical learning support vector machines (SVMs) are used in our approach as the classifiers. The different sources of the extracted image features, as well as the binary classification nature of SVM, require our system to be able to integrate the outputs from multiple classifiers. Both static combiner (averaging) and trainable combiner (also based on SVMs) are proposed and evaluated in this work. Furthermore, two rejection options (regular and re-enforced ambiguity rejections) are employed to improve orientation detection accuracy by sieving out images with low confidence values during the classification. Large amounts of experiments have been conducted on a database of more than 14,000 images to validate our approaches. Discussions and future directions for this work are also addressed at the end of the paper.     

Automatic content based image retrieval using semantic analysis

We present a new text-to-image re-ranking approach for improving the relevancy rate in searches. In particular, we focus on the fundamental semantic gap that exists between the low-level visual features of the image and high-level textual queries by dynamically maintaining a connected hierarchy in the form of a concept database. For each textual query, we take the results from popular search engines as an initial retrieval, followed by a semantic analysis to map the textual query to higher level concepts. In order to do this, we design a two-layer scoring system which can identify the relationship between the query and the concepts automatically. We then calculate the image feature vectors and compare them with the classifier for each related concept. An image is relevant only when it is related to the query both semantically and content-wise. The second feature of this work is that we loosen the requirement for query accuracy from the user, which makes it possible to perform well on users’ queries containing less relevant information. Thirdly, the concept database can be dynamically maintained to satisfy the variations in user queries, which eliminates the need for human labor in building a sophisticated initial concept database. We designed our experiment using complex queries (based on five scenarios) to demonstrate how our retrieval results are a significant improvement over those obtained from current state-of-the-art image search engines.     

场景语义树图像标注方法

自动图像标注是一项具有挑战性的工作,它对于图像分析理解和图像检索都有着重要的意义.在自动图像标注领域,通过对已标注图像集的学习,建立语义概念空间与视觉特征空间之间的关系模型,并用这个模型对未标注的图像集进行标注.由于低高级语义之间错综复杂的对应关系,使目前自动图像标注的精度仍然较低.而在场景约束条件下可以简化标注与视觉特征之间的映射关系,提高自动标注的可靠性.因此提出一种基于场景语义树的图像标注方法.首先对用于学习的标注图像进行自动的语义场景聚类,对每个场景语义类别生成视觉场景空间,然后对每个场景空间建立相应的语义树.对待标注图像,确定其语义类别后,通过相应的场景语义树,获得图像的最终标注.在Corel5K图像集上,获得了优于TM(translation model)、CMRM(cross media relevance model)、CRM(continous-space relevance model)、PLSA-GMM(概率潜在语义分析-高期混合模型)等模型的标注结果.     

Single image face orientation and gaze detection

We introduce a system to compute both head orientation and gaze detection from a single image. The system uses a camera with fixed parameters and requires no user calibration. Our approach to head orientation is based on a geometrical model of the human face, and is derived form morphological and physiological data. Eye gaze detection is based on a geometrical model of the human eye. Two new algorithms are introduced that require either two or three feature points to be extracted from each image. Our algorithms are robust and run in real-time on a typical PC, which makes our system useful for a large variety of needs, from driver attention monitoring to machine-human interaction.     

Extracting image orientation feature by using integration operator

This paper presents an orientation operator to extract image local orientation features. We show that a proper employment of image integration leads to an unbiased orientation estimate, based on which an orientation operator is proposed. The resulting discrete operator has flexibility in the scale selection as the scale change does not violate the bias minimization criteria. An analytical formula is developed to compare orientation biases of various discrete operators. The proposed operator shows lower bias than eight well-known gradient operators. Experiments further demonstrate higher orientation accuracy of the proposed operator than these gradient operators.     

Automatic image semantic interpretation using social action and tagging data

The plethora of social actions and annotations (tags, comments, ratings) from online media sharing Websites and collaborative games have induced a paradigm shift in the research on image semantic interpretation. Social inputs with their added context represent a strong substitute for expert annotations. Novel algorithms have been designed to fuse visual features with noisy social labels and behavioral signals. In this survey, we review nearly 200 representative papers to identify the current trends, challenges as well as opportunities presented by social inputs for research on image semantics. Our study builds on an interdisciplinary confluence of insights from image processing, data mining, human computer interaction, and sociology to describe the folksonomic features of users, annotations and images. Applications are categorized into four types: concept semantics, person identification, location semantics and event semantics. The survey concludes with a summary of principle research directions for the present and the future.     

部件检测和语义网络的细粒度鞋类图像检索

目的 细粒度图像检索是当前细粒度图像分析和视觉领域的热点问题。以鞋类图像为例,传统方法仅提取其粗粒度特征且缺少关键的语义属性,难以区分部件间的细微差异,不能有效用于细粒度检索。针对鞋类图像检索大多基于简单款式导致检索效率不高的问题,提出一种结合部件检测和语义网络的细粒度鞋类图像检索方法。方法 结合标注后的鞋类图像训练集对输入的待检鞋类图像进行部件检测;基于部件检测后的鞋类图像和定义的语义属性训练语义网络,以提取待检图像和训练图像的特征向量,并采用主成分分析进行降维;通过对鞋类图像训练集中每个候选图像与待检图像间的特征向量进行度量学习,按其匹配度高低顺序输出检索结果。结果 实验在UT-Zap50K数据集上与目前检索效果较好的4种方法进行比较,检索精度提高近6%。同时,与同任务的SHOE-CNN（semantic hierarchy of attribute convolutional neural network）检索方法比较,本文具有更高的检索准确率。结论 针对传统图像特征缺少细微的视觉描述导致鞋类图像检索准确率低的问题,提出一种细粒度鞋类图像检索方法,既提高了鞋类图像检索的精度和准确率,又能较好地满足实际应用需求。     

Concept-concept association information integration and multi-model collaboration for multimedia semantic concept detection

The recent development of the digital camera technology and the popularity of social network websites such as Facebook and Youtube have created huge amounts of multimedia data. Multimedia information is ubiquitous and essential in many applications. In order to fill the gap between data and application requirements (or the so-called semantic gap), advanced methods and tools are needed to automatically mine and annotate high-level concepts to assist in associating the low-level features to the high-level concepts directly. It has been shown that concept-concept association can be effective in bridging the semantic gap in multimedia data. In this paper, a concept-concept association information integration and multi-model collaboration framework is proposed to enhance high-level semantic concept detection from multimedia data. Several experiments are conducted and the comparison results demonstrate that the proposed framework outperforms those approaches in the comparison in terms of the Mean Average Precision (MAP) values.     

Democratic integration: self-organized integration of adaptive cues

Sensory integration or sensor fusion -- the integration of information from different modalities, cues, or sensors -- is among the most fundamental problems of perception in biological and artificial systems. We propose a new architecture for adaptively integrating different cues in a self-organized manner. In Democratic Integration different cues agree on a result, and each cue adapts toward the result agreed on. In particular, discordant cues are quickly suppressed and recalibrated, while cues having been consistent with the result in the recent past are given a higher weight in the future. The architecture is tested in a face tracking scenario. Experiments show its robustness with respect to sudden changes in the environment as long as the changes disrupt only a minority of cues at the same time, although all cues may be disrupted at one time or another.     

多态细菌趋药性的传感器图像自动配准

传统的图像配准的相似性测度函数对噪声过于敏感,且需要先验知识约束。对此加以改进,提出一种新的相似性测度模型。为了对模型求解,引入一种新的优化算法——细菌趋药性算法,并对其做出改进,得到多态细菌趋药性算法。实验表明,修正的相似性测度模型对噪声免疫;同时多态细菌趋药性算法比精英遗传算法、蚁群算法、粒子群算法、细菌群体趋药性算法等收敛更快,且能以更大概率收敛到全局最优。     

Kernel-based semantic relation detection and classification via enriched parse tree structure

This paper proposes a tree kernel method of semantic relation detection and classification(RDC) between named entities.It resolves two critical problems in previous tree kernel methods of RDC.First,a new tree kernel is presented to better capture the inherent structural information in a parse tree by enabling the standard convolution tree kernel with context-sensitiveness and approximate matching of sub-trees.Second,an enriched parse tree structure is proposed to well derive necessary structural informat...     

伪随机编码图像的特征点自动检测方法

针对伪随机阵列编码的彩色结构光系统,提出了一种角点自动检测、识别方法。它首先利用聚类分析算法,确定摄像机图像角点和编码模板图像的特征点匹配关系,实现编码图像的识码、解码;然后获取编码图像中有效的、非冗余的角点,并自动输出角点的行列数目及其位置坐标,为亚像素角点检测提供了充分的输入数据。最后,将该算法应用于自主开发的彩色结构光系统上,并用实验检验,结果表明该方法准确、有效,有助于结构光系统的自动化标定。     

基于语料库的潜语义信息度量

为关键词定义了与主题或语义相关联的信息度量.首先获取基于主题的语料库,然后建立语料库的潜语义向量空间模型,通过该模型定义关键词的信息度量.由此可以计算任意文档包含该主题的信息量,定义文档对主题的隶属度.设定文档对主题隶属度阈值,从而判断文档是否属于该主题类.实验表明,与主题或语义关联的信息度量可以克服搜索中"词匹配"的不足,达到"语义匹配"的搜索.     

一种情感词语义加权的句子倾向性识别方法

互联网上大量的主观评论性信息蕴含着巨大的商业价值,同时也促使了倾向性识别研究的兴起。句子倾向性识别是文本倾向性识别的基础,现有句子倾向性识别方法存在着识别效果不理想、模式抽取困难等问题。将情感词视为基因,在不同的语境下呈现出不同的性状,通过构建情感词语义倾向分析器,先确定情感词的静态显性,然后根据不同的语境确定情感词的动态显性,最后提出基于情感词语义加权的句子倾向性识别算法。实验结果显示,该方法提高了句子倾向性识别的判全率和判准率,是合理和有效的。