Distance metric between 3D models and 2D images for recognition andclassification

Similarity measurements between 3D objects and 2D images are useful for the tasks of object recognition and classification. The authors distinguish between two types of similarity metrics: metrics computed in image-space (image metrics) and metrics computed in transformation-space (transformation metrics). Existing methods typically use image metrics; namely, metrics that measure the difference in the image between the observed image and the nearest view of the object. Example for such a measure is the Euclidean distance between feature points in the image and their corresponding points in the nearest view. (This measure can be computed by solving the exterior orientation calibration problem.) In this paper the authors introduce a different type of metrics: transformation metrics. These metrics penalize for the deformations applied to the object to produce the observed image. In particular, the authors define a transformation metric that optimally penalizes for “affine deformations” under weak-perspective. A closed-form solution, together with the nearest view according to this metric, are derived. The metric is shown to be equivalent to the Euclidean image metric, in the sense that they bound each other from both above and below. It therefore provides an easy-to-use closed-form approximation for the commonly-used least-squares distance between models and images. The authors demonstrate an image understanding application, where the true dimensions of a photographed battery charger are estimated by minimizing the transformation metric     

Surface reconstruction by multiaxial triangulation

Outlines for reconstructing object surfaces are traditionally drawn from sequential images in parallel planes. The method presented here instead supports complex object topologies by drawing contours from multiaxial image planes. Multiaxial triangulation of an object in a given data volume involves four steps. First, the user generates contours interactively by selecting sample planes inside the data volume, then drawing object contours from the image corresponding to this sample plane. Our algorithm for multiaxial triangulation then processes these contours to verify consistency within and between sample planes. Second, it uses the sample planes containing the contours to partition the data volume into a divided volume. The contours are partitioned against the plane boundaries, and the contour parts (chains) are associated with faces, edges, and vertices in the divided volume. Third, these chains are joined into closed loops in the divided volume. Fourth, the loops are triangulated patchwise to create the surface model     

融合直接度量和间接度量的行人再识别

当前行人再识别的度量算法在计算相似性时主要依据两幅图像自身的判别信息(直接度量),较少依据与两幅图像相关的其它图像的判别信息(间接度量).针对此种情况,文中提出加权融合直接度量和间接度量的度量方法.首先提取图像的局部最大概率特征和突出性颜色名称特征,融合两者作为图像的最终特征.然后分别计算两幅图像的直接相似性和间接相似性,利用序列排序方法对数据库样本进行训练,得到权值参数,从而得到两幅图像的最终相似性.在Market-1501数据库和CUHK03数据库上的实验表明,融合后的度量识别能力明显高于单个度量的识别能力.     

Accelerated Backward Warping

In this paper a plane-based backward warping algorithm is proposed to generate novel viwews from multiple reference images.First.depth information is employed to reconstruct space planes from individual refernce images and calculate the potential occluding relationship between these planes.Then the planes which represent each identical space plane from different reference images are compared with each other to decide the one with the best sample rate to be preserved and used in the later warping period while the other samples are abandoned.While the image of a novel view is produced,traditional methods in computer graphics,such as visibility test.and clipping,are used to process the planes reconstructed.Then the planes processed are projected onto the desired image form the knowledge on which plane the desired image pixels are warped from can be acquired.Finally,pixels‘ depth of the desired image is calculated and then a ackwared warping is performed from these pixels to the reference images to obtain their colors.The storage requirement in the algorithm is small and increases slowly with the number of reference images increases.By combining the strategy of only preserving the best sample parts and the backward warping algorithm ,the sample problem could be well tackled.     

Graph-based representation for similarity retrieval of symbolic images

Image retrieval from an image database by the image objects and their spatial relationships has emerged as an important research subject in these decades. To retrieve images similar to a given query image, retrieval methods must assess the similarity degree between a database image and the query image by the extracted features with acceptable efficiency and effectiveness. This paper proposes a graph-based model SRG (spatial relation graph) to represent the semantic information of the contained objects and their spatial relationships in an image with no file annotation. In an SRG graph, the image objects are symbolized by the predefined class names as vertices and the spatial relations between object pairs are represented as arcs. The proposed model assesses the similarity degree between two images by calculating the maximum common subgraph of two corresponding SRG’s through intersection, which has quadratic time complexity owing to the characteristics of SRG. Its efficiency remains quadratic regardless of the duplication rate of the object symbols. The extended model SRG_T is also proposed, with the same time complexity, for the applications that need to consider the topological relations among objects. A synthetic symbolic image database and an existing image dataset are used in the conducted experiments to verify the performance of the proposed models. The experimental results show that the proposed models have compatible retrieval quality with remarkable efficiency improvements compared with three well-known methods LCS_Clique, SIM_R, and 2D Be-string, where LCS_Clique utilizes the number of objects in the maximum common subimage as its similarity function, SIM_R uses accumulation-based similarity function of similar object pairs, and 2D Be-string calculates the similarity of 2D patterns by the linear combination of two 1D similarities.     

基于平面的Warping技术

提出了一种基于平面的逆向Warping算法,用于根据多幅参考图像生成任意视点下的新图像.首先通过参考图像的深度信息来重建三维平面,然后寻找这些重建平面间的对应关系,并比较它们对空间平面同一部分的采样密度,以获得最好的采样结果.在生成新视点图像时,首先对那些采样密度最好的重建平面进行可见性判断,然后将其投影到新视点下,在此基础上求得目标图像上各点的深度,最后将目标图像上的点逆向Warping到相应的参考图像中,以获取它们的颜色值.同时,对于参考图像中不能被重建成平面的像素点,用正向Warping的方法对其进     

Detection and recognition of contour parts based on shape similarity

Due to distortion, noise, segmentation errors, overlap, and occlusion of objects in digital images, it is usually impossible to extract complete object contours or to segment the whole objects. However, in many cases parts of contours can be correctly reconstructed either by performing edge grouping or as parts of boundaries of segmented regions. Therefore, recognition of objects based on their contour parts seems to be a promising as well as a necessary research direction.The main contribution of this paper is a system for detection and recognition of contour parts in digital images. Both detection and recognition are based on shape similarity of contour parts. For each contour part produced by contour grouping, we use shape similarity to retrieve the most similar contour parts in a database of known contour segments. A shape-based classification of the retrieved contour parts performs then a simultaneous detection and recognition.An important step in our approach is the construction of the database of known contour segments. First complete contours of known objects are decomposed into parts using discrete curve evolution. Then, their representation is constructed that is invariant to scaling, rotation, and translation.     

Learning similarity measure for natural image retrieval with relevance feedback

A new scheme of learning similarity measure is proposed for content-based image retrieval (CBIR). It learns a boundary that separates the images in the database into two clusters. Images inside the boundary are ranked by their Euclidean distances to the query. The scheme is called constrained similarity measure (CSM), which not only takes into consideration the perceptual similarity between images, but also significantly improves the retrieval performance of the Euclidean distance measure. Two techniques, support vector machine (SVM) and AdaBoost from machine learning, are utilized to learn the boundary. They are compared to see their differences in boundary learning. The positive and negative examples used to learn the boundary are provided by the user with relevance feedback. The CSM metric is evaluated in a large database of 10009 natural images with an accurate ground truth. Experimental results demonstrate the usefulness and effectiveness of the proposed similarity measure for image retrieval.     

一种基于对象空间关系的图像检索模型

介绍一种基于空间相似的图像数据库检索模型，提出了空间相似度计算的算法，对对象的空间方位、拓扑关系、数量和空间关系局部高度相似等因素都作了全面考虑，详细阐述空间相似度与局部相似对象簇的求解过程，建立的模型不仅能识别经平移、缩放和旋转交换的图像，而且能识别图像的对称变换，通过实验验证，表明检索模型的图像搜索效率高。     

邻域排斥稀疏判决单样本亲属关系认证算法

针对如何利用人脸图像进行亲属关系认证问题,提出邻域排斥测度学习稀疏判决的单样本亲属关系认证算法。学习能使具有亲属关系样本距离变小,而非亲属关系样本距离变大的变换矩阵,目的是利用已有数据样本间相似程度的先验知识学习最佳相似性度量,使之能更好地刻画亲属样本间的相似关系。在新的测度空间下采用稀疏表示方法用父母样本集建立过完备字典来线性表示子女图像,并以稀疏系数大小衡量样本间相似程度。针对亲属样本间相似性不明显问题提出子模块综合稀疏认证方法,通过多重稀疏系数综合判别两输入样本的亲属关系。在KinFaceW-I和KinFaceW-II两个亲属图像库上的实验结果表明,采用测度学习空间下稀疏系数判决的方法相比已有亲属关系人脸认证方法具有更好的性能。     

Using an image-extended relational database to support content-based image retrieval in a PACS

This paper presents a new Picture Archiving and Communication System (PACS), called cbPACS, which has content-based image retrieval capabilities. The cbPACS answers range and k-nearest- neighbor similarity queries, employing a relational database manager extended to support images. The images are compared through their features, which are extracted by an image-processing module and stored in the extended relational database. The database extensions were developed aiming at efficiently answering similarity queries by taking advantage of specialized indexing methods. The main concept supporting the extensions is the definition, inside the relational manager, of distance functions based on features extracted from the images. An extension to the SQL language enables the construction of an interpreter that intercepts the extended commands and translates them to standard SQL, allowing any relational database server to be used. By now, the system implemented works on features based on color distribution of the images through normalized histograms as well as metric histograms. Metric histograms are invariant regarding scale, translation and rotation of images and also to brightness transformations. The cbPACS is prepared to integrate new image features, based on texture and shape of the main objects in the image.     

Content-based Access in Image Database by Quantitative Relationships

In this paper, we describe a novel technique to perform content-based access in image databases using quantitative spatial relationships. Usually, spatial relation-based indexing methods fail if the metric spatial information contained in the images must be preserved. In order to provide a more robust approach to directional relations indexing with respect to metric differences in images, this paper introduces an improvement of the virtual image index, namely quantitative virtual image, using a quantitative methodology. A scalar quantitative measure is associated with each spatial relation, in order to discriminate among images of the image database having the same objects and spatial relationships, but different degree of similarity if we also consider distance relationships. The measure we introduce does not correspond to any significant increase of complexity with respect to the standard virtual image providing a more precise answer set.     

未标定单幅结构场景图像的三维重构

提出了一种单视三维重构方法，该方法是利用用户提供图像点及其对应的三维点之间几何信息。由于结构场景是由大量平面构成的，存在大量的平行性、正交性约束，因此该方法主要应用于结构场景的三维重构。首先，相机定标和计算每个平面的度量信息，即先基于3组互相垂直方向的影灭点，对方形像素相机标定，再利用影灭线和圆环点像，对每个平面度量校正；然后考虑每个校正平面的尺度因子和非正交平面间的相对面向，从而将所有校正后的平面缝合起来。采用真实图像进行实验，实验结果表明，该方法简单易用。     

Improved Evaluation and Generation Of Grid Layouts Using Distance Preservation Quality and Linear Assignment Sorting

Images sorted by similarity enables more images to be viewed simultaneously, and can be very useful for stock photo agencies or e-commerce applications. Visually sorted grid layouts attempt to arrange images so that their proximity on the grid corresponds as closely as possible to their similarity. Various metrics exist for evaluating such arrangements, but there is low experimental evidence on correlation between human perceived quality and metric value. We propose distance preservation quality (DPQ) as a new metric to evaluate the quality of an arrangement. Extensive user testing revealed stronger correlation of DPQ with user-perceived quality and performance in image retrieval tasks compared to other metrics. In addition, we introduce Fast linear assignment sorting (FLAS) as a new algorithm for creating visually sorted grid layouts. FLAS achieves very good sorting qualities while improving run time and computational resources.     

Mobile multi-view object image search

High user interaction capability of mobile devices can help improve the accuracy of mobile visual search systems. At query time, it is possible to capture multiple views of an object from different viewing angles and at different scales with the mobile device camera to obtain richer information about the object compared to a single view and hence return more accurate results. Motivated by this, we propose a new multi-view visual query model on multi-view object image databases for mobile visual search. Multi-view images of objects acquired by the mobile clients are processed and local features are sent to a server, which combines the query image representations with early/late fusion methods and returns the query results. We performed a comprehensive analysis of early and late fusion approaches using various similarity functions, on an existing single view and a new multi-view object image database. The experimental results show that multi-view search provides significantly better retrieval accuracy compared to traditional single view search.     

Visual image retrieval by elastic matching of user sketches

Effective image retrieval by content from database requires that visual image properties are used instead of textual labels to properly index and recover pictorial data. Retrieval by shape similarity, given a user-sketched template is particularly challenging, owing to the difficulty to derive a similarity measure that closely conforms to the common perception of similarity by humans. In this paper, we present a technique which is based on elastic matching of sketched templates over the shapes in the images to evaluate similarity ranks. The degree of matching achieved and the elastic deformation energy spent by the sketch to achieve such a match are used to derive a measure of similarity between the sketch and the images in the database and to rank images to be displayed. The elastic matching is integrated with arrangements to provide scale invariance and take into account spatial relationships between objects in multi-object queries. Examples from a prototype system are expounded with considerations about the effectiveness of the approach and comparative performance analysis     

Dynamic learning from multiple examples for semantic object segmentation and search

We present a novel “dynamic learning” approach for an intelligent image database system to automatically improve object segmentation and labeling without user intervention, as new examples become available, for object-based indexing. The proposed approach is an extension of our earlier work on “learning by example,” which addressed labeling of similar objects in a set of database images based on a single example. The proposed dynamic learning procedure utilizes multiple example object templates to improve the accuracy of existing object segmentations and labels. Multiple example templates may be images of the same object from different viewing angles, or images of related objects. This paper also introduces a new shape similarity metric called normalized area of symmetric differences (NASD), which has desired properties for use in the proposed “dynamic learning” scheme, and is more robust against boundary noise that results from automatic image segmentation. Performance of the dynamic learning procedures has been demonstrated by experimental results.     

Efficient extraction of metric measurements for planar scene under 2D homography with the help of planar circles

Extraction of metric properties from perspective view is a challenging task in many machine vision applications. Most conventional approaches typically first recover the perspective transformation parameters up to a similarity transform and make measurements in the resulting rectified image. In this paper, a new approach is proposed to allow quick and reliable Euclidean measures to be made directly from a perspective view without explicitly recovering the world plane. Unlike previous planar rectification strategies, our approach makes use of planar circles to help identify the image of the absolute conic, which makes it capable of performing effective rectification under many difficult cases that are unable to be treated with other rectification approaches. This is made possible by solving the images of the circular points in closed-form from the vanishing line and the image of one arbitrary planar circle and by exploiting the invariant relationship between the circular points and the absolute conic under projective transformation. Subsequently, planar Euclidean measures can be made directly from the image plane. The practical advantages and the efficiency of this method are demonstrated by experiments on both synthetic and real scenes.     

图像检索中的动态相似性度量方法

为提高图像检索的效率,近年来相关反馈机制被引入到了基于内容的图像检索领域。该文提出了一种新的相关反馈方法－－动态相似性度量方法。该方法建立在目前被广泛采用的图像相拟性度量方法的基础上,结合了相关反馈图像检索系统的时序特性,通过捕获用户的交互信息,动态地修正图像的相似性度量公式,从而把用户模型嵌入到了图像检索系统,在某种程度上使图像检索结果与人的主观感知更加接近。实验结果表明该方法的性能明显优于其它图像检索系统所采用的方法。     

基于单像视觉的平面透视影像度量纠正方法

该文针对基于单幅影像的建筑物三维重建,提出一种对平面的透视影像进行度量纠正的方法。利用建筑物立面上的平行线作为约束条件,计算相应平面的透视影像上的灭点、灭线、投影中心及其他特征点线,以二维直接线性变换表达平面的透视影像与其平行影像间的变换关系,对建筑物立面影像进行纠正,通过纠正生成具有度量特性的平行影像。文章重点阐述了生成平行影像的原理和算法,同时也介绍了获取平行影像的长度比例的方法,并进行了算法的实现和实验,最后根据实验结果进行了误差分析。