The visual hull concept for silhouette-based image understanding

Many algorithms for both identifying and reconstructing a 3-D object are based on the 2-D silhouettes of the object. In general, identifying a nonconvex object using a silhouette-based approach implies neglecting some features of its surface as identification clues. The same features cannot be reconstructed by volume intersection techniques using multiple silhouettes of the object. This paper addresses the problem of finding which parts of a nonconvex object are relevant for silhouette-based image understanding. For this purpose, the geometric concept of visual hull of a 3-D object is introduced. This is the closest approximation of object S that can be obtained with the volume intersection approach; it is the maximal object silhouette-equivalent to S, i.e., which can be substituted for S without affecting any silhouette. Only the parts of the surface of S that also lie on the surface of the visual hull can be reconstructed or identified using silhouette-based algorithms. The visual hull depends not only on the object but also on the region allowed to the viewpoint. Two main viewing regions result in the external and internal visual hull. In the former case the viewing region is related to the convex hull of S, in the latter it is bounded by S. The internal visual hull also admits an interpretation not related to silhouettes. Algorithms for computing visual hulls are presented and their complexity analyzed. In general, the visual hull of a 3-D planar face object turns out to be bounded by planar and curved patches     

基于特征线条的手势识别

单目视觉的手势识别系统,通常把人手建模成一个像素或者一块,从整体上分析手势的运动参数并提取表观特征.从细微之处着手,融合颜色、运动和边缘等多种信息以提取能够反应人手结构特性的特征线条,并将特征线条分割成小的曲线段,跟踪这些曲线段的运动.采用平面模型对手势图像表观的变化建模,根据各个曲线段的运动,融合成手的整体运动分量.同时分析了图像坐标系的选取和对运动参数的影响,提出了随手运动的坐标系,以抽取平移不变的平面模型参数,进行手势识别.     

Synthesis of Silhouettes and Visual Hull Reconstruction for Articulated Humans

In this paper, we propose a complete framework for improved synthesis and understanding of the human pose from a limited number of silhouette images. It combines the active image-based visual hull (IBVH) algorithm and a contour-based body part segmentation technique. We derive a simple, approximate algorithm to decide the extrinsic parameters of a virtual camera, and synthesize the turntable image collection of the person using the IBVH algorithm by actively moving the virtual camera on a properly computed circular trajectory around the person. Using the turning function distance as the silhouette similarity measurement, this approach can be used to generate the desired pose-normalized images for recognition applications. In order to overcome the inability of the visual hull (VH) method to reconstruct concave regions, we propose a contour-based human body part localization algorithm to segment the silhouette images into convex body parts. The body parts observed from the virtual view are generated separately from the corresponding body parts observed from the input views and then assembled together for a more accurate VH reconstruction. Furthermore, the obtained turntable image collection helps to improve the body part segmentation and identification process. By using the inner distance shape context (IDSC) measurement, we are able to estimate the body part locations more accurately from a synthesized view where we can localize the body part more precisely. Experiments show that the proposed algorithm can greatly improve body part segmentation and hence shape reconstruction results.      

一种传统中国书画图像的二分类方法

传统的中国画和书法是我国的艺术瑰宝。随着数字技术的迅速发展,越来越多的传统中国书画作品被数字化,如何快速有效地检索这些数字图像已成为一个热门的研究课题。如果能够准确地将中国画和书法图像首先进行二分类,将为中国书画图像的进一步检索和分类打下坚实的基础。提出了一种基于主体颜色特征的中国传统书画图像的二分类方法。该算法首先对书画图像中的留白区域进行检测,然后将其去掉,因为历史久远,这些留白区域含有过多的噪声;其次,从处理后的书画图像中提取灰度特征作为二分类的基础;最后,利用这些特征训练分类器,并使用训练好的分类器对中国画和书法图像进行二分类。实验结果表明,该算法达到了比较理想的中国书画图像二分类结果。     

Region level annotation by fuzzy based contextual cueing label propagation

This paper investigates the challenging issue of assigning given image-level annotations to precise regions on images. We propose a novel label to region assignment (LRA) technique called Fuzzy-based Contextual-cueing Label Propagation (FCLP) with four parts: First, an image is over-segmented into a set of atomic patches and the local visual information of color features and texture features are extracted. Second, fuzzy representation and fuzzy logic are used to model spatial invariants of contextual cueing information, especially for the imprecise position information and ambiguous spatial topological relationships. Third, labels are propagated inter images and intra images in visual space and in contextual cueing space. Finally, the fuzzy C-means clustering based on K-nearest neighbor (KNN-FCM) is utilized to segment the images into semantic regions and associate with corresponding annotations. Experiments on two public datasets demonstrate the effectiveness of the proposed technique.     

GPU-based parallel construction of compact visual hull meshes

Building a visual hull model from multiple two-dimensional images provides an effective way of understanding the three-dimensional geometries inherent in the images. In this paper, we present a GPU accelerated algorithm for volumetric visual hull reconstruction that aims to harness the full compute power of the many-core processor. From a set of binary silhouette images with respective camera parameters, our parallel algorithm directly outputs the triangular mesh of the resulting visual hull in the indexed face set format for a compact mesh representation. Unlike previous approaches, the presented method extracts a smooth silhouette contour on the fly from each binary image, which markedly reduces the bumpy artifacts on the visual hull surface due to a simple binary in/out classification. In addition, it applies several optimization techniques that allow an efficient CUDA implementation. We also demonstrate that the compact mesh construction scheme can easily be modified for also producing a time- and space-efficient GPU implementation of the marching cubes algorithm.     

Carved Visual Hulls for Image-Based Modeling

This article presents a novel method for acquiring high-quality solid models of complex 3D shapes from multiple calibrated photographs. After the purely geometric constraints associated with the silhouettes found in each image have been used to construct a coarse surface approximation in the form of a visual hull, photoconsistency constraints are enforced in three consecutive steps: (1) the rims where the surface grazes the visual hull are first identified through dynamic programming; (2) with the rims now fixed, the visual hull is carved using graph cuts to globally optimize the photoconsistency of the surface and recover its main features; (3) an iterative (local) refinement step is finally used to recover fine surface details. The proposed approach has been implemented, and experiments with seven real data sets are presented, along with qualitative and quantitative comparisons with several state-of-the-art image-based-modeling algorithms.     

Reduced Depth and Visual Hulls of Complex 3D Scenes

Depth and visual hulls are useful for quick reconstruction and rendering of a 3D object based on a number of reference views. However, for many scenes, especially multi‐object, these hulls may contain significant artifacts known as phantom geometry. In depth hulls the phantom geometry appears behind the scene objects in regions occluded from all the reference views. In visual hulls the phantom geometry may also appear in front of the objects because there is not enough information to unambiguously imply the object positions. In this work we identify which parts of the depth and visual hull might constitute phantom geometry. We define the notion of reduced depth hull and reduced visual hull as the parts of the corresponding hull that are phantom‐free. We analyze the role of the depth information in identification of the phantom geometry. Based on this, we provide an algorithm for rendering the reduced depth hull at interactive frame‐rates and suggest an approach for rendering the reduced visual hull. The rendering algorithms take advantage of modern GPU programming techniques. Our techniques bypass explicit reconstruction of the hulls, rendering the reduced depth or visual hull directly from the reference views.     

Vessel Inspection: A Micro-Aerial Vehicle-based Approach

Vessel maintenance entails periodic visual inspections of the internal and external parts of the hull in order to detect the typical defective situations affecting metallic structures, such as coating breakdown, corrosion, cracks, etc. The main goal of project MINOAS is the automation of the inspection process, currently undertaken by human surveyors, by means of a fleet of robotic agents. This paper overviews an approach to the inspection problem based on an autonomous Micro Aerial Vehicle (MAV) which, as part of this fleet, is in charge of regularly supplying images that can teleport the surveyor from a base station to the areas of the hull to be inspected. The control software approach adopted for the MAV is fully described, with a special emphasis on the self-localization capabilities of the vehicle. Experimental results showing the suitability of the platform to the application are as well reported and discussed.     

Three-dimensional Visual Mapping of Underwater Ship Hull Surface Using Piecewise-planar SLAM

In-water visual ship hull inspection using unmanned underwater vehicles needs to be performed at very close range to the target surface because of the visibility limitations in underwater environments mainly due to light attenuation, scattering, and water turbidity. These environmental challenges result in ineffective photometric and geometric information in hull surface images and, therefore, the performance of conventional three-dimensional (3D) reconstruction techniques is often unsatisfactory. This paper addresses a visual mapping method for 3D reconstruction of underwater ship hull surface using a monocular camera as a primary mapping sensor. The main idea of the proposed approach is to model the moderately curved hull surface as a combination of piecewise-planar panels, and to generate a global map by aligning the local images in a two-dimensional reference frame and correcting them appropriately to reflect the information of perspective projections of the 3D panels. The estimated 3D panels associated with the local images are used to extract the loop-closure relative measurements in the framework of simultaneous localization and mapping (SLAM) for precise camera trajectory estimation and 3D reconstruction results. The validity and practical feasibility of the proposed method are demonstrated using a dataset obtained in a field experiment with a full-scale ship in a real sea environment.

     

Two- and three-dimensional image reconstructions from stained and autoradiographed histological sections

A complete system has been developed to utilize histological serial sections for two- and three-dimensional image reconstructions. Eighty to 120 sections are digitized using a personal computing system augmented with a imaging board and CCD camera. The image files are transmitted to a VAX computer for processing and image reconstruction, and the processed images are transmitted back to the personal computer for display and recording using a film recorder or PostScript printer. The software developed for the system allows serial sections to be placed into proper registration in a 256(3) array, 256 grey levels. Autoradiographs of the sections are obtained in the presence of appropriate standards which are used to recalibrate grey levels to represent linearly the radioactivity of each pixel in the sections and scale the values to allow maximum use of the grey scale. Starting from coronally sectioned material the system has been used to analyse and reconstruct rat nasal turbinates. In two dimensions horizontal and sagittal sections have been obtained while in three dimensions back-to-front and surface-rendered images have been constructed. Useful rendering of differential metabolic activity within an organ of complex geometry has been obtained, and there appears to be no reason why the system cannot be used for any material for which serial sectioning is appropriate.     

Noisy Image Decomposition: A New Structure,Texture and Noise Model Based on Local Adaptivity

These last few years, image decomposition algorithms have been proposed to split an image into two parts: the structures and the textures. These algorithms are not adapted to the case of noisy images because the textures are corrupted by noise. In this paper, we propose a new model which decomposes an image into three parts (structures, textures and noise) based on a local regularization scheme. We compare our results with the recent work of Aujol and Chambolle. We finish by giving another model which combines the advantages of the two previous ones.     

Incorporating object-centered sampling and Delaunay tetrahedrization for visual hull reconstruction

In this paper we present a novel shape from silhouette algorithm. For an object to be modeled, the algorithm first computes a cloud of points located on a pencil of rays and distributed evenly on the visual hull surface, inside and outside the visual hull. Then Delaunay tetrahedrization is applied to the point cloud to partition its convex hull into a set of tetrahedrons. Finally, outlier tetrahedrons are removed by tetrahedron peeling, and a mesh model of the visual hull is extracted. The algorithm is robust, free from discretization artifacts, and produces a mesh model composed of well-shaped triangles.     

A coherent computational approach to model bottom-up visual attention

Visual attention is a mechanism which filters out redundant visual information and detects the most relevant parts of our visual field. Automatic determination of the most visually relevant areas would be useful in many applications such as image and video coding, watermarking, video browsing, and quality assessment. Many research groups are currently investigating computational modeling of the visual attention system. The first published computational models have been based on some basic and well-understood human visual system (HVS) properties. These models feature a single perceptual layer that simulates only one aspect of the visual system. More recent models integrate complex features of the HVS and simulate hierarchical perceptual representation of the visual input. The bottom-up mechanism is the most occurring feature found in modern models. This mechanism refers to involuntary attention (i.e., salient spatial visual features that effortlessly or involuntary attract our attention). This paper presents a coherent computational approach to the modeling of the bottom-up visual attention. This model is mainly based on the current understanding of the HVS behavior. Contrast sensitivity functions, perceptual decomposition, visual masking, and center-surround interactions are some of the features implemented in this model. The performances of this algorithm are assessed by using natural images and experimental measurements from an eye-tracking system. Two adequate well-known metrics (correlation coefficient and Kullbacl-Leibler divergence) are used to validate this model. A further metric is also defined. The results from this model are finally compared to those from a reference bottom-up model.     

Remote sensing and field data integration in the definition of hydrothermally altered areas in vegetated terrain,central Brazil

A Landsat Thematic Mapper (TM) natural colour composite allowed the discrimination of areas of hydrothermally altered materials, even where vegetation (mainly herbaceous plants) covered portions of the terrain. Field spectra data showed that broad iron-oxide absorption features in TM 1 and TM 2 bands enabled the spectral discrimination between areas of hydrothermally altered materials and areas of soils derived from biotite-granites. In order to improve the definition of the target areas, the TM images were merged with a digitized aerial photograph through IHS technique. The resulting high resolution hybrid images were segmented using a region growing method, which generated images partitioned into a number of homogeneous regions. The segmented images were classified using an unsupervised clustering region classifier algorithm. The result, compared with field observations, demonstrated that the method eliminated the subjectivity of the visual image interpretation and increased the accuracy in the delineation of the hydrothermally altered areas.     

Invariant 2D object recognition using KRA and GRA

Computer vision has been extensively adopted in industry for the last two decades. It enhances productivity and quality management, and is flexibility, efficient, fast, inexpensive, reliable and robust. This study presents a new translation, rotation and scaling-free object recognition method for 2D objects. The proposed method comprises two parts: KRA feature extractor and GRA classifier. The KRA feature extractor employs K-curvature, re-sampling, and autocorrelation transformation to extract unique features of objects, and then gray relational analysis (GRA) classifies the extracted invariant features. The boundary of the digital object was first represented as the form of the K-curvature over a given region of support, and was then re-sampled and transformed with autocorrelation function. After that, the extracted features own the unique property that is invariant to translation, rotation and scaling. To verify and validate the proposed method, 50 synthetic and 50 real objects were digitized as standard patterns, and 10 extra images of each object (test images) which were taken at different positions, orientations and scales, were acquired and compared with the standard patterns. The experimental results reveal that the proposed method with either GRA or MD methods is effective and reliable for part recognition.     

Combining global, regional and contextual features for automatic image annotation

This paper presents a novel approach to automatic image annotation which combines global, regional, and contextual features by an extended cross-media relevance model. Unlike typical image annotation methods which use either global or regional features exclusively, as well as neglect the textual context information among the annotated words, the proposed approach incorporates the three kinds of information which are helpful to describe image semantics to annotate images by estimating their joint probability. Specifically, we describe the global features as a distribution vector of visual topics and model the textual context as a multinomial distribution. The global features provide the global distribution of visual topics over an image, while the textual context relaxes the assumption of mutual independence among annotated words which is commonly adopted in most existing methods. Both the global features and textual context are learned by a probability latent semantic analysis approach from the training data. The experiments over 5k Corel images have shown that combining these three kinds of information is beneficial in image annotation.     

Association and content-based retrieval

In spite of important efforts in content-based indexing and retrieval during these last years, seeking relevant and accurate images remains a very difficult query. In the state-of-the-art approaches, the retrieval task may be efficient for some queries in which the semantic content of the query can be easily translated into visual features. For example, finding images of fires is simple because fires are characterized by specific colors (yellow and red). However, it is not efficient in other application fields in which the semantic content of the query is not easily translated into visual features. For example, finding images of birds during migrations is not easy because the system has to understand the query semantic. In the query, the basic visual features may be useful (a bird is characterized by a texture and a color), but they are not sufficient. What is missing is the generalization capability. Birds during migrations belong to the same repository of birds, so they share common associations among basic features (e.g., textures and colors) that the user cannot specify explicitly. We present an approach that discovers hidden associations among features during image indexing. These associations discriminate image repositories. The best associations are selected on the basis of measures of confidence. To reduce the combinatory explosion of associations, because images of the database contain very large numbers of colors and textures, we consider a visual dictionary that group together similar colors and textures.