一种处理部分遮挡表情图像的方法

针对目前表情识别中眼部因头发、帽子等物体而存在部分遮挡的问题,提出了一种基于对称变换的眼部遮挡处理方法。方法针对二值化的人脸表情图像,参照人脸几何特征对眼部区域进行垂直积分投影;通过中心线检测算法,确定人脸的中心线并判断是否存在遮挡。对于不可容忍的遮挡,进行对称变换处理以修复表情图像。实验表明,在相同的特征提取方法和分类器选择情况下,该方法可有效提高部分遮挡人脸表情的识别效果,并可容忍头部一定范围内的偏转。     

Recovering intrinsic images from a single image

Interpreting real-world images requires the ability distinguish the different characteristics of the scene that lead to its final appearance. Two of the most important of these characteristics are the shading and reflectance of each point in the scene. We present an algorithm that uses multiple cues to recover shading and reflectance intrinsic images from a single image. Using both color information and a classifier trained to recognize gray-scale patterns, given the lighting direction, each image derivative is classified as being caused by shading or a change in the surface's reflectance. The classifiers gather local evidence about the surface's form and color, which is then propagated using the Generalized Belief Propagation algorithm. The propagation step disambiguates areas of the image where the correct classification is not clear from local evidence. We use real-world images to demonstrate results and show how each component of the system affects the results.     

Three-dimensional object recognition from single two-dimensional images

A computer vision system has been implemented that can recognize three-dimensional objects from unknown viewpoints in single gray-scale images. Unlike most other approaches, the recognition is accomplished without any attempt to reconstruct depth information bottom-up from the visual input. Instead, three other mechanisms are used that can bridge the gap between the two-dimensional image and knowledge of three-dimensional objects. First, a process of perceptual organization is used to form groupings and structures in the image that are likely to be invariant over a wide range of viewpoints. Second, a probabilistic ranking method is used to reduce the size of the search space during model-based matching. Finally, a process of spatial correspondence brings the projections of three-dimensional models into direct correspondence with the image by solving for unknown viewpoint and model parameters. A high level of robustness in the presence of occlusion and missing data can be achieved through full application of a viewpoint consistency constraint. It is argued that similar mechanisms and constraints form the basis for recognition in human vision.     

Estimating the surface radiance function from single images

This paper describes a simple method for estimating the surface radiance function from single images of smooth surfaces made of materials whose reflectance function is isotropic and monotonic. The method makes implicit use of the Gauss map between the surface and a unit sphere. We assume that the material brightness is monotonic with respect to the angle between the illuminant direction and the surface normal. Under conditions in which the light source and the viewer directions are identical, we show how a tabular representation of the surface radiance function can be estimated using the cumulative distribution of image gradients. Using this tabular representation of the radiance function, surfaces may be rendered under varying light source direction by rotating the corresponding reflectance map on the Gauss sphere about the specular spike direction. We present a sensitivity study on synthetic and real-world imagery. We also present two applications which make use of the estimated radiance function. The first of these illustrates how the radiance function estimates can be used to render objects when the light and viewer directions are no longer coincident. The second application involves applying corrected Lambertian radiance to rough and shiny surfaces.     

Tracking multiple people with recovery from partial and total occlusion

Robust tracking of multiple people in video sequences is a challenging task. In this paper, we present an algorithm for tracking faces of multiple people even in cases of total occlusion. Faces are detected first; then a model for each person is built. The models are handed over to the tracking module which is based on the mean shift algorithm, where each face is represented by the non-parametric distribution of the colors in the face region. The mean shift tracking algorithm is robust to partial occlusion and rotation, and is computationally efficient, but it does not deal with the problem of total occlusion. Our algorithm overcomes this problem by detecting the occlusion using an occlusion grid, and uses a non-parametric distribution of the color of the occluded person's cloth to distinguish that person after the occlusion ends. Our algorithm uses the speed and the trajectory of each occluded person to predict the locations that should be searched after occlusion ends. It integrates multiple features to handle tracking multiple people in cases of partial and total occlusion. Experiments on a large set of video clips demonstrate the robustness of the algorithm, and its capability to correctly track multiple people even when faces are temporarily occluded by other faces or by other objects in the scene.     

Curve segmentation under partial occlusion

2-D shape boundary segmentation is required as a fundamental and important stage in the recognition of partially occluded objects. We introduce here a new segmentation method capable of extracting a controlled number of segments along a smooth boundary curve. This new approach is invariant to similarity transformation, and partial occlusion has only marginal influence on the segmentation of the visible part. The basic concept is to transform the curve into another one which intersects itself. Points of intersection of the new curve are re-transformed to the original curve and serve as endpoints of segments. Properties of the transform are discussed, and conditions for existence of intersection points are given. Simulation results of gray level images are presented, and advantages of our method over conventional approaches relying on singular points of the curvature are discussed     

Model-based recognition of 3D objects from single images

In this work, we treat major problems of object recognition which have received relatively little attention lately. Among them are the loss of depth information in the projection from a 3D object to a single 2D image, and the complexity of finding feature correspondences between images. We use geometric invariants to reduce the complexity of these problems. There are no geometric invariants of a projection from 3D to 2D. However, given certain modeling assumptions about the 3D object, such invariants can be found. The modeling assumptions can be either a particular model or a generic assumption about a class of models. Here, we use such assumptions for single-view recognition. We find algebraic relations between the invariants of a 3D model and those of its 2D image under general projective projection. These relations can be described geometrically as invariant models in a 3D invariant space, illuminated by invariant “light rays,” and projected onto an invariant version of the given image. We apply the method to real images     

Removal of achromatic reflections from a single color image

In this paper we consider the problem of removing achromatic reflections from a picture of a scene taken through a semi-transparent medium, assuming that the reflection pattern is due to a light source or another object located in front of the object of interest. While other works assume the availability of multiple observations, we consider the more challenging problem of having as data a single color image. We suppose a data model where the virtual reflected image combines additively with the real transmitted image of the object, through unknown coefficients. This highly underdetermined problem is handled by means of a blind estimation technique that exploits the strict dependence of the gradients of the three color channels of the ideal image, and their independence from the gradient of the grayscale reflected image. The model parameters are estimated through independent component analysis, and then the component images are estimated through a regularization technique. The whole algorithm is very fast, and its performance is quantitatively evaluated on numerically generated images, and qualitatively tested on real images.     

Restoration of brick and stone relief from single rubbing images

We present a two-level approach for height map estimation from single images, aiming at restoring brick and stone relief(BSR) from their rubbing images in a visually plausible manner. In our approach, the base relief of the low frequency component is estimated automatically with a partial differential equation (PDE)-based mesh deformation scheme. A few vertices near the central area of the object region are selected and assigned with heights estimated by an erosion-based contour map. These vertices together with object boundary vertices, boundary normals as well as the partial differential properties of the mesh are taken as constraints to deform the mesh by minimizing a least-squares error functional. The high frequency detail is estimated directly from rubbing images automatically or optionally with minimal interactive processing. The final height map for a restored BSR is obtained by blending height maps of the base relief and high frequency detail. We demonstrate that our method can not only successfully restore several BSR maps from their rubbing images, but also restore some relief-like surfaces from photographic images.     

局部遮挡下的目标识别算法

规格化互相关算法是用得较普遍的目标识别方法,但是当目标区域被局部遮挡时,该方法通常不能正确定位目标.提出了一种新的基于选择互相关系数的目标识别算法用于搜索有局部遮挡的目标区域.算法分两步进行:用增量互相关算法计算出模板图和场景图的增量图像,比较二者增量图像的一致性,计算出选择互相关系数矩阵;结合选择互相关系数矩阵,用规格化互相关算法在场景图中搜索目标区域.当场景图存在较严重的噪声时,可对选择互相关系数矩阵进行修正以克服噪声的影响.实验结果表明,基于选择互相关系数的目标识别算法对局部遮挡和高亮度变化情况有较强的鲁棒性.     

Depth estimation from single monocular images using deep hybrid network

Multimedia Tools and Applications - Depth estimation is a significant task in the robotics vision. In this paper, we address the depth estimation from a single monocular image, which is a...     

3D head tracking under partial occlusion

A new algorithm for 3D head tracking under partial occlusion from 2D monocular image sequences is proposed. The extended superquadric (ESQ) is used to generate a geometric 3D face model in order to reduce the shape ambiguity during tracking. Optical flow is then regularized by this model to estimate the 3D rigid motion. To deal with occlusion, a new motion segmentation algorithm using motion residual error analysis is developed. The occluded areas are successfully detected and discarded as noise. Furthermore, accumulation error is heavily reduced by a new post-regularization process based on edge flow. This makes the algorithm more stable over long image sequences. The algorithm is applied to both synthetic occlusion sequence and real image sequences. Comparisons with the ground truth indicate that our method is effective and is not sensitive to occlusion during head tracking.     

Automatic classification of single facial images

We propose a method for automatically classifying facial images based on labeled elastic graph matching, a 2D Gabor wavelet representation, and linear discriminant analysis. Results of tests with three image sets are presented for the classification of sex, “race”, and expression. A visual interpretation of the discriminant vectors is provided     

Calibration, pose and novel views from single images of constrained scenes

We exploit the common constraint of having a right-angle corner of two rectangular planes in the scene in order to calibrate a perspective projection camera and compute its pose relative to the coordinate system defined by the corner. No metric information about the corner is assumed. The camera is constrained to have its image x- and y-axes to be orthogonal with the same scale factor, which is valid for most real-world cameras. We then reproject the image of the corner to an arbitrary viewpoint. We can also compute the metric properties of the scene to scale. We report experimental results with subjectively acceptable quality. The approach shows the power of exploiting constraints that are abundant in typical architectural scenes.     

3D scene analysis from a single range image through occlusion graphs

This paper presents a new strategy to extract knowledge about the objects and their relative location in a complex scene when a single range image is taken. The analysis process is based on a range data distributed segmentation technique, which separates the components of the scene, and on a silhouette segmentation method, which classified the silhouette in real (non occluded) and false (occluded) parts. Finally, an occlusion graph provides a compact representation about the layout and relationship of the objects in the scene. This information is essential before higher level tasks in complex scenes – like recognition, understanding and robot interaction – are carried out. An extensive experimentation has been accomplished under real conditions in scenes of up to 12 objects yielding a very good performance. The experiments and results carried out validate the goodness of this approach in 3D environments.     

Re-illuminating single images using Albedo estimation

Predicting the appearance of a scene under novel lighting conditions is of growing interest at the convergence of vision, graphics and virtual reality. In this paper, we develop a method for appearance prediction from a single image using the apparatus of shape from shading (SFS). We re-visit the reflectance estimation process first proposed by Blake (Graphics Image Process. 32 (1985) 314), and develop a novel approach to parameter selection within the Blake method based on the quality of images which can be produced by re-illuminating the recovered needle-map. Combining Blake's method with recent advances in SFS is demonstrated to yield significant improvements in the appearance prediction of real images under varying lighting conditions.     

Vanishing point detection using cascaded 1D Hough Transform from single images

Vanishing point detection algorithms based on 2D histogramming techniques have been employed in a variety of computer vision systems. Previous algorithms achieved some good results but still failed to maintain a balanced performance in both accuracy and time. Recent research (Li et al., 2010) shows that, vanishing point detection could be converted to a 1D histogram search problem, which largely accelerates the procedure. In this paper, we further improve this idea and propose a complete scheme for vanishing point detection from images of the so called “Manhattan world”. We test our algorithm and some commonly used vanishing point detection methods on public database YorkUrbanDB and our own implemented database PKUCampusDB. Our algorithm shows significant performance improvements.     

基于遮挡关系推理的输电线路图像金具检测

实现输电线路图像典型金具的精准检测是进行其缺陷检测的前提。针对通用目标检测模型对密集分布、遮挡严重的金具检测精度较低、易出现漏检等问题,提出了一种结合金具间遮挡结构信息和场景关联信息的典型金具检测方法。基于经典的Faster R-CNN模型提取金具特征作为节点,提取整张图像特征作为金具场景关联信息,学习金具标注框间相交区域信息作为金具遮挡关系信息,并采用图同时建模金具特征、场景关联信息和遮挡关系信息,通过门控循环单元信息传递机制构建结构推理模块完成金具类别和位置的联合推理检测。为了验证所提方法的有效性,选取了8类存在遮挡连接关系的金具进行实验,其中,原始Faster R-CNN模型的mAP值为81.30%,改进模型的mAP值为84.15%。实验结果表明,本文方法一定程度上提高了遮挡严重金具的检测精度,为后续的金具故障诊断奠定良好的基础。     

Prior model evaluation from Null Space Compensation perspective with application to surface reconstruction from single images

Prior model is widely applied in the area of computer vision and computer graphics. However, there is still a lack of a general theoretical scheme for evaluating the performance of the priors and a guidance for choosing suitable models. In this paper, a general scheme is proposed for linear singular problems based on the idea of Null Space Compensation. It is proved that for a linear prior model the principal directions obtained from the singular value decomposition of the model shall not be parallel to those of the system matrix determined by the problem. It is also suggested that for a nonlinear prior, higher correlation between the null space components of the estimate data based on the given prior and those of the ground truth or controlled data indicate the better suitability of the prior. The proposed evaluation scheme is demonstrated through an application to a linearized shape from shading problem, where surface shall be reconstructed from single 2D images. Both linear model and nonlinear constraints are evaluated with experiments on both synthetic images and real images. The results validate the proposed evaluation scheme and its capability for guiding in choosing a good prior model structure.