Garment Modeling from a Single Image

Modeling of realistic garments is essential for online shopping and many other applications including virtual characters. Most of existing methods either require a multi‐camera capture setup or a restricted mannequin pose. We address the garment modeling problem according to a single input image. We design an all‐pose garment outline interpretation, and a shading‐based detail modeling algorithm. Our method first estimates the mannequin pose and body shape from the input image. It further interprets the garment outline with an oriented facet decided according to the mannequin pose to generate the initial 3D garment model. Shape details such as folds and wrinkles are modeled by shape‐from‐shading techniques, to improve the realism of the garment model. Our method achieves similar result quality as prior methods from just a single image, significantly improving the flexibility of garment modeling.     

先验形状约束的超声前列腺图像分割方法

提出一种结合超声前列腺图像的局部特征和前列腺的先验形状知识的分割方法。该方法在传统图像分割方法中引入了前列腺的先验形状约束,使得分割能够一定程度地避免由于超声图像中噪声、伪影、灰度分布不均匀等因素对前列腺分割所造成的影响。算法分为两个部分：先验形状模型的学习和先验形状约束的分割。在先验形状模型学习阶段,采用主成分分析方法对形状作特征提取,以高斯分布作为形变参数的估计;在先验形状约束分剖阶段,将基于局部高斯拟合特征的活动轮廓模型与形状模型相结合对前列腺图像分割。实验表明,所提出的方法在超声前列腺图像中取得了良好的分割效果,为临床诊断和治疗提供了定量分析的工具。     

Learning Layered Motion Segmentations of Video

We present an unsupervised approach for learning a layered representation of a scene from a video for motion segmentation. Our method is applicable to any video containing piecewise parametric motion. The learnt model is a composition of layers, which consist of one or more segments. The shape of each segment is represented using a binary matte and its appearance is given by the rgb value for each point belonging to the matte. Included in the model are the effects of image projection, lighting, and motion blur. Furthermore, spatial continuity is explicitly modeled resulting in contiguous segments. Unlike previous approaches, our method does not use reference frame(s) for initialization. The two main contributions of our method are: (i) A novel algorithm for obtaining the initial estimate of the model by dividing the scene into rigidly moving components using efficient loopy belief propagation; and (ii) Refining the initial estimate using α β-swap and α-expansion algorithms, which guarantee a strong local minima. Results are presented on several classes of objects with different types of camera motion, e.g. videos of a human walking shot with static or translating cameras. We compare our method with the state of the art and demonstrate significant improvements.     

Face recognition from a single training image under arbitrary unknown lighting using spherical harmonics

In this paper, we propose two novel methods for face recognition under arbitrary unknown lighting by using spherical harmonics illumination representation, which require only one training image per subject and no 3D shape information. Our methods are based on the result which demonstrated that the set of images of a convex Lambertian object obtained under a wide variety of lighting conditions can be approximated accurately by a low-dimensional linear subspace. We provide two methods to estimate the spherical harmonic basis images spanning this space from just one image. Our first method builds the statistical model based on a collection of 2D basis images. We demonstrate that, by using the learned statistics, we can estimate the spherical harmonic basis images from just one image taken under arbitrary illumination conditions if there is no pose variation. Compared to the first method, the second method builds the statistical models directly in 3D spaces by combining the spherical harmonic illumination representation and a 3D morphable model of human faces to recover basis images from images across both poses and illuminations. After estimating the basis images, we use the same recognition scheme for both methods: we recognize the face for which there exists a weighted combination of basis images that is the closest to the test face image. We provide a series of experiments that achieve high recognition rates, under a wide range of illumination conditions, including multiple sources of illumination. Our methods achieve comparable levels of accuracy with methods that have much more onerous training data requirements. Comparison of the two methods is also provided.     

Estimation of the epipole using optical flow at antipodal points

We present algorithms for estimating the epipole or direction of translation of a moving camera. We use constraints arising from two points that are antipodal on the image sphere in order to decouple rotation from translation. One pair of antipodal points constrains the epipole to lie on a plane, and two such pairs will correspondingly give two planes. The intersection of these two planes is an estimate of the epipole. This means we require image motion measurements at two pairs of antipodal points to obtain an estimate. Two classes of algorithms are possible and we present two simple yet extremely robust algorithms representative of each class. These are shown to have comparable accuracy with the state of the art when tested in simulation under noise and with real image sequences.     

Deformable model for estimating clothed and naked human shapes from a single image

Estimation of human shape from images has numerous applications ranging from graphics to surveillance. A single image provides insufficient constraints (e.g. clothing), making human shape estimation more challenging. We propose a method to simultaneously estimate a person’s clothed and naked shapes from a single image of that person wearing clothing. The key component of our method is a deformable model of clothed human shape. We learn our deformable model, which spans variations in pose, body, and clothes, from a training dataset. These variations are derived by the non-rigid surface deformation, and encoded in various low-dimension parameters. Our deformable model can be used to produce clothed 3D meshes for different people in different poses, which neither appears in the training dataset. Afterward, given an input image, our deformable model is initialized with a few user-specified 2D joints and contours of the person. We optimize the parameters of the deformable model by pose fitting and body fitting in an iterative way. Then the clothed and naked 3D shapes of the person can be obtained simultaneously. We illustrate our method for texture mapping and animation. The experimental results on real images demonstrate the effectiveness of our method.     

A shape prior constraint for implicit active contours

We present a shape prior constraint to guide the evolution of implicit active contours. Our method includes three core techniques. Firstly, a rigid registration is introduced, using a line search method within a level set framework. The method automatically finds the time step for the iterative optimization processes. The order for finding the optimal translation, rotation and scale is derived experimentally. Secondly, a single reconstructed shape is created from a shape distribution of a previously acquired learning set. The reconstructed shape is applied to guide the active contour evolution. Thirdly, our method balances the impact of the shape prior versus the image guidance of the active contour. A mixed stopping condition is defined based on the stationarity of the evolving curve and the shape prior constraint. Our method is completely non-parametric and avoids taking linear combinations of non-linear signed distance functions, which would cause problems because distance functions are not closed under linear operations. Experimental results show that our method is able to extract the desired objects in several circumstances, namely when noise is present in the image, when the objects are in slightly different poses and when parts of the object are invisible in the image.     

Face hallucination based on two-dimensional joint learning

In this paper, a face hallucination method based on two-dimensional joint learning is presented. Unlike the existing works on face super-resolution algorithms that first reshape the image or image patch into 1D vector, in our study the spatial construction of the high resolution (HR) and the low resolution (LR) face image are efficiently maintained in the reconstruction procedure. Enlightened by the 1D joint learning approach for image super-resolution, we propose a 2D joint learning algorithm to map the original 2D LR and HR image patch spaces onto a unified feature subspace. Subsequently, the neighbor-embedding (NE) based super-resolution algorithm can be conducted on the unified feature subspace to estimate the reconstruction weights. With these weights, the initial HR facial image can be generated. To refine further the initial HR estimate, the global reconstruction constraint is exploited to improve the quality of reconstruction result. Experiments on the face databases and real-world face images demonstrate the effectiveness of the proposed algorithm.     

基于主动学习的高光谱图像分类方法

高光谱图像监督分类中,为了避免休斯效应需要大量的训练样本,但在实际应用中对样本进行标注成本非常高,因此,得到高质量的训练样本显得十分重要。提出一种基于主动学习的高光谱图像分类方法,通过对区域关注度的统计,有效地结合图像光谱和空间特性,基于主动学习方法获取信息量较大的训练样本,从而较大幅度提高了分类的精确度。实验结果表明,所提算法比传统的随机取样监督分类法和主动学习方法在分类精确度上有较大的优势。     

形状的全尺度可视化表示与识别

目的视觉目标的形状特征表示和识别是图像领域中的重要问题。在实际应用中,视角、形变、遮挡和噪声等干扰因素造成识别精度较低,且大数据场景需要算法具有较高的学习效率。针对这些问题,本文提出一种全尺度可视化形状表示方法。方法在尺度空间的所有尺度上对形状轮廓提取形状的不变量特征,获得形状的全尺度特征。将获得的全部特征紧凑地表示为单幅彩色图像,得到形状特征的可视化表示。将表示形状特征的彩色图像输入双路卷积网络模型,完成形状分类和检索任务。结果通过对原始形状加入旋转、遮挡和噪声等不同干扰的定性实验,验证了本文方法具有旋转和缩放不变性,以及对铰接变换、遮挡和噪声等干扰的鲁棒性。在通用数据集上进行形状分类和形状检索的定量实验,所得准确率在不同数据集上均超过对比算法。在MPEG-7数据集上精度达到99.57%,对比算法的最好结果为98.84%。在铰接和射影变换数据集上皆达到100%的识别精度,而对比算法的最好结果分别为89.75%和95%。结论本文提出的全尺度可视化形状表示方法,通过一幅彩色图像紧凑地表达了全部形状信息。通过卷积模型既学习了轮廓点间的形状特征关系,又学习了不同尺度间的形状特征关系。本文方法...     

Metric rectification of curved document images

In this paper, we propose a metric rectification method to restore an image from a single camera-captured document image. The core idea is to construct an isometric image mesh by exploiting the geometry of page surface and camera. Our method uses a general cylindrical surface (GCS) to model the curved page shape. Under a few proper assumptions, the printed horizontal text lines are shown to be line convergent symmetric. This property is then used to constrain the estimation of various model parameters under perspective projection. We also introduce a paraperspective projection to approximate the nonlinear perspective projection. A set of close-form formulas is thus derived for the estimate of GCS directrix and document aspect ratio. Our method provides a straightforward framework for image metric rectification. It is insensitive to camera positions, viewing angles, and the shapes of document pages. To evaluate the proposed method, we implemented comprehensive experiments on both synthetic and real-captured images. The results demonstrate the efficiency of our method. We also carried out a comparative experiment on the public CBDAR2007 data set. The experimental results show that our method outperforms the state-of-the-art methods in terms of OCR accuracy and rectification errors.     

Automatic 3D object segmentation in multiple views using volumetric graph-cuts

We propose an algorithm for automatically obtaining a segmentation of a rigid object in a sequence of images that are calibrated for camera pose and intrinsic parameters. Until recently, the best segmentation results have been obtained by interactive methods that require manual labelling of image regions. Our method requires no user input but instead relies on the camera fixating on the object of interest during the sequence. We begin by learning a model of the object’s colour, from the image pixels around the fixation points. We then extract image edges and combine these with the object colour information in a volumetric binary MRF model. The globally optimal segmentation of 3D space is obtained by a graph-cut optimisation. From this segmentation an improved colour model is extracted and the whole process is iterated until convergence.     

Shape space estimation by higher-rank of SOM

The aim of this study is to develop an estimation method for a shape space. In this work, “shape space” means a nonlinear subspace formed by a class of visual shapes, in which the continuous change in shapes is naturally represented. By using the shape space, various operations dealing with shapes, such as identification, classification, recognition, and interpolation can be carried out in the shape space. This paper introduces an algorithm based on a generative model of shapes. A higher-rank of the self-organizing map (SOM²) is used to implement the shape space estimation method. We use this method to estimate the shape space of artificial contours. In addition, we present results from a simulation of omnidirectional camera images taken from mobile robots. Our technique accurately predicts changes in image properties as the robot’s attitude changes. Finally, we consider the addition of local features to our method. We show that the inclusion of local features solves the correspondence problem. These results suggest the potential of our technique in the future.     

Particle-SfT: A Provably-Convergent,Fast Shape-from-Template Algorithm

The Shape-from-Template (SfT) problem is to recover the 3D shape of a deformable object from a single image, given a 3D template and a deformation constraint. We propose Particle-SfT, a new SfT algorithm which handles isometric and non-isometric deformations. We build Particle-SfT upon a particle system guided by deformation and reprojection constraint projections. Reconstruction is achieved by evolving particles to a globally attractive equilibrium, while taking observable external forces such as gravity into account, if any. Particle-SfT may be used to refine an existing initial shape. However, in practice we simply use the template as initial guess. This is because, as opposed to the existing refining methods, Particle-SfT has an extremely wide convergence basin. Particle-SfT is also faster than the existing refining methods. This is because it moves pieces of the shape’s mesh independently to achieve larger step size by optimal constraint projection. We proved its convergence to a fixed-point. We experimented it with synthetic and real data. It has the same accuracy as the best performing isometric method and consistently outperforms all existing elastic methods in almost all cases, while being much faster.     

DeepGarment : 3D Garment Shape Estimation from a Single Image

3D garment capture is an important component for various applications such as free‐view point video, virtual avatars, online shopping, and virtual cloth fitting. Due to the complexity of the deformations, capturing 3D garment shapes requires controlled and specialized setups. A viable alternative is image‐based garment capture. Capturing 3D garment shapes from a single image, however, is a challenging problem and the current solutions come with assumptions on the lighting, camera calibration, complexity of human or mannequin poses considered, and more importantly a stable physical state for the garment and the underlying human body. In addition, most of the works require manual interaction and exhibit high run‐times. We propose a new technique that overcomes these limitations, making garment shape estimation from an image a practical approach for dynamic garment capture. Starting from synthetic garment shape data generated through physically based simulations from various human bodies in complex poses obtained through Mocap sequences, and rendered under varying camera positions and lighting conditions, our novel method learns a mapping from rendered garment images to the underlying 3D garment model. This is achieved by training Convolutional Neural Networks (CNN‐s) to estimate 3D vertex displacements from a template mesh with a specialized loss function. We illustrate that this technique is able to recover the global shape of dynamic 3D garments from a single image under varying factors such as challenging human poses, self occlusions, various camera poses and lighting conditions, at interactive rates. Improvement is shown if more than one view is integrated. Additionally, we show applications of our method to videos.     

Participation recommendation system for crowdsourcing contests

We propose a novel participation recommendation approach for crowdsourcing contests including probabilistic modeling of contest participation and winner determination. Our method estimates the winning and participation probability of each worker and offers ranked lists of recommended contests. Since there is only one winner in most contests, standard recommendation techniques fail to estimate the accurate winning probability using only the extremely sparse winning information of completed contests. Our solution is to utilize contest participation information and features of workers and contests as auxiliary information. We use the concept of a transfer learning method for matrices and a feature-based matrix factorization method. Experiments conducted using real crowdsourcing contest datasets show that the use of auxiliary information is crucial for improving the performance of contest recommendation, and also reveal several important common skills.     

On Estimating the Uncertainty in the Location of Image Points in 3D Recognition from Match Sets of Different Sizes

Efficient and robust model-based recognition systems need to be able to estimate reliably and quickly the possible locations of other model features in the image when a match of several model points to image points is given. Errors in the sensed data lead to uncertainty in the computed pose of the object, which in turn lead to uncertainty in these positions. We present an efficient and accurate method for estimating these uncertainty regions. Our basic method deals with an initial match of three points. With a small additional computational cost it can be used to compute the uncertainty regions of the projections of many model points using the same match triplet. The basic method is then extended employing statistical methods to estimate uncertainty regions when given initial matches of any size. This is the major practical contribution of the paper because when the number of points in the match increases, the size of the uncertainty region decreases dramatically, which helps to discriminate much better between correct and incorrect matches in model-based recognition algorithms.     

基于DDPG的三维重建模糊概率点推理

单视图物体三维重建是一个长期存在的具有挑战性的问题. 为了解决具有复杂拓扑结构的物体以及一些高保真度的表面细节信息仍然难以准确进行恢复的问题, 本文提出了一种基于深度强化学习算法深度确定性策略梯度 (Deep deterministic policy gradient, DDPG)的方法对三维重建中模糊概率点进行再推理, 实现了具有高保真和丰富细节的单视图三维重建. 本文的方法是端到端的, 包括以下四个部分: 拟合物体三维形状的动态分支代偿网络的学习过程, 聚合模糊概率点周围点的邻域路由机制, 注意力机制引导的信息聚合和基于深度强化学习算法的模糊概率调整. 本文在公开的大规模三维形状数据集上进行了大量的实验证明了本文方法的正确性和有效性. 本文提出的方法结合了强化学习和深度学习, 聚合了模糊概率点周围的局部信息和图像全局信息, 从而有效地提升了模型对复杂拓扑结构和高保真度的细节信息的重建能力.     

Shape from Texture without Boundaries

We describe a shape from texture method that constructs an estimate of surface geometry using only the deformation of individual texture elements. Our method does not need to use either the boundary of the observed surface or any assumption about the overall distribution of elements.The method assumes that surface texture elements are drawn from a number of different types, each of fixed shape. Neither the shape of the elements nor the number of types need be known in advance. We show that, with this assumption and assuming a generic, scaled orthographic view and texture, each type of texture element can be reconstructed in a frontal coordinate system from image instances. Interest-point methods supply a method of simultaneously obtaining instances of each texture element automatically and defining each type of element. Furthermore, image instances that have been marked in error can be identified and ignored using the Expectation-Maximization algorithm. A further EM procedure yields a surface reconstruction and a relative irradiance map from the data. We provide numerous examples of reconstructions for images of real scenes, show a comparison between our reconstruction and range maps, and demonstrate that the reconstructions display geometric and irradiance phenomena that can be observed in the original image. First online version published in February, 2006     

Stable grasping under pose uncertainty using tactile feedback

This paper deals with the problem of stable grasping under pose uncertainty. Our method utilizes tactile sensing data to estimate grasp stability and make necessary hand adjustments after an initial grasp is established. We first discuss a learning approach to estimating grasp stability based on tactile sensing data. This estimator can be used as an indicator to the stability of the current grasp during a grasping procedure. We then present a tactile experience based hand adjustment algorithm to synthesize a hand adjustment and optimize the hand pose to achieve a stable grasp. Experiments show that our method improves the grasping performance under pose uncertainty.