Recovering Shape by Shading and Stereo Under Lambertian Shading Model

A method that integrates shape from shading and stereo is reported for Lambertian objects. A rectification is proposed to convert any lighting direction from oblique to orthographic. A sparse stereo method is reported that directly uses depth information and has no foreshortening problem. The method completely solves three difficult problems in stereo, namely, recovering depth at occlusion; matching at places with similar shading and matching at smooth silhouettes. The method has been tested on both synthetic and real images. It shows superior performance compared with two recent stereo algorithms. It is also a method based on the physics of image formation. The work described in this article was fully supported by a grant from CityU (7002128).     

SFS方法及其与立体视觉方法的集成方案综述

立体视觉(StereoVision)方法是目前利用图象数据获取物体三维信息的主要方法之一。但该方法在图象灰度变化较缓慢的区域,由于难以准确地进行图象间的象素配对,而严重影响了它的效果。利用从明暗重构物体三维表面形状(ShapefromShading,简称SFS)的方法与该方法相结合,是改善重构结果的主要途径之一。文章通过分析SFS问题本身的不适定性,揭示了目前几类主要的SFS算法在可靠性、稳定性、局限性以及实用性方面所存在的问题,并在此基础上,简要地介绍了四类SFS与立体视觉方法相结合的形式,说明了通过利用立体视觉为SFS补充辅助的信息来消除SFS问题的不适定性,并对过去SFS的实现方法进行有效的改进,它是提高集成系统准确性的关键。     

基于表面局部最高点估计的明暗恢复形状算法

基于数据逼近强约束的针图恢复算法是近年来提出的一种较为成功的从明暗恢复形状（shape from shading）的算法,但由于该算法在非垂直光线下得到的初始化针图的误差较大,并且不能保证法向量有解或有唯一解,为了解决SFS算法存在的问题,提出了一种改进的SFS算法。该改进算法从分析非垂直光线下图像梯度图与针图之间的关系入手,首先检测图像局部最亮点位置;然后根据照度方程估计表面局部最高点的位置,同时对梯度方向进行调整,并建立方程组;最后针对方程组解的不同情况,提出了相应的处理方法。改进后的算法,对于垂直光线和非垂直光线下的情况同样有效,从而扩大了基于数据逼近强约束的SFS算法的适用范围。从合成图像和实际图像的实验结果可以看出,采用改进的算法可以得到比基于数据逼近强约束的算法更接近真实表面的初始化针图和初始化高度。     

基于阴影恢复形状的起皱织物表面形态重建研究

阴影恢复形状算法是计算机视觉中的一个重要研究课题 ,该算法应用于织物三维表面重建 ,为客观评价织物起皱等级奠定了基础 .提出了一种新的阴影恢复重建算法 ,并阐述了该算法的实现步骤和计算方法 .先利用合成图象对算法进行迭代计算并验证 ,获得了较为准确的重建图象 ,然后再结合真实模板进行重建 ,重建准确度较高 .同时说明了该算法可应用于真实织物的表面重建之中 ,并可从获得的织物三维轮廓数据中提取特征值 ,实验表明 ,这些特征值均可从不同侧面表征织物的褶皱程度 ,特征值与织物的褶皱程度基本呈线性相关     

Surface Texture Using Photometric Stereo Data: Classification and Direction of Illumination Detection

This paper generalizes the recently proposed sinusoidal model used for modeling the variation of texture features under changes in illumination direction, so that it can handle surfaces which are very rough and of variable albedo. It deals with the problem of identifying the direction of illumination of a rough surface from a single image, using information from a photometric stereo set of images. In addition, it presents methodology for classifying the texture of a rough surface, using generalized normals that capture both shape and albedo information. It assumes that the surface is Lambertian and is presented to the camera in a fronto-parallel view.     

Shape from Shading Using Probability Functions and Belief Propagation

Shape-from-shading (SFS) aims to reconstruct the three-dimensional shape of an object from a single shaded image. This article proposes an improved framework based on belief propagation for computing SFS. The implementation of the well-known brightness, integrability and smoothness constraints inside this framework is shown. We implement the constraints as probability density functions. For example, the brightness constraint is a two-dimensional probability density function that relates all possible surface gradients at a pixel to their probability given the pixel intensity. A straightforward extension of the framework to photometric stereo is presented, where multiple images of the same scene taken under different lighting conditions are available. The results are promising, especially since the solution is obtained by iteratively applying simple operations on a regular grid of points. The presented framework therefore can be implemented in parallel and is a reasonably likely biological scheme.     

从遮挡边缘和局部阴影恢复表面形状

遮挡边缘和局部阴影是表面形状估计的重要线索。本文利用不同视点下的遮挡边缘序列，获取它们所对应的表面点的三维几何关系。并结合局部阴影分析，对物体表面的非暴露区域进行深度（高度）计算。最终通过插值，获得表面三维模型，实现真实目标建模。     

Direct Differential Photometric Stereo Shape Recovery of Diffuse and Specular Surfaces

Recovering the 3D shape of an object from shading is a challenging problem due to the complexity of modeling light propagation and surface reflections. Photometric Stereo (PS) is broadly considered a suitable approach for high-resolution shape recovery, but its functionality is restricted to a limited set of object surfaces and controlled lighting setup. In particular, PS models generally consider reflection from objects as purely diffuse, with specularities being regarded as a nuisance that breaks down shape reconstruction. This is a serious drawback for implementing PS approaches, since most common materials have prominent specular components. In this paper, we propose a PS model that solves the problem for both diffuse and specular components aimed at shape recovery of generic objects with the approach being independent of the albedo values thanks to the image ratio formulation used. Notably, we show that by including specularities, it is possible to solve the PS problem for a minimal number of three images using a setup with three calibrated lights and a standard industrial camera. Even if an initial separation of diffuse and specular components is still required for each input image, experimental results on synthetic and real objects demonstrate the feasibility of our approach for shape reconstruction of complex geometries.     

Photometric Stereo by Hemispherical Metric Embedding

Photometric stereo methods seek to reconstruct the 3d shape of an object from motionless images obtained with varying illumination. Most existing methods solve a restricted problem where the physical reflectance model, such as Lambertian reflectance, is known in advance. In contrast, we do not restrict ourselves to a specific reflectance model. Instead, we offer a method that works on a wide variety of reflectances. Our approach uses a simple yet uncommonly used property of the problem— the sought after normals are points on a unit hemisphere. We present a novel embedding method that maps pixels to normals on the unit hemisphere. Our experiments demonstrate that this approach outperforms existing manifold learning methods for the task of hemisphere embedding. We further show successful reconstructions of objects from a wide variety of reflectances including smooth, rough, diffuse and specular surfaces, even in the presence of significant attached shadows. Finally, we empirically prove that under these challenging settings we obtain more accurate shape reconstructions than existing methods.     

Shape From Shading by Using Neural Based Colour Reflectance Model

In this Letter, a new methodology for Colour Shape From Shading problem is proposed. The problem of colour SFS refers to the well-known fact that most real objects usually contain mixtures of diffuse and specular colour reflections. In this paper, these limitations are addressed and a new colour neural based model is proposed. The proposed approach focuses on developing a generalized neural based colour reflectance model. Experimental results on synthetic coloured objects and a real coloured object were performed to demonstrate the performance of the proposed methodology.     

Optimal Reconstruction of Approximate Planar Surfaces Using Photometric Stereo

Photometric stereo can be used to obtain a fast and noncontact surface reconstruction of Lambertian surfaces. Despite several published works concerning the uncertainties and optimal light configurations of photometric stereo, no solutions for optimal surface reconstruction from noisy real images have been proposed. In this paper, optimal surface reconstruction methods for approximate planar textured surfaces using photometric stereo are derived, given that the statistics of imaging errors are measurable. Simulated and real surfaces are experimentally studied, and the results validate that the proposed approaches improve the surface reconstruction especially for the high-frequency height variations.     

Direct Solvers for the Biharmonic Eigenvalue Problems Using Legendre Polynomials

We propose an efficient algorithm based on the Legendre–Galerkin approximations for the direct solution of the biharmonic eigenvalue problems with the boundary conditions of the clamped plate, the simply supported plate and the Cahn–Hilliard type. The key point to the efficiency of our algorithm is to construct appropriate basis functions which satisfying the corresponding boundary condition automatically and leading to linear systems with sparse matrices for the discrete variational formulations. In addition, the error estimate was driven by the minimax principle. Finally, the numerical results demonstrate the accuracy and the efficiency of this method.     

一种鲁棒的基于光度立体视觉的表面重建方法

提出一种基于先进的凸优化技术的光度立体视觉重建框架. 首先通过鲁棒的主成分分析(Robust principle component analysis, RPCA)祛除图像噪声, 得到低秩矩阵和物体表面向量场, 然后再通过表面重建算法从向量场来恢复物体形状. 相对于先前的一些使用最小二乘或者一些启发式鲁棒技术的方法, 该方法使用了所有可用的信息, 可以同时修复数据中的丢失和噪声数据, 显示出了较高的计算效率以及对于大的稀疏噪声的鲁棒性. 实验结果表明, 本文提出的框架大大提高了在噪声存在情况下物体表面的重建精度.     

Combining Shape from Shading and Stereo: A Joint Variational Method for Estimating Depth,Illumination and Albedo

Shape from shading (SfS) and stereo are two fundamentally different strategies for image-based 3-D reconstruction. While approaches for SfS infer the depth solely from pixel intensities, methods for stereo are based on a matching process that establishes correspondences across images. This difference in approaching the reconstruction problem yields complementary advantages that are worthwhile being combined. So far, however, most “joint” approaches are based on an initial stereo mesh that is subsequently refined using shading information. In this paper we follow a completely different approach. We propose a joint variational method that combines both cues within a single minimisation framework. To this end, we fuse a Lambertian SfS approach with a robust stereo model and supplement the resulting energy functional with a detail-preserving anisotropic second-order smoothness term. Moreover, we extend the resulting model in such a way that it jointly estimates depth, albedo and illumination. This in turn makes the approach applicable to objects with non-uniform albedo as well as to scenes with unknown illumination. Experiments for synthetic and real-world images demonstrate the benefits of our combined approach: They not only show that our method is capable of generating very detailed reconstructions, but also that joint approaches are feasible in practice.     

Uniqueness in Shape from Shading Revisited

We analyse the problem of representing solutions of first-orderpartial differential equations in terms of complete integrals and envelopes. In this context, we revisit the uniqueness results alreadyexisting in the shape-from-shading literature that concern eikonalequations corresponding to the images of a Lambertian hemi-sphere and aLambertian plane. We show that the approach adopted by Brooks in [2, 3] isincomplete and subsequently re-establish its uniqueness claims.     

Solving Uncalibrated Photometric Stereo Using Fewer Images by Jointly Optimizing Low-rank Matrix Completion and Integrability

We introduce a new, integrated approach to uncalibrated photometric stereo. We perform 3D reconstruction of Lambertian objects using multiple images produced by unknown, directional light sources. We show how to formulate a single optimization that includes rank and integrability constraints, allowing also for missing data. We then solve this optimization using the Alternating Direction Method of Multipliers (ADMM). We conduct extensive experimental evaluation on real and synthetic data sets. Our integrated approach is particularly valuable when performing photometric stereo using as few as 4–6 images, since the integrability constraint is capable of improving estimation of the linear subspace of possible solutions. We show good improvements over prior work in these cases.     

二次曲线的多项式逼近

研究用B啨ｚｉｅｒ曲线或样条逼近任意长二次曲线弧的方法 对不同曲线类型 ,均得到具有 6阶逼近精度的误差函数 并且相邻的B啨ｚｉｅｒ曲线间ＧＣ1连续 最后给出任意二次曲线弧近似多项式或多项式样条参数化的算法     

椭球的高精度多项式逼近

给出了用双三次多项式逼近椭球的一种简明方法.逼近椭圆的误差为273×10^-6,逼近椭球的误差为545×10^-6     

双曲面片的高精度多项式逼近

用三次Bezier曲线逼近双曲线段，在端点保持GC^1插值，给出单边逼近的误差，并进行最优插值点的选择，得到最优的误差估计；在此基础上，用双三次Bezier多项式逼近单叶和双叶双曲面片，给出误差估计，逼近六到六阶精度。相邻的逼近片之间GC^1连续。