Extracting the shape and roughness of specular lobe objects usingfour light photometric stereo

We propose a noncontact method for the measurement of surface shape and surface roughness. The method, which we call “four light photometric stereo”, uses four lights, which sequentially illuminate the object under inspection, and a video camera for taking images of the object. The method has successfully been applied to a number of real objects     

An improved method of photometric stereo using local shape from shading

This paper presents an improved photometric stereo (PS) method by integrating it with a local shape from shading (SFS) algorithm. PS produces the initial estimate of image for the global accuracy and also provides the recovery of albedo, SFS supplies the more detailed information within each homogeneous area. The quality of depth obtained by integrating PS and SFS is compared with the real depth using absolute dept error function, and the improvement ranging from 2.3 to 14% over PS is obtained.     

Integrability disambiguates surface recovery in two-image photometric stereo

Two images of a Lambertian surface obtained under different illumination conditions, determine the local surface normals up to two possible orientations. We show that for smooth surfaces, the local integrability constraints usually resolve the problem of deciding between the two possibilities. We also provide a complete characterization of the surfaces that remain ambiguous under given illumination conditions.     

Auto-calibrating photometric stereo using ring light constraints

We propose an auto-calibration method for photometric stereo. Our method exploits constraints placed on light sources to recover their positions and the surface normals of an object up to a scaling and planar rotation ambiguity. The ambiguity is resolved with multi-view consistency constraints leading to a Euclidean reconstruction of the geometric shape of the object. Auto-calibration methods are helpful to overcome the limitations imposed by calibrating objects. We evaluate our algorithm with experiments on real world scenes.     

Application of computer algebra for the reconstruction of surfaces from their photometric stereo images

The problem of reconstructing a Lambertian surface from its two photometric stereo images is discussed. Previously, the solution to this problem was only obtained for a special choice of two light source directions. In this paper, using the computer algebra system Mathematica, the necessary and sufficient conditions for the unique reconstruction of the surface from its two images is analyzed in a more general setting. Photometric images of various surfaces are simulated, and the validity of the theoretical results is demonstrated.     

Robust 3D face capture using example-based photometric stereo

We show that using example-based photometric stereo, it is possible to achieve realistic reconstructions of the human face. The method can handle non-Lambertian reflectance and attached shadows after a simple calibration step. We use spherical harmonics to model and de-noise the illumination functions from images of a reference object with known shape, and a fast grid technique to invert those functions and recover the surface normal for each point of the target object. The depth coordinate is obtained by weighted multi-scale integration of these normals, using an integration weight mask obtained automatically from the images themselves. We have applied these techniques to improve the PhotoFace system of Hansen et al. (2010).     

The 4-source photometric stereo technique for three-dimensional surfaces in the presence of highlights and shadows

We present an algorithm for separating the local gradient information and Lambertian color by using 4-source color photometric stereo in the presence of highlights and shadows. We assume that the surface reflectance can be approximated by the sum of a Lambertian and a specular component. The conventional photometric method is generalized for color images. Shadows and highlights in the input images are detected using either spectral or directional cues and excluded from the recovery process, thus giving more reliable estimates of local surface parameters.     

Light source position calibration method for photometric stereo in capsule endoscopy

ABSTRACT

For photometric stereo in capsule endoscopy, calibration of light source is crucial for improving the precision of surface normal estimation. Therefore, this paper presents an improved planar-mirror-based light source position calibration method: from captured images of light source and detected poses of planar mirror, light paths are retraced from camera to light source, and position of light source is triangulated with least square method. The contribution of this paper is that a refraction model of the planar mirror is employed in the retracement of light paths, thus the bias of light paths caused by refraction can be compensated and the position of light source can be estimated more precisely. The results of simulation and experiment show that the proposed method provides higher calibration accuracy than the current planar mirror-based calibration method and can improve the precision of subsequent photometric stereo-based 3D reconstruction.     

Measurement of mirror surfaces using specular reflection and analytical computation

In many applications, it is desirable to know three-dimensional information of mirror surfaces. However, the reflective characteristic of mirror surfaces makes many traditional three-dimensional reconstruction methods, such as stereovision or structured light scanning, fail. In this paper, a novel method is proposed to measure mirror surfaces. It is implemented with practical hardware and software that directly obtain analytic solutions for three-dimensional points on a mirror surface. Experimental results verified the effectiveness of the proposed method. In particular, this proposed method aims at providing an effective method to reconstruct specular weld pool surfaces which typically are highly dynamic and irregular. The capability to directly obtain analytic solutions for a set of points from a mirror surface as proposed and developed in this paper is a key to effectively measuring such highly dynamic and irregular mirror surfaces.     

Detection and localization of specular surfaces using image motion cues

Successful identification of specularities in an image can be crucial for an artificial vision system when extracting the semantic content of an image or while interacting with the environment. We developed an algorithm that relies on scale and rotation invariant feature extraction techniques and uses motion cues to detect and localize specular surfaces. Appearance change in feature vectors is used to quantify the appearance distortion on specular surfaces, which has previously been shown to be a powerful indicator for specularity (Doerschner et al. in Curr Biol, 2011). The algorithm combines epipolar deviations (Swaminathan et al. in Lect Notes Comput Sci 2350:508–523, 2002) and appearance distortion, and succeeds in localizing specular objects in computer-rendered and real scenes, across a wide range of camera motions and speeds, object sizes and shapes, and performs well under image noise and blur conditions.     

Fast panoramic stereo matching using cylindrical maximum surfaces.

This paper presents a fast panoramic stereo matching algorithm using a cylindrical maximum surface technique. The disparity for a pair of panoramic images is found in a cylindrical shaped correlation coefficient volume by obtaining the maximum surface rather than simply choosing a position that gives the maximum correlation coefficient value. The use of our cylindrical maximum surface technique ensures that the disparities obtained at the left and the right columns of the panoramic stereo images are properly constrained. Typical running time for a pair of 1324 x 120 images is about 0.33 s on a 1.7-GHz PC. A variety of real images have been tested, and good results have been obtained.     

Face recognition in 2D and 2.5D using ridgelets and photometric stereo

A new technique for face recognition – Ridgefaces – is presented. The method combines the well-known Fisherface method with the ridgelet transform and high-speed Photometric Stereo (PS). The paper first derives ridgelet projections for 2D/2.5D face images before the Fisherface approach is used to reduce the dimensionality and increase the spread of the resulting feature vectors. The ridgelet transform is attractive because it is efficient at extracting highly discriminating low-frequency directional features. Best recognition is obtained when Ridgefaces is performed on surface normals acquired from PS, although good results are also found using standard 2D images and PS-derived albedo maps.     

3D face reconstructions from photometric stereo using near infrared and visible light

This paper seeks to advance the state-of-the-art in 3D face capture and processing via novel Photometric Stereo (PS) hardware and algorithms. The first contribution is a new high-speed 3D data capture system, which is capable of acquiring four raw images in approximately 20 ms. The results presented in this paper demonstrate the feasibility of deploying the device in commercial settings. We show how the device can operate with either visible light or near infrared (NIR) light. The NIR light sources offer the advantages of being less intrusive and more covert than most existing face recognition methods allow. Furthermore, our experiments show that the accuracy of the reconstructions is also better using NIR light. The paper also presents a modified four-source PS algorithm which enhances the surface normal estimates by assigning a likelihood measure for each pixel being in a shadowed region. This likelihood measure is determined by the discrepancies between measured pixel brightnesses and expected values. Where the likelihood of shadow is high, then one light source is omitted from the computation for that pixel, otherwise a weighted combination of pixels is used to determine the surface normal. This means that the precise shadow boundary is not required by our method. The results section of the paper provides a detailed analysis of the methods presented and a comparison to ground truth. We also analyse the reflectance properties of a small number of skin samples to test the validity of the Lambertian model and point towards potential improvements to our method using the Oren–Nayar model.     

Evaluating the effect of diffuse light on photometric stereo reconstruction

Photometric stereo surface reconstruction requires each input image to be associated with a particular 3D illumination vector. This signifies that the subject should be illuminated in turn by various directional illumination sources. In real life, this directionality may be reduced by ambient illumination, which is typically present as a diffuse component of the incident light. This work assesses the photometric stereo reconstruction quality for various ratios of ambient to directional illuminance and provides a reference for the robustness of photometric stereo with respect to that illuminance ratio. In our analysis, we focus on the face reconstruction application of photometric stereo, as faces are convex objects with rich surface variation, thus providing a suitable platform for photometric stereo reconstruction quality evaluation. Results demonstrate that photometric stereo renders realistic reconstructions of the given surface for ambient illuminance as high as nine times the illuminance of the directional light component.     

MASR-PSN：低分光度立体图像的高分法向重建深度学习模型

目的 光度立体算法是一种单视角下的稠密三维重建方法,其利用相同视角下来自不同光照方向的一系列图像恢复像素级的表面法向。拍摄光度立体图像所用的高分辨率线性响应相机的成本十分昂贵且难以获取,很难通过传感器直接获取超高分辨率图像来恢复高分辨率表面法向。因此,提出一种基于深度神经网络的光度立体超分算法,以从低分光度立体图像中恢复出准确的高分表面法向。方法 首先,对原始的低分光度立体图像进行归一化预处理操作,以消除剧烈变化的表面反射率影响,并消减过饱和镜面反射的影响。随后,提出多层聚合超分光度立体网络（multi-level aggregation super resolution photometric stereo network,MASR-PSN）。MASR-PSN包含一个新颖的深浅层融合的最大池化聚合框架、权值共享的特征回归器、并行设计的不同尺寸卷积核的并行回归器结构,能够在保留多尺度信息的同时,增强特征表示,防止模式坍塌学习到某一固定尺度相关的非重要特征,以及防止3×3卷积核带来空间域上的过度平滑。结果 广泛的消融实验证明了提出的深浅层聚合层和并行权值共享回归器的有效性,能明显减少生成表面法向的平均角度误差（mean angular error,MAE）。本文方法仅需其他方法一半分辨率的光度立体图像,而能准确地恢复出复杂表面的结构。DiLiGenT benchmark数据集的定量实验和Light Stage Data Gallery数据集、 Gourd数据集的定性实验显示,MASR-PSN在预测表面法向精确度方面有明显提升。在DiLiGenT benchmark数据集中,本文方法在仅使用其他方法一半分辨率的光度立体图像的情况下,以96幅图像为输入时,取得7.31°的平均角度误差,比最佳方法提升0.08°,以10幅图像为输入时,取得9.00°的平均角度误差,比最佳方法提升0.43°。结论 提出的MASR-PSN方法提升了光度立体任务表面法向重建的准确性,在低分辨率的输入图像下,依然可以恢复出细节清晰的超分辨率表面法向。     

基于二维纹理映射的镜面反射加速绘制

针对物体表面镜面反射现象的绘制,提出了一种新的基于二维纹理映射技术的加速算法。将具有镜面反射表达式的Phong模型分解成多个简单的数学函数,预先离散采样计算后作为纹理对象传给显卡。使用OpenGL多重纹理技术,将每个纹理对象放入相应的纹理单元,分别设置其组合函数,将多个纹理的运算组合起来。将三角片顶点坐标及多组纹理坐标传入,从而实现了物体镜面反射现象的实时绘制。实验结果表明,该算法大大提高了绘制的性能。