基于混合反射模型的SFS有限元方法的研究

提出了基于混合反射模型的由明暗恢复物体三维形状的有限元算法。用正方形面元逼近光滑曲面,把曲面表示为所有节点基函数的线性组合;基于既含有漫反射成分又有镜面反射成分的混合模型,结合节点基函数,将反射图线性化。考虑数字图像的特点,直接使用离散形式的SFS问题的亮度约束形式,用最小化方法得到高度满足的线性方程;使用Kaczmarz算法计算出表面三维形状。使用合成图像和实际图像验证该文算法的有效性,探讨了该算法的性能。     

Reflectance and Shape from Images Using a Collinear Light Source

In this paper, a novel technique, called the photogeometric technique, is presented for surface reflectance extraction and surface recovery from an image sequence of a rotating object illuminated under a collinear light source (where the illuminant direction of the light source lies on or near the viewing direction of the camera). The rotation of the object is precisely controlled. The object surface is assumed to be smooth and uniform. The technique first computes the 3D locations of some surface points which give singular brightness values and builds the surface reflectance function by extracting the brightness values of these surface points from the image sequence. Then the technique uses the surface reflectance function and two images of the surface to recover surface depth and orientation simultaneously. The technique has been tested on real images of surfaces with different reflectance properties and geometric structures. The experimental results and comprehensive analysis show that the proposed technique is efficient and robust.     

On Uniqueness of Solutions of the Three-Light-Source Photometric Stereo: Conditions on Illumination Configuration and Surface Reflectance

This paper is concerned with photometric methods using three images with different lighting direction to obtain shape information of an object. Such methods are based on the photometric equation that relates the normal of the object surface to the triplet of the image brightness. This paper discusses the issue of whether the surface normal and the orientation of the 3-vector formed by the image brightness triplet is one-to-one in the equation. Several types of photometric methods require this relation to be one-to-one. We mainly consider the case where the reflectance map is an increasing function of the angle between the surface normal and the illuminant direction. We first point out that even in this simple case, it is possible that the relation is not one-to-one. Then we derive several sufficient conditions on the reflectance as well as the illumination configuration for the one-to-one relation.     

Stereo Image Analysis of Non-Lambertian Surfaces

Stereo image analysis is based on establishing correspondences between a pair of images by determining similarity measures for potentially corresponding image parts. Such similarity criteria are only strictly valid for surfaces with Lambertian (diffuse) reflectance characteristics. Specular reflections are viewpoint dependent and may thus cause large intensity differences at corresponding image points. In the presence of specular reflections, traditional stereo approaches are often unable to establish correspondences at all, or the inferred disparity values tend to be inaccurate, or the established correspondences do not belong to the same physical surface point. The stereo image analysis framework for non-Lambertian surfaces presented in this contribution combines geometric cues with photometric and polarimetric information into an iterative scheme that allows to establish stereo correspondences in accordance with the specular reflectance behaviour and at the same time to determine the surface gradient field based on the known photometric and polarimetric reflectance properties. The described approach yields a dense 3D reconstruction of the surface which is consistent with all observed geometric and photopolarimetric data. Initially, a sparse 3D point cloud of the surface is computed by traditional blockmatching stereo. Subsequently, a dense 3D profile of the surface is determined in the coordinate system of camera 1 based on the shape from photopolarimetric reflectance and depth technique. A synthetic image of the surface is rendered in the coordinate system of camera 2 using the illumination direction and reflectance properties of the surface material. Point correspondences between the rendered image and the observed image of camera 2 are established with the blockmatching technique. This procedure yields an increased number of 3D points of higher accuracy, compared to the initial 3D point cloud. The improved 3D point cloud is used to compute a refined dense 3D surface profile. These steps are iterated until convergence of the 3D reconstruction. An experimental evaluation of our method is provided for areas of several square centimetres of forged and cast iron objects with rough surfaces displaying both diffuse and significant specular reflectance components, where traditional stereo image analysis largely fails. A comparison to independently measured ground truth data reveals that the root-mean-square error of the 3D reconstruction results is typically of the order 30–100 μm at a lateral pixel resolution of 86 μm. For two example surfaces, the number of stereo correspondences established by the specular stereo algorithm is several orders of magnitude higher than the initial number of 3D points. For one example surface, the number of stereo correspondences decreases by a factor of about two, but the 3D point cloud obtained with the specular stereo method is less noisy, contains a negligible number of outliers, and shows significantly more surface detail than the initial 3D point cloud. For poorly known reflectance parameters we observe a graceful degradation of the accuracy of 3D reconstruction.     

Direct passive navigation

In this correspondence, we show how to recover the motion of an observer relative to a planar surface from image brightness derivatives. We do not compute the optical flow as an intermediate step, only the spatial and temporal brightness gradients (at a minimum of eight points). We first present two iterative schemes for solving nine nonlinear equations in terms of the motion and surface parameters that are derived from a least-squares fomulation. An initial pass over the relevant image region is used to accumulate a number of moments of the image brightness derivatives. All of the quantities used in the iteration are efficiently computed from these totals without the need to refer back to the image. We then show that either of two possible solutions can be obtained in closed form. We first solve a linear matrix equation for the elements of a 3 × 3 matrix. The eigenvalue decomposition of the symmetric part of the matrix is then used to compute the motion parameters and the plane orientation. A new compact notation allows us to show easily that there are at most two planar solutions.     

Shape from intensity gradient

We propose a new shape-from-shading (SFS) algorithm which replaces the brightness constraint with an intensity gradient constraint. This is a global approach which obtains the solution by the minimization of an error function over the entire image. Through the linearization of the gradient of the reflectance map and the discretization of the surface gradient, the intensity gradient can be expressed as a linear function of the surface height. A quadratic error function, which involves the intensity gradient constraint and the traditional smoothness constraint, is minimized efficiently by solving a sparse linear system using the multigrid technique. Neither the information at singular points nor the information at occluding boundaries is needed for the initialization of the height. Results for real images are presented to show the robustness of the algorithm, and the execution time is demonstrated to prove its efficiency     

Shape from shading with a linear triangular element surface model

The authors propose to combine a triangular element surface model with a linearized reflectance map to formulate the shape-from-shading problem. The main idea is to approximate a smooth surface by the union of triangular surface patches called triangular elements and express the approximating surface as a linear combination of a set of nodal basis functions. Since the surface normal of a triangular element is uniquely determined by the heights of its three vertices (or nodes), image brightness can be directly related to nodal heights using the linearized reflectance map. The surface height can then be determined by minimizing a quadratic cost functional corresponding to the squares of brightness errors and solved effectively with the multigrid computational technique. The proposed method does not require any integrability constraint or artificial assumptions on boundary conditions. Simulation results for synthetic and real images are presented to illustrate the performance and efficiency of the method     

Estimating the surface radiance function from single images

This paper describes a simple method for estimating the surface radiance function from single images of smooth surfaces made of materials whose reflectance function is isotropic and monotonic. The method makes implicit use of the Gauss map between the surface and a unit sphere. We assume that the material brightness is monotonic with respect to the angle between the illuminant direction and the surface normal. Under conditions in which the light source and the viewer directions are identical, we show how a tabular representation of the surface radiance function can be estimated using the cumulative distribution of image gradients. Using this tabular representation of the radiance function, surfaces may be rendered under varying light source direction by rotating the corresponding reflectance map on the Gauss sphere about the specular spike direction. We present a sensitivity study on synthetic and real-world imagery. We also present two applications which make use of the estimated radiance function. The first of these illustrates how the radiance function estimates can be used to render objects when the light and viewer directions are no longer coincident. The second application involves applying corrected Lambertian radiance to rough and shiny surfaces.     

基于Laplace的LSSIM图像配准算法

提出一种基于Laplace变换的图像配准算法. 首先利用经典的角点检测算法提取待匹配图像的特征点或角点; 其次利用相位相关法估算出两幅图像的重叠区域, 以缩小匹配范围; 然后对角点邻域模板区域施行Laplace变换; 最后利用基于改进的SSIM (结构相似性)作为相似性度量准则建立特征点之间的匹配关系. 实验结果表明, 该方法可以很好的完成特征点匹配, 匹配点对充足且具有很高的准确率, 而且对亮度差异具有一定的鲁棒性, 从而保证图像配准精度.     

单幅植物叶片图像的3维重建

目的 植物叶片形态复杂,在虚拟场景中很难真实表现。为了从信息量有限的单幅图像中恢复植物叶片的3维形状,本文基于从明暗恢复形状（shape from shading,SFS）的方法,利用亮度统计规律和植物形态特征恢复叶片的3维形状。方法 在SFS的基础上,设计基于图像骨架的距离场偏置加强表面细节;针对SFS对恢复宏观几何形状的不足,提出根据图像亮度统计分布选取控制点控制表面宏观形状变化,并利用叶片中轴的距离场约束恢复宏观几何形状,每种方法对于表面宏观几何形状恢复的权重基于恢复的反射图和输入图像间的相似度设定;将表面细节添加到宏观几何形状上得到目标对象的3维形状。结果 选取植物叶片图像进行实验,并与其他方法进行比较,实验结果表明本文方法增强了表面细节显示,并有明显的宏观几何形状变化。同时为了验证本文方法对其他物体表面细节恢复的适用性,分别对硬币和恐龙恢复表面细节,实验结果表明提出的增强表面细节的方法同样适用于其他物体。结论 针对单幅植物叶片图像的3维重建,在SFS的基础上提出了根据骨架特征加强表面细节,根据图像亮度统计分布和叶片中轴距离场约束共同恢复表面宏观几何形状的算法,实验结果验证了本文方法的可行性。     

基于图像的点云建模及其真实感绘制

从视觉凸壳的理论出发,提出了一种基于图像的真实物体的点云建模和绘制方法。在建模方面,首先从不同视点采集目标物体的图像,然后对采样图像所形成的视觉凸壳进行均匀的点采样。同时,利用等间隔索引表来组织每幅采样图像的轮廓边,从而提高了建模效率。在绘制方面,首先分析目标物体在各种不同光照条件下的图像,并得到点云模型的离散反射属性,然后通过优化的插值方法实现真实感的绘制效果。实验表明,该方法的建模速度快,绘制结果具有很强的真实感。     

Reflectance based object recognition

Neighboring points on a smoothly curved surface have similar surface normals and illumination conditions. Therefore, their brightness values can be used to compute the ratio of their reflectance coefficients. Based on this observation, we develop an algorithm that estimates a reflectance ratio for each region in an image with respect to its background. The algorithm is efficient as it computes ratios for all image regions in just two raster scans. The region reflectance ratio represents a physical property that is invariant to illumination and imaging parameters. Several experiments are conducted to demonstrate the accuracy and robustness of ratio invariant.The ratio invariant is used to recognize objects from a single brightness image of a scene. Object models are automatically acquired and represented using a hash table. Recognition and pose estimation algorithms are presented that use ratio estimates of scene regions as well as their geometric properties to index the hash table. The result is a hypothesis for the existence of an object in the image. This hypothesis is verified using the ratios and locations of other regions in the scene. This approach to recognition is effective for objects with printed characters and pictures. Recognition experiments are conducted on images with illumination variations, occlusions, and shadows. The paper is concluded with a discussion on the simultaneous use of reflectance and geometry for visual perception.     

Corner validation based on extracted corner properties

We developed a method to validate and filter a large set of previously obtained corner points. We derived the necessary relationships between image derivatives and estimates of corner angle, orientation and contrast. Commonly used cornerness measures of the auto-correlation matrix estimates of image derivatives are expressed in terms of these estimated corner properties. A candidate corner is validated if the cornerness score directly obtained from the image is sufficiently close to the cornerness score for an ideal corner with the estimated orientation, angle and contrast. We tested this algorithm on both real and synthetic images and observed that this procedure significantly improves the corner detection rates based on human evaluations. We tested the accuracy of our corner property estimates under various noise conditions. Extracted corner properties can also be used for tasks like feature point matching, object recognition and pose estimation.     

Ambiguous Shape from Shading with Critical Points

The eikonal equation can have an infinite number of solutions when the image has critical points. We exhibit a family, indexed by a continuous parameter, of non isomorphic surfaces with one critical point, which give the same simple image. Hence, shape from shading can be an ill-posed problem when no additional condition on the shape is imposed, even when the image has critical points. Also, deformations without distortion are possible, i.e., there can exist a continuous deformation of the surface which does not modify its image.     

基于表面局部最高点估计的明暗恢复形状算法

基于数据逼近强约束的针图恢复算法是近年来提出的一种较为成功的从明暗恢复形状（shape from shading）的算法,但由于该算法在非垂直光线下得到的初始化针图的误差较大,并且不能保证法向量有解或有唯一解,为了解决SFS算法存在的问题,提出了一种改进的SFS算法。该改进算法从分析非垂直光线下图像梯度图与针图之间的关系入手,首先检测图像局部最亮点位置;然后根据照度方程估计表面局部最高点的位置,同时对梯度方向进行调整,并建立方程组;最后针对方程组解的不同情况,提出了相应的处理方法。改进后的算法,对于垂直光线和非垂直光线下的情况同样有效,从而扩大了基于数据逼近强约束的SFS算法的适用范围。从合成图像和实际图像的实验结果可以看出,采用改进的算法可以得到比基于数据逼近强约束的算法更接近真实表面的初始化针图和初始化高度。     

Determining reflectance properties of an object using range andbrightness images

The authors discuss a method of recovering reflectance properties of a surface from a range image given by a range finder and a brightness image given by a standard TV camera. The Torrance-Sparrow model is used for the reflectance model. The model consists of the Lambertian and specular components: its reflectance properties consist of the relative strength between the Lambertian and specular components and specular sharpness as well as light source direction. An iterative least square fitting method is used to obtain these parameters based on the range and brightness images. An input image is segmented into four different parts using the parameters: Lambertian reflection, specular reflection, interreflection, and shadow part. The authors also reconstruct ideal images that consist of only Lambertian or specular reflection     

Correcting brightness gradients in hyperspectral data from urban areas

The analysis of airborne hyperspectral data is often affected by brightness gradients that are caused by directional surface reflectance. For line scanners these gradients occur in across-track direction and depend on the sensor's view-angle. They are greatest whenever the flight path is perpendicular to the sun-target-observer plane. A common way to correct these gradients is to normalize the reflectance factors to nadir view. This is especially complicated for data from spatially and spectrally heterogeneous urban areas and requires surface type specific models. This paper presents a class-wise empirical approach that is adapted to meet the needs of such images.Within this class-wise approach, empirical models are fit to the brightness gradients of spectrally pure pixels from classes after a spectral angle mapping (SAM). Compensation factors resulting from these models are then assigned to all pixels of the image, both in a discrete manner according the SAM and in a weighted manner based on information from the SAM rule images. The latter scheme is designed in consideration of the great number of mixed pixels.The method is tested on data from the Hyperspectral Mapper (HyMap) that was acquired over Berlin, Germany. It proves superior to a common global approach based on a thorough assessment using a second HyMap image as reference. The weighted assignment of compensation factors is adequate for the correction of areas that are characterized by mixed pixels.A remainder of the original brightness gradient cannot be found in the corrected image, which can then be used for any subsequent qualitative and quantitative analyses. Thus, the proposed method enables the comparison and composition of airborne data sets with similar recording conditions and does not require additional field or laboratory measurements.     

Determining plane orientation from specular reflectance

This paper describes methods of estimating the orientation of a planar surface from the shapes of the contours of constant brightness on the surface, using perspective projection. It is assumed that the illumination is from a distant point source in a known direction. Two reflectance models, proposed respectively by Horn and Pentland, are used.     

A Green's function approach to shape from shading

We present a new algorithm for shape from shading, inspired on the recently introduced disparity-based approach to photometric stereo (DBPS). Assuming that the single input image will be matched to a second image through a uniform disparity field, we construct an irradiance conservation equation and solve it for the matching image, via Green's function. When a linear expansion of the reflectance map is considered, this leads to a closed-form approximate expression for the surface function, whose parameters can be estimated via a structure-from-motion approach already used for DBPS.     

Color Subspaces as Photometric Invariants

Complex reflectance phenomena such as specular reflections confound many vision problems since they produce image ‘features’ that do not correspond directly to intrinsic surface properties such as shape and spectral reflectance. A common approach to mitigate these effects is to explore functions of an image that are invariant to these photometric events. In this paper we describe a class of such invariants that result from exploiting color information in images of dichromatic surfaces. These invariants are derived from illuminant-dependent ‘subspaces’ of RGB color space, and they enable the application of Lambertian-based vision techniques to a broad class of specular, non-Lambertian scenes. Using implementations of recent algorithms taken from the literature, we demonstrate the practical utility of these invariants for a wide variety of applications, including stereo, shape from shading, photometric stereo, material-based segmentation, and motion estimation.