3D Reconstruction by Shadow Carving: Theory and Practical Evaluation

Cast shadows are an informative cue to the shape of objects. They are particularly valuable for discovering object’s concavities which are not available from other cues such as occluding boundaries. We propose a new method for recovering shape from shadows which we call shadow carving. Given a conservative estimate of the volume occupied by an object, it is possible to identify and carve away regions of this volume that are inconsistent with the observed pattern of shadows. We prove a theorem that guarantees that when these regions are carved away from the shape, the shape still remains conservative. Shadow carving overcomes limitations of previous studies on shape from shadows because it is robust with respect to errors in shadows detection and it allows the reconstruction of objects in the round, rather than just bas-reliefs. We propose a reconstruction system to recover shape from silhouettes and shadow carving. The silhouettes are used to reconstruct the initial conservative estimate of the object’s shape and shadow carving is used to carve out the concavities. We have simulated our reconstruction system with a commercial rendering package to explore the design parameters and assess the accuracy of the reconstruction. We have also implemented our reconstruction scheme in a table-top system and present the results of scanning of several objects.     

Point set silhouettes via local reconstruction

We present an algorithm to compute the silhouette set of a point cloud. Previous methods extract point set silhouettes by thresholding point normals, which can lead to simultaneous over- and under-detection of silhouettes. We argue that additional information such as surface curvature is necessary to resolve these issues. To this end, we develop a local reconstruction scheme using Gabriel and intrinsic Delaunay criteria and define point set silhouettes based on the notion of a silhouette-generating set. The mesh umbrellas, or local reconstructions of one-ring triangles surrounding each point sample, generated by our method enable accurate silhouette identification near sharp features and close-by surface sheets, and provide the information necessary to detect other characteristic curves such as creases and boundaries. We show that these curves collectively provide a sparse and intuitive visualisation of point-cloud data.     

基于多尺度图像块的SAR图像无监督分割

提出了一种基于多尺度图像块的SAR图像无监督分割方法。在利用高斯混合模型进行图像分割时,大多采用的是基于单个像素的分割方法,这种方法由于未考虑像素周围邻域结点的信息,分割精度往往不高。论文考虑到SAR图像具有很强的斑点噪声,为了更好地抑制斑点噪声对分割结果的影响,在多分辨分析的基础上提出了一种基于多尺度图像块的图像分割新方法。实验表明,这种基于多尺度图像块的分割较在单个像素下多尺度Markov模型的MPM分割好,分割精度有了较大的提高。     

Fuzzy classification of earth terrain covers using complex polarimetric SAR data

The conventional approach of terrain image classification that assigns a specific class for each pixel is inadequate, because the area covered by each pixel may embrace more than a single class. Fuzzy set theory which has been developed to deal with imprecise information can be incorporate in the analysis for a more appropriate solution to this problem. In the current state of imaging radar technology, polarimetric synthetic aperture radar (SAR) is unique in providing complete polarization information of ground covers for more effective classification than a single polarization radar. In this paper, we use the fuzzy c-means clustering algorithm for unsupervised segmentation of multi-look polarimetric SAR images. A statistical distance measure adopted in this algorithm is derived from the complex Wishart distribution of the complex covariance matrix. In classifying polarimetric SAR imagery, each terrain class is characterized by its own feature covariance matrix. The algorithm searches for cluster centres for each class and generates a fuzzy partition for the whole image. Membership grades obtained for each pixel provide detailed information about spatial terrain variations. Classification of the image is achieved by choosing a defuzzification criterion. When the back-scattering characteristics of two or more classes are not well distinguished from each other, a divisive hierarchical clustering procedure is adopted to locate their respective feature covariance matrices. NASA/JPL AIRSAR data is used to substantiate this fuzzy classification algorithm.     

基于全局图像结构信息的Seam Carving算法*

本文提出了一种基于全局图像结构信息的Seam Carving算法,它根据像素的重要性修改图像尺寸和比例。通过从图像提取特定方向的边缘结构信息,再利用每个像素的梯度信息,从全局和局部两方面定义新的像素能量计算函数,以此来阻止Seam通路与特定方向图像边缘的交叉,避免边缘像素的不一致位移,以此保持图像的边缘结构。实验结果证明,本文算法减少了处理后图像的结构形变,有效改进了Seam Carving算法的处理效果。     

Detection and classification of vehicles from omnidirectional videos using multiple silhouettes

To detect and classify vehicles in omnidirectional videos, we propose an approach based on the shape (silhouette) of the moving object obtained by background subtraction. Different from other shape-based classification techniques, we exploit the information available in multiple frames of the video. We investigated two different approaches for this purpose. One is combining silhouettes extracted from a sequence of frames to create an average silhouette, the other is making individual decisions for all frames and use consensus of these decisions. Using multiple frames eliminates most of the wrong decisions which are caused by a poorly extracted silhouette from a single video frame. The vehicle types we classify are motorcycle, car (sedan) and van (minibus). The features extracted from the silhouettes are convexity, elongation, rectangularity and Hu moments. We applied two separate methods of classification. First one is a flowchart-based method that we developed and the second is K-nearest neighbour classification. 60% of the samples in the dataset are used for training. To ensure randomization in the experiments, threefold cross-validation is applied. The results indicate that using multiple silhouettes increases the classification performance.     

Three-dimensional human shape inference from silhouettes: reconstruction and validation

Silhouettes are robust image features that provide considerable evidence about the three-dimensional (3D) shape of a human body. The information they provide is, however, incomplete and prior knowledge has to be integrated to reconstruction algorithms in order to obtain realistic body models. This paper presents a method that integrates both geometric and statistical priors to reconstruct the shape of a subject assuming a standardized posture from a frontal and a lateral silhouette. The method is comprised of three successive steps. First, a non-linear function that connects the silhouette appearances and the body shapes is learnt and used to create a first approximation. Then, the body shape is deformed globally along the principal directions of the population (obtained by performing principal component analysis over 359 subjects) to follow the contours of the silhouettes. Finally, the body shape is deformed locally to ensure it fits the input silhouettes as well as possible. Experimental results showed a mean absolute 3D error of 8 mm with ideal silhouettes extraction. Furthermore, experiments on body measurements (circumferences or distances between two points on the body) resulted in a mean error of 11 mm.     

结合色度和纹理不变性的运动阴影检测

目的：在运动检测中,运动物体产生的阴影常常被错误地检测为运动物体本身,为了将阴影从检测结果中消除,本文提出了一种色度不变性和纹理不变性相结合的运动阴影检测方法。方法：首先从阴影的物理模型出发,直接在RGB颜色空间利用色度不变性来获得候选阴影区域,然后根据颜色信息对候选阴影区域进行分割,对每个子区域,利用一种基于局部二值模式的指标来度量其与对应背景区域的纹理相似程度,进而判断该子区域是否是阴影,从而得到最终的检测结果。结果：在公开测试集上的实验结果表明我们的方法可以有效地检测出运动阴影,相对于几种常用的阴影检测算法具有一定的优势。结论：本文将像素级水平和区域级水平阴影检测方法结合起来,提出了一种结合色度不变性和纹理不变性的运动阴影检测方法。实验结果表明,在多类复杂场景中,本文方法都能有效地将运动阴影检测出来,具有较强的鲁棒性。     

Bayesian 3D shape from silhouettes

This paper introduces a smooth posterior density function for inferring shapes from silhouettes. Both the likelihood and the prior are modelled using kernel density functions and optimisation is performed using gradient ascent algorithms. Adding a prior allows for the recovery of concave areas of the shape that are usually lost when estimating the visual hull. This framework is also extended to use colour information when it is available in addition to the silhouettes. In these cases, the modelling not only allows for the shape to be recovered but also its colour information. Our new algorithms are assessed by reconstructing 2D shapes from 1D silhouettes and 3D faces from 2D silhouettes. Experimental results show that using the prior can assist in reconstructing concave areas and also illustrate the benefits of using colour information even when only small numbers of silhouettes are available.     

Methods for Volumetric Reconstruction of Visual Scenes

In this paper, we present methods for 3D volumetric reconstruction of visual scenes photographed by multiple calibrated cameras placed at arbitrary viewpoints. Our goal is to generate a 3D model that can be rendered to synthesize new photo-realistic views of the scene. We improve upon existing voxel coloring/space carving approaches by introducing new ways to compute visibility and photo-consistency, as well as model infinitely large scenes. In particular, we describe a visibility approach that uses all possible color information from the photographs during reconstruction, photo-consistency measures that are more robust and/or require less manual intervention, and a volumetric warping method for application of these reconstruction methods to large-scale scenes.     

Spatio-temporal pattern of urban land cover evolvement with urban renewal and expansion in Shanghai based on mixed-pixel classification for remote sensing imagery

Research into pixel unmixing in remote sensing imagery led to the development of soft classification methods. In this article, we propose a possibilistic c repulsive medoids (PCRMdd) clustering algorithm which attempts to find c repulsive medoids as a minimal solution of a particular objective function. The PCRMdd algorithm is applied to predict the proportion of each land use class within a single pixel, and generate a set of endmember fraction images. The clustering results obtained on multi-temporal Landsat Thematic Mapper (TM)/Enhanced Thematic Mapper plus (ETM+) images of Shanghai city in China reveal the spatio-temporal pattern of Shanghai land use evolvement and urban land spatial sprawl in course of urbanization from 1989 to 2002. The spatial pattern of land use transformation with urban renewal and expansion indicates the urban land use structure is gradually optimized during vigorous urban renewal and large-scale development of Pudong area, which will have an active influence on improving urban space landscape and enhancing the quality of the ecological environment. In addition, accuracy analysis demonstrates that PCRMdd represents a robust and effective tool for mixed-pixel classification on remote sensing imagery to obtain reliable soft classification results and endmember spectral information in a noisy environment.     

A study on multimedia file carving method

File carving is a method that recovers files at unallocated space without any file information and used to recover data and execute a digital forensic investigation. In general, the file carving recovers files using the inherent header and footer in files or the entire file size determined in the file header. The largely used multimedia files, such as AVI, WAV, and MP3, can be exactly recovered using an internal format in files as they are continuously allocated. In the case of the NTFS, which is one of the most widely used file system, it supports an internal data compression function itself, but the NTFS compression function has not been considered in file carving. Thus, a large part of file carving tools cannot recover NTFS compressed files. Also, for carving the multimedia files compressed by the NTFS, a recovery method for such NTFS compressed files is required. In this study, we propose a carving method for multimedia files and represent a recovery plan for deleted NTFS compressed files. In addition, we propose a way to apply such a recovery method to the carving of multimedia files.     

A Surface Reconstruction Method Using Global Graph Cut Optimization

Surface reconstruction from multiple calibrated images has been mainly approached using local methods, either as a continuous optimization problem driven by level sets, or by discrete volumetric methods such as space carving. We propose a direct surface reconstruction approach which starts from a continuous geometric functional that is minimized up to a discretization by a global graph-cut algorithm operating on a 3D embedded graph. The method is related to the stereo disparity computation based on graph-cut formulation, but fundamentally different in two aspects. First, existing stereo disparity methods are only interested in obtaining layers of constant disparity, while we focus on high resolution surface geometry. Second, most of the existing graph-cut algorithms only reach approximate solutions, while we guarantee a global minimum. The whole procedure is consistently incorporated into a voxel representation that handles both occlusions and discontinuities. We demonstrate our algorithm on real sequences, yielding remarkably detailed surface geometry up to 1/10th of a pixel. Author has worked on this project during his Ph. D. at ARTIS     

基于统计模式识别的岩石节理图像分割方法

岩石节理裂隙形状复杂,图像中含有很多噪声。而使用统计模式识别方法在分割图像时,可以首先使分类器学习图像中不同样本的特征,进而利用这些特征对图像中每个像素进行分类,实现分割。在设计统计模式识别的分类器时,我们提出使用基于核函数Fisher判别法构造分类器。使用该方法可以将图像高维的属性空间上的非线形判别转化为图像特征空间上的线形判别,而不需要知道从属性空间到特征空间的具体映射形式。通过对岩石节理裂隙图像分割实验表明,该方法能达到较其他方法更好的分割效果。     

一种鲁棒的自适应空间域隐藏算法

提出了一种新的图像空域信息隐藏算法,算法利用广义Arnold猫映射,将待隐藏的二进制比特序列的每一个比特的嵌入位置,随机置乱到整个载体图像的像素空间,再用Logistic混沌映射并结合像素点最高非0位,采用自适应策略,将待嵌入的比特随机嵌入到像素点的某一个不会明显影响视觉感知效果的比特位,提取水印只需要密钥,且密钥空间大。算法采用了纠错码进行纠错来抵御噪声攻击,还采用最小像素改变量的优化策略减少视觉感知上的变化。实验结果表明,本算法信息隐藏量大,视觉不可感知性好,对剪切攻击和椒盐噪声攻击有很好的鲁棒性。     

结合像元形状特征分割的高分辨率影像面向对象分类

针对高分辨率遥感影像空间分辨率高,结构形状、纹理、细节信息丰富等特点,提出一种新的融合特征的面向对象影像分类方法来提取城市空间信息。基本过程包含以下4个方面:①提取影像的几何纹理等结构;②融合几何与纹理特征的面向对象影像分割;③提取对象的形状、纹理和光谱特征,并优选最佳特征子集;④最后基于支持向量机(SVM)完成面向对象的影像分类。通过对福州IKONOS影像数据实验,结果表明融入影像特征后的分割效果明显优于原始影像的分割结果,而信息最大化(mRMR)的特征选择能够快速地获得较好的特征子集。通过与eCognition最邻近分类方法比较,表明本文方法的分类总体精度大约提高了6%,效果显著。     

一种基于复合混沌序列的图像加密方法

针对低维度混沌系统的密钥空间小、加密系统安全性较低的不足。提出一种由sine混沌改变均匀分布logistic混沌排列次序形成复合混沌序列的图像加密方法。首先,产生服从均匀分布的logistic混沌序列,用sine混沌序列重排该序列整数化后的重复部分,以此无重复数值的复合混沌序列进行像素位置置乱;之后,由于仅进行位置置乱不能改变图像的灰度统计直方图特征,用sine混沌重排整个logistic混沌序列形成复合混沌序列,以此进行像素扩散完成图像加密。对方法安全性从密钥空间、密钥敏感性、差分分析、统计直方图、相邻像素相关性、信息熵方面进行了测量。实验结果表明该方法密钥空间大、敏感性高,能有效地抵抗穷举分析、差分分析和统计分析。     

Using low‐spectral‐resolution images to acquire simulated hyperspectral images

We propose a method to acquire simulated hyperspectral images using low‐spectral‐resolution images. Hyperspectral images provide more spectral information than low‐spectral‐resolution images, because of the additional spectral bands used for data acquisition in hyperspectral imaging. Unfortunately, original hyperspectral images are more expensive and more difficult to acquire. However, some research questions require an abundance of spectral information for ground monitoring, which original hyperspectral images can easily provide. Hence, we need to propose a method to acquire simulated hyperspectral images, when original hyperspectral images are especially necessary. Since low‐spectral‐resolution images are readily available and cheaper, we develop a method to acquire simulated hyperspectral images using low‐spectral‐resolution images. With simulated hyperspectral images, we can acquire more ‘hidden’ information from low‐spectral‐resolution images. Our method uses the principles of pixel‐mixing to understand the compositional relationship of spectrum data to an image pixel, and to simulate radiation transmission processes. To this end, we use previously obtained data (i.e. spectrum library) and the sorting data of objects that are derived from a low‐spectral‐resolution image. Using the simulation of radiation transmission processes and these different data, we acquire simulated hyperspectral images. In addition, previous analyses of simulated remotely sensed images do not use quantitative statistical measures, but use qualitative methods, describing simulated images by sight. Here, we quantitatively assess our simulation by comparing the correlation coefficients of simulated images and real images. Finally, we use simulated hyperspectral images, real Hyperion images, and their corresponding ALI images to generate several classification images. The classification results demonstrate that simulated hyperspectral data contain additional information not available in the multispectral data. We find that our method can acquire simulated hyperspectral images quickly.     

A conceptual framework to define the spatial resolution requirements for agricultural monitoring using remote sensing

Satellite remote sensing is an invaluable tool to monitor agricultural resources. However, spatial patterns in agricultural landscapes vary significantly across the Earth resulting in different imagery requirements depending on what part of the globe is observed. Furthermore, there is an increasing diversity of Earth observation instruments providing imagery with various configurations of spatial, temporal, spectral and angular resolutions. In terms of spatial resolution, the choice of imagery should be conditioned by knowing the appropriate spatial frequency at which the landscape must be sampled with the imaging instrument in order to provide the required information from the targeted fields. This paper presents a conceptual framework to define quantitatively such requirements for both crop area estimation and crop growth monitoring based on user-defined constraints. The methodological development is based on simulating how agricultural landscapes, and more specifically the fields covered by a crop of interest, are seen by instruments with increasingly coarser resolving power. The results are provided not only in terms of acceptable pixel size but also of pixel purity which is the degree of homogeneity with respect to the target crop. This trade-off between size and purity can be adjusted according to the end-user's requirements. The method is implemented over various agricultural landscapes with contrasting spatial patterns, demonstrating its operational applicability. This diagnostic approach can be used: (i) to guide users in choosing the most appropriate imagery for their application, (ii) to evaluate the adequacy of existing remote sensing systems for monitoring agriculture in different regions of the world and (iii) to provide guidelines for space agencies to design future instruments dedicated to agriculture monitoring.