Hallucinating Stereoscopy from a Single Image

We introduce a novel method for enabling stereoscopic viewing of a scene from a single pre‐segmented image. Rather than attempting full 3D reconstruction or accurate depth map recovery, we hallucinate a rough approximation of the scene's 3D model using a number of simple depth and occlusion cues and shape priors. We begin by depth‐sorting the segments, each of which is assumed to represent a separate object in the scene, resulting in a collection of depth layers. The shapes and textures of the partially occluded segments are then completed using symmetry and convexity priors. Next, each completed segment is converted to a union of generalized cylinders yielding a rough 3D model for each object. Finally, the object depths are refined using an iterative ground fitting process. The hallucinated 3D model of the scene may then be used to generate a stereoscopic image pair, or to produce images from novel viewpoints within a small neighborhood of the original view. Despite the simplicity of our approach, we show that it compares favorably with state‐of‐the‐art depth ordering methods. A user study was conducted showing that our method produces more convincing stereoscopic images than existing semi‐interactive and automatic single image depth recovery methods.     

3D object model reconstruction from image sequence based on photometric consistency in volume space

This paper presents a system that can reconstruct a photorealistic 3D object model from an image sequence captured at arbitrary viewpoints. The whole system consists of four steps: camera calibration, volumetric modeling, polygonal model formation and texture mapping. We adopt the shape-from-silhouette approach for volumetric modeling. There are two common types of object surface that are difficult to reconstruct—textureless surface and concave surface. To tackle the problems, we propose to perform the volumetric modeling based on the constraints of viewpoint proximity and photometric consistency in the volume space. The volumetric model is converted to the mesh model for efficient manipulation. Finally, the texture map is generated from the image sequence to give the 3D model a photorealistic appearance. Some reconstructed object models are presented to demonstrate the superior performance of our system as compared with the conventional modeling technique based on the photo-consistency in the image space.     

Stereoscopic Scene Flow Computation for 3D Motion Understanding

Building upon recent developments in optical flow and stereo matching estimation, we propose a variational framework for the estimation of stereoscopic scene flow, i.e., the motion of points in the three-dimensional world from stereo image sequences. The proposed algorithm takes into account image pairs from two consecutive times and computes both depth and a 3D motion vector associated with each point in the image. In contrast to previous works, we partially decouple the depth estimation from the motion estimation, which has many practical advantages. The variational formulation is quite flexible and can handle both sparse or dense disparity maps. The proposed method is very efficient; with the depth map being computed on an FPGA, and the scene flow computed on the GPU, the proposed algorithm runs at frame rates of 20 frames per second on QVGA images (320×240 pixels). Furthermore, we present solutions to two important problems in scene flow estimation: violations of intensity consistency between input images, and the uncertainty measures for the scene flow result.     

Synthesizing Novel Views from Unregistered 2-D Images

Synthesizing the image of a 3-D scene as it would be captured by a camera from an arbitrary viewpoint is a central problem in Computer Graphics. Given a complete 3-D model, it is possible to render the scene from any viewpoint. The construction of models is a tedious task. Here, we propose to bypass the model construction phase altogether, and to generate images of a 3-D scene from any novel viewpoint from prestored images. Unlike methods presented so far, we propose to completely avoid inferring and reasoning in 3-D by using projective invariants. These invariants are derived from corresponding points in the prestored images. The correspondences between features are established off-line in a semi-automated way. It is then possible to generate wireframe animation in real time on a standard computing platform. Well understood texture mapping methods can be applied to the wireframes to realistically render new images from the prestored ones. The method proposed here should allow the integration of computer generated and real imagery for applications such as walkthroughs in realistic virtual environments. We illustrate our approach on synthetic and real indoor and outdoor images.     

基于全局优化策略的场景分类算法

提出一种基于全局优化策略的场景分类算法.该算法基于整幅图像提取全局场景特征——空间包络特征.从图像块中提取视觉单词,且定义隐变量表示该视觉单词语义,然后引入隐状态结构图描述整幅图像的视觉单词上下文;在场景分类策略上,构造由相容函数组成的目标函数,其中相容函数度量全局场景特征、隐变量与场景类别标记的相容度,通过求解目标函数的全局最优解推断图像的场景类别标记.在标准场景图像库上的对比实验表明该算法优于当前有代表性的场景分类算法.     

Correcting interperspective aliasing in autostereoscopic displays

An image presented on an autostereoscopic system should not contain discontinuities between adjacent views. A viewer should experience a continuous scene when moving from one view to the next. If corresponding points in two perspectives do not spatially abut, a viewer will experience jumps in the scene. This is known as interperspective aliasing. Interperspective aliasing is caused by object features far away from the stereoscopic screen being too small, which results in visual artifacts. By modeling a 3D point as a defocused image point, we can adapt Fourier analysis to devise a depth-dependent filter kernel that allows filtering of a stereoscopic 3D image. For synthetic 3D data, we use a simpler approach, which is to smear the data by a distance proportional to its depth     

Three-dimensional image mosaicking using multiple projection planes for 3-D visualization of roadside standing buildings.

A novel image-mosaicking technique suitable for 3-D visualization of roadside buildings on websites or mobile systems is proposed. Our method was tested on a roadside building scene taken using a side-looking video camera employing a continuous set of vertical-textured planar faces. A vertical plane approximation of the scene geometry for each frame was calculated using sparsely distributed feature points that were assigned 3-D data through bundle adjustments. These vertical planes were concatenated to create an approximate model on which the images could be backprojected as textures and blended together. Additionally, our proposed method includes an expanded crossed-slits projection around far-range areas to reduce the "ghost effect," a phenomenon in which a particular object appears repeatedly in a created image mosaic. The final step was to produce seamless image mosaics using Dijkstra's algorithm to find the optimum seam line to blend overlapping images. We used our algorithm to create efficient image mosaics in 3-D space from a sequence of real images.     

On 3-D scene flow and structure recovery from multiview image sequences

Two novel systems computing dense three-dimensional (3-D) scene flow and structure from multiview image sequences are described in this paper. We do not assume rigidity of the scene motion, thus allowing for nonrigid motion in the scene. The first system, integrated model-based system (IMS), assumes that each small local image region is undergoing 3-D affine motion. Non-linear motion model fitting based on both optical flow constraints and stereo constraints is then carried out on each local region in order to simultaneously estimate 3-D motion correspondences and structure. The second system is based on extended gradient-based system (EGS), a natural extension of two-dimensional (2-D) optical flow computation. In this method, a new hierarchical rule-based stereo matching algorithm is first developed to estimate the initial disparity map. Different available constraints under a multiview camera setup are further investigated and utilized in the proposed motion estimation. We use image segmentation information to adopt and maintain the motion and depth discontinuities. Within the framework for EGS, we present two different formulations for 3-D scene flow and structure computation. One formulation assumes that initial disparity map is accurate, while the other does not. Experimental results on both synthetic and real imagery demonstrate the effectiveness of our 3-D motion and structure recovery schemes. Empirical comparison between IMS and EGS is also reported.     

Representation and shape matching of 3-d objects

A three-dimensional scene analysis system for the shape matching of real world 3-D objects is presented. Various issues related to representation and modeling of 3-D objects are addressed. A new method for the approximation of 3-D objects by a set of planar faces is discussed. The major advantage of this method is that it is applicable to a complete object and not restricted to single range view which was the limitation of the previous work in 3-D scene analysis. The method is a sequential region growing algorithm. It is not applied to range images, but rather to a set of 3-D points. The 3-D model of an object is obtained by combining the object points from a sequence of range data images corresponding to various views of the object, applying the necessary transformations and then approximating the surface by polygons. A stochastic labeling technique is used to do the shape matching of 3-D objects. The technique matches the faces of an unknown view against the faces of the model. It explicitly maximizes a criterion function based on the ambiguity and inconsistency of classification. It is hierarchical and uses results obtained at low levels to speed up and improve the accuracy of results at higher levels. The objective here is to match the individual views of the object taken from any vantage point. Details of the algorithm are presented and the results are shown on several unknown views of a complicated automobile casting.     

基于空域自然场景统计的无参考立体图像质量评价模型

针对现有的评价方法大都将图像变换到不同的坐标域问题,提出一种基于空域自然场景统计(NSS)的通用型无参考立体图像质量评价模型。在评价中为了更好地结合人类双目视觉特性, 将左右图像融合成一幅独眼图;评价模型首先统计独眼图归一化亮度(CMSCN)系数分布规律,进而对独眼图提取空域自然场景统计特征;其次,统计视差图归一化亮度(DMSCN)系数的分布规律,并对用光流法得到的视差图提取同样的特征;最后,通过支持向量回归(SVR)建立立体图像特征信息与主观评价值(DMOS)之间的关系,从而预测得到图像质量的客观评价值。实验结果表明,该评价模型对立体数据测试库进行评价,其Pearson线性相关系数(PLCC)和Spearman等级相关系数(SROCC)值均在0.94以上;对于非对称立体图像库,PLCC和SROCC值分别接近0.91和0.93。该模型能够很好地预测人眼对立体图像的主观感知。     

基于特定场景的图像篡改检测算法的研究

提出一种针对特定场景的图像篡改检测方法。针对场景中存在有反射现象的图像对象．通过对图像进行内部相似性搜索，找出所有的相似子区域对，根据描线确定中心点．根据连线的中心是否在反射面附近，来判断图像有无遭遇篡改。通过MatLab对算法进行仿真．验证算法对该类型图像的检测效率。     

Motif analysis for automatic segmentation of CT surface contours into individual surface features

This paper presents a new layer-based technique for automatic high-level segmentation of 3-D surface contours into individual surface features through motif analysis. The procedure starts from a contour-based surface model representing a composite surface area of an object. For each of the surface contours, a relative turning angle (RTA) map is derived. The RTA map usually contains noise and minor features. Algorithms based on motif analysis are applied for extracting a main profile of the RTA map free from background noise and other minor features. All feature points on the extracted profile are further identified from the extracted main profile through further motif analysis. The original contour is thus partitioned into individual segments with the identified feature points. A collection of consecutive contour segments among different layers form an individual 3-D surface feature of the original composite surface. The developed approach using motif analysis is particularly useful for the identification of smooth joins between individual surface features and for the elimination of superposed noise and unwanted minor features.     

Superresolution modeling using an omnidirectional image sensor

Recently, many virtual reality and robotics applications have been called on to create virtual environments from real scenes. A catadioptric omnidirectional image sensor composed of a convex mirror can simultaneously observe a 360-degree field of view making it useful for modeling man-made environments such as rooms, corridors, and buildings, because any landmarks around the sensor can be taken in and tracked in its large field of view. However, the angular resolution of the omnidirectional image is low because of the large field of view captured. Hence, the resolution of surface texture patterns on the three-dimensional (3-D) scene model generated is not sufficient for monitoring details. To overcome this, we propose a high resolution scene texture generation method that combines an omnidirectional image sequence using image mosaic and superresolution techniques.     

基于形变模型由立体序列图象恢复物体的3D形状

结合立体视觉和形变模型提出了一种新的物体3D形状的恢复方法。采用立体视觉方法导出物体表面的3D坐标;利用光流模型估计物体的3D运动,根据此运动移动形变模型,使其对准物体的表面块;由形变模型将由各幅图象得到的离散的3D点融为一起,得到物体的表面形状。实验结果表明该方法能用于形状复杂的物体恢复。     

Surround structured lighting: 3-D scanning with orthographic illumination

This paper presents a new system for rapidly acquiring complete 3-D surface models using a single orthographic structured light projector, a pair of planar mirrors, and one or more synchronized cameras. Using the mirrors, we project structured light patterns that illuminate the object from all sides (not just the side of the projector) and are able to observe the object from several vantage points simultaneously. This system requires that projected planes of light to be parallel, so we construct an orthographic projector using a Fresnel lens and a commercial DLP projector. A single Gray code sequence is used to encode a set of vertically-spaced light planes within the scanning volume, and five views of the illuminated object are obtained from a single image of the planar mirrors located behind it. From each real and virtual camera we recover a dense 3-D point cloud spanning the entire object surface using traditional structured light algorithms. A key benefit of this design is to ensure that each point on the object surface can be assigned an unambiguous Gray code sequence, despite the possibility of being illuminated from multiple directions. In addition to presenting a prototype implementation, we also develop a complete set of mechanical alignment and calibration procedures for utilizing orthographic projectors in computer vision applications. As we demonstrate, the proposed system overcomes a major hurdle to achieving full 360° reconstructions using a single structured light sequence by eliminating the need for merging multiple scans or multiplexing several projectors.     

Integrated 3-D analysis and analysis-guided synthesis of flightimage sequences

This paper is concerned with three-dimensional (3D) analysis, and analysis-guided syntheses, of images showing 3-D motion of an observer relative to a scene. There are two objectives of the paper. First, it presents an approach to recovering 3D motion and structure parameters from multiple cues present in a monocular image sequence, such as point features, optical flow, regions, lines, texture gradient, and vanishing line. Second, it introduces the notion that the cues that contribute the most to 3-D interpretation are also the ones that would yield the most realistic synthesis, thus suggesting an approach to analysis guided 3-D representation. For concreteness, the paper focuses on flight image sequences of a planar, textured surface. The integration of information in these diverse cues is carried out using optimization. For reliable estimation, a sequential batch method is used to compute motion and structure. Synthesis is done by using (i) image attributes extracted from the image sequence, and (ii) simple, artificial image attributes which are not present in the original images. For display, real and/or artificial attributes are shown as a monocular or a binocular sequence. Performance evaluation is done through experiments with one synthetic sequence, and two real image sequences digitized from a commercially available video tape and a laserdisc. The attribute based representation of these sequences compressed their sizes by 502 and 367. The visualization sequence appears very similar to the original sequence in informal, monocular as well as stereo viewing on a workstation monitor     

实时交互的带浮雕纹理的三维模型构建方法

为了快速生成带浮雕纹理的三维模型,提出一种实时交互的浮雕纹理模型构建方法。方法分两步：第一步,将生成浮雕的源模型或图像转换为初始深度图,并进一步转换为梯度图,再通过梯度域的压缩、过滤,求解线性方程重建出整体连续的浮雕深度图;第二步,借助基于网格求交的浮雕纹理映射算法将浮雕深度图贴在目标模型表面,并通过移动、旋转、缩放等操作实时在目标模型三维空间上修改浮雕效果,最终重建目标模型网格,生成浮雕纹理模型。实验表明,所提方法可快速实现在一个目标模型上生成凹浮雕、凸浮雕、多浮雕等效果,所得模型无需经过其他处理,可直接应用于3D打印,打印效果较好。