3-D Scene Data Recovery Using Omnidirectional Multibaseline Stereo

A traditional approach to extracting geometric information from a large scene is to compute multiple 3-D depth maps from stereo pairs or direct range finders, and then to merge the 3-D data. However, the resulting merged depth maps may be subject to merging errors if the relative poses between depth maps are not known exactly. In addition, the 3-D data may also have to be resampled before merging, which adds additional complexity and potential sources of errors.This paper provides a means of directly extracting 3-D data covering a very wide field of view, thus by-passing the need for numerous depth map merging. In our work, cylindrical images are first composited from sequences of images taken while the camera is rotated 360° about a vertical axis. By taking such image panoramas at different camera locations, we can recover 3-D data of the scene using a set of simple techniques: feature tracking, an 8-point structure from motion algorithm, and multibaseline stereo. We also investigate the effect of median filtering on the recovered 3-D point distributions, and show the results of our approach applied to both synthetic and real scenes.     

Tracking 3-D motion from straight lines with trifocal tensors

We present a novel approach to track the position and orientation of a stereo camera using line features in the images. The method combines the strengths of trifocal tensors and Bayesian filtering. The trifocal tensor provides a geometric constraint to lock line features among every three frames. It eliminates the explicit reconstruction of the scene even if the 3-D scene structure is not known. Such a trifocal constraint thus makes the algorithm fast and robust. The twist motion model is applied to further improve its computation efficiency. Another major contribution is that our approach can obtain the 3-D camera motion using as little as 2 line correspondences instead of 13 in the traditional approaches. This makes the approach attractive for realistic applications. The performance of the proposed method has been evaluated using both synthetic and real data with encouraging results. Our algorithm is able to estimate 3-D camera motion in real scenarios accurately having little drifting from an image sequence longer than a 1,000 frames.     

3D Reconstruction of Indoor Scenes via Image Registration

With the development of computer vision technologies, 3D reconstruction has become a hotspot. At present, 3D reconstruction relies heavily on expensive equipment and has poor real-time performance. In this paper, we aim at solving the problem of 3D reconstruction of an indoor scene with large vertical span. In this paper, we propose a novel approach for 3D reconstruction of indoor scenes with only a Kinect. Firstly, this method uses a Kinect sensor to get color images and depth images of an indoor scene. Secondly, the combination of scale-invariant feature transform and random sample consensus algorithm is used to determine the transformation matrix of adjacent frames, which can be seen as the initial value of iterative closest point (ICP). Thirdly, we establish the relative coordinate relation between pair-wise frames which are the initial point cloud data by using ICP. Finally, we achieve the 3D visual reconstruction model of indoor scene by the top-down image registration of point cloud data. This approach not only mitigates the sensor perspective restriction and achieves the indoor scene reconstruction of large vertical span, but also develops the fast algorithm of indoor scene reconstruction with large amount of cloud data. The experimental results show that the proposed algorithm has better accuracy, better reconstruction effect, and less running time for point cloud registration. In addition, the proposed method has great potential applied to 3D simultaneous location and mapping.     

Synthesizing Novel Views from Unregistered 2-D Images

Synthesizing the image of a 3-D scene as it would be captured by a camera from an arbitrary viewpoint is a central problem in Computer Graphics. Given a complete 3-D model, it is possible to render the scene from any viewpoint. The construction of models is a tedious task. Here, we propose to bypass the model construction phase altogether, and to generate images of a 3-D scene from any novel viewpoint from prestored images. Unlike methods presented so far, we propose to completely avoid inferring and reasoning in 3-D by using projective invariants. These invariants are derived from corresponding points in the prestored images. The correspondences between features are established off-line in a semi-automated way. It is then possible to generate wireframe animation in real time on a standard computing platform. Well understood texture mapping methods can be applied to the wireframes to realistically render new images from the prestored ones. The method proposed here should allow the integration of computer generated and real imagery for applications such as walkthroughs in realistic virtual environments. We illustrate our approach on synthetic and real indoor and outdoor images.     

Acquiring a radiance distribution to superimpose virtual objectsonto a real scene

This paper describes a new method for superimposing virtual objects with correct shadings onto an image of a real scene. Unlike the previously proposed methods, our method can measure a radiance distribution of a real scene automatically and use it for superimposing virtual objects appropriately onto a real scene. First, a geometric model of the scene is constructed from a pair of omnidirectional images by using an omnidirectional stereo algorithm. Then, radiance of the scene is computed from a sequence of omnidirectional images taken with different shutter speeds and mapped onto the constructed geometric model. The radiance distribution mapped onto the geometric model is used for rendering virtual objects superimposed onto the scene image. As a result, even for a complex radiance distribution, our method can superimpose virtual objects with convincing shadings and shadows cast onto the real scene. We successfully tested the proposed method by using real images to show its effectiveness     

一种圆柱形全景图生成新算法及其实现

利用基于图象的绘制(IBR)技术来建立具有真实感的虚拟环境是当前计算机图形学研究领域中的一项热门课题，也是虚拟现实技术中的关键技术。它以简单的图象合成代替基于三维几何的建模和渲染，加快了画面的显示速度，是建立特定三维场景的一种高效率的方法。基于这种方法，我们提出了一种生成圆柱形全景图的新算法，它通过快速查找
相邻两幅图象的匹配区域而实现图象的无缝拼接，建立圆柱形全景图。本算法计算速度快，效率高，能生成逼真的全景图。文章的最后给出了基于这种算法生成的部分全景图图实例。     

Bayes smoothing algorithms for segmentation of binary images modeled by markov random fields

A new image segmentation algorithm is presented, based on recursive Bayes smoothing of images modeled by Markov random fields and corrupted by independent additive noise. The Bayes smoothing algorithm yields the a posteriori distribution of the scene value at each pixel, given the total noisy image, in a recursive way. The a posteriori distribution together with a criterion of optimality then determine a Bayes estimate of the scene. The algorithm presented is an extension of a 1-D Bayes smoothing algorithm to 2-D and it gives the optimum Bayes estimate for the scene value at each pixel. Computational concerns in 2-D, however, necessitate certain simplifying assumptions on the model and approximations on the implementation of the algorithm. In particular, the scene (noiseless image) is modeled as a Markov mesh random field, a special class of Markov random fields, and the Bayes smoothing algorithm is applied on overlapping strips (horizontal/vertical) of the image consisting of several rows (columns). It is assumed that the signal (scene values) vector sequence along the strip is a vector Markov chain. Since signal correlation in one of the dimensions is not fully used along the edges of the strip, estimates are generated only along the middle sections of the strips. The overlapping strips are chosen such that the union of the middle sections of the strips gives the whole image. The Bayes smoothing algorithm presented here is valid for scene random fields consisting of multilevel (discrete) or continuous random variables.     

Method for HDR-like imaging using industrial digital cameras

Dynamic range of the scene can be significantly wider than the dynamic range of an image because of limitations of A/D conversion. In such a situation, numerous details of the scene cannot be adequately shown on the image. Standard industrial digital cameras are equipped with an auto-exposure function that automatically sets both the aperture value and cameras exposure time. When measuring a scene with atypical distribution of light and dark elements, the indicated auto-exposure time may not be optimal. The aim of work was to improve, with minimal cost, the performance of standard industrial digital cameras. We propose a low complexity method for creating HDR-like image using three images captured with different exposure times. The proposed method consists of three algorithms: (1) algorithm for estimating whether the auto-exposure time is optimal, (2) algorithm which determines exposure times for two additional images (one with shorter and another with longer than auto-exposure time), and (3) algorithm for HDR-like imaging based on fusion of three previously obtained images. Method is implemented on FPGA inserted into standard industrial digital camera. Results show that the proposed approach produces high quality HDR-like scene-mapped 8-bit images with minimal computational cost. All improvements may be noticed through the performance evaluation.

     

3-D face recognition method based on optimum 3-D image measurement technology

Most face recognition systems employ 2-D color or gray-scale images. However, face recognition based on 2-D images is adversely affected by 3-D movement, variable lighting, and the use of cosmetics. 3-D image measurement technology has the potential to overcome these limitations of face recognition based on 2-D images since it can perform geometric analysis. We propose a method that is capable of recognizing a person from a 3-D facial image obtained using a 3-D shape measurement system by employing a technique that optimizes the intensity-modulation pattern projection. This face recognition method is based on the iterative closest point algorithm. It is robust to changes in reflectivity and color. Since the 3-D facial information can be registered, this method can estimate rotations and translations to compensate for different positions or directions. In order to prove the validity of the proposed technique, a verification experiment was conducted which used 105 sample 3-D images obtained from 15 subjects. It achieved a detection rate of 96% when heads were turned at an angle of 20° or less relative to the camera.     

An evaluation of 3-D scene exploration using a multiperspective image framework

Multiperspective images (MPIs) show more than what is visible from a single viewpoint and are a promising approach for alleviating the problem of occlusions. We present a comprehensive user study that investigates the effectiveness of MPIs for 3-D scene exploration. A total of 47 subjects performed searching, counting, and spatial orientation tasks using both conventional and multiperspective images. We use a flexible MPI framework that allows trading off disocclusion power for image simplicity. The framework also allows rendering MPI images at interactive rates, which enables investigating interactive navigation and dynamic 3-D scenes. The results of our experiments show that MPIs can greatly outperform conventional images. For searching, subjects performed on average 28% faster using an MPI. For counting, accuracy was on average 91% using MPIs as compared to 42% for conventional images.     

利用不均匀散焦模型获取景物深度信息

景物三维重建的关键之一是从景物图像中获得景物目标到摄像机的距离。研究了一种基于散焦图像的计算景物距离的算法。该方法基于不均匀散焦模型,并且只需两幅改变光圈指数得到的散焦程度不同的图像,因此可以避免图像的大小匹配问题。根据图像的点扩散函数的形式,可以通过优化的方法求得深度。模拟和真实实验表明了算法的有效性。     

A virtual reality system using the concentric mosaic: construction, rendering, and data compression

This paper proposes a new image-based rendering (IBR) technique called "concentric mosaic" for virtual reality applications. IBR using the plenoptic function is an efficient technique for rendering new views of a scene from a collection of sample images previously captured. It provides much better image quality and lower computational requirement for rendering than conventional three-dimensional (3-D) model-building approaches. The concentric mosaic is a 3-D plenoptic function with viewpoints constrained on a plane. Compared with other more sophisticated four-dimensional plenoptic functions such as the light field and the lumigraph, the file size of a concentric mosaic is much smaller. In contrast to a panorama, the concentric mosaic allows users to move freely in a circular region and observe significant parallax and lighting changes without recovering the geometric and photometric scene models. The rendering of concentric mosaics is very efficient, and involves the reordering and interpolating of previously captured slit images in the concentric mosaic. It typically consists of hundreds of high-resolution images which consume a significant amount of storage and bandwidth for transmission. An MPEG-like compression algorithm is therefore proposed in this paper taking into account the access patterns and redundancy of the mosaic images. The compression algorithms of two equivalent representations of the concentric mosaic, namely the multiperspective panoramas and the normal setup sequence, are investigated. A multiresolution representation of concentric mosaics using a nonlinear filter bank is also proposed.     

自然场景图像与合成图像的快速分类

目的 随着现代通信和传感技术的快速发展,互联网上多媒体数据日益增长,既为人们生活提供了便利,又给信息有效利用提出了挑战。为充分挖掘网络图像中蕴含的丰富信息,同时考虑到网络中图像类型的多样性,以及不同类型的图像需要不同的处理方法,本文针对当今互联网中两种主要的图像类型：自然场景图像与合成图像,设计层次化的快速分类算法。方法 该算法包括两层,第1层利用两类图像在颜色,饱和度以及边缘对比度上表现出来的差异性提取全局特征,并结合支持向量机（SVM）进行初步分类,第1层分类结果中低置信度的图像会被送到第2层中。在第2层中,系统基于词袋模型（bag-of-words）对图像不同类型的局部区域的纹理信息进行编码得到局部特征并结合第2个SVM分类器完成最终分类。针对层次化分类框架,文中还提出两种策略对两个分类器进行融合,分别为分类器结果融合与全局+局部特征融合。为测试算法的实用性,同时收集并发布了一个包含超过30 000幅图像的数据库。结果 本文设计的全局与局部特征对两类图像具有较强的判别性。在单核Intel Xeon（R）（2.50 GHz）CPU上,分类精度可达到98.26%,分类速度超过40帧/s。另外通过与基于卷积神经网络的方法进行对比实验可发现,本文提出的算法在性能上与浅层网络相当,但消耗更少的计算资源。结论 本文基于自然场景图像与合成图像在颜色、饱和度、边缘对比度以及局部纹理上的差异,设计并提取快速有效的全局与局部特征,并结合层次化的分类框架,完成对两类图像的快速分类任务,该算法兼顾分类精度与分类速度,可应用于对实时性要求较高的图像检索与数据信息挖掘等实际项目中。     

On 3-D scene flow and structure recovery from multiview image sequences

Two novel systems computing dense three-dimensional (3-D) scene flow and structure from multiview image sequences are described in this paper. We do not assume rigidity of the scene motion, thus allowing for nonrigid motion in the scene. The first system, integrated model-based system (IMS), assumes that each small local image region is undergoing 3-D affine motion. Non-linear motion model fitting based on both optical flow constraints and stereo constraints is then carried out on each local region in order to simultaneously estimate 3-D motion correspondences and structure. The second system is based on extended gradient-based system (EGS), a natural extension of two-dimensional (2-D) optical flow computation. In this method, a new hierarchical rule-based stereo matching algorithm is first developed to estimate the initial disparity map. Different available constraints under a multiview camera setup are further investigated and utilized in the proposed motion estimation. We use image segmentation information to adopt and maintain the motion and depth discontinuities. Within the framework for EGS, we present two different formulations for 3-D scene flow and structure computation. One formulation assumes that initial disparity map is accurate, while the other does not. Experimental results on both synthetic and real imagery demonstrate the effectiveness of our 3-D motion and structure recovery schemes. Empirical comparison between IMS and EGS is also reported.     

DLSLA 3-D SAR imaging algorithm for off-grid targets based on pseudo-polar formatting and atomic norm minimization基于伪极坐标变换和原子范数最小化的网格偏离目标DLSLA 3-D SAR成像方法

This paper concerns the imaging problem for downward looking sparse linear array three-dimensional synthetic aperture radar (DLSLA 3-D SAR) under the circumstance of sparse and non-uniform cross-track dimensional virtual phase centers configuration. Since the 3-D imaging scene behaves typical sparsity in a certain domain, sparse recovery approaches hold the potential to achieve a better reconstruction performance. However, most of the existing compressive sensing (CS) algorithms assume the scatterers located on the pre-discretized grids, which is often violated by the off-grid effect. By contrast, atomic norm minimization (ANM) deals with sparse recovery problem directly on continuous space instead of discrete grids. This paper firstly analyzes the off-grid effect in DLSLA 3-D SAR sparse image reconstruction, and then introduces an imaging method applied to off-gird targets reconstruction which combines 3-D pseudo-polar formatting algorithm (pseudo-PFA) with ANM. With the proposed method, wave propagation and along-track image reconstruction are operated with pseudo-PFA, then the cross-track reconstruction is implemented with semidefinite programming (SDP) based on the ANM model. The proposed method holds the advantage of avoiding the off-grid effect and managing to locate the off-grid targets to accurate locations in different imaging scenes. The performance of the proposed method is verified and evaluated by the 3-D image reconstruction of different scenes, i.e., point targets and distributed scene.     

高斯二阶差分特征算子在图像拼接中的应用

为了将同一场景中具有重叠区域序列的图像快速准确合成一幅具有宽视角、高分辨率的图像,提出了基于高斯二阶差分(D2oG)特征检测算子的SIFT算法.采用高斯二阶差分(D2oG)金字塔的过零点检测提取图像尺度不变特征点,并选用RANSAC算法对特征点匹配对进行提纯,在此基础上计算不变换矩阵H,最后,用渐进渐出平滑算法完成图像的无缝拼接.实验中分别采用所提方法和SIFT算法对具有典型变换的4种图像进行拼接与测试,结果表明:所提方法提取的匹配点数、拼接所消耗时间明显低于采用SIFT算法,同时匹配效率也高于后者.此方法降低了运算复杂度的同时,图像拼接实时性也得到提高.     

一种实现森林场景漫游的几何与图像混合绘制方法*

提出了一种基于粒子系统和Particle System API的景物模拟方法,并采用Line方式取代传统的Point方式渲染粒子,结合纹理映射方法实现了多种喷泉模拟。实验证明用该方法模拟喷泉效果比较真实,速度快,在普通微机上可以得到令人满意的效果。     

基于轴对称窗口文字图像检测快速算法研究

研究了图像定位的问题,由于存在污渍干扰等影响图像定位,针对以往单一特征进行图像文字定位及识别的系统中容易受到各种环境因素干扰的缺陷,提出了一种利用轴对称窗口进行边缘检测的图像文字检测算法。首先将利用轴对称滑动窗口提取水平和竖直方向上的边缘特征,利用连通域确定初始的图像文字位置;通过对可能的图像文字区域进行颜色色调验证,区域内垂直方向直方图投影,从而确定最终的图像文字位置。由于利用多种特征综合检测图像文字进行仿真。仿真结果表明改进方法能准确检测出复杂场景下图像文字所在区域。