图像3D轮廓提取

针对现有3D轮廓提取方法计算量大、图像立体匹配难、图片中含有大量噪音的问题,提出了一种利用早期认知视觉系统提取图像3D轮廓的方法。早期认知视觉是连接早期视觉和认知视觉的中间平台,通过早期认知视觉系统提取出图像的2D基元,2D基元是图片小块,有丰富的语义信息包括位置,方向,相位和三色值;再由两个立体图像的相一致的2D基元配对来构建3D基元,通过3D基元的共面性和共色性得到图像的3D轮廓。在Ubuntu10.04下,通过输入立体图片对,对这种方法进行了实验,实验结果表明图像的3D轮廓能完好地提取出来,有效地保留图像的必要信息并具有强抗噪性。     

Constructing a 3D trunk model from two images

Trees stand for a key component in the natural environment, thus modeling realistic trees has received much attentions of researchers in computer graphics. However, most trees in computer graphics are generated according to some procedural rules in conjunction with some random perturbations, thus they are generally different from the real trees in the natural environment. In this paper, we propose a systematic approach to create a 3D trunk graphical model from two images so that the created trunk has a similar 3D trunk structure to the real one. In the proposed system, the trunk is first segmented from the image via an interactive segmentation tool and its skeleton is then extracted. Some points on the skeleton are selected and their context relations are established for representing the 2D trunk structure. A camera self-calibration algorithm appropriate for the two-view case is developed, and a minimum curvature constraint is employed to recover the 3D trunk skeleton from the established 2D trunk structure and the calibrated camera. The trunk is then modeled by a set of generalized cylinders around the recovered 3D trunk skeleton. A polygonal mesh representing the trunk is finally generated and a textured 3D trunk model is also produced by mapping the image onto the surface of the 3D trunk model. We have conducted some experiments and the results demonstrated that the proposed system can actually yield a visually plausible 3D trunk model which is similar to the real one in the image.     

First-order and second-order statistical analysis of 3D and 2D image structure

In the first part of this article, we analyze the relation between local image structures (i.e., homogeneous, edge-like, corner-like or texture-like structures) and the underlying local 3D structure (represented in terms of continuous surfaces and different kinds of 3D discontinuities) using range data with real-world color images. We find that homogeneous image structures correspond to continuous surfaces, and discontinuities are mainly formed by edge-like or corner-like structures, which we discuss regarding potential computer vision applications and existing assumptions about the 3D world. In the second part, we utilize the measurements developed in the first part to investigate how the depth at homogeneous image structures is related to the depth of neighbor edges. For this, we first extract the local 3D structure of regularly sampled points, and then, analyze the coplanarity relation between these local 3D structures. We show that the likelihood to find a certain depth at a homogeneous image patch depends on the distance between the image patch and a neighbor edge. We find that this dependence is higher when there is a second neighbor edge which is coplanar with the first neighbor edge. These results allow deriving statistically based prediction models for depth interpolation on homogeneous image structures.     

A patch-based approach to 3D plant shoot phenotyping

The emerging discipline of plant phenomics aims to measure key plant characteristics, or traits, though as yet the set of plant traits that should be measured by automated systems is not well defined. Methods capable of recovering generic representations of the 3D structure of plant shoots from images would provide a key technology underpinning quantification of a wide range of current and future physiological and morphological traits. We present a fully automatic approach to image-based 3D plant reconstruction which represents plants as series of small planar sections that together model the complex architecture of leaf surfaces. The initial boundary of each leaf patch is refined using a level set method, optimising the model based on image information, curvature constraints and the position of neighbouring surfaces. The reconstruction process makes few assumptions about the nature of the plant material being reconstructed. As such it is applicable to a wide variety of plant species and topologies, and can be extended to canopy-scale imaging. We demonstrate the effectiveness of our approach on real images of wheat and rice plants, an artificial plant with challenging architecture, as well as a novel virtual dataset that allows us to compute distance measures of reconstruction accuracy. We also illustrate the method’s potential to support the identification of individual leaves, and so the phenotyping of plant shoots, using a spectral clustering approach.     

丝路文化虚拟体验中的多视角立体重建技术研究

丝路文化是联系一带一路战略的重要纽带,其传承意义重大,但是由于历史地理原因,丝路文化中代表性的历史遗产分散或损坏,难以有效地呈现,因此,本文面向丝路文化的虚拟展示与数字化,提出并实现了基于虚拟现实技术的丝路文化传承平台,通过历史遗迹复原以及基于图像的三维重建,还原了丝路文化中重要节点宁夏固原有关的历史遗迹、文物和事件....     

Weakly Supervised Part-wise 3D Shape Reconstruction from Single-View RGB Images

In order for the deep learning models to truly understand the 2D images for 3D geometry recovery, we argue that single-view reconstruction should be learned in a part-aware and weakly supervised manner. Such models lead to more profound interpretation of 2D images in which part-based parsing and assembling are involved. To this end, we learn a deep neural network which takes a single-view RGB image as input, and outputs a 3D shape in parts represented by 3D point clouds with an array of 3D part generators. In particular, we devise two levels of generative adversarial network (GAN) to generate shapes with both correct part shape and reasonable overall structure. To enable a self-taught network training, we devise a differentiable projection module along with a self-projection loss measuring the error between the shape projection and the input image. The training data in our method is unpaired between the 2D images and the 3D shapes with part decomposition. Through qualitative and quantitative evaluations on public datasets, we show that our method achieves good performance in part-wise single-view reconstruction.     

A learning-based visual saliency prediction model for stereoscopic 3D video (LBVS-3D)

Saliency prediction models provide a probabilistic map of relative likelihood of an image or video region to attract the attention of the human visual system. Over the past decade, many computational saliency prediction models have been proposed for 2D images and videos. Considering that the human visual system has evolved in a natural 3D environment, it is only natural to want to design visual attention models for 3D content. Existing monocular saliency models are not able to accurately predict the attentive regions when applied to 3D image/video content, as they do not incorporate depth information. This paper explores stereoscopic video saliency prediction by exploiting both low-level attributes such as brightness, color, texture, orientation, motion, and depth, as well as high-level cues such as face, person, vehicle, animal, text, and horizon. Our model starts with a rough segmentation and quantifies several intuitive observations such as the effects of visual discomfort level, depth abruptness, motion acceleration, elements of surprise, size and compactness of the salient regions, and emphasizing only a few salient objects in a scene. A new fovea-based model of spatial distance between the image regions is adopted for considering local and global feature calculations. To efficiently fuse the conspicuity maps generated by our method to one single saliency map that is highly correlated with the eye-fixation data, a random forest based algorithm is utilized. The performance of the proposed saliency model is evaluated against the results of an eye-tracking experiment, which involved 24 subjects and an in-house database of 61 captured stereoscopic videos. Our stereo video database as well as the eye-tracking data are publicly available along with this paper. Experiment results show that the proposed saliency prediction method achieves competitive performance compared to the state-of-the-art approaches.

     

Learning 3D Deformation of Animals from 2D Images

Understanding how an animal can deform and articulate is essential for a realistic modification of its 3D model. In this paper, we show that such information can be learned from user‐clicked 2D images and a template 3D model of the target animal. We present a volumetric deformation framework that produces a set of new 3D models by deforming a template 3D model according to a set of user‐clicked images. Our framework is based on a novel locally‐bounded deformation energy, where every local region has its own stiffness value that bounds how much distortion is allowed at that location. We jointly learn the local stiffness bounds as we deform the template 3D mesh to match each user‐clicked image. We show that this seemingly complex task can be solved as a sequence of convex optimization problems. We demonstrate the effectiveness of our approach on cats and horses, which are highly deformable and articulated animals. Our framework produces new 3D models of animals that are significantly more plausible than methods without learned stiffness.     

基于多约束融合的人手臂三维运动分析

为了在没有任何特殊标志的情况下,实现从单目序列图象中分析、估计人手臂的三维运动,提出了一种多约束融合的方法,该方法是利用棍棒模型来模拟人的手臂,首先通过处理单目图象序列来自动获取图象序列中手臂关节点的对应;然后再利用多约束融合及基于图象序列中关节点的对应,即估计尺度意义下关节点的三维相对运动轨迹;最后利用真实图象来获得相应人手臂的三维运动轨迹,并将其与通过运动捕捉系统获得的人手臂的真实三维运动轨迹进行了比较实验。实验结果表明,该方法用于对人手臂的运动分析非常有效。     

Real-time video-based modeling and rendering of 3D scenes

Our work targets 3D scenes in motion. In this article, we propose a method for view-dependent layered representation of 3D dynamic scenes. Using densely arranged cameras, we've developed a system that can perform processing in real time from image pickup to interactive display, using video sequences instead of static images, at 10 frames per second. In our system, images on layers are view dependent, and we update both the shape and image of each layer in real time. This lets us use the dynamic layers as the coarse structure of the dynamic 3D scenes, which improves the quality of the synthesized images. In this sense, our prototype system may be one of the first full real-time image -based modelling and rendering systems. Our experimental results show that this method is useful for interactive 3D rendering of real scenes     

Geometric fusion for a hand-held 3D sensor

Abstract. This article presents a geometric fusion algorithm developed for the reconstruction of 3D surface models from hand-held sensor data. Hand-held systems allow full 3D movement of the sensor to capture the shape of complex objects. Techniques previously developed for reconstruction from conventional 2.5D range image data cannot be applied to hand-held sensor data. A geometric fusion algorithm is introduced to integrate the measured 3D points from a hand-held sensor into a single continuous surface. The new geometric fusion algorithm is based on the normal-volume representation of a triangle, which enables incremental transformation of an arbitrary mesh into an implicit volumetric field function. This system is demonstrated for reconstruction of surface models from both hand-held sensor data and conventional 2.5D range images. Received: 30 August 1999 / Accepted: 21 January 2000     

2D/3D image registration on the GPU

We present a method that performs a rigid 2D/3D image registration efficiently on the Graphical Processing Unit (GPU). As one main contribution of this paper, we propose an efficient method for generating realistic DRRs that are visually similar to x-ray images. Therefore, we model some of the electronic post-processes of current x-ray C-arm-systems. As another main contribution, the GPU is used to compute eight intensity-based similarity measures between the DRR and the x-ray image in parallel. A combination of these eight similarity measures is used as a new similarity measure for the optimization. We evaluated the performance and the precision of our 2D/3D image registration algorithm using two phantom models. Compared to a CPU + GPU algorithm, which calculates the similarity measures on the CPU, our GPU algorithm is between three and six times faster. In contrast to single similarity measures, our new similarity measure achieved precise and robust registration results for both phantom models.     

Image-Driven Furniture Style for Interactive 3D Scene Modeling

Creating realistic styled spaces is a complex task, which involves design know-how for what furniture pieces go well together. Interior style follows abstract rules involving color, geometry and other visual elements. Following such rules, users manually select similar-style items from large repositories of 3D furniture models, a process which is both laborious and time-consuming. We propose a method for fast-tracking style-similarity tasks, by learning a furniture's style-compatibility from interior scene images. Such images contain more style information than images depicting single furniture. To understand style, we train a deep learning network on a classification task. Based on image embeddings extracted from our network, we measure stylistic compatibility of furniture. We demonstrate our method with several 3D model style-compatibility results, and with an interactive system for modeling style-consistent scenes.     

基于光线的全局优化多视图三维重建方法

利用光线跟踪原理,提出一种全局优化的多视图三维重建方法。根据图像轮廓得到物体的包围盒,采用体素离散物体所在的几何空间。从相机中心向图像上每个像素发射一条光线,为确定光线达到的体素,使用归一化互相关（NCC）值度量光线一体素的一致性,并估计采样空间中面片的法向信息,以提高NCC值的可信度。设计基于因子图的全局优化模型得到物体体素,针对光线因子的特殊性,设计一种高效的置信度传播算法,使重建方法的时间复杂度从指数阶降为线性阶。实验结果表明,与基于马尔可夫场的重建方法相比,该方法的鲁棒性较好,可提高重建模型的准确度和完整性。     

DeepGarment : 3D Garment Shape Estimation from a Single Image

3D garment capture is an important component for various applications such as free‐view point video, virtual avatars, online shopping, and virtual cloth fitting. Due to the complexity of the deformations, capturing 3D garment shapes requires controlled and specialized setups. A viable alternative is image‐based garment capture. Capturing 3D garment shapes from a single image, however, is a challenging problem and the current solutions come with assumptions on the lighting, camera calibration, complexity of human or mannequin poses considered, and more importantly a stable physical state for the garment and the underlying human body. In addition, most of the works require manual interaction and exhibit high run‐times. We propose a new technique that overcomes these limitations, making garment shape estimation from an image a practical approach for dynamic garment capture. Starting from synthetic garment shape data generated through physically based simulations from various human bodies in complex poses obtained through Mocap sequences, and rendered under varying camera positions and lighting conditions, our novel method learns a mapping from rendered garment images to the underlying 3D garment model. This is achieved by training Convolutional Neural Networks (CNN‐s) to estimate 3D vertex displacements from a template mesh with a specialized loss function. We illustrate that this technique is able to recover the global shape of dynamic 3D garments from a single image under varying factors such as challenging human poses, self occlusions, various camera poses and lighting conditions, at interactive rates. Improvement is shown if more than one view is integrated. Additionally, we show applications of our method to videos.     

图像质量评价研究综述——从失真的角度

随着多媒体技术的快速发展及广泛应用,图像质量评价因其在多媒体处理中的重要作用得到越来越多的关注,其作用包括图像数据筛选、算法参数选择与优化等。根据图像质量评价应用时是否需要参考信息,它可分为全参考图像质量评价、半参考图像质量评价和无参考图像质量评价,前两类分别需要全部参考信息和部分参考信息,而第3类不需要参考信息。无论是全参考、半参考还是无参考图像质量评价,图像失真对图像质量评价的影响均较大,主要体现在图像质量评价数据库构建和图像质量评价模型设计两方面。本文从图像失真的角度,主要概述2011—2021年国内外公开发表的图像质量评价模型,涵盖全参考、半参考和无参考模型。根据图像的失真类型,将图像质量评价模型分为针对合成失真的图像质量评价模型、针对真实失真的图像质量评价模型和针对算法相关失真的图像质量评价模型。其中,合成失真是指人工添加噪声,如高斯噪声和模糊失真,通常呈现均匀分布;真实失真是指在图像的获取中,由于环境、拍摄设备或拍摄操作不当等因素所引入的失真类型。相对合成失真,真实失真更为复杂,可能包括一种或多种失真,数据收集难度更大;算法相关失真是指图像处理算法或计算机视觉算法在处理图像...     

基于3D模型和AdaBoost算法的绝缘子检测

随着电力系统直升飞机巡线的不断发展与应用,对于输电线路关键部件的检测与识别越来越受到图像处理工作者的青睐。提出了一种利用3D模型制作训练样本及Ada Boost算法实现的航拍图像绝缘子自动检测方法。根据绝缘子3D模型图像的空间结构特征,提出了能反映这些结构的Haar矩形特征,从中挑选对绝缘子航拍图像有最好区分的特征构成弱分类器,再组合生成强分类器。使用正负样本图像训练后,由强分类器级联组成了一个多层分类器系统。实验结果表明,该方法有效地提升了绝缘子的识别效果,为后续的故障检测工作提供了良好的铺垫。     

Compression-Based 3D Texture Mapping for Real-Time Rendering

While 2D texture mapping is one of the most effective of the rendering techniques that make 3D objects appear visually interesting, it often suffers from visual artifacts produced when 2D image patterns are wrapped onto the surfaces of objects with arbitrary shapes. On the other hand, 3D texture mapping generates highly natural visual effects in which objects appear carved from lumps of materials rather than laminated with thin sheets as in 2D texture mapping. Storing 3D texture images in a table for fast mapping computations, instead of evaluating procedures on the fly, however, has been considered impractical due to the extremely high memory requirement. In this paper, we present a new effective method for 3D texture mapping designed for real-time rendering of polygonal models. Our scheme attempts to resolve the potential texture memory problem by compressing 3D textures using a wavelet-based encoding method. The experimental results on various nontrivial 3D textures and polygonal models show that high compression rates are achieved with few visual artifacts in the rendered images and a small impact on rendering time. The simplicity of our compression-based scheme will make it easy to implement practical 3D texture mapping in software/hardware rendering systems including real-time 3D graphics APIs such as OpenGL and Direct3D.     

Automatic generation of 3D outdoor and indoor building scenes from a single image

In this paper, a novel approach for creating 3D models of building scenes is presented. The proposed method is fully automated and fast, and accurately reconstructs both outdoor images of a building and indoor scenes, with perspective cues in real-time, using only one image. It combines the extracted line segments to identify the vanishing points of the image, the orientation, the different planes that are depicted in the image and concludes whether the image depicts indoor or outdoor scenes. In addition, the proposed method efficiently eliminates the perspective distortion and produces an accurate 3D model of the scene without any intervention from the user. The main innovation of the method is that it uses only one image for the 3D reconstruction, while other state-of-the-art methods rely on the processing of multiple images. A website and a database of 100 images were created to prove the efficiency of the proposed method in terms of time needed for the 3D reconstruction, its automation and 3D model accuracy and can be used by anyone so as to easily produce user-generated 3D content: http://3d-test.iti.gr:8080/3d-test/3D_recon/