A patch-based approach to 3D plant shoot phenotyping

The emerging discipline of plant phenomics aims to measure key plant characteristics, or traits, though as yet the set of plant traits that should be measured by automated systems is not well defined. Methods capable of recovering generic representations of the 3D structure of plant shoots from images would provide a key technology underpinning quantification of a wide range of current and future physiological and morphological traits. We present a fully automatic approach to image-based 3D plant reconstruction which represents plants as series of small planar sections that together model the complex architecture of leaf surfaces. The initial boundary of each leaf patch is refined using a level set method, optimising the model based on image information, curvature constraints and the position of neighbouring surfaces. The reconstruction process makes few assumptions about the nature of the plant material being reconstructed. As such it is applicable to a wide variety of plant species and topologies, and can be extended to canopy-scale imaging. We demonstrate the effectiveness of our approach on real images of wheat and rice plants, an artificial plant with challenging architecture, as well as a novel virtual dataset that allows us to compute distance measures of reconstruction accuracy. We also illustrate the method’s potential to support the identification of individual leaves, and so the phenotyping of plant shoots, using a spectral clustering approach.     

Direct Estimation of Nonrigid Registrations with Image-Based Self-Occlusion Reasoning

The registration problem for images of a deforming surface has been well studied. External occlusions are usually well handled. In 2D image-based registration, self-occlusions are more challenging. Consequently, the surface is usually assumed to be only slightly self-occluding. This paper is about image-based nonrigid registration with self-occlusion reasoning. A specific framework explicitly modeling self-occlusions is proposed. It is combined with an intensity-based, “direct” data term for registration. Self-occlusions are detected as shrinkage areas in the 2D warp. Experimental results on several challenging data sets show that our approach successfully registers images with self-occlusions while effectively detecting the self-occluded regions.     

Special relativistic visualization by local ray tracing

Special relativistic visualization offers the possibility of experiencing the optical effects of traveling near the speed of light, including apparent geometric distortions as well as Doppler and searchlight effects. Early high-quality computer graphics images of relativistic scenes were created using offline, computationally expensive CPU-side 4D ray tracing. Alternate approaches such as image-based rendering and polygon-distortion methods are able to achieve interactivity, but exhibit inferior visual quality due to sampling artifacts. In this paper, we introduce a hybrid rendering technique based on polygon distortion and local ray tracing that facilitates interactive high-quality visualization of multiple objects moving at relativistic speeds in arbitrary directions. The method starts by calculating tight image-space footprints for the apparent triangles of the 3D scene objects. The final image is generated using a single image-space ray tracing step incorporating Doppler and searchlight effects. Our implementation uses GPU shader programming and hardware texture filtering to achieve high rendering speed.     

Prediction-based realistic 3D model compression

The benefit of using the geometry image to represent an arbitrary 3D mesh is that the 3D mesh can be re-sampled as a completely regular structure and coded efficiently by common image compression methods. For geometry image-based 3D mesh compression, we need to code the normal-map images while coding geometry images to improve the subjective quality and realistic effects of the reconstructed model. In traditional methods, a geometry image and a normal-map image are coded independently. However a strong correlation exists between these two kinds of images, because both of them are generated from the same 3D mesh and share the same parameterization. In this paper we propose a predictive coding framework, in which the normal-map image is predicted based on the geometric correlation between them. Additionally we utilize the strong geometric correlation among three components of normal-map image to improve the predicting accuracy. The experimental results show the proposed coding framework improves the coding efficiency of normal-map image, meanwhile the realistic effect of a 3D mesh is significantly enhanced.     

Multiview Visibility Estimation for Image-Based Modeling

In this paper, we investigate the problem of determining regions in 3D scene visible to some given viewpoints when obstacles are present in the scene. We assume that the obstacles are composed of some opaque objects with closed surfaces. The problem is formulated in an implicit framework where the obstacles are represented by a level set function. The visible and invisible regions of the given viewpoints are determined through an efficient implicit ray tracing technique. As an extension of our approach, we apply the multiview visibility estimation to an image-based modeling technique. The unknown scene geometry and multiview visibility information are incorporated into a variational energy functional. By minimizing the energy functional, the true scene geometry as well as the accurate visibility information of the multiple views can be recovered from a number of scene images. This makes it feasible to handle the visibility problem of multiple views by our approach when the true scene geometry is unknown.     

Make3D: Learning 3D Scene Structure from a Single Still Image

We consider the problem of estimating detailed 3D structure from a single still image of an unstructured environment. Our goal is to create 3D models that are both quantitatively accurate as well as visually pleasing. For each small homogeneous patch in the image, we use a Markov Random Field (MRF) to infer a set of "plane parameters” that capture both the 3D location and 3D orientation of the patch. The MRF, trained via supervised learning, models both image depth cues as well as the relationships between different parts of the image. Other than assuming that the environment is made up of a number of small planes, our model makes no explicit assumptions about the structure of the scene; this enables the algorithm to capture much more detailed 3D structure than does prior art and also give a much richer experience in the 3D flythroughs created using image-based rendering, even for scenes with significant nonvertical structure. Using this approach, we have created qualitatively correct 3D models for 64.9 percent of 588 images downloaded from the Internet. We have also extended our model to produce large-scale 3D models from a few images.     

A video prediction approach for animating single face image

Generating dynamic 2D image-based facial expressions is a challenging task for facial animation. Much research work focused on performance-driven facial animation from given videos or images of a target face, while animating a single face image driven by emotion labels is a less explored problem. In this work, we treat the task of animating single face image from emotion labels as a conditional video prediction problem, and propose a novel framework by combining factored conditional restricted boltzmann machines (FCRBM) and reconstruction contractive auto-encoder (RCAE). A modified RCAE with an associated efficient training strategy is used to extract low dimensional features and reconstruct face images. FCRBM is used as animator to predict facial expression sequence in the feature space given discrete emotion labels and a frontal neutral face image as input. Both quantitative and qualitative evaluations on two facial expression databases, and comparison to state-of-the-art showed the effectiveness of our proposed framework for animating frontal neutral face image from given emotion labels.

     

从单幅高动态范围图像恢复环境中物体的材质

提出一种从单幅高动态范围图像恢复一般环境中物体材质的方法,适用于单一材质物体,对物体形状和光照条件没有任何特殊要求.在一般光照环境中,获取被考察物体的一幅高动态范围图像以及用来近似物体光照的一个或几个高动态范围环境映照,然后用模拟退火算法求解逆向绘制问题.在求解过程中采用了基于图像的光照和光线跟踪技术,充分考虑了物体自身互反射的影响.最后得到了物体表面反射模型的最优参数.若与基于图像的建模技术相结合,可以根据真实物体的照片建立真实感模型.     

Aerial full spherical HDR imaging and display

This paper describes a framework for aerial imaging of high dynamic range (HDR) scenes for use in virtual reality applications, such as immersive panorama applications and photorealistic superimposition of virtual objects using image-based lighting. We propose a complete and practical system to acquire full spherical HDR images from the sky, using two omnidirectional cameras mounted above and below an unmanned aircraft. The HDR images are generated by combining multiple omnidirectional images captured with different exposures controlled automatically. Our system consists of methods for image completion, alignment, and color correction, as well as a novel approach for automatic exposure control, which selects optimal exposure so as to avoid banding artifacts. Experimental results indicated that our system generated better spherical images compared to an ordinary spherical image completion system in terms of naturalness and accuracy. In addition to proposing an imaging method, we have carried out an experiment about display methods for aerial HDR immersive panoramas utilizing spherical images acquired by the proposed system. The experiment demonstrated HDR imaging is beneficial to immersive panorama using an HMD, in addition to ordinary uses of HDR images.     

Generation and application of hyperspectral 3D plant models: methods and challenges

Hyperspectral imaging sensors have been introduced for measuring the health status of plants. Recently, they also have been used for close-range sensing of plant canopies with a highly complex architecture. However, the complex geometry of plants and their interaction with the illumination setting severely affect the spectral information obtained. Furthermore, the spatial component of analysis results gain in importance as higher plants are represented by multiple plant organs as leaves, stems and seed pods. The combination of hyperspectral images and 3D point clouds is a promising approach to face these problems. We present the generation and application of hyperspectral 3D plant models as a new, interesting application field for computer vision with a variety of challenging tasks. We sum up a geometric calibration method for hyperspectral pushbroom cameras using a reference object for the combination of spectral and spatial information. Furthermore, we show exemplarily new calibration and analysis methods enabled by the hyperspectral 3D models in an experiment with sugar beet plants. An improved normalization, a comparison of image and 3D analysis and the density estimation of infected surface points underline some of the new capabilities gained using this new data type. Based on such hyperspectral 3D models the effects of plant geometry and sensor configuration can be quantified and modeled. In future, reflectance models can be used to remove or weaken the geometry-related effects in hyperspectral images and, therefore, have the potential to improve automated plant phenotyping significantly.     

三维分层分支树状模式的复原:等式与推导

本文提出一种三维分层分支结构建模与复原算法．该算法利用模型成功地估计树图象的结构参数，进行运动模式的光流分析，且可由树类植物的单幅图象复原其３Ｄ模式的几何特性.     

基于VHDL的系统规范语言的研究

将ＶＨＤＬ语言应用到嵌入式系统的设计提供支持是扩展ＶＨＤＬ对系统级设计支持的一种研究途径．本文考察了几个典型的对系统级设计提供支持的规范语言，它们都将ＶＨＤＬ集成到自己的设计方法之中．在此基础上，作者总结了支持系统级设计的规范语言的基本要素，指出将形式描述方法与ＶＨＤＬ相结合可提供一种更有效的设计支持工具．     

Foveated Real‐Time Ray Tracing for Head‐Mounted Displays

Head‐mounted displays with dense pixel arrays used for virtual reality applications require high frame rates and low latency rendering. This forms a challenging use case for any rendering approach. In addition to its ability of generating realistic images, ray tracing offers a number of distinct advantages, but has been held back mainly by its performance. In this paper, we present an approach that significantly improves image generation performance of ray tracing. This is done by combining foveated rendering based on eye tracking with reprojection rendering using previous frames in order to drastically reduce the number of new image samples per frame. To reproject samples a coarse geometry is reconstructed from a G‐Buffer. Possible errors introduced by this reprojection as well as parts that are critical to the perception are scheduled for resampling. Additionally, a coarse color buffer is used to provide an initial image, refined smoothly by more samples were needed. Evaluations and user tests show that our method achieves real‐time frame rates, while visual differences compared to fully rendered images are hardly perceivable. As a result, we can ray trace non‐trivial static scenes for the Oculus DK2 HMD at 1182 × 1464 per eye within the the VSync limits without perceived visual differences.     

Plenoptic Image Editing

This paper presents a new class of interactive image editing operations designed to maintain consistency between multiple images of a physical 3D scene. The distinguishing feature of these operations is that edits to any one image propagate automatically to all other images as if the (unknown) 3D scene had itself been modified. The modified scene can then be viewed interactively from any other camera viewpoint and under different scene illuminations. The approach is useful first as a power-assist that enables a user to quickly modify many images by editing just a few, and second as a means for constructing and editing image-based scene representations by manipulating a set of photographs. The approach works by extending operations like image painting, scissoring, and morphing so that they alter a scene's plenoptic function in a physically-consistent way, thereby affecting scene appearance from all viewpoints simultaneously. A key element inrealizing these operations is a new volumetric decomposition technique for reconstructing an scene's plenoptic function from an incomplete set of camera viewpoints.     

Flexible three-dimensional modeling of plants using low- resolution cameras and visual odometry

The three-dimensional reconstruction of plants using computer vision methods is a promising alternative to non-destructive metrology in plant phenotyping. However, diversity in plants form and size, different surrounding environments (laboratory, greenhouse or field), and occlusions impose challenging issues. We propose the use of state-of-the-art methods for visual odometry to accurately recover camera pose and preliminary three-dimensional models on image acquisition time. Specimens of maize and sunflower were imaged using a single free-moving camera and a software tool with visual odometry capabilities. Multiple-view stereo was employed to produce dense point clouds sampling the plant surfaces. The produced three-dimensional models are accurate snapshots of the shoot state and plant measurements can be recovered in a non-invasive way. The results show a free-moving low-resolution camera is able to handle occlusions and variations in plant size and form, allowing the reconstruction of different species, and specimens in different stages of development. It is also a cheap and flexible method, suitable for different phenotyping needs. Plant traits were computed from the point clouds and compared to manually measured reference, showing millimeter accuracy. All data, including images, camera calibration, pose, and three-dimensional models are publicly available.     

Template-based quadrilateral mesh generation from imaging data

This paper describes a novel template-based meshing approach for generating good quality quadrilateral meshes from 2D digital images. This approach builds upon an existing image-based mesh generation technique called Imeshp, which enables us to create a segmented triangle mesh from an image without the need for an image segmentation step. Our approach generates a quadrilateral mesh using an indirect scheme, which converts the segmented triangle mesh created by the initial steps of the Imesh technique into a quadrilateral one. The triangle-to-quadrilateral conversion makes use of template meshes of triangles. To ensure good element quality, the conversion step is followed by a smoothing step, which is based on a new optimization-based procedure. We show several examples of meshes generated by our approach, and present a thorough experimental evaluation of the quality of the meshes given as examples.