基于运动恢复结构方法的植株三维重建

为了探究基于运动恢复结构(Structure From Motion, SFM)方法的植株三维重建模型的效果,为植物三维重建工作提供研究案例,本文以紫叶鸭跖草(setcreasea pallida)为研究对象,在搭建序列图像获取平台的基础上,选取35幅、75幅、105幅序列图像进行三维重建的对比分析;同时从植株表型参数方面,对植株三维重建模型进行精度评价。结果表明：75幅图像序列的重建效果最好;不同图像序列的模型计算的植株高度相对误差（Relative Error, RE）均小于2.5%,决定系数（coefficient of determination, R2）均大于0.998;不同图像序列的模型提取叶片长和叶片宽的RE均小于2.89%,R2均大于0.958。因此,序列图像的数量与重建模型的效果有关,但二者并非呈正相关关系;序列图像的数量对重建叶片的长与宽的误差影响较小;SFM方法应用于结构比较复杂的植株的三维重建可以取得较好的重建效果。     

基于叶片识别的侵入式室内植物建模

由于室内植物的叶片存在大量自遮挡,为了得到植物的完整三维信息,往往需要用户手动裁剪、扫描和配准叶片.针对该问题,提出了利用实例分割网络进行叶片识别并选取被裁剪叶片的方法.通过总结植物叶片形状与分布特征,建立虚拟植物模型,渲染大量带叶片轮廓信息的图片来训练实例分割网络,避免了耗费大量精力标注真实植物叶片.进一步提出了自动...     

基于非线性重构模型的植物叶片图像集分类方法

提出一种基于非线性重构模型的植物叶片图像集的分类识别方法。该方法首先使用高斯受限玻尔兹曼机(GRBMs)通过非监督预训练来初始化模型的权值;然后针对每一个植物叶片图像集用初始化的模型训练得到一个特定的模型;最后根据测试样本的最小重构误差和测试样本集的最多投票策略来判定测试样本集的类别。该方法通过图像预处理来处理图像,避免了图像在缩放时发生形变,并采用基于k-means的特征提取方法来提取植物叶片图像特征。实验结果表明,该方法能够准确地对植物叶片图像集进行分类识别。     

Validation of plant part measurements using a 3D reconstruction method suitable for high-throughput seedling phenotyping

In plant phenotyping, there is a demand for high-throughput, non-destructive systems that can accurately analyse various plant traits by measuring features such as plant volume, leaf area, and stem length. Existing vision-based systems either focus on speed using 2D imaging, which is consequently inaccurate, or on accuracy using time-consuming 3D methods. In this paper, we present a computer-vision system for seedling phenotyping that combines best of both approaches by utilizing a fast three-dimensional (3D) reconstruction method. We developed image processing methods for the identification and segmentation of plant organs (stem and leaf) from the 3D plant model. Various measurements of plant features such as plant volume, leaf area, and stem length are estimated based on these plant segments. We evaluate the accuracy of our system by comparing the measurements of our methods with ground truth measurements obtained destructively by hand. The results indicate that the proposed system is very promising.     

A computational model of bounded developable surfaces with application to image‐based three‐dimensional reconstruction

Developable surfaces have been extensively studied in computer graphics because they are involved in a large body of applications. This type of surfaces has also been used in computer vision and document processing in the context of three‐dimensional (3D) reconstruction for book digitization and augmented reality. Indeed, the shape of a smoothly deformed piece of paper can be very well modeled by a developable surface. Most of the existing developable surface parameterizations do not handle boundaries or are driven by overly large parameter sets. These two characteristics become issues in the context of developable surface reconstruction from real observations. Our main contribution is a generative model of bounded developable surfaces that solves these two issues. Our model is governed by intuitive parameters whose number depends on the actual deformation and including the “flat shape boundary”. A vast majority of the existing image‐based paper 3D reconstruction methods either require a tightly controlled environment or restricts the set of possible deformations. We propose an algorithm for reconstructing our model's parameters from a general smooth 3D surface interpolating a sparse cloud of 3D points. The latter is assumed to be reconstructed from images of a static piece of paper or any other developable surface. Our 3D reconstruction method is well adapted to the use of keypoint matches over multiple images. In this context, the initial 3D point cloud is reconstructed by structure‐from‐motion for which mature and reliable algorithms now exist and the thin‐plate spline is used as a general smooth surface model. After initialization, our model's parameters are refined with model‐based bundle adjustment. We experimentally validated our model and 3D reconstruction algorithm for shape capture and augmented reality on seven real datasets. The first six datasets consist of multiple images or videos and a sparse set of 3D points obtained by structure‐from‐motion. The last dataset is a dense 3D point cloud acquired by structured light. Our implementation has been made publicly available on the authors' web home pages. Copyright © 2012 John Wiley & Sons, Ltd.     

基于改进VGGNet模型的外来入侵植物叶片识别方法

针对自然界中不同种类植物的叶片可能存在类间差异小而导致一些边缘轮廓相似的本土植物和外来入侵植物叶片识别错误的问题,提出一种PF-VGGNet模型。常用的VGGNet模型在图像分类上表现优秀,采用顺次连接的结构,可以很好地提取图像的高级语义信息特征,但一些图像浅层的轮廓和纹理特征也对分类起到关键作用。PF-VGGNet模型可以将浅层轮廓和纹理特征与网络深层高级语义信息融合,实现对植物叶片的自动识别。实验结果表明,PF-VGGNet模型对比其它算法在自建的外来入侵植物叶片数据集上取得了较好的识别效果,在训练集和测试集上的准确率分别为99.89%和99.63%。PF-VGGNet可以有效降低因叶片边缘轮廓相近导致识别错误的问题,能够快速识别外来入侵植物叶片,为防治外来植物入侵提供支持。     

Photo Hull regularized stereo

A regularization-based approach to 3D reconstruction from multiple images is proposed. As one of the most widely used multiple-view 3D reconstruction algorithms, Space Carving can produce a Photo Hull of a scene, which is at best a coarse volumetric model. The two-view stereo algorithm, on the other hand, can generate a more accurate reconstruction of the surfaces, provided that a given surface is visible to both views. The proposed method is essentially a data fusion approach to 3D reconstruction, combining the above two algorithms by means of regularization. The process is divided into two steps: (1) computing the Photo Hull from multiple calibrated images and (2) selecting two of the images as input and solving the two-view stereo problem by global optimization, using the Photo Hull as the regularizer. Our dynamic programming implementation of this regularization-based stereo approach potentially provides an efficient and robust way of reconstructing 3D surfaces. The results of an implementation of this theory is presented on real data sets and compared with peer algorithms.     

Goal‐based Caustics

We propose a novel system for designing and manufacturing surfaces that produce desired caustic images when illuminated by a light source. Our system is based on a nonnegative image decomposition using a set of possibly overlapping anisotropic Gaussian kernels. We utilize this decomposition to construct an array of continuous surface patches, each of which focuses light onto one of the Gaussian kernels, either through refraction or reflection. We show how to derive the shape of each continuous patch and arrange them by performing a discrete assignment of patches to kernels in the desired caustic. Our decomposition provides for high fidelity reconstruction of natural images using a small collection of patches. We demonstrate our approach on a wide variety of caustic images by manufacturing physical surfaces with a small number of patches.     

Generation and application of hyperspectral 3D plant models: methods and challenges

Hyperspectral imaging sensors have been introduced for measuring the health status of plants. Recently, they also have been used for close-range sensing of plant canopies with a highly complex architecture. However, the complex geometry of plants and their interaction with the illumination setting severely affect the spectral information obtained. Furthermore, the spatial component of analysis results gain in importance as higher plants are represented by multiple plant organs as leaves, stems and seed pods. The combination of hyperspectral images and 3D point clouds is a promising approach to face these problems. We present the generation and application of hyperspectral 3D plant models as a new, interesting application field for computer vision with a variety of challenging tasks. We sum up a geometric calibration method for hyperspectral pushbroom cameras using a reference object for the combination of spectral and spatial information. Furthermore, we show exemplarily new calibration and analysis methods enabled by the hyperspectral 3D models in an experiment with sugar beet plants. An improved normalization, a comparison of image and 3D analysis and the density estimation of infected surface points underline some of the new capabilities gained using this new data type. Based on such hyperspectral 3D models the effects of plant geometry and sensor configuration can be quantified and modeled. In future, reflectance models can be used to remove or weaken the geometry-related effects in hyperspectral images and, therefore, have the potential to improve automated plant phenotyping significantly.     

Flexible three-dimensional modeling of plants using low- resolution cameras and visual odometry

The three-dimensional reconstruction of plants using computer vision methods is a promising alternative to non-destructive metrology in plant phenotyping. However, diversity in plants form and size, different surrounding environments (laboratory, greenhouse or field), and occlusions impose challenging issues. We propose the use of state-of-the-art methods for visual odometry to accurately recover camera pose and preliminary three-dimensional models on image acquisition time. Specimens of maize and sunflower were imaged using a single free-moving camera and a software tool with visual odometry capabilities. Multiple-view stereo was employed to produce dense point clouds sampling the plant surfaces. The produced three-dimensional models are accurate snapshots of the shoot state and plant measurements can be recovered in a non-invasive way. The results show a free-moving low-resolution camera is able to handle occlusions and variations in plant size and form, allowing the reconstruction of different species, and specimens in different stages of development. It is also a cheap and flexible method, suitable for different phenotyping needs. Plant traits were computed from the point clouds and compared to manually measured reference, showing millimeter accuracy. All data, including images, camera calibration, pose, and three-dimensional models are publicly available.     

Field‐based robotic phenotyping of sorghum plant architecture using stereo vision

Sorghum (Sorghum bicolor) is known as a major feedstock for biofuel production. To improve its biomass yield through genetic research, manually measuring yield component traits (e.g. plant height, stem diameter, leaf angle, leaf area, leaf number, and panicle size) in the field is the current best practice. However, such laborious and time‐consuming tasks have become a bottleneck limiting experiment scale and data acquisition frequency. This paper presents a high‐throughput field‐based robotic phenotyping system which performed side‐view stereo imaging for dense sorghum plants with a wide range of plant heights throughout the growing season. Our study demonstrated the suitability of stereo vision for field‐based three‐dimensional plant phenotyping when recent advances in stereo matching algorithms were incorporated. A robust data processing pipeline was developed to quantify the variations or morphological traits in plant architecture, which included plot‐based plant height, plot‐based plant width, convex hull volume, plant surface area, and stem diameter (semiautomated). These image‐derived measurements were highly repeatable and showed high correlations with the in‐field manual measurements. Meanwhile, manually collecting the same traits required a large amount of manpower and time compared to the robotic system. The results demonstrated that the proposed system could be a promising tool for large‐scale field‐based high‐throughput plant phenotyping of bioenergy crops.     

Fusing multiview and photometric stereo for 3D reconstruction under uncalibrated illumination

We propose a method to obtain a complete and accurate 3D model from multiview images captured under a variety of unknown illuminations. Based on recent results showing that for Lambertian objects, general illumination can be approximated well using low-order spherical harmonics, we develop a robust alternating approach to recover surface normals. Surface normals are initialized using a multi-illumination multiview stereo algorithm, then refined using a robust alternating optimization method based on the l(1) metric. Erroneous normal estimates are detected using a shape prior. Finally, the computed normals are used to improve the preliminary 3D model. The reconstruction system achieves watertight and robust 3D reconstruction while neither requiring manual interactions nor imposing any constraints on the illumination. Experimental results on both real world and synthetic data show that the technique can acquire accurate 3D models for Lambertian surfaces, and even tolerates small violations of the Lambertian assumption.     

Plant species identification using digital morphometrics: A review

Plants are of fundamental importance to life on Earth. The shapes of leaves, petals and whole plants are of great significance to plant science, as they can help to distinguish between different species, to measure plant health, and even to model climate change. The growing interest in biodiversity and the increasing availability of digital images combine to make this topic timely. The global shortage of expert taxonomists further increases the demand for software tools that can recognize and characterize plants from images. A robust automated species identification system would allow people with only limited botanical training and expertise to carry out valuable field work.We review the main computational, morphometric and image processing methods that have been used in recent years to analyze images of plants, introducing readers to relevant botanical concepts along the way. We discuss the measurement of leaf outlines, flower shape, vein structures and leaf textures, and describe a wide range of analytical methods in use. We also discuss a number of systems that apply this research, including prototypes of hand-held digital field guides and various robotic systems used in agriculture. We conclude with a discussion of ongoing work and outstanding problems in the area.     

单幅植物叶片图像的3维重建

目的 植物叶片形态复杂,在虚拟场景中很难真实表现。为了从信息量有限的单幅图像中恢复植物叶片的3维形状,本文基于从明暗恢复形状（shape from shading,SFS）的方法,利用亮度统计规律和植物形态特征恢复叶片的3维形状。方法 在SFS的基础上,设计基于图像骨架的距离场偏置加强表面细节;针对SFS对恢复宏观几何形状的不足,提出根据图像亮度统计分布选取控制点控制表面宏观形状变化,并利用叶片中轴的距离场约束恢复宏观几何形状,每种方法对于表面宏观几何形状恢复的权重基于恢复的反射图和输入图像间的相似度设定;将表面细节添加到宏观几何形状上得到目标对象的3维形状。结果 选取植物叶片图像进行实验,并与其他方法进行比较,实验结果表明本文方法增强了表面细节显示,并有明显的宏观几何形状变化。同时为了验证本文方法对其他物体表面细节恢复的适用性,分别对硬币和恐龙恢复表面细节,实验结果表明提出的增强表面细节的方法同样适用于其他物体。结论 针对单幅植物叶片图像的3维重建,在SFS的基础上提出了根据骨架特征加强表面细节,根据图像亮度统计分布和叶片中轴距离场约束共同恢复表面宏观几何形状的算法,实验结果验证了本文方法的可行性。     

Spatio‐temporal geometry fusion for multiple hybrid cameras using moving least squares surfaces

Multi‐view reconstruction aims at computing the geometry of a scene observed by a set of cameras. Accurate 3D reconstruction of dynamic scenes is a key component for a large variety of applications, ranging from special effects to telepresence and medical imaging. In this paper we propose a method based on Moving Least Squares surfaces which robustly and efficiently reconstructs dynamic scenes captured by a calibrated set of hybrid color+depth cameras. Our reconstruction provides spatio‐temporal consistency and seamlessly fuses color and geometric information. We illustrate our approach on a variety of real sequences and demonstrate that it favorably compares to state‐of‐the‐art methods.     

Registration of technical drawings and calibrated images for industrial augmented reality

Despite tremendous progress in 3D modelling technology, most sites in traditional industries do not have a computer model of their facilities at their disposal. In these industries, 2D technical drawings are typically the most commonly used documents. In many cases, a database of fully calibrated and oriented photogrammetric images of parts of the plant is also available. These images are often used for metric measurement and 3D as-built modelling. For planning revamps and maintenance, it is necessary to use industrial drawings as well as images and 3D models represented in a common “world” coordinate system. This paper proposes a method for full integration of technical drawings, calibrated images and as-built 3D models. A new algorithm is developed in order to use only a few correspondences between points on a technical drawing and multiple images to estimate a metric planar transformation between the drawing and the world coordinate system. The paper describes the mathematical relationship between this transformation and the set of homographies needed for merging the technical drawing with all the calibrated images. The method is implemented and fully integrated into an industrial software we developed for 3D as-built reconstruction. We present examples of a real application, in which the method is successfully applied to create an augmented reality representation of a waste water plant. Accepted: 13 August 2001     

Omnidirectional Image Stabilization for Visual Object Recognition

In this paper, we present a pipeline for camera pose and trajectory estimation, and image stabilization and rectification for dense as well as wide baseline omnidirectional images. The proposed pipeline transforms a set of images taken by a single hand-held camera to a set of stabilized and rectified images augmented by the computed camera 3D trajectory and a reconstruction of feature points facilitating visual object recognition. The paper generalizes previous works on camera trajectory estimation done on perspective images to omnidirectional images and introduces a new technique for omnidirectional image rectification that is suited for recognizing people and cars in images. The performance of the pipeline is demonstrated on real image sequences acquired in urban as well as natural environments.     

Surface deformation models for nonrigid 3D shape recovery

Three-dimensional detection and shape recovery of a nonrigid surface from video sequences require deformation models to effectively take advantage of potentially noisy image data. Here, we introduce an approach to creating such models for deformable 3D surfaces. We exploit the fact that the shape of an inextensible triangulated mesh can be parameterized in terms of a small subset of the angles between its facets. We use this set of angles to create a representative set of potential shapes, which we feed to a simple dimensionality reduction technique to produce low-dimensional 3D deformation models. We show that these models can be used to accurately model a wide range of deforming 3D surfaces from video sequences acquired under realistic conditions.