Automated crop plant detection based on the fusion of color and depth images for robotic weed control

Robotic weeding enables weed control near or within crop rows automatically, precisely and effectively. A computer‐vision system was developed for detecting crop plants at different growth stages for robotic weed control. Fusion of color images and depth images was investigated as a means of enhancing the detection accuracy of crop plants under conditions of high weed population. In‐field images of broccoli and lettuce were acquired 3–27 days after transplanting with a Kinect v2 sensor. The image processing pipeline included data preprocessing, vegetation pixel segmentation, plant extraction, feature extraction, feature‐based localization refinement, and crop plant classification. For the detection of broccoli and lettuce, the color‐depth fusion algorithm produced high true‐positive detection rates (91.7% and 90.8%, respectively) and low average false discovery rates (1.1% and 4.0%, respectively). Mean absolute localization errors of the crop plant stems were 26.8 and 7.4 mm for broccoli and lettuce, respectively. The fusion of color and depth was proved beneficial to the segmentation of crop plants from background, which improved the average segmentation success rates from 87.2% (depth‐based) and 76.4% (color‐based) to 96.6% for broccoli, and from 74.2% (depth‐based) and 81.2% (color‐based) to 92.4% for lettuce, respectively. The fusion‐based algorithm had reduced performance in detecting crop plants at early growth stages.     

Bayesian classification and unsupervised learning for isolating weeds in row crops

This paper presents a weed/crop classification method using computer vision and morphological analysis. Subsequent supervised and unsupervised learning methods are applied to extract dominant morphological characteristics of weeds present in corn and soybean fields. The novelty of the presented technique resides in the feature extraction process that is based on spatial localization of vegetation in fields. Features from the weed leaf area distribution are extracted from the cultivation inter-rows, then features from the crop are inferred from the mixture model equation. Those extracted features are then passed to a naive bayesian classifier and a gaussian mixture clustering algorithm to discriminate weed from crop plant. The presented technique correctly classifies an average of 94 % of corn and soybean plants and 85 % of the weed (multiple species) without any prior knowledge on the species present in the field.     

基于多特征融合与极限学习机的植物叶片分类方法

对植物的分类多通过对植物叶片的分类来实现,为提高植物叶片分类的准确率提出了一种基于多特征融合与极限学习机的植物叶片分类方法.首先对植物叶片彩色图像进行预处理,得到去除叶片颜色与背景的二值图像和灰度图像;然后从二值图像中提取植物叶片的形状特征和不变矩特征,利用灰度图像提取灰度共生矩阵参数作为叶片图像的纹理特征,共得到28...     

一种结合形状与纹理特征的植物叶片分类方法

根据植物叶片识别植物种类对于生物科学与生态科学具有重要的辅助作用。针对叶片分类,提出了一种基于形状与纹理特征的分类算法。在进行了去噪等预处理后,通过阈值分割和数学形态学方法获取叶片区域;在分割得到的二值区域图像上提取了形状特征,在灰度图像上提取了纹理特征;在所得特征的基础上,利用BP网络对叶片进行分类。在实际图片上的实验结果表明,相比于已有算法,该方法可以达到更高的正确分类率。     

基于降维LBP与叶片形状特征的植物叶片识别方法

为解决由于植物叶片特征的相似性以及叶片旋转导致植物识别率较低的问题,提出一种基于降维局部二值模式（LBP）与叶片形状特征相结合的植物叶片识别方法。首先利用LBP算法提取高维叶片纹理信息,通过主成分分析方法（PCA）对高维叶片特征降维;同时考虑叶片的形状特征,将LBP旋转不变性特征与叶片形状特征有效结合,在低维空间利用k近邻法（KNN）实现叶片的分类与识别。实验结果表明该方法具有较好的识别效果。     

基于形状特征的叶片图像识别算法比较研究

植物是生命的主要形态之一,其种类已达40多万种,对其进行分类识别在生物多样性保护,生态农业,生物安全中有着重要的意义。不同的种类的植物一般有着不同的叶片形状,因此叶片的形状特征在植物分类中扮演着重要的角色。作为计算机视觉的一个重要应用的植物叶片图像识别,近些年来受到了学者们的关注,产生了大量的研究成果。但由于植物种类巨大,叶片图像存在的类内差异大、类间差异小和叶片的自遮挡等问题等诸多问题,使得叶片图像的识别仍然是目前计算机视觉应用研究的一个热点。对近些年来的基于形状特征的叶片图像识别算法进行了综述和比较,对现有的算法进行了分类,对目前各类最先进的识别算法进行了分析和比较。此外,还介绍了常用的叶片图像测试集和性能评估方法,并将各类算法进行了实验结果的比较研究。研究工作既为现有的植物叶片识别算法的实际应用提供了指导,又为今后进一步研究新的高性能的识别算法提出了努力的方向。     

Crop recognition under weedy conditions based on 3D imaging for robotic weed control

A 3D time‐of‐flight camera was applied to develop a crop plant recognition system for broccoli and green bean plants under weedy conditions. The developed system overcame the previously unsolved problems caused by occluded canopy and illumination variation. An efficient noise filter was developed to remove the sparse noise points in 3D point cloud space. Both 2D and 3D features including the gradient of amplitude and depth image, surface curvature, amplitude percentile index, normal direction, and neighbor point count in 3D space were extracted and found effective for recognizing these two types of plants. Separate segmentation algorithms were developed for each of the broccoli and green bean plant in accordance with their 3D geometry and 2D amplitude characteristics. Under the experimental condition where the crops were heavily infested by various types of weed plants, detection rates over 88.3% and 91.2% were achieved for broccoli and green bean plant leaves, respectively. Additionally, the crop plants were segmented out with nearly complete shape. Moreover, the algorithms were computationally optimized, resulting in an image processing speed of over 30 frames per second.     

Object‐oriented classification of sidescan sonar data for mapping benthic marine habitats

This research presents an object‐oriented technique for habitat classification at different segmentation levels based on the use of imagery from an Edgetech 272 side scan sonar. We investigate the success of object parameters such as shape and size as well as texture in discriminating reef from sand habitat. The results are evaluated using traditional digitization, based on visual assessment of the sidescan imagery, and video transects. Whereas the application of traditional pixel‐based classification results in a pixelized (salt and pepper) representation of habitat distribution, the object‐based classification technique results in habitat objects (raster or vector). The object‐oriented classification results are cross‐validated using confusion matrices in image classification software and error matrices from underwater video transects showing an overall accuracy of 80% based on two classes within the image at three segmentation levels and an overall accuracy of 60% based on three classes at two segmentation levels. This is compared with the digitized layer accuracy of 81% for two classes and 72% for three classes, and this demonstrates the successful application of object‐oriented methods for habitat mapping. This technique retains spatially discrete habitat pattern information in a classified vector shape file with methods that are automated, repeatable, objective, and capable of processing many sidescan records in a more efficient manner.     

Robust joint stem detection and crop‐weed classification using image sequences for plant‐specific treatment in precision farming

Conventional farming still relies on large quantities of agrochemicals for weed management which have several negative side‐effects on the environment. Autonomous robots offer the potential to reduce the amount of chemicals applied, as robots can monitor and treat each plant in the field individually and thereby circumventing the uniform chemical treatment of the whole field. Such agricultural robots need the ability to identify individual crops and weeds in the field using sensor data and must additionally select effective treatment methods based on the type of weed. For example, certain types of weeds can only be effectively treated mechanically due to their resistance to herbicides, whereas other types can be treated trough selective spraying. In this article, we present a novel system that provides the necessary information for effective plant‐specific treatment. It estimates the stem location for weeds, which enables the robots to perform precise mechanical treatment, and at the same time provides the pixel‐accurate area covered by weeds for treatment through selective spraying. The major challenge in developing such a system is the large variability in the visual appearance that occurs in different fields. Thus, an effective classification system has to robustly handle substantial environmental changes including varying weed pressure, various weed types, different growth stages, changing visual appearance of the plants and the soil. Our approach uses an end‐to‐end trainable fully convolutional network that simultaneously estimates plant stem positions as well as the spatial extent of crop plants and weeds. It jointly learns how to detect the stems and the pixel‐wise semantic segmentation and incorporates spatial information by considering image sequences of local field strips. The jointly learned feature representation for both tasks furthermore exploits the crop arrangement information that is often present in crop fields. This information is considered even if it is only observable from the image sequences and not a single image. Such image sequences, as typically provided by robots navigating over the field along crop rows, enable our approach to robustly estimate the semantic segmentation and stem positions despite the large variations encountered in different fields. We implemented and thoroughly tested our approach on images from multiple farms in different countries. The experiments show that our system generalizes well to previously unseen fields under varying environmental conditions—a key capability to deploy such systems in the real world. Compared to state‐of‐the‐art approaches, our approach generalizes well to unseen fields and not only substantially improves the stem detection accuracy, that is, distinguishing crop and weed stems, but also improves the semantic segmentation performance.     

基于改进VGGNet模型的外来入侵植物叶片识别方法

针对自然界中不同种类植物的叶片可能存在类间差异小而导致一些边缘轮廓相似的本土植物和外来入侵植物叶片识别错误的问题,提出一种PF-VGGNet模型。常用的VGGNet模型在图像分类上表现优秀,采用顺次连接的结构,可以很好地提取图像的高级语义信息特征,但一些图像浅层的轮廓和纹理特征也对分类起到关键作用。PF-VGGNet模型可以将浅层轮廓和纹理特征与网络深层高级语义信息融合,实现对植物叶片的自动识别。实验结果表明,PF-VGGNet模型对比其它算法在自建的外来入侵植物叶片数据集上取得了较好的识别效果,在训练集和测试集上的准确率分别为99.89%和99.63%。PF-VGGNet可以有效降低因叶片边缘轮廓相近导致识别错误的问题,能够快速识别外来入侵植物叶片,为防治外来植物入侵提供支持。     

Vegetation classification method with biochemical composition estimated from remote sensing data

In this article, a vegetation classification hypothesis based on plant biochemical composition is presented. The basic idea of this hypothesis is that the vegetation species/crops have their own biochemical composition characteristics, which are separable from each other for those co-existing species at a specific region. Therefore, vegetation species can be classified based on the biochemical composition characteristics, which can be retrieved from hyperspectral remote-sensing data. In order to test this hypothesis, an experiment was conducted in north-western China. Field data on the biochemical compositions and spectral responses of different plants and an Earth-observing 1 (EO-1) Hyperion image were simultaneously collected. After analysing the relationship between biochemical composition and spectral data collected from Hyperion, the vegetation biochemical compositions were estimated using sample biochemical data and bands of Hyperion data. The vegetation classification was completed using the biochemical content classifier (BCC) and maximum-likelihood classifier (MLC) with all Hyperion bands (MLC_A) and selected bands (MLC_S), which were used for estimating considered biochemical contents (cellulose and carotenoid). The overall classification accuracy of the BCC (95.2%) was as good as MLC_S (95.2%) and better than MLC_A (91.1%), as was the kappa value (BCC 92.849%, MLC_S 92.845%, MLC_A 86.637%), suggesting that the BCC was a feasible classification method. The biochemical-based classification method has higher vegetation classification accuracy and execution speed, reduces data dimension and redundancy and needs only a few spectral bands to retrieve biochemical contents instead of using all of the spectral bands. It is an effective method to classify vegetation based on plant biochemical composition characteristics.     

A robot to detect and control broad‐leaved dock (Rumex obtusifolius L.) in grassland

Broad‐leaved dock is a common and troublesome grassland weed with a wide geographic distribution. In conventional farming the weed is normally controlled by using a selective herbicide, but in organic farming manual removal is the best option to control this weed. The objective of our work was to develop a robot that can navigate a pasture, detect broad‐leaved dock, and remove any weeds found. A prototype robot was constructed that navigates by following a predefined path using centimeter‐precision global positioning system (GPS). Broad‐leaved dock is detected using a camera and image processing. Once detected, weeds are destroyed by a cutting device. Tests of aspects of the system showed that path following accuracy is adequate but could be improved through tuning of the controller or adoption of a dynamic vehicle model, that the success rate of weed detection is highest when the grass is short and when the broad‐leaved dock plants are in rosette form, and that 75% of weeds removed did not grow back. An on‐farm field test of the complete system resulted in detection of 124 weeds of 134 encountered (93%), while a weed removal action was performed eight times without a weed being present. Effective weed control is considered to be achieved when the center of the weeder is positioned within 0.1 m of the taproot of the weed—this occurred in 73% of the cases. We conclude that the robot is an effective instrument to detect and control broad‐leaved dock under the conditions encountered on a commercial farm. © 2010 Wiley Periodicals, Inc.     

基于图像特征的植物形态相似度算法

目的 研究不同植物形态之间的相似度是有效区分植物种类或科属的一个重要依据。目前的植物形态相似度计算方法,大多只考虑了植物拓扑结构或者外围轮廓等几何形状方面的相似性,而未涉及叶片颜色、冠层叶片的稠密状态及株型的松散状态等因素。因此,基于植物图像的形状特征和颜色特征,本文提出一种基于图像特征的植物形态相似度计算方法。方法 首先,获取图像的轮廓特征和区域特征。轮廓特征用植物枝条的松散程度表示,具体包括植物的高宽比、轮廓四边形和第1个1级侧枝的高度;区域特征用叶片稠密度表示,计算叶片所占整个包围矩形面积的比例。其次,获取图像的颜色特征,使用基于HSV和YUV颜色空间的颜色直方图,统计图像的颜色分布。最后,利用信息熵分析数据的离散程度,据此确定各部分对应的权重大小,加权得到总体的相似度值。结果 实验在人工采集的数据集上进行,得出松散度、稠密度和颜色对应的权重分别为0.62、0.17和0.21。在此基础上得到的相似度计算结果符合实际,可以有效度量植物之间的相似程度。同时,将提出的算法应用于图像检索,并与常见的5种方法进行比较。实验得出该算法查准率都在0.747 7以上。在同一查准率水平下,相比于其他方法,查全率也都处于较高水平。尤其在相似度阈值大于0.8时,查准率可以达到0.910 8以上。另外,该方法对植物图像缩放不敏感,同类植物的相似度依然接近于1。结论 本文提出的植物形态相似度算法,结合了形状特征和颜色特征,计算结果符合人的视觉感受。与其他方法相比,可以更有效区分植物种类或科属。算法主要适用于背景单一的单株植物图像,可为研究植物形态的相似性提供技术参考。     

干旱区植物叶片识别研究

现有植物叶片识别方法都是基于扁平状叶片，而干旱区植物叶片多呈针叶，因此不适合干旱区植物叶片的识别，使得对于干旱区植物研究主要依靠专家识别，不利于对干旱区植物叶片的进一步研究。提出使用差异性值监督局部线性嵌入算法D-LLE，充分挖掘样本之间的类别信息，提高干旱区植物叶片的识别效率。首先利用金字塔梯度方向直方图（PHOG）的方法提取叶片图像特征，再使用PCA、LLE、WLLE、D-LLE等主流的降维算法，对提取的PHOG特征进行降维，最后建立支持向量机（SVM）的分类模型对植物叶片图像分类。经过这四种降维算法后的平均识别率分别为76.3%、85.3%、89.1%、95.5% ；骆驼刺、苦豆子和沙枣的叶片正确识别率，相对其他植物叶片较低。通过实验证明了PHOG特征在植物叶片特征提取的可行性，使用D-LLE算法相比传统特征降维的算法具有更高的效率，且较适合于干旱区植物叶片的自动识别分类。     

叶片图像特征提取与识别技术的研究

文章介绍了一种基于叶片图像的形状特征对叶片进行识别的方法。首先对叶片图像进行预处理并且提取出叶片的轮廓,然后利用轮廓计算得到叶片的矩形度、圆形度、偏心率等8项几何特征和7个图像不变矩,同时在文中提出了一种新的移动中心超球分类器,利用其对得到的形状特征进行分类从而实现了对20多种植物叶片的快速识别,并且平均识别率达到了92%。     

Fractal dimension applied to plant identification

This article discusses methods to identify plants by analysing leaf complexity based on estimating their fractal dimension. Leaves were analyzed according to the complexity of their internal and external shapes. A computational program was developed to process, analyze and extract the features of leaf images, thereby allowing for automatic plant identification. Results are presented from two experiments, the first to identify plant species from the Brazilian Atlantic forest and Brazilian Cerrado scrublands, using fifty leaf samples from ten different species, and the second to identify four different species from genus Passiflora, using twenty leaf samples for each class. A comparison is made of two methods to estimate fractal dimension (box-counting and multiscale Minkowski). The results are discussed to determine the best approach to analyze shape complexity based on the performance of the technique, when estimating fractal dimension and identifying plants.     

Species classification of aquatic plants using GRNN and BPNN

Computer-aided plant species identification acts significantly on plant digital museum system and systematic botany, which is the groundwork for research and development of plants. This work presents a method for plant species identification using the images of flowers. It focuses on the stable feature extraction of flowers such as color, texture and shape features. Color-based segmentation using k-means clustering is used to extract the color features. Texture segmentation using texture filter is used to segment the image and obtain texture features. Sobel, Prewitt and Robert operators are used to extract the boundary of image and to obtain the shape features. From 405 images of flowers, color, texture and shape features are extracted. Classification of the plants into dry land plants and aquatic plants, the aquatic plant species into wet and marsh aquatic plants, wet aquatic plants into Iridaceae and Epilobium family and marsh aquatic plants into Malvaceae and Onagraceae family, the Iridaceae family is again classified into Babiana and Crocus species, the family Epilobium into Canum and Hirsutum, the family Malvaceae into Mallow and Pavonia, the family Onagraceae into Fuschia and Ludwigia species are done using general regression neural network and backpropagation neural network classifiers.     

基于叶片识别的侵入式室内植物建模

由于室内植物的叶片存在大量自遮挡,为了得到植物的完整三维信息,往往需要用户手动裁剪、扫描和配准叶片.针对该问题,提出了利用实例分割网络进行叶片识别并选取被裁剪叶片的方法.通过总结植物叶片形状与分布特征,建立虚拟植物模型,渲染大量带叶片轮廓信息的图片来训练实例分割网络,避免了耗费大量精力标注真实植物叶片.进一步提出了自动...     

Usefulness of remote sensing for the assessment of growth traits in individual cereal plants grown in the field

Biomass determination usually involves destructive and tedious measurements. This study was conducted to evaluate the usefulness of the Normalized Difference Vegetation Index (NDVI) and the Simple Ratio (SR), calculated from the spectra of individual plants, for the assessment of leaf area per plant (LAP), green area per plant (GAP) and plant dry weight (W) at different growth stages. Two varieties of four cereal species (barley, bread wheat, durum wheat and triticale) were sown in a field experiment at a density of 25 plants m^?2. The spectra were captured on three plants per plot on eight occasions from the beginning of jointing to heading using a narrow‐bandwidth visible‐near‐infrared portable field spectroradiometer adapted for measurements at plant level. Strong associations were found between NDVI and SR and growth traits, both indices being better estimators of GAP and W than of LAP. Exponential models fitted to NDVI data were more useful for a wide number of situations than the linear models fitted to SR data. However, SR was able to discriminate between genotypes within a species. The accuracy of the reflectance measurements was comparable to that obtained by destructive measurements of growth traits, in which differences between varieties of over 24% were needed to be statistically significant. However, differences in SR of only 18% were statistically significant (P<0.05). The reliability of the spectral reflectance measurements and the non‐destructive nature convert this methodology into a promising tool for the assessment of growth traits in spaced individual plants.