Livestock detection in aerial images using a fully convolutional network

Computational Visual Media - In order to accurately count the number of animals grazing on grassland, we present a livestock detection algorithm using modified versions of U-net and Google...     

Evidential fully convolutional network for semantic segmentation

Applied Intelligence - We propose a hybrid architecture composed of a fully convolutional network (FCN) and a Dempster-Shafer layer for image semantic segmentation. In the so-called evidential FCN...     

结合全卷积网络和GrowCut的肾皮质分割算法

目的 肾脏图像分割对于肾脏疾病的诊断有着重要意义,临床上通过测量肾皮质的体积和厚度可判断肾脏是否有肿瘤、慢性动脉硬化性肾病和肾移植急性排斥反应等。现有的肾脏分割算法大多针对一种模态,且只能分割出肾脏整体。本文提出一种基于全卷积网络和GrowCut的肾皮质自动分割算法,用于多模态肾脏图像分割。方法 首先用广义霍夫变换对肾脏进行检测,提取出感兴趣区域,通过数据增强扩充带标签数据;然后用VGG-16预训练模型进行迁移学习,构建适用于肾皮质分割的全卷积网络,设置网络训练参数,使用扩充数据训练网络。最后用全卷积网络分割图像,提取最后一层卷积层的特征图得到种子点标记,结合肾脏图像的先验知识纠正错误种子点,将该标记图作为GrowCut初始种子点可实现肾皮质准确分割。结果 实验数据为30组临床CT和MRI图像,其中一组有标记的CT图像用于训练网络并测试算法分割准确性,该文算法分割准确率IU（region intersection over union）和DSC（Dice similarity coefficient）分别达到91.06%±2.34%和91.79%±2.39%。与全卷积网络FCN-32s相比,本文提出的网络参数减少,准确率更高,可实现肾皮质分割。GrowCut算法考虑像素间的邻域信息,与全卷积网络结合可进一步将分割准确率提高3%。结论 该方法可准确分割多模态肾脏图像,包括正常和变异肾脏的图像,说明该方法优于主流方法,能够为临床诊断提供可靠依据。     

C-FCN: Corners-based fully convolutional network for visual object detection

Object detection has achieved significantly progresses in recent years. Proposal-based methods have become the mainstream object detectors, achieving excellent performance on accurate recognition and localization of objects. However, region proposal generation is still a bottleneck. In this paper, to address the limitations of conventional region proposal network (RPN) that defines dense anchor boxes with different scales and aspect ratios, we propose an anchor-free proposal generator named corner region proposal network (CRPN) which is based on a pair of key-points, including top-left corner and bottom-right corner of an object bounding box. First, we respectively predict the top-left corners and bottom-right corners by two sibling convolutional layers, then we obtain a set of object proposals by grouping strategy and non-maximum suppression algorithm. Finally, we further merge CRPN and fully convolutional network (FCN) into a unified network, achieving an end-to-end object detection. Our method has been evaluated on standard PASCAL VOC and MS COCO datasets using a deep residual network. Experiment results present that the proposed method outperforms previous detectors in the term of precision. Additionally, it runs with a speed of 76 ms per image on a single GPU by using ResNet-50 as the backbone, which is faster than other detectors.

     

全卷积语义分割与物体检测网络

目的 目前主流物体检测算法需要预先划定默认框,通过对默认框的筛选剔除得到物体框。为了保证足够的召回率,就必须要预设足够密集和多尺度的默认框,这就导致了图像中各个区域被重复检测,造成了极大的计算浪费。提出一种不需要划定默认框,实现完全端到端深度学习语义分割及物体检测的多任务深度学习模型（FCDN）,使得检测模型能够在保证精度的同时提高检测速度。方法 首先分析了被检测物体数量不可预知是目前主流物体检测算法需要预先划定默认框的原因,由于目前深度学习物体检测算法都是由图像分类模型拓展而来,被检测数量的无法预知导致无法设置检测模型的输出,为了保证召回率,必须要对足够密集和多尺度的默认框进行分类识别;物体检测任务需要物体的类别信息以实现对不同类物体的识别,也需要物体的边界信息以实现对各个物体的区分、定位;语义分割提取了丰富的物体类别信息,可以根据语义分割图识别物体的种类,同时采用语义分割的思想,设计模块提取图像中物体的边界关键点,结合语义分割图和边界关键点分布图,从而完成物体的识别和定位。结果 为了验证基于语义分割思想的物体检测方法的可行性,训练模型并在VOC（visual object classes）2007 test数据集上进行测试,与目前主流物体检测算法进行性能对比,结果表明,利用新模型可以同时实现语义分割和物体检测任务,在训练样本相同的条件下训练后,其物体检测精度优于经典的物体检测模型;在算法的运行速度上,相比于FCN,减少了8 ms,比较接近于YOLO（you only look once）等快速检测算法。结论 本文提出了一种新的物体检测思路,不再以图像分类为检测基础,不需要对预设的密集且多尺度的默认框进行分类识别;实验结果表明充分利用语义分割提取的丰富信息,根据语义分割图和边界关键点完成物体检测的方法是可行的,该方法避免了对图像的重复检测和计算浪费;同时通过减少语义分割预测的像素点数量来提高检测效率,并通过实验验证简化后的语义分割结果仍足够进行物体检测任务。     

A simple and effective table detection system from document images

The requirement of detection and identification of tables from document images is crucial to any document image analysis and digital library system. In this paper we report a very simple but extremely powerful approach to detect tables present in document pages. The algorithm relies on the observation that the tables have distinct columns which implies that gaps between the fields are substantially larger than the gaps between the words in text lines. This deceptively simple observation has led to the design of a simple but powerful table detection system with low computation cost. Moreover, mathematical foundation of the approach is also established including formation of a regular expression for ease of implementation.     

基于HarDNet全卷积网络的道路路面语义分割方法

道路路面可行驶区域识别是无人驾驶环境感知的重要组成部分.针对计算资源有限的车载设备,设计了基于HarDNet全卷积网络的道路路面语义分割方法.首先,在U-Net语义分割框架中使用低内存消耗的HarDNet卷积神经网络结构提取卷积特征进行路面分割;其次,在模型训练时对图像标签进行权重增强,提升路面边缘分割精度;然后,针对国内道路特点,构建国内道路路面分割数据集;最后,使用基于像素的交叉熵函数和Softmax的损失函数,结合平移、形变、填充、灰度处理、"复制粘贴"的数据增强方法进行模型训练.在构造的国内道路数据集上进行算法测试,实验结果表明所提方法的平均交并比值为94.5,在AGX Xavier设备上运行速度为10.6帧/秒.在满足无人车计算力要求的前提下,尽可能提升了路面可行驶区域分割的精度.     

全卷积神经网络下的多光谱遥感影像分割

目的 传统的遥感影像分割方法需要大量人工参与特征选取以及参数选择,同时浅层的机器学习算法无法取得高精度的分割结果。因此,利用卷积神经网络能够自动学习特征的特性,借鉴处理自然图像语义分割的优秀网络结构,针对遥感数据集的特点提出新的基于全卷积神经网络的遥感影像分割方法。方法 针对遥感影像中目标排列紧凑、尺寸变化大的特点,提出基于金字塔池化和DUC（dense upsampling convolution）结构的全卷积神经网络。该网络结构使用改进的DenseNet作为基础网络提取影像特征,使用空间金字塔池化结构获取上下文信息,使用DUC结构进行上采样以恢复细节信息。在数据处理阶段,结合遥感知识将波段融合生成多源数据,生成植被指数和归一化水指数,增加特征。针对遥感影像尺寸较大、采用普通预测方法会出现拼接痕迹的问题,提出基于集成学习的滑动步长预测方法,对每个像素预测14次,每次预测像素都位于不同图像块的不同位置,对多次预测得到的结果进行投票。在预测结束后,使用全连接条件随机场（CRFs）对预测结果进行后处理,细化地物边界,优化分割结果。结果 结合遥感知识将波段融合生成多源数据可使分割精度提高3.19%;采用基于集成学习的滑动步长预测方法可使分割精度较不使用该方法时提高1.44%;使用全连接CRFs对预测结果进行后处理可使分割精度提高1.03%。结论 针对宁夏特殊地形的遥感影像语义分割问题,提出基于全卷积神经网络的新的网络结构,在此基础上采用集成学习的滑动步长预测方法,使用全连接条件随机场进行影像后处理可优化分割结果,提高遥感影像语义分割精度。     

Multi-atlas segmentation of optic disc in retinal images via convolutional neural network

Multi-atlas segmentation is widely accepted as an essential image segmentation approach. Through leveraging on the information from the atlases instead of utilizing the model-based segmentation techniques, the multi-atlas segmentation could significantly enhance the accuracy of segmentation. However, label fusion, which plays an important role for multi-atlas segmentation still remains the primary challenge. Bearing this in mind, a deep learning-based approach is presented through integrating feature extraction and label fusion. The proposed deep learning architecture consists of two independent channels composing of continuous convolutional layers. To evaluate the performance our approach, we conducted comparison experiments between state-of-the-art techniques and the proposed approach on publicly available datasets. Experimental results demonstrate that the accuracy of the proposed approach outperforms state-of-the-art techniques both in efficiency and effectiveness.

     

Gland segmentation in colorectal cancer histopathological images using U-net inspired convolutional network

Neural Computing and Applications - The accurate gland segmentation from digitized H&E (hematoxylin and eosin) histology images with a wide range of histologic grades of cancer is quite...     

A convolutional neural network model for semantic segmentation of mitotic events in microscopy images

Neural Computing and Applications - Mitosis, which has important effects such as healing and growing for human body, has attracted considerable attention in recent years. Especially, cell division...     

深度全卷积网络的IVUS图像内膜与中—外膜边界检测

目的 心血管内超声（IVUS）图像内膜和中—外膜（MA）轮廓勾画是冠脉粥样硬化和易损斑块定量评估的必要过程。由于存在斑点噪声、图像伪影和各类斑块,重要组织边界的自动分割是一个非常困难的任务。为此,提出一种用于检测20 MHz心电门控IVUS图像内膜和MA边界方法。方法 首先利用深度全卷积网络（DFCN）学习原始IVUS图像与所对应手动分割图像之间映射,预测出目标或者背景的概率图,实现医学图像语义分割。然后在此基础上,结合心血管先验形状信息,采用数学形态学闭、开操作,平滑内膜和MA边界,降低分割过程中错误分类像素或区域的影响。结果 针对来自10位病人的IVUS图像及其标注信息所组成的435幅国际标准公开数据集,从线性回归、Bland-Altman分析和面积交并比（JM）、面积差异百分比（PAD）、Hausdorff距离（HD）、平均距离（AD）等性能指标上,评价本文方法。实验结果表明,算法检测结果与手动勾画结果的相关性可达到0.94,其超过94.71%的结果落在95%置信区域内,具有良好一致性。内膜和MA边界的AD指标分别为：0.07 mm和0.08 mm;HD指标分别为：0.21 mm和0.30 mm。JM指标分别为0.92和0.93;PAD指标分别为5%和4%。此外,对临床所采集的100幅IVUS图像进行了测试,证明本文学习的模型在跨数据集上具有较好的泛化能力。结论 与现有的国际算法比较,本文方法提高了各类斑块、声影区域和血管分支等因素的识别能力,不受超声斑点的影响,能准确地、可重复地检测出IVUS图像中的关键目标边界。     

Cell detection in pathology and microscopy images with multi-scale fully convolutional neural networks

Automated nucleus/cell detection is usually considered as the basis and a critical prerequisite step of computer assisted pathology and microscopy image analysis. However, due to the enormous variability (cell types, stains and different microscopes) and data complexity (cell overlapping, inhomogeneous intensities, background clutters and image artifacts), robust and accurate nucleus/cell detection is usually a difficult problem. To address this issue, we propose a novel multi-scale fully convolutional neural networks approach for regression of a density map to robustly detect the nuclei of pathology and microscopy images. The procedure can be divided into three main stages. Initially, instead of working on the simple dot label space, regression on the proposed structured proximity space for patches is performed so that centers of image patches are explicitly forced to produce larger values than their adjacent areas. Then, several multi-scale fully convolutional regression networks are developed for this task; this will enlarge the receptive field and not only can detect the single, small size cells, but also benefit to detecting cells with big size and overlapping states. In this stage, we copy the full feature maps from the contracting path and merge with the feature maps of the expansive path. This operation will make full use of shallow and deep semantic information of the networks. The networks do not have any fully connected layers; this strategy allows the seamless probability map prediction of arbitrarily large images. At the same time, data augmentations (e.g., small range shift, zoom and randomly flip) are carefully used to enhance the robustness of detection. Finally, morphological operations and suitable filters are employed and some prior information is introduced to find the centers of the cells more robustly. Our method achieves about 99.25% detection precision and the F1-measure is 0.9924 on fluorescence microscopy cell images; about 85.90% detection precision and the F1-measure is 0.9020 on Lymphocyte cell images and about 78.41% detection precision and the F1-measure is 0.8440 on breast histopathological images. This result leads to a promising detection performance that equates and sometimes exceeds the recently published leading detection approaches with the same benchmark datasets.     

基于全卷积神经网络和多核学习的显著性检测*

针对显著性检测过程中特征选择的个人主观片面性和预测过程中特征权重的难以协调性问题,提出了一种基于全卷积神经网络和多核学习的监督学习算法。首先通过MSRA10K图像数据库训练出的全卷积神经网络(FCNN),预测待处理图像的初步显著性区域图;然后在多尺度上选择置信度高的前景、背景超像素块作为多核SVM分类器的训学习样本集,选择并提取八种典型特征代表对应样本训练SVM,接着通过多核SVM分类器预测各超像素显著值;最后融合初步显著图和多核学习显著图,改善FCNN网络输出图的不足,得到最终的显著性目标。方法在SOD数据库和DUT-OMRON数据库上有更高的AUC值,F-Measure值,综合性能均优于对比方法,验证了方法在显著性检测中准确性的提高,为目标识别、机器视觉等应用提供更可靠的预处理结果。     

一阶全卷积遥感影像倾斜目标检测

目的 主流深度学习的目标检测技术对自然影像的识别精度依赖于锚框设置的好坏,并使用平行于坐标轴的正框表示物体位置,而遥感影像中地物目标具有尺寸多变、分布密集、长宽比悬殊且朝向不定的特点,更宜通过与物体朝向一致的斜框表示其位置。本文试图结合无锚框和斜框检测技术,在遥感影像上实现高精度目标识别。方法 使用斜框标注能够更为紧密地贴合目标边缘,有效减少识别干扰因素。本文基于单阶段无锚框目标检测算法：一阶全卷积目标检测网络(fully convolutional one-stage object detector, FCOS),通过引入滑动点结构,在遥感影像上实现高效率、高精度的斜框目标检测。与FCOS的不同之处在于,本文改进的检测算法增加了用于斜框检测的两个分支,通过在正框的两邻边上回归滑动顶点比率产生斜框,并预测斜框与正框的面积比以减少极端情况下的检测误差。结果 在当前最大、最复杂的斜框遥感目标检测数据集DOTA(object detection in aerial images)上对本文方法进行评测,使用ResNet50作为骨干网络,平均精确率(mean average precision,...     

A subclass supported convolutional neural network for object detection and localization in remote-sensing images

This article proposes a novel subclass-based classifier based on convolutional neural networks (CNNs) for detecting objects more accurately on remote-sensing images. The proposed classifier, called subclass supported CNN (SSCNN), is used to separate the representation of the objects into subclasses such as nearcentre, centre, and border depending on the distance of the object centre to obtain more effective feature extractor. A three-stage object recognition framework is used to evaluate the performance of the proposed classifier. In the first of these stages, the Selective Search algorithm generates object proposals from the image. Then, the proposed SSCNN classifies the proposals. Finally, subclass-based localization evaluation function has been proposed to calculate the localization of the object with classification results. Due to the limited number of satellite image samples, pretrained AlexNet is used by transfer learning approach to build effective feature extractor. The proposed method has been compared with region-based CNN (R-CNN) on a four-class remote-sensing test dataset consisting of 411 airplanes, 240 baseball diamonds, 468 storage tanks, and 83 ground track fields. In addition, Faster R-CNN has been trained with SSCNN features and the performances of the trained Faster R-CNNs are comparatively evaluated on 10-class remote-sensing image dataset. Experiment results have shown that the proposed framework can locate the objects precisely.     

联合显著性特征与卷积神经网络的遥感影像舰船检测

目的 针对高分辨率遥感影像舰船检测受云雾、海浪以及海岛等复杂因素干扰,存在虚警率高、漏检率高、目标检测和识别困难等问题,提出一种联合视觉显著性特征与卷积神经网络的海面舰船目标检测方法。方法 基于频率域相位谱显著性检测能够有效抑制高分辨率遥感影像上云层、海面杂波干扰的特点,计算影像多尺度显著图并进行加权融合。采用对数变换对融合后的图像进行空间域灰度增强以提高目标与背景的区分度,利用灰度形态学闭运算填充舰船目标孔洞,采用大津分割法来提取疑似舰船目标作为兴趣区域。最后构建舰船样本库,利用迁移学习的思想训练卷积神经网络模型,对所有兴趣区域切片进行分类判断和识别,得到最终检测结果。结果 利用多幅不同背景下的高分辨率遥感影像,分别从视觉显著性检测、舰船粗检测与船只类型识别3个方面进行实验验证,选取检测率、虚警率、识别率3个指标进行定量评价。结果表明,本文方法相比于其他方法能有效排除云雾、海岛等多种因素的干扰,检测率、虚警率、识别率分别为93.63%、3.01%、90.09%,明显优于其他算法,能够实现大范围影像上多种类型舰船的快速准确检测和识别。结论 本文将图像视觉显著性检测快速获取图像显著目标的特点与卷积神经网络在图像分类的优势相结合,应用于遥感影像的海域舰船目标检测,能够实现对复杂背景下舰船目标的检测和船只类型的精细化识别。     

基于密集卷积网络的X线气胸检测与定位

现有X线气胸检测存在两个主要问题：一是由于气胸通常与肋骨、锁骨等组织重叠,在临床上存在较大的漏诊,而现有算法的检测性能仍有待提高;二是现有基于卷积神经网络的算法无法给出可疑的气胸区域,缺乏可解释性。针对上述问题,提出了一种结合密集卷积网络（DenseNet）与梯度加权类激活映射的方法用于X线气胸的检测与定位。首先,构建了一个较大规模的胸部X线数据集PX-ray用于模型的训练和测试。其次,修改DenseNet的输出节点并在全连接层后添加一个sigmoid函数对胸片进行二分类（气胸/非气胸）。在训练过程中通过设置交叉熵损失函数的权重来缓解数据不平衡问题,提高模型准确率。最后,提取网络最后一个卷积层的参数以及对应的梯度,通过梯度加权类激活映射算法获得气胸类别的粗略定位图。在PX-ray测试集上的实验结果表明,所提方法的检测准确率为95.45%,并且在曲线下面积（AUC）、敏感度、特异性等指标上均高于0.9,优于VGG19、GoogLeNet以及ResNet算法,同时实现了对气胸区域的可视化。     

A new convolutional neural network model for peripapillary atrophy area segmentation from retinal fundus images

Peripapillary atrophy (PPA) is a clinical finding, which reflects the atrophy of retina layer and retinal pigment epithelium. The size of PPA area is a useful medical indicator, as it is highly associated with many diseases such as glaucoma and myopia. Therefore, separating the PPA area from retinal images, which is called PPA area segmentation, is very important. It is a challenging task, because PPA areas are irregular and non-uniform, and their contours are blurry and change gradually. To solve these issues, we transform the PPA area segmentation task into a task of segmenting another two areas with relatively regular and uniform shapes, and then propose a novel multi-task fully convolutional network (MFCN) model to jointly extract them from retinal images. Meanwhile, we take edge continuity of the target area into consideration. To evaluate the performance of the proposed model, we conduct experiments on images with PPA areas labelled by experts and achieve an average precision of 0.8928, outperforming the state-of-the-art models. To demonstrate the application of PPA segmentation in medical research, we apply PPA related features based on the segmented PPA area on differentiating glaucomatous and physiologic large cup cases. Experiment conducted on real datasets confirms the effectiveness of using these features for glaucoma diagnosis.