基于改进U-Net的视网膜血管图像分割

在眼科疾病的诊断中,对视网膜血管进行分割是非常有效的一种方法。在方法使用中,经常会遇到由于视网膜血管背景对比度低及血管末梢细节复杂导致的血管分割难度较大的问题,通过在设计网络的过程中在基础U-net网络中引入残差学习,注意力机制等模块,并将两者巧妙地结合在一起,提出一种新型的基于U-net的RAU-net视网膜血管图像分割算法。首先,在网络的编码器阶段加入残差模块,解决了模型网络加深导致梯度爆炸以及梯度消失的问题。其次,在网络的解码器阶段引入注意力门（attention gate, AU）模块,用来抑制不必要的特征,从而使模型产生更高的精度。通过在DRIVE数据集上进行验证,该算法的准确率、灵敏度、特异性和F₁-score分别达到了0.7832,0.9815,0.9568和0.8192。分割效果相对于普通监督学习算法较为良好。     

Textureless macula swelling detection with multiple retinal fundus images

Retinal fundus images acquired with nonmydriatic digital fundus cameras are versatile tools for the diagnosis of various retinal diseases. Because of the ease of use of newer camera models and their relatively low cost, these cameras can be employed by operators with limited training for telemedicine or point-of-care (PoC) applications. We propose a novel technique that uses uncalibrated multiple-view fundus images to analyze the swelling of the macula. This innovation enables the detection and quantitative measurement of swollen areas by remote ophthalmologists. This capability is not available with a single image and prone to error with stereo fundus cameras. We also present automatic algorithms to measure features from the reconstructed image, which are useful in PoC automated diagnosis of early macular edema, e.g., before the appearance of exudation. The technique presented is divided into three parts: first, a preprocessing technique simultaneously enhances the dark microstructures of the macula and equalizes the image; second, all available views are registered using nonmorphological sparse features; finally, a dense pyramidal optical flow is calculated for all the images and statistically combined to build a naive height map of the macula. Results are presented on three sets of synthetic images and two sets of real-world images. These preliminary tests show the ability to infer a minimum swelling of 300 μm and to correlate the reconstruction with the swollen location.     

Contextual information enhanced convolutional neural networks for retinal vessel segmentation in color fundus images

Accurate retinal vessel segmentation is a challenging problem in color fundus image analysis. An automatic retinal vessel segmentation system can effectively facilitate clinical diagnosis and ophthalmological research. In general, this problem suffers from various degrees of vessel thickness, perception of details, and contextual feature fusion in technique. For addressing these challenges, a deep learning based method has been proposed and several customized modules have been integrated into the well-known U-net with encoder–decoder architecture, which is widely employed in medical image segmentation. In the network structure, cascaded dilated convolutional modules have been integrated into the intermediate layers, for obtaining larger receptive field and generating denser encoded feature maps. Also, the advantages of the pyramid module with spatial continuity have been taken for multi-thickness perception, detail refinement, and contextual feature fusion. Additionally, the effectiveness of different normalization approaches has been discussed on different datasets with specific properties. Finally, sufficient comparative experiments have been enforced on three retinal vessel segmentation datasets, DRIVE, CHASE_DB1, and the STARE dataset with unhealthy samples. As a result, the proposed method outperforms the work of predecessors and achieves state-of-the-art performance.     

Integrated analysis of vascular and nonvascular changes from color retinal fundus image sequences

Algorithms are presented for integrated analysis of both vascular and nonvascular changes observed in longitudinal time-series of color retinal fundus images, extending our prior work. A Bayesian model selection algorithm that combines color change information, and image understanding systems outputs in a novel manner is used to analyze vascular changes such as increase/decrease in width, and disappearance/appearance of vessels, as well as nonvascular changes such as appearance/disappearance of different kinds of lesions. The overall system is robust to false changes due to inter-image and intra-image nonuniform illumination, imaging artifacts such as dust particles in the optical path, alignment errors and outliers in the training-data. An expert observer validated the algorithms on 54 regions selected from 34 image pairs. The regions were selected such that they represented diverse types of vascular changes of interest, as well as no-change regions. The algorithm achieved a sensitivity of 82% and a 9% false positive rate for vascular changes. For the nonvascular changes, 97% sensitivity and a 10% false positive rate are achieved. The combined system is intended for diverse applications including computer-assisted retinal screening, image-reading centers, quantitative monitoring of disease onset and progression, assessment of treatment efficacy, and scoring clinical trials.     

注意力残差链式融合的彩色眼底图像硬性渗出检测

硬性渗出物(Hard Exudate,HE)对糖尿病视网膜病 变(Diabetic Retinopathy,DR)的早期诊断具有重要意义,从彩色眼底图像中准确检测出HE 是诊断和筛查DR的重要步骤。针对以往自动检测方法中存在的分割效果不佳和对小目标误检 率高的问题,本文在经典U-Net生成网络的基础上,引入注意力残差链式融合机制,构造出 一种适用于视网膜HE检测的注意力残差链式融合生成对抗网络(Chain fusion of attention residuals GAN,CFAR-GAN)。该网络在生成网络编码过程的每个子模块后添加一个残差网 络(Residual Network,ResNet)结构,并采用带有残差连接的卷积层链(Residual convolut ional layer path,Res path)建立不同层间的跳跃连接,同时将全局最大池化注意力机制 用于刻画不同深度特征的权重,以防止训练过程的过拟合从而提高网络的泛化能力。将CFAR -GAN用e-ophtha EX数据库的训练集进行训练,在其测试集上,检测敏感性、PPV和F-sco re分别为92.5%、88.7%和90.6%;将训练好的网络在另一个独立的DIARETDB1数据库上进行测试,敏感性、特异性和 准确性分别为100%、98.5%和99.1%,表明本文所提出的方法具有 理想的泛化能力,这对于准确高效地检测眼底图 像中的HE,实现DR早期的自动诊断具有积极 意义。     

Detection and classification of edges in color images

Up to now, most of the color edge detection methods are monochromatic-based techniques, which produce, in general, better than when traditional gray-value techniques are applied. In this overview, we focus mainly on vector-valued techniques because it is easy to understand how to apply common edge detection schemes to every color component. Opposed to this, vector-valued techniques are new and different. The second part of the article addresses the topic of edge classification. While edges are often classified into step edges and ramp edges, we address the topic of physical edge classification based on their origin into shadow edges, reflectance edges, orientation edges, occlusion edges, and specular edges. In the rest of this article we discuss various vector-valued techniques for detecting discontinuities in color images. Then operators are presented based on vector order statistics, followed by presentation by examples of a couple of results of color edge detection. We then discuss different approaches to a physical classification of edges by their origin.     

Robust skin detection in real-world images

Human skin detection in images is desirable in many practical applications, e.g., human–computer interaction and adult-content filtering. However, existing methods are mainly suffer from confusing backgrounds in real-world images. In this paper, we try to address this issue by exploring and combining several human skin properties, i.e. color property, texture property and region property. First, images are divided into superpixels, and robust skin seeds and background seeds are acquired through color property and texture property of skin. Then we combining color, region and texture properties of skin by proposing a novel skin color and texture based graph cuts (SCTGC) to acquire the final skin detection results. Comprehensive and comparative experiments show that the proposed method achieves promising performance and outperforms many state-of-the-art methods over publicly available challenging datasets with a great part of hard images.     

Optimal scheduling of tracing computations for real-time vascularlandmark extraction from retinal fundus images

This group published fast algorithms for automatic tracing (vectorization) of the vasculature in live retinal angiograms, and for the extraction of visual landmarks formed by vascular bifurcations and crossings. These landmarks are used for feature-based image matching for controlling a computer-assisted laser retinal surgery instrument under development. This paper describes methods to schedule the vascular tracing computations to maximize the rate of growth of quality of the partial tracing results within a frame cycle. There are two main advantages. First, progressive image matching from partially extracted landmark sets can be faster, and provide an earlier indication of matching failure. Second, the likelihood of successful image matching is greatly improved since the extracted landmarks are of the highest quality for the given computational budget. The scheduling method is based on quantitative measures for the computational work and the quality of landmarks. A coarse grid-based analysis of the image is used to generate seed points for the tracing computations, along with estimates of local edge strengths, orientations, and vessel thickness. These estimates are used to define criteria for real-time preemptive scheduling of the tracing computations     

Automatic detection of red lesions in digital color fundus photographs

The robust detection of red lesions in digital color fundus photographs is a critical step in the development of automated screening systems for diabetic retinopathy. In this paper, a novel red lesion detection method is presented based on a hybrid approach, combining prior works by Spencer et al. (1996) and Frame et al. (1998) with two important new contributions. The first contribution is a new red lesion candidate detection system based on pixel classification. Using this technique, vasculature and red lesions are separated from the background of the image. After removal of the connected vasculature the remaining objects are considered possible red lesions. Second, an extensive number of new features are added to those proposed by Spencer-Frame. The detected candidate objects are classified using all features and a k-nearest neighbor classifier. An extensive evaluation was performed on a test set composed of images representative of those normally found in a screening set. When determining whether an image contains red lesions the system achieves a sensitivity of 100% at a specificity of 87%. The method is compared with several different automatic systems and is shown to outperform them all. Performance is close to that of a human expert examining the images for the presence of red lesions.     

Cooperation of color pixel classification schemes and color watershed: a study for microscopic images

We study the ability of the cooperation of two-color pixel classification schemes (Bayesian and K-means classification) with color watershed. Using color pixel classification alone does not sufficiently accurately extract color regions so we suggest to use a strategy based on three steps: simplification, classification, and color watershed. Color watershed is based on a new aggregation function using local and global criteria. The strategy is performed on microscopic images. Quantitative measures are used to evaluate the resulting segmentations according to a learning set of reference images.     

A contribution of image processing to the diagnosis of diabetic retinopathy--detection of exudates in color fundus images of the human retina

In the framework of computer assisted diagnosis of diabetic retinopathy, a new algorithm for detection of exudates is presented and discussed. The presence of exudates within the macular region is a main hallmark of diabetic macular edema and allows its detection with a high sensitivity. Hence, detection of exudates is an important diagnostic task, in which computer assistance may play a major role. Exudates are found using their high grey level variation, and their contours are determined by means of morphological reconstruction techniques. The detection of the optic disc is indispensable for this approach. We detect the optic disc by means of morphological filtering techniques and the watershed transformation. The algorithm has been tested on a small image data base and compared with the performance of a human grader. As a result, we obtain a mean sensitivity of 92.8% and a mean predictive value of 92.4%. Robustness with respect to changes of the parameters of the algorithm has been evaluated.     

An ensemble-based system for microaneurysm detection and diabetic retinopathy grading

Reliable microaneurysm detection in digital fundus images is still an open issue in medical image processing. We propose an ensemble-based framework to improve microaneurysm detection. Unlike the well-known approach of considering the output of multiple classifiers, we propose a combination of internal components of microaneurysm detectors, namely preprocessing methods and candidate extractors. We have evaluated our approach for microaneurysm detection in an online competition, where this algorithm is currently ranked as first, and also on two other databases. Since microaneurysm detection is decisive in diabetic retinopathy (DR) grading, we also tested the proposed method for this task on the publicly available Messidor database, where a promising AUC 0.90 ± 0.01 is achieved in a "DR/non-DR"-type classification based on the presence or absence of the microaneurysms.     

Automatic assessment of macular edema from color retinal images

Diabetic macular edema (DME) is an advanced symptom of diabetic retinopathy and can lead to irreversible vision loss. In this paper, a two-stage methodology for the detection and classification of DME severity from color fundus images is proposed. DME detection is carried out via a supervised learning approach using the normal fundus images. A feature extraction technique is introduced to capture the global characteristics of the fundus images and discriminate the normal from DME images. Disease severity is assessed using a rotational asymmetry metric by examining the symmetry of macular region. The performance of the proposed methodology and features are evaluated against several publicly available datasets. The detection performance has a sensitivity of 100% with specificity between 74% and 90%. Cases needing immediate referral are detected with a sensitivity of 100% and specificity of 97%. The severity classification accuracy is 81% for the moderate case and 100% for severe cases. These results establish the effectiveness of the proposed solution.     

Robust model-based vasculature detection in noisy biomedical images

This paper presents a set of algorithms for robust detection of vasculature in noisy retinal video images. Three methods are studied for effective handling of outliers. The first method is based on Huber's censored likelihood ratio test. The second is based on the use of a alpha-trimmed test statistic. The third is based on robust model selection algorithms. All of these algorithms rely on a mathematical model for the vasculature that accounts for the expected variations in intensity/texture profile, width, orientation, scale, and imaging noise. These unknown parameters are estimated implicitly within a robust detection and estimation framework. The proposed algorithms are also useful as nonlinear vessel enhancement filters. The proposed algorithms were evaluated over carefully constructed phantom images, where the ground truth is known a priori, as well as clinically recorded images for which the ground truth was manually compiled. A comparative evaluation of the proposed approaches is presented. Collectively, these methods outperformed prior approaches based on Chaudhuri et al. (1989) matched filtering, as well as the verification methods used by prior exploratory tracing algorithms, such as the work of Can et aL (1999). The Huber censored likelihood test yielded the best overall improvement, with a 145.7% improvement over the exploratory tracing algorithm, and a 43.7% improvement in detection rates over the matched filter.     

A successive clutter-rejection-based approach for early detection of diabetic retinopathy

The presence of microaneurysms (MAs) is usually an early sign of diabetic retinopathy and their automatic detection from color retinal images is of clinical interest. In this paper, we present a new approach for automatic MA detection from digital color fundus images. We formulate MA detection as a problem of target detection from clutter, where the probability of occurrence of target is considerably smaller compared to the clutter. A successive rejection-based strategy is proposed to progressively lower the number of clutter responses. The processing stages are designed to reject specific classes of clutter while passing majority of true MAs, using a set of specialized features. The true positives that remain after the final rejector are assigned a score which is based on its similarity to a true MA. Results of extensive evaluation of the proposed approach on three different retinal image datasets are reported, and used to highlight the promise in the presented strategy.     

A sorting system for hierarchical grading of diabetic fundus images: a preliminary study

Diabetic retinopathy is a leading cause of blindness in developed countries. Diabetic patients can prevent severe visual loss by attending regular eye examinations and receiving timely treatments. In the United States, standard protocols have been developed and refined for years to provide better screening and evaluation procedures of the fundus images. Due to the emerging number of diabetic retinopathy cases, accurate and efficient evaluations of the fundus images have become a serious burden for the ophthalmologists or care providers. While diabetic retinopathy remains too complicated to call for an automatic diagnosis system, an efficient tool to facilitate the grading process with a limited number of personnel is in great demand. The current study is to develop a sorting system with a user-friendly interface, based upon the standardized early treatment diabetic retinopathy study (ETDRS) protocol, to assist the professional graders. The raw fundus images will be screened and assigned to different graders according to their skill levels and experiences. The developed hierarchical sorting process will greatly support the graders and enhance their efficiency and throughput. The proposed hybrid intelligent system with multilevel knowledge representation is used to construct this sorting system. A preliminary case study is conducted using only the features of the spot lesion group coupled with the ETDRS standard to demonstrate its feasibility and performance. The results obtained from the case study show a promising future.     

Optic disk and cup segmentation from monocular color retinal images for glaucoma assessment

Automatic retinal image analysis is emerging as an important screening tool for early detection of eye diseases. Glaucoma is one of the most common causes of blindness. The manual examination of optic disk (OD) is a standard procedure used for detecting glaucoma. In this paper, we present an automatic OD parameterization technique based on segmented OD and cup regions obtained from monocular retinal images. A novel OD segmentation method is proposed which integrates the local image information around each point of interest in multidimensional feature space to provide robustness against variations found in and around the OD region. We also propose a novel cup segmentation method which is based on anatomical evidence such as vessel bends at the cup boundary, considered relevant by glaucoma experts. Bends in a vessel are robustly detected using a region of support concept, which automatically selects the right scale for analysis. A multi-stage strategy is employed to derive a reliable subset of vessel bends called r-bends followed by a local spline fitting to derive the desired cup boundary. The method has been evaluated on 138 images comprising 33 normal and 105 glaucomatous images against three glaucoma experts. The obtained segmentation results show consistency in handling various geometric and photometric variations found across the dataset. The estimation error of the method for vertical cup-to-disk diameter ratio is 0.09/0.08 (mean/standard deviation) while for cup-to-disk area ratio it is 0.12/0.10. Overall, the obtained qualitative and quantitative results show effectiveness in both segmentation and subsequent OD parameterization for glaucoma assessment.     

Detection and measurement of retinal vessels in fundus images using amplitude modified second-order Gaussian filter

In this paper, the fitness of estimating vessel profiles with Gaussian function is evaluated and an amplitude-modified second-order Gaussian filter is proposed for the detection and measurement of vessels. Mathematical analysis is given and supported by a simulation and experiments to demonstrate that the vessel width can be measured in linear relationship with the "spreading factor" of the matched filter when the magnitude coefficient of the filter is suitably assigned. The absolute value of vessel diameter can be determined simply by using a precalibrated line, which is typically required since images are always system dependent. The experiment shows that the inclusion of the width measurement in the detection process can improve the performance of matched filter and result in a significant increase in success rate of detection.     

Robust simultaneous detection of coronary borders in complex images

Visual estimation of coronary obstruction severity from angiograms suffers from poor inter- and intraobserver reproducibility and is often inaccurate. In spite of the widely recognized limitations of visual analysis, automated methods have not found widespread clinical use, in part because they too frequently fail to accurately identify vessel borders. The authors have developed a robust method for simultaneous detection of left and right coronary borders that is suitable for analysis of complex images with poor contrast, nearby or overlapping structures, or branching vessels. The reliability of the simultaneous border detection method and that of the authors' previously reported conventional border detection method were tested in 130 complex images, selected because conventional automated border detection might be expected to fail. Conventional analysis failed to yield acceptable borders in 65/130 or 50% of images. Simultaneous border detection was much more robust (p<.001) and failed in only 15/130 or 12% of complex images. Simultaneous border detection identified stenosis diameters that correlated significantly better with observer-derived stenosis diameters than did diameters obtained with conventional border detection (p<0.001), Simultaneous detection of left and right coronary borders is highly robust and has substantial promise for enhancing the utility of quantitative coronary angiography in the clinical setting.