Detection of anatomic structures in human retinal imagery

The widespread availability of electronic imaging devices throughout the medical community is leading to a growing body of research on image processing and analysis to diagnose retinal disease such as diabetic retinopathy (DR). Productive computer-based screening of large, at-risk populations at low cost requires robust, automated image analysis. In this paper we present results for the automatic detection of the optic nerve and localization of the macula using digital red-free fundus photography. Our method relies on the accurate segmentation of the vasculature of the retina followed by the determination of spatial features describing the density, average thickness, and average orientation of the vasculature in relation to the position of the optic nerve. Localization of the macula follows using knowledge of the optic nerve location to detect the horizontal raphe of the retina using a geometric model of the vasculature. We report 90.4% detection performance for the optic nerve and 92.5% localization performance for the macula for red-free fundus images representing a population of 345 images corresponding to 269 patients with 18 different pathologies associated with DR and other common retinal diseases such as age-related macular degeneration.     

A Web-based system for the quantitative and reproducible assessment of clinical indexes from the retinal vasculature

A novel system for the vascular tree identification and the quantitative estimation of arteriolar venular ratio clinical index in retinal fundus images is presented. The system is composed of a module for automatic vascular tracking, an interactive editing interface to correct errors and set the required parameters of analysis, and a module for the computation of clinical indexes. The system was organized as a client-server structure to allow clinicians and researchers from all over the world to work remotely. The system was evaluated by three graders analyzing 30 fundus images. The evaluation of the Pearson's correlation coefficient and p-value of a paired t-test for each pair of graders demonstrates the high reproducibility of the measures provided by the system.     

基于视频序列的数字图像拼接技术研究进展

基于视频序列的数字图像拼接是指将具有重叠区的多帧视频通过数字配准和融合获得单幅宽视场静态全景图或动态全景图.基于视频序列的数字图像拼接技术主要包括全局快速配准算法、运动目标分割算法和无缝融合算法.首先分析理想数字图像拼接系统的特性,然后介绍近年来基于视频序列的数字图像拼接技术的研究进展,最后分析其研究动向.     

Retinal fundus vasculature multilevel segmentation using whale optimization algorithm

The aim was to present a novel automated approach for extracting the vasculature of retinal fundus images. The proposed vasculature extraction method on retinal fundus images consists of two phases: preprocessing phase and segmentation phase. In the first phase, brightness enhancement is applied for the retinal fundus images. For the vessel segmentation phase, a hybrid model of multilevel thresholding along with whale optimization algorithm (WOA) is performed. WOA is used to improve the segmentation accuracy through finding the \(n{-}1\) optimal n-level threshold on the fundus image. To evaluate the accuracy, sensitivity, specificity, accuracy, receiver operating characteristic (ROC) curve analysis measurements are used. The proposed approach achieved an overall accuracy of 97.8%, sensitivity of 88.9%, and specificity of 98.7% for the identification of retinal blood vessels by using a dataset that was collected from Bostan diagnostic center in Fayoum city. The area under the ROC curve reached a value of 0.967. Automated identification of retinal blood vessels based on whale algorithm seems highly successful through a comprehensive optimization process of operational parameters.     

Predictive scheduling algorithms for real-time feature extraction and spatial referencing: application to retinal image sequences

Real-time spatial referencing is an important alternative to tracking for designing spatially aware ophthalmic instrumentation for procedures such as laser photocoagulation and perimetry. It requires independent, fast registration of each image frame from a digital video stream (1024 x 1024 pixels) to a spatial map of the retina. Recently, we have introduced a spatial referencing algorithm that works in three primary steps: 1) tracing the retinal vasculature to extract image feature (landmarks); 2) invariant indexing to generate hypothesized landmark correspondences and initial transformations; and 3) alignment and verification steps to robustly estimate a 12-parameter quadratic spatial transformation between the image frame and the map. The goal of this paper is to introduce techniques to minimize the amount of computation for successful spatial referencing. The fundamental driving idea is to make feature extraction subservient to registration and, therefore, only produce the information needed for verified, accurate transformations. To this end, the image is analyzed along one-dimensional, vertical and horizontal grid lines to produce a regular sampling of the vasculature, needed for step 3) and to initiate step 1). Tracing of the vascular is then prioritized hierarchically to quickly extract landmarks and groups (constellations) of landmarks for indexing. Finally, the tracing and spatial referencing computations are integrated so that landmark constellations found by tracing are tested immediately. The resulting implementation is an order-of-magnitude faster with the same success rate. The average total computation time is 31.2 ms per image on a 2.2-GHz Pentium Xeon processor.     

Real-time multimodal retinal image registration for a computer-assisted laser photocoagulation system

An algorithm for the real-time registration of a retinal video sequence captured with a scanning digital ophthalmoscope (SDO) to a retinal composite image is presented. This method is designed for a computer-assisted retinal laser photocoagulation system to compensate for retinal motion and hence enhance the accuracy, speed, and patient safety of retinal laser treatments. The procedure combines intensity and feature-based registration techniques. For the registration of an individual frame, the translational frame-to-frame motion between preceding and current frame is detected by normalized cross correlation. Next, vessel points on the current video frame are identified and an initial transformation estimate is constructed from the calculated translation vector and the quadratic registration matrix of the previous frame. The vessel points are then iteratively matched to the segmented vessel centerline of the composite image to refine the initial transformation and register the video frame to the composite image. Criteria for image quality and algorithm convergence are introduced, which assess the exclusion of single frames from the registration process and enable a loss of tracking signal if necessary. The algorithm was successfully applied to ten different video sequences recorded from patients. It revealed an average accuracy of 2.47 ± 2.0 pixels (～23.2 ± 18.8 μm) for 2764 evaluated video frames and demonstrated that it meets the clinical requirements.     

Integrated analysis of vascular and nonvascular changes from color retinal fundus image sequences

Algorithms are presented for integrated analysis of both vascular and nonvascular changes observed in longitudinal time-series of color retinal fundus images, extending our prior work. A Bayesian model selection algorithm that combines color change information, and image understanding systems outputs in a novel manner is used to analyze vascular changes such as increase/decrease in width, and disappearance/appearance of vessels, as well as nonvascular changes such as appearance/disappearance of different kinds of lesions. The overall system is robust to false changes due to inter-image and intra-image nonuniform illumination, imaging artifacts such as dust particles in the optical path, alignment errors and outliers in the training-data. An expert observer validated the algorithms on 54 regions selected from 34 image pairs. The regions were selected such that they represented diverse types of vascular changes of interest, as well as no-change regions. The algorithm achieved a sensitivity of 82% and a 9% false positive rate for vascular changes. For the nonvascular changes, 97% sensitivity and a 10% false positive rate are achieved. The combined system is intended for diverse applications including computer-assisted retinal screening, image-reading centers, quantitative monitoring of disease onset and progression, assessment of treatment efficacy, and scoring clinical trials.     

Contextual information enhanced convolutional neural networks for retinal vessel segmentation in color fundus images

Accurate retinal vessel segmentation is a challenging problem in color fundus image analysis. An automatic retinal vessel segmentation system can effectively facilitate clinical diagnosis and ophthalmological research. In general, this problem suffers from various degrees of vessel thickness, perception of details, and contextual feature fusion in technique. For addressing these challenges, a deep learning based method has been proposed and several customized modules have been integrated into the well-known U-net with encoder–decoder architecture, which is widely employed in medical image segmentation. In the network structure, cascaded dilated convolutional modules have been integrated into the intermediate layers, for obtaining larger receptive field and generating denser encoded feature maps. Also, the advantages of the pyramid module with spatial continuity have been taken for multi-thickness perception, detail refinement, and contextual feature fusion. Additionally, the effectiveness of different normalization approaches has been discussed on different datasets with specific properties. Finally, sufficient comparative experiments have been enforced on three retinal vessel segmentation datasets, DRIVE, CHASE_DB1, and the STARE dataset with unhealthy samples. As a result, the proposed method outperforms the work of predecessors and achieves state-of-the-art performance.     

DBMap: a space-conscious data visualization and knowledge discovery framework for biomedical data warehouse

Advances in digital imaging modalities as well as other diagnosis and therapeutic techniques have generated a massive amount of diverse data for clinical research. The purpose of this study is to investigate and implement a new intuitive and space-conscious visualization framework, called DBMap, to facilitate efficient multidimensional data visualization and knowledge discovery against the large-scale data warehouses of integrated image and nonimage data. The DBMap framework is built upon the TreeMap concept. TreeMap is a space constrained graphical representation of large hierarchical data sets, mapped to a matrix of rectangles, whose size and color represent interested database fields. It allows the display of a large amount of numerical and categorical information in limited real estate of the computer screen with an intuitive user interface. DBMap has been implemented and integrated into a large brain research data warehouse to support neurologic and neuroradiologic research at the University of California, San Francisco Medical Center. For imaging specialists and clinical researchers, this novel DBMap framework facilitates another way to better explore and classify the hidden knowledge embedded in medical image data warehouses.     

视标引导的自适应光学眼底成像视场精确定位

自适应光学眼底相机,由于较高的成像分辨率和人眼等晕角的存在,单次成像的视场被限制在1左右。必须实现单个视场的精确定位和多个视场的图像拼接,才能得到完整的眼底图像。为了精确定位,文中分析视标引导成像视场的原理,设计了新型的视标引导系统。平行光照明视标,并通过透镜聚焦于人眼瞳孔中心,这样能够精确测量眼底成像视场的位置。基于此搭建的自适应光学系统可在22.6的眼底范围内成像,精度达到0.003。这套系统成功实现了单个细胞的追踪和眼底血管的大视场拼接,这将有益于液晶自适应光学系统在临床眼科的应用和推广。     

基于互信息的荧光素眼底血管造影图像序列的自动配准方法

荧光素眼底血管造影技术(FFA)是眼底疾病诊断的金标准,但是造影过程中病人不可避免地转动眼球,造成FFA图像序列中感兴趣区域(例如视网膜血管分支、新生血管)的位置发生变化,给后续的图像定量分析与病情准确评估诊断带来困难。针对上述问题,该文提出一种基于互信息的FFA图像序列配准方法。首先采用多尺度线性滤波方法分割出图像中的血管,并利用图像金字塔对分割后的图像进行下采样,然后利用互信息计算待配准图像与参考图像的相似性,通过进化策略对配准参数进行优化,获得互信息最大时图像的空间变换矩阵,实现FFA图像的配准。采用上述方法,对4位患者共计1039帧FFA图像进行测试,总体配准率达到93%,失败率仅为1%;与常用的配准方法相比,所提方法的配准率、配准速度和鲁棒性等综合性能良好,为FFA影像的定量分析在未来的临床应用奠定了基础。     

Robust detection and classification of longitudinal changes in color retinal fundus images for monitoring diabetic retinopathy

A fully automated approach is presented for robust detection and classification of changes in longitudinal time-series of color retinal fundus images of diabetic retinopathy. The method is robust to: 1) spatial variations in illumination resulting from instrument limitations and changes both within, and between patient visits; 2) imaging artifacts such as dust particles; 3) outliers in the training data; 4) segmentation and alignment errors. Robustness to illumination variation is achieved by a novel iterative algorithm to estimate the reflectance of the retina exploiting automatically extracted segmentations of the retinal vasculature, optic disk, fovea, and pathologies. Robustness to dust artifacts is achieved by exploiting their spectral characteristics, enabling application to film-based, as well as digital imaging systems. False changes from alignment errors are minimized by subpixel accuracy registration using a 12-parameter transformation that accounts for unknown retinal curvature and camera parameters. Bayesian detection and classification algorithms are used to generate a color-coded output that is readily inspected. A multiobserver validation on 43 image pairs from 22 eyes involving nonproliferative and proliferative diabetic retinopathies, showed a 97% change detection rate, a 3% miss rate, and a 10% false alarm rate. The performance in correctly classifying the changes was 99.3%. A self-consistency metric, and an error factor were developed to measure performance over more than two periods. The average self consistency was 94% and the error factor was 0.06%. Although this study focuses on diabetic changes, the proposed techniques have broader applicability in ophthalmology.     

An integrated computer supported acquisition, handling, and characterization system for pigmented skin lesions in dermatological images.

This paper describes an integrated prototype computer-based system for the characterization of skin digital images. The first stage includes an image acquisition arrangement designed for capturing skin images, under reproducible conditions. The system processes the captured images and performs unsupervised image segmentation and image registration utilizing an efficient algorithm based on the log-polar transform of the images' Fourier spectrum. Border- and color-based features, extracted from the digital images of skin lesions, were used to construct a classification module for the recognition of malignant melanoma versus dysplastic nevus. Different methods, drawn from the fields of artificial intelligence (neural networks) and statistical modeling (discriminant analysis), were used in order to find the best classification rules and to compare the results of different approaches to the problem.     

3-D Retinal Curvature Estimation

We study 3-D retinal curvature estimation from multiple images that provides the fundamental geometry of the human retina and could be used for 3-D retina visualization and disease diagnosis purposes. An affine camera model is used for 3-D reconstruction due to its simplicity, linearity, and robustness. A major challenge is that a series of optics is involved in the retinal imaging process, including an actual fundus camera, a digital camera, and the optics of the human eye, all of which cause significant nonlinear distortions in retinal images. In this paper, we develop a new constrained optimization method that considers both the geometric shape of the human retina and nonlinear lens distortions. Moreover, we examine a variety of lens distortion models to approximate the optics of the human eye in order to create a smooth spherical surface for curvature estimation. The experimental results on both synthetic data and real retinal images validate the proposed algorithm.      

Elastic registration for retinal images based on reconstructed vascular trees

The vascular tree of the retina is likely the most representative and stable feature for eye fundus images in registration. Based on the reconstructed vascular tree, we propose an elastic matching algorithm to register pairs of fundus images. The identified vessels are thinned and approximated using short line segments of equal length that results a set of elements. The set of elements corresponding to one vascular tree are elastically deformed to optimally match the set of elements of another vascular tree, with the guide of an energy function to finally establish pixel relationship between both vascular trees. The mapped positions of pixels in the transformed retinal image are computed to be the sum of their original locations and corresponding displacement vectors. For the purpose of performance comparison, a weak affine model based fast chamfer matching technique is proposed and implemented. Experiment results validated the effectiveness of the elastic matching algorithm and its advantage over the weak affine model for registration of retinal fundus images.     

基于多路径输入和多尺度特征融合的视网膜血管分割

视网膜血管的形态变化,如分叉角度、扩张程度等 ,可为眼底疾病的诊断提供依据。 使用深度学习技术对视网膜病变程度进行评估成为目前研究的重点。提出了一种基于多 路径输入和多尺度特征融合的视网膜血管分割方法来解决视网膜血管分割问题。采用了 多路径输入和多特征融合的方式改进了U-Net模型,使本文的网络能够有效的解决眼底视网 膜图像的分割效果差的 问题。实验结果表明,算法在DRIVE和CHASE_DB1数据集上,敏 感性分别取得0.814和0.813,特异性 分别取得0.984和0.986,在分割准确率指标上 分别取得0.969和0.975,所提方法相较于其他方法较优。     

CTA-based angle selection for diagnostic and interventionalangiography of saccular intracranial aneurysms

Coil embolization is a safe treatment for cerebral aneurysms only if the width of the neck in relation to the fundus of the aneurysm is small. Therefore, accurate visualization of the aneurysmal neck is required both in the diagnostic process and during the intervention. Conventional digital subtraction angiography (DSA) is still the preferred modality for the examination of cerebrovascular abnormalities like aneurysms, but it often does not provide the required morphological characteristics due to the suboptimal selection of projection angles and resulting overprojections of surrounding vasculature. This paper presents a method for performing a computer-assisted calculation of the optimal projection angles for DSA by post-processing computed tomographic angiography (CTA) volume data using ray-casting techniques and a combination of image processing algorithms. By means of phantom studies, retrospective simulations of angiograms, and in vivo applications of calculated optimal viewing angles, it is demonstrated that the proposed method results in better angiographic projections of the neck of saccular aneurysms with small neck-fundus ratio than those acquired at standard angles prescribed by clinical protocols     

免散瞳眼底照相机自动对焦算法设计

眼底图像的自动对焦是眼底照相机的重要组成部分。在分析各种应用于眼底图像自动对焦算法的基础上,针对免散瞳眼底照相机及眼底图像对比度特点,在现有低对比度图像自动对焦算法的基础上,提出了基于区域分块的自动对焦改进算法。算法结合无重叠型的采样方式与梯度向量平方函数,在保证对焦灵敏度的基础上,提升了自动对焦的运算速度。仿真实验证明,该算法与原算法及常用自动对焦算法相比具有较好的灵敏度及运算速度,能够应用于实际的眼底图像自动对焦中。     

Automatic detection of red lesions in digital color fundus photographs

The robust detection of red lesions in digital color fundus photographs is a critical step in the development of automated screening systems for diabetic retinopathy. In this paper, a novel red lesion detection method is presented based on a hybrid approach, combining prior works by Spencer et al. (1996) and Frame et al. (1998) with two important new contributions. The first contribution is a new red lesion candidate detection system based on pixel classification. Using this technique, vasculature and red lesions are separated from the background of the image. After removal of the connected vasculature the remaining objects are considered possible red lesions. Second, an extensive number of new features are added to those proposed by Spencer-Frame. The detected candidate objects are classified using all features and a k-nearest neighbor classifier. An extensive evaluation was performed on a test set composed of images representative of those normally found in a screening set. When determining whether an image contains red lesions the system achieves a sensitivity of 100% at a specificity of 87%. The method is compared with several different automatic systems and is shown to outperform them all. Performance is close to that of a human expert examining the images for the presence of red lesions.     

A successive clutter-rejection-based approach for early detection of diabetic retinopathy

The presence of microaneurysms (MAs) is usually an early sign of diabetic retinopathy and their automatic detection from color retinal images is of clinical interest. In this paper, we present a new approach for automatic MA detection from digital color fundus images. We formulate MA detection as a problem of target detection from clutter, where the probability of occurrence of target is considerably smaller compared to the clutter. A successive rejection-based strategy is proposed to progressively lower the number of clutter responses. The processing stages are designed to reject specific classes of clutter while passing majority of true MAs, using a set of specialized features. The true positives that remain after the final rejector are assigned a score which is based on its similarity to a true MA. Results of extensive evaluation of the proposed approach on three different retinal image datasets are reported, and used to highlight the promise in the presented strategy.