时间序列脑功能成象中的图象配准

脑功能成角是研究脑科学和生命科学的重要工具,在对脑功能图象进行统计分析时,有必要对时间序列脑功能成象中的图象进行配准,研究采用修正的Ｇａｕｓｓ－Ｎｅｗｔｏｎ最优化方法,通过计算配准图象之间残差平方和的极值,实现了时间序列脑功能图象的高精度配准。     

Super-resolution: a comprehensive survey

Super-resolution, the process of obtaining one or more high-resolution images from one or more low-resolution observations, has been a very attractive research topic over the last two decades. It has found practical applications in many real-world problems in different fields, from satellite and aerial imaging to medical image processing, to facial image analysis, text image analysis, sign and number plates reading, and biometrics recognition, to name a few. This has resulted in many research papers, each developing a new super-resolution algorithm for a specific purpose. The current comprehensive survey provides an overview of most of these published works by grouping them in a broad taxonomy. For each of the groups in the taxonomy, the basic concepts of the algorithms are first explained and then the paths through which each of these groups have evolved are given in detail, by mentioning the contributions of different authors to the basic concepts of each group. Furthermore, common issues in super-resolution algorithms, such as imaging models and registration algorithms, optimization of the cost functions employed, dealing with color information, improvement factors, assessment of super-resolution algorithms, and the most commonly employed databases are discussed.     

Development and evaluation of satellite image data analysis infrastructure

Tokyo University of Information Sciences (TUIS) receives moderate resolution imaging spectroradiometer (MODIS) data, and provides the processed data to universities and research institutes as part of the academic frontier project. One of the major fields of research using MODIS data is the analysis of changes in the environment. We are currently developing applications to analyze environmental changes. These applications run on our satellite image data analysis system, which is implemented in a parallel distributed system and a database server. When using satellite data, one common problem is the interference of clouds. In order to remove this interference, the standard solution is to create composite data of the same regions during a selected time span, and to patch together data which are not covered by clouds to create a clear image. We introduced a piece-processing algorithm which separates one set of satellite image data into many small pieces of image data, making it quicker and easier to analyze and process the time-series satellite data. In this research, we implemented the pieceprocessing and composite-processing algorithms in order to increase the speed of analysis within the satellite image database. We tested the proposed processing and verified its effectiveness for target applications.     

图像融合在手术导航中的应用研究

普通脑外导航手术中是利用脑结构像来进行引导的,切除肿瘤时有可能损伤脑中的听力、视力、运动、语言、思维、记忆等功能区,造成病人终身残疾,因为这些功能区在脑结构像中是不可见的。笔者将核磁共振功能像与结构像进行融合,在手术导航系统中,用含有功能像和结构像的融合图像进行引导,避开关键组织和功能区,从而使手术中的危险和手术后伤残率降至最低。主要内容如下:(1)运动功能区磁共振成像过程和图像获取;(2)图像标志点的自动识别;(3)功能像与结构像快速配准算法;(4)融合图像在导航手术中应用等。临床试用表明,基于论文提出的各种算法开发出的图像融合软件,速度快,可靠性强,融合可以在数秒钟内完成,精度达到1mm,完全适用于临床。     

Calibrated,Registered Images of an Extended Urban Area

We describe a dataset of several thousand calibrated, time-stamped, geo-referenced, high dynamic range color images, acquired under uncontrolled, variable illumination conditions in an outdoor region spanning several hundred meters. The image data is grouped into several regions which have little mutual inter-visibility. For each group, the calibration data is globally consistent on average to roughly five centimeters and 0 1°, or about four pixels of epipolar registration. All image, feature and calibration data is available for interactive inspection and downloading at http://city.lcs.mit.edu/data.Calibrated imagery is of fundamental interest in a variety of applications. We have made this data available in the belief that researchers in computer graphics, computer vision, photogrammetry and digital cartography will find it of value as a test set for their own image registration algorithms, as a calibrated image set for applications such as image-based rendering, metric 3D reconstruction, and appearance recovery, and as input for existing GIS applications.     

Image registration by local histogram matching

We previously presented an image registration method, referred to hierarchical attribute matching mechanism for elastic registration (HAMMER), which demonstrated relatively high accuracy in inter-subject registration of MR brain images. However, the HAMMER algorithm requires the pre-segmentation of brain tissues, since the attribute vectors used to hierarchically match the corresponding pairs of points are defined from the segmented image. In many applications, the segmentation of tissues might be difficult, unreliable or even impossible to complete, which potentially limits the use of the HAMMER algorithm in more generalized applications. To overcome this limitation, we have used local spatial intensity histograms to design a new type of attribute vector for each point in an intensity image. The histogram-based attribute vector is rotationally invariant, and importantly it also captures spatial information by integrating a number of local intensity histograms from multi-resolution images of original intensity image. The new attribute vectors are able to determine the corresponding points across individual images. Therefore, by hierarchically matching new attribute vectors, the proposed method can perform as successfully as the previous HAMMER algorithm did in registering MR brain images, while providing more generalized applications in registering images of various organs. Experimental results show good performance of the proposed method in registering MR brain images, DTI brain images, CT pelvis images, and MR mouse images.     

Iconic feature based nonrigid registration: the PASHA algorithm

In this paper, we first propose a new subdivision of the image information axis used for the classification of nonrigid registration algorithms. Namely, we introduce the notion of iconic feature based (IFB) algorithms, which lie between geometrical and standard intensity based algorithms for they use both an intensity similarity measure and a geometrical distance. Then we present a new registration energy for IFB registration that generalizes some of the existing techniques. We compare our algorithm with other registration approaches, and show the advantages of this energy. Besides, we also present a fast technique for the computation of local statistics between images, which turns out to be useful on pairs of images having a complex, nonstationary relationship between their intensities, as well as an hybrid regularization scheme mixing elastic and fluid components. The potential of the algorithm is finally demonstrated on a clinical application, namely deep brain stimulation of a Parkinsonian patient. Registration of pre- and immediate postoperative MR images allow to quantify the range of the deformation due to pneumocephalus over the entire brain, thus yielding to measurement of the deformation around the preoperatively computed stereotactic targets.     

基于脑磁共振图像配准的动态联合角点检测算法

角点检测算法是基于角特征点的图像配准方法的核心。Harris和Susan是两种重要的角点检测算法,有较好的检测能力,但是其在描述角点信息方面都不全面。因此,联合Harris、Susan两种算法是一种较好的解决思路。其中,如何确定在联合算法中Harris、Susan两种算法的权重是一个关键。设计了一种联合算法,并通过统计实验获取两者的权重,通过引入两个加权因子ω1和ω2分别对Harris角点响应值与Susan角点响应值进行加权计算,获得其角点强度,从而筛选出新的角点集合,使该联合算法的角点检测能力明显提高。最后将该方法用于脑磁共振图像配准实验中。实验比较结果表明,该联合角点检测算法在脑磁共振图像配准的应用中,相对于目前已有角点检测算法,能获得较高的配准精度和较好的稳定性。     

Spatial transformation and registration of brain images using elastically deformable models.

The development of algorithms for the spatial transformation and registration of tomographic brain images is a key issue in several clinical and basic science medical applications, including computer-aided neurosurgery, functional image analysis, and morphometrics. This paper describes a technique for the spatial transformation of brain images, which is based on elastically deformable models. A deformable surface algorithm is used to find a parametric representation of the outer cortical surface and then to define a map between corresponding cortical regions in two brain images. Based on the resulting map, a three-dimensional elastic warping transformation is then determined, which brings two images into register. This transformation models images as inhomogeneous elastic objects which are deformed into registration with each other by external force fields. The elastic properties of the images can vary from one region to the other, allowing more variable brain regions, such as the ventricles, to deform more freely than less variable ones. Finally, the framework of prestrained elasticity is used to model structural irregularities, and in particular the ventricular expansion occurring with aging or diseases, and the growth of tumors. Performance measurements are obtained using magnetic resonance images.     

A Theory of Minimal 3D Point to 3D Plane Registration and Its Generalization

Registration of 3D data is a key problem in many applications in computer vision, computer graphics and robotics. This paper provides a family of minimal solutions for the 3D-to-3D registration problem in which the 3D data are represented as points and planes. Such scenarios occur frequently when a 3D sensor provides 3D points and our goal is to register them to a 3D object represented by a set of planes. In order to compute the 6 degrees-of-freedom transformation between the sensor and the object, we need at least six points on three or more planes. We systematically investigate and develop pose estimation algorithms for several configurations, including all minimal configurations, that arise from the distribution of points on planes. We also identify the degenerate configurations in such registrations. The underlying algebraic equations used in many registration problems are the same and we show that many 2D-to-3D and 3D-to-3D pose estimation/registration algorithms involving points, lines, and planes can be mapped to the proposed framework. We validate our theory in simulations as well as in three real-world applications: registration of a robotic arm with an object using a contact sensor, registration of planar city models with 3D point clouds obtained using multi-view reconstruction, and registration between depth maps generated by a Kinect sensor.     

配准算法对 PCA 单幅投影肺部重建的影响

肺癌放射治疗中,肺部肿瘤位置实成像对于临床意义重大。在一种利用单X射线投影进行成像的实时肺部3D成像算法中,图像配准过程引入的不准确对于PCA模型构建以及重建过程有重大影响。文章分析了光流法、Demons算法、水平集算法三种配准算法对重建效果的影响,并通过定性以及定量实验分析验证。结果表明,光流法配准在配准结果以及模型构建方面有较好的效果。     

Configurable real-time motion estimation for medical imaging: application to X-ray and ultrasound

Motion estimation is a key building block of image processing pipelines in many different contexts, ranging from efficient coding of video sequences in the consumer electronics domain (TV, DVD, BD) to professional medical applications. Many block-matching approaches have been proposed in the literature for motion detection and compensation in general, including both lossless and lossy algorithms. However, in real-time medical imaging applications, characterized by high frame rates, the needs for low latency and jitter, accuracy and robustness against noise are quite difficult to achieve with standard block-matching methods. We introduce a new hybrid image processing approach to block-matching that takes advantage of both types of algorithms (lossless and lossy), adapts to the image content and noise, and provides high flexibility for the speed/accuracy tradeoff. The presented approach has been successfully tested on interventional X-ray fluoroscopy and cardiac ultrasound images sequences.     

A review of atlas-based segmentation for magnetic resonance brain images

Normal and abnormal brains can be segmented by registering the target image with an atlas. Here, an atlas is defined as the combination of an intensity image (template) and its segmented image (the atlas labels). After registering the atlas template and the target image, the atlas labels are propagated to the target image. We define this process as atlas-based segmentation. In recent years, researchers have investigated registration algorithms to match atlases to query subjects and also strategies for atlas construction. In this paper we present a review of the automated approaches for atlas-based segmentation of magnetic resonance brain images. We aim to point out the strengths and weaknesses of atlas-based methods and suggest new research directions. We use two different criteria to present the methods. First, we refer to the algorithms according to their atlas-based strategy: label propagation, multi-atlas methods, and probabilistic techniques. Subsequently, we classify the methods according to their medical target: the brain and its internal structures, tissue segmentation in healthy subjects, tissue segmentation in fetus, neonates and elderly subjects, and segmentation of damaged brains. A quantitative comparison of the results reported in the literature is also presented.     

深度学习与生物医学图像分析2020年综述

医学大数据主要包括电子健康档案数据（electronic health record,EHR）、医学影像数据和基因信息数据等,其中医学影像数据占现阶段医学数据的绝大部分。如何将医学大数据应用于临床实践?这是计算机科学研究人员非常关注的问题,医学人工智能提供了一个很好的答案。通过结合医学图像大数据分析方向截至2020年的最新研究进展,以及医学图像大数据分析领域最近的工作,梳理了当前在医学图像领域以核磁共振影像、超声影像、病理和电信号为代表的4个子领域以及部分其他方向使用深度学习进行图像分析的方法理论和主要流程,对不同算法进行结果评价。本文分析了现有算法的优缺点以及医学影像领域的重难点,介绍了智能成像和深度学习在大数据分析以及疾病早期诊断领域的应用,同时展望了本领域未来的发展热点。深度学习在医学影像领域发展迅速,发展前景广阔,对疾病的早期诊断有重要作用,能有效提高医生工作效率并减轻负担,具有重要的理论研究和实际应用价值。     

A flexible registration and evaluation engine (f.r.e.e.)

We present an image registration framework which offers effective assistance for solving current registration problems. This work was motivated by the huge amount of registration problems in clinical applications and the problem of finding adequate solutions and properly comparing them. We have therefore designed a framework that supports the establishment, evaluation and comparison of registration approaches. Flexible registration and evaluation engine (f.r.e.e.) achieves a broad basis of algorithms by utilizing the insight segmentation and registration toolkit (ITK). This basis can be extended by virtually any new approach or algorithm, which then becomes seamlessly integrated into the method set of the f.r.e.e. framework. The framework offers suitable tools for an easy integration, optimization and proper evaluation of registration approaches, as well as an efficient utilization of the results in clinical routine. The framework is currently being evaluated at the Heidelberg University Hospital, Germany. The first results were gathered with an application implemented for the Neurosurgical Department of the hospital. In these tests the framework concept, along with its specific tools, was very promising for establishing clinical applications (e.g. preoperative neurosurgical planning; registration of cardiac images) and therefore motivated further development. The ability to automatically optimize the parameterization of registration methods regarding a given test set also proved useful, allowing more concentration on scientific problems themselves and not on the laborious task of parameter tweaking. Due to implemented abstraction layers, f.r.e.e. also allows a high degree of transparency and thus good comparability of registration approaches and results.     

Influence of image fusion approaches on classification accuracy: a case study

While many studies in the field of image fusion of remotely sensed data aim towards deriving new algorithms for visual enhancement, there is little research on the influence of image fusion on other applications. One major application in earth science is land cover mapping. The concept of sensors with multiple spatial resolutions provides a potential for image fusion. It minimises errors of geometric alignment and atmospheric or temporal changes.

This study focuses on the influence of image fusion on spectral classification algorithms and their accuracy. A Landsat 7 ETM+ image was used, where six multispectral bands (30 m) were fused with the corresponding 15 m panchromatic channel. The fusion methods comprise rather common techniques like Brovey, hue‐saturation‐value transform, and principal component analysis, and more complex approaches, including adaptive image fusion, multisensor multiresolution image fusion technique, and wavelet transformation. Image classification was performed with supervised methods, e.g. maximum likelihood classifier, object‐based classification, and support vector machines. The classification was assessed with test samples, a clump analysis, and techniques accounting for classification errors along land cover boundaries. It was found that the adaptive image fusion approach shows best results with low noise content. It resulted in a major improvement when compared with the reference, especially along object edges. Acceptable results were achieved by wavelet, multisensor multiresolution image fusion, and principal component analysis. Brovey and hue‐saturation‐value image fusion performed poorly and cannot be recommended for classification of fused imagery.     

深度医学图像配准研究进展：迈向无监督学习

在疾病诊断、手术引导及放射性治疗等图像辅助诊疗场景中,将不同时间、不同模态或不同设备的图像通过合理的空间变换进行配准是必要的处理流程之一。随着深度学习的快速发展,基于深度学习的医学图像配准研究以其耗时短、精度高的优势吸引了研究者的广泛关注。本文全面整理了2015—2019年深度医学图像配准方向的论文,系统地分析了深度医学图像配准领域的最新研究进展,展现了深度配准算法研究从迭代优化到一步预测、从有监督学习到无监督学习的总体发展趋势。具体来说,本文在界定深度医学图像配准问题和介绍配准研究分类方法的基础上,以相关算法的网络训练过程中所使用的监督信息多少作为分类标准,将深度医学图像配准划分为全监督、双监督与弱监督、无监督医学图像配准方法。全监督配准方法通过采用随机变换、传统算法和模型生成等方式获取近似的金标准作为监督信息;双监督、无监督配准方法通过引入图像相似度损失、标签相似度损失等其他监督信息以降低对金标准的依赖;无监督配准方法则完全消除对标注数据的需要,仅使用图像相似度损失和正则化损失监督网络训练。目前,无监督医学图像算法已经成为医学图像配准领域的研究重点,在无需获得代价高昂的标注信息下就能够取得与有监督和传统方法相当甚至更高的配准精度。在此基础上,本文进一步讨论了医学图像配准研究后续可能的4个未来挑战,希望能够为更高精度、更高效率的深度医学图像配准算法的研究提供方向,并推动深度医学图像配准技术在临床诊疗中落地应用。     

Accelerated multiplicative updates and hierarchical ALS algorithms for nonnegative matrix factorization

Nonnegative matrix factorization (NMF) is a data analysis technique used in a great variety of applications such as text mining, image processing, hyperspectral data analysis, computational biology, and clustering. In this letter, we consider two well-known algorithms designed to solve NMF problems: the multiplicative updates of Lee and Seung and the hierarchical alternating least squares of Cichocki et al. We propose a simple way to significantly accelerate these schemes, based on a careful analysis of the computational cost needed at each iteration, while preserving their convergence properties. This acceleration technique can also be applied to other algorithms, which we illustrate on the projected gradient method of Lin. The efficiency of the accelerated algorithms is empirically demonstrated on image and text data sets and compares favorably with a state-of-the-art alternating nonnegative least squares algorithm.     

Query optimization in information integration

The problem of decentralized data sharing, which is relevant to a wide range of applications, is still a source of major theoretical and practical challenges, in spite of many years of sustained research. In this paper we focus on the challenge of efficiency of query evaluation in information integration systems that use the global-as-view approach, with the objective of developing query-processing strategies that would be widely applicable and easy to implement in real-life applications. Our algorithms take into account important features of today’s data sharing applications: XML as likely interface or representation for data sources; the potential for information overlap across data sources; and the need for inter-source processing, as in joins of data across sources. The focus of this paper is on performance-related characteristics of several alternative approaches that we propose for efficient query processing in information integration, including an approach that uses materialized restructured views. We use synthetic and real-life datasets in our implementation of an information integration system shell to provide experimental results that demonstrate that our algorithms are efficient and competitive in the information integration setting. In addition, our experimental results allow us to make context-specific recommendations on selecting query-processing approaches from our proposed alternatives. As such, our approaches could form a basis for scalable query processing in information integration and interoperability in many practical settings.     

Clustered sampling improves random subspace brain mapping

Intuitive and efficient, the random subspace ensemble approach provides an appealing solution to the problem of the vast dimensionality of functional magnetic resonance imaging (fMRI) data for maximal-accuracy brain state decoding. Recently, efforts to generate biologically plausible and interpretable maps of brain regions which contribute information to the ensemble decoding task have been made and two approaches have been introduced: globally multivariate random subsampling and locally multivariate Monte Carlo mapping. Both types of maps reflect voxel-wise decoding accuracies averaged across repeatedly randomly sampled voxel subsets, highlighting voxels which consistently participate in high-classification subsets. We compare the mapping sensitivities of the approaches on realistic simulated data containing both locally and globally multivariate information and demonstrate that utilizing the inherent volumetric nature of fMRI through clustered Monte Carlo mapping yields dramatically improved performances in terms of voxel detection sensitivity and efficiency. These results suggest that, unless a priori information specifically dictates a global search, variants of clustered sampling should be the priority for random subspace brain mapping.