红外与可见光图像实时配准融合系统

描述了一个自主研制的基于实时分布式多处理机的图像配准和融合系统的设计与实现方案。本系统是具有并行计算机体系结构的通用高速实时图像融合处理系统,选择VxWorks实时操作系统和VEM64x总线的软硬件平台,采用AD公司新型的TS101DSP处理器为核心,多DSP处理器分布并行进行处理,完成多源图像实时高速配准和融合需要进行的大量运算,CPLD芯片完成了采集控制以及多传感器视频同步。由于采用了基于高性能DSP的实时嵌入式系统和通用标准化总线结构设计,该系统可以灵活地应用多种配准和融合算法来实现可见光和红外双通道数字图像的高速实时融合处理,比较好地解决多尺度图像配准融合算法的大数据量计算处理与系统实时性要求之间的矛盾,为多传感器实时图像配准融合处理系统的研制奠定了良好的技术基础。     

Masked object registration in the Fourier domain

Registration is one of the most common tasks of image analysis and computer vision applications. The requirements of most registration algorithms include large capture range and fast computation so that the algorithms are robust to different scenarios and can be computed in a reasonable amount of time. For these purposes, registration in the Fourier domain using normalized cross-correlation is well suited and has been extensively studied in the literature. Another common requirement is masking, which is necessary for applications where certain regions of the image that would adversely affect the registration result should be ignored. To address these requirements, we have derived a mathematical model that describes an exact form for embedding the masking step fully into the Fourier domain so that all steps of translation registration can be computed efficiently using Fast Fourier Transforms. We provide algorithms and implementation details that demonstrate the correctness of our derivations. We also demonstrate how this masked FFT registration approach can be applied to improve the Fourier-Mellin algorithm that calculates translation, rotation, and scale in the Fourier domain. We demonstrate the computational efficiency, advantages, and correctness of our algorithm on a number of images from real-world applications. Our framework enables fast, global, parameter-free registration of images with masked regions.     

A novel FPGA-based architecture for Sobel edge detection operator

A novel FPGA-based architecture for Sobel edge detection algorithm has been proposed. The Sobel algorithm is chosen due to its property of providing a differencing as well as noise smoothing operation in the single kernel. Thus, noise sensitivity of first gradient based operations can be avoided by the use of this algorithm. The implementation of edge detection algorithms on a field programmable gate array (FPGA) is motivated by the fact that large memory FPGAs are now available, providing a platform for processing real time algorithms on application-specific hardware with substantially higher performance than programmable digital signal processors (DSPs). This architecture can be used as a building block of a pattern recognition system, autonomous robot navigation, and also as a system for creating an image dazzling effect in multimedia graphics. This architecture is implicitly pipelined to provide a system capable of operating at a clock speed of 99.499?MHz which is a significant improvement over programmable DSPs implementation.     

基于椭圆傅里叶描述子的遥感图像配准算法

传统的基于点特征的图像配准算法在遥感图像配 准时提取的特征点数量大、分布密,易产生误匹配,导致特 征匹配的效率和精度低。为此,本文利用椭圆傅里叶描述子(EFD)能较好保留 形状信息的特点,提出了一种基于EFD的 图像配准算法,根据匹配的边界对预估变换参数,给出特征搜索范围,从而有效地提 高特征搜索的效率。实 验结果表明,所提出算法能有效抑制误匹配的产生,提升特征匹配的效率,尤其是对于纹理 丰富的大尺寸图像,提 升效果更加明显,验证了本文算法对遥感图像配准的适用性。     

A general fast registration framework by learning deformation-appearance correlation

In this paper, we propose a general framework for performance improvement of the current state-of-the-art registration algorithms in terms of both accuracy and computation time. The key concept involves rapid prediction of a deformation field for registration initialization, which is achieved by a statistical correlation model learned between image appearances and deformation fields. This allows us to immediately bring a template image as close as possible to a subject image that we need to register. The task of the registration algorithm is hence reduced to estimating small deformation between the subject image and the initially warped template image, i.e., the intermediate template (IT). Specifically, to obtain a good subject-specific initial deformation, support vector regression is utilized to determine the correlation between image appearances and their respective deformation fields. When registering a new subject onto the template, an initial deformation field is first predicted based on the subject's image appearance for generating an IT. With the IT, only the residual deformation needs to be estimated, presenting much less challenge to the existing registration algorithms. Our learning-based framework affords two important advantages: 1) by requiring only the estimation of the residual deformation between the IT and the subject image, the computation time can be greatly reduced; 2) by leveraging good deformation initialization, local minima giving suboptimal solution could be avoided. Our framework has been extensively evaluated using medical images from different sources, and the results indicate that, on top of accuracy improvement, significant registration speedup can be achieved, as compared with the case where no prediction of initial deformation is performed.     

High-performance medical image registration using new optimization techniques.

Optimization of a similarity metric is an essential component in intensity-based medical image registration. The increasing availability of parallel computers makes parallelizing some registration tasks an attractive option to increase speed. In this paper, two new deterministic, derivative-free, and intrinsically parallel optimization methods are adapted for image registration. DIviding RECTangles (DIRECT) is a global technique for linearly bounded problems, and multidirectional search (MDS) is a recent local method. The performance of DIRECT, MDS, and hybrid methods using a parallel implementation of Powell's method for local refinement, are compared. Experimental results demonstrate that DIRECT and MDS are robust, accurate, and substantially reduce computation time in parallel implementations.     

FAIR: a hardware architecture for real-time 3-D image registration

Mutual information-based image registration, shown to be effective in registering a range of medical images, is a computationally expensive process, with a typical execution time on the order of minutes on a modern single-processor computer. Accelerated execution of this process promises to enhance efficiency and therefore promote routine use of image registration clinically. This paper presents details of a hardware architecture for real-time three-dimensional (3-D) image registration. Real-time performance can be achieved by setting up a network of processing units, each with three independent memory buses: one each for the two image memories and one for the mutual histogram memory. Memory access parallelization and pipelining, by design, allow each processing unit to be 25 times faster than a processor with the same bus speed, when calculating mutual information using partial volume interpolation. Our architecture provides superior per-processor performance at a lower cost compared to a parallel supercomputer.     

Registration algorithm based on image matching for outdoor AR system with fixed viewing position

An image registration method based on the Fourier-Mellin transform is introduced for an outdoor augmented reality (AR) system. For this type of AR system, the observation position is fixed, and a complex 3-D registration problem can be reduced to a 2-D image registration for this fixed viewing position system. An observation globe model for this method is proposed. Under this supposition, a Fourier-Mellin transform is used in image registration, and the architecture of this system is illustrated. Experimental results show that this image registration algorithm is accurate and robust. It is effective for an outdoor AR system with a fixed viewing position.     

Reducing registration traffic for multitier personal communicationsservices

This paper presents an architecture for the multitier personal communications system (PCS) and intelligent algorithms for mobility management (specifically, the registration procedure). The multitier PCS system architecture considered in this study integrates three individual tiers: a high-tier system, a licensed low tier, and a unlicensed low tier. These three tiers are integrated into a single system by using a single home-location register (HLR) or the multitier HLR (MHLR). Under this architecture, we describe a registration protocol, where the mobile station (MS) is allowed to register to the MHLR on only one tier at any given time. We propose several intelligent algorithms for the MS to determine whether to perform registration or not in various situations to reduce the registration traffic     

Adaptive search space scaling in digital image registration

An image registration technique for application in X-ray, gamma-ray, and magnetic resonance imaging is described. The technique involves searching a real-valued, multidimensional, rectangular, symmetric space of bilinear geometrical transformations for a globally optimal transformation. Physical considerations provide theoretical limits on the search space, but the theoretically maximum allowable space is still often much larger than the smallest rectangular symmetric subspace that contains the optimal transformation. To reduce the search time, the current practice is to guess an optimal subspace from the maximum allowable space. This reduced space is then discretized and searched. An automatic technique to estimate adaptively a subspace from the maximum space during the search process itself is described. This adaptive technique is tested with two quite different types of search algorithms, namely, genetic algorithms and simulated annealing.     

基于深度学习的电力设备红外与可见光图像配准

针对现有电力设备红外与可见光图像配准难度大、配准时间长等问题,提出一种基于深度学习的电力设备红外与可见光图像配准融合的方法。本文将特征提取与特征匹配联合在深度学习框架中,直接学习图像块对与匹配标签之间的映射关系,用于后续的配准。此外为了缓解训练时红外图像样本不足的问题,提出一种利用红外图像及其变换图像学习映射函数的自学习方法,同时采用迁移学习来减少训练时间,加速网络框架。实验结果表明：本文方法与其他4种配准算法相比性能指标均有显著提升,本文平均准确率为89.909,同其余4种算法相比分别提高了2.31%、3.36%、2.67%、0.82%,本文平均RMSE(Root Mean Square Error)为2.521,同其余4种配准算法相比分别降低了14.68%、15.24%、4.90%、1.04%,算法平均用时为5.625 s,较其余4种算法分别降低了5.57%、6.82%、2.45%、1.75%,有效提高了电力设备红外与红外可见光图像配准的效率。     

Genetic algorithms for a robust 3-D MR-CT registration

Presents an original usage of genetic algorithms as a robust search space sampler in an application to 3D medical image elastic registration. An overview of the standard steps of a registration algorithm is given. We focus on the genetic algorithm use, and particularly on the problem of extracting the optimal solution among the final genetic population. We provide an original encoding scheme relying on a structural approach of point matching and then point out the need for a local optimization process. We then illustrate the algorithm with a concrete registration example and assert the results with a direct multi-volume rendering tool. Finally, the algorithm is applied to the Vanderbilt medical image database to assert its robustness and in order to compare it with other techniques     

JAGUAR: a fully pipelined VLSI architecture for JPEG imagecompression standard

In this paper, we describe a fully pipelined single chip VLSI architecture for implementing the JPEG baseline image compression standard. The architecture exploits the principles of pipelining and parallelism to the maximum extent in order to obtain high speed and throughput. The architecture for discrete cosine transform and the entropy encoder are based on efficient algorithms designed for high speed VLSI implementation. The entire architecture can be implemented on a single VLSI chip to yield a clock rate of about 100 MHz which would allow an input rate of 30 frames per second for 1024×1024 color images     

Image registration for image-based rendering.

Image-based rendering (IBR) has received much attention in recent years for its ability to synthesize photo-realistic novel views. To support translational motion, existing IBR methods either require a large amount of reference images or assume that some geometric information is available. However, rendering with a large amount of images is very expensive in terms of image acquisition, data storage, and memory costs. As IBR accepts various kinds of geometric proxy, we may use image registration techniques, such as stereo matching and structure and motion recognition, to obtain geometric information to help reduce the number of images required. Unfortunately, existing image registration techniques only support a small search range and require closely sampled reference images. This results in a high spatial sampling rate, making IBR impractical for use in scalable walkthrough environments.Our primary objective of this project is to develop an image registration technique that would recover the geometric proxy for IBR while, at the same time, reducing the number of reference images required. In this paper, we analyze the roles and requirements of an image registration technique for reducing the spatial sampling rate. Based on these requirements, we present a novel image registration technique to automatically recover the geometric proxy from reference images. With the distinguishing feature of supporting a large search range, the new method can accurately identify correspondences even though the reference images may only be sparsely sampled. This can significantly reduce the acquisition effort, the model size, and the memory cost.     

基于Logistic回归模型的景像匹配算法性能评估

景像匹配算法性能评估的目的是通过科学客观地评估算法性能,选择满足应用需要的匹配算法及进一步改进算法.将自动目标识别(ATR)算法性能评估中常用的Logistic回归模型应用于景像匹配算法性能的定量评估,进一步深入研究了匹配试验所需的样本量和试验结果的定量分析方法,并通过对两种匹配算法的定量评估试验验证了方法的有效性.     

Stochastic methods for joint registration, restoration, and interpolation of multiple undersampled images.

Using a stochastic framework, we propose two algorithms for the problem of obtaining a single high-resolution image from multiple noisy, blurred, and undersampled images. The first is based on a Bayesian formulation that is implemented via the expectation maximization algorithm. The second is based on a maximum a posteriori formulation. In both of our formulations, the registration, noise, and image statistics are treated as unknown parameters. These unknown parameters and the high-resolution image are estimated jointly based on the available observations. We present an efficient implementation of these algorithms in the frequency domain that allows their application to large images. Simulations are presented that test and compare the proposed algorithms.     

Implementation and validation of architectural space exploration techniques for domain-specific reconfigurable computing

Domain specific coarse-grained reconfigurable architectures (CGRAs) have great promise for energy-efficient flexible designs for a suite of applications. Designing such a reconfigurable device for an application domain is very challenging because the needs of different applications must be carefully balanced to achieve the targeted design goals. It requires the evaluation of many potential architectural options to select an optimal solution. Exploring the design space manually would be very time consuming and may not even be feasible for very large designs. Even mapping one algorithm onto a customized architecture can require time ranging from minutes to hours. Running a full power simulation on a complete suite of benchmarks for various architectural options require several days. Finding the optimal point in a design space could require a very long time. We have designed a framework/tool that made such design space exploration (DSE) feasible. The resulting framework allows testing a family of algorithms and architectural options in minutes rather than days and can allow rapid selection of architectural choices. In this paper, we describe our DSE framework for domain specific reconfigurable computing where the needs of the application domain drive the construction of the device architecture. The framework has been developed to automate design space case studies, allowing application developers to explore architectural tradeoffs efficiently and reach solutions quickly. We selected some of the core signal processing benchmarks from the MediaBench benchmark suite and some edge-detection benchmarks from the image processing domain for our case studies. We describe two search algorithms: a stepped search algorithm motivated by our manual design studies and a more traditional gradient based optimization. Approximate energy models are developed in each case to guide the search toward a minimal energy solution. We validate our search results by comparing the architectural solutions selected by our tool to an architecture optimized manually and by performing sensitivity tests to evaluate the ability of our algorithms to find good quality minima in the design space. All selected fabric architectures were synthesized on 130 nm cell-based ASIC fabrication process from IBM. These architectures consume almost same amount of energy on average, but the gradient based approach is more general and promises to extend well to new problem domains. We expect these or similar heuristics and the overall design flow of the system to be useful for a wide range of architectures, including mesh based and other commonly used architectures for CGRAs.     

复眼图像超分辨率重构中配准算法研究进展

高分辨率图像能够提供更多的图像细节和更清晰的图像质量,因此模仿生物复眼高分辨率这一特性、研究复眼超分辨率对于航天侦查和军事目标的识别具有重要意义。近年来亚像素级图像配准作为超分辨率重构中的关键步骤成为了研究热点,新的配准算法层出不穷。图像配准作为复眼图像超分辨率重构技术中至关重要的一步也是超分辨率重构中的一个难点,图像配准的精度以及图像配准算法的运算复杂程度直接影响着超分辨率重构的质量和效率。文中总结了近年来国内外超分辨率重构中配准算法的研究进展,介绍了图像配准技术和复眼超分辨率重构技术的基本原理和应用背景,阐明了课题的研究目的、意义以及发展前景,并且重点研究与分析了目前主流的配准算法以及各自的优缺点,并对今后的研究趋势进行了展望,同时为今后的配准算法研究提供了重要参考。     

基于同名点追踪的空间相机成像拼接配准模型

分析了已有图像配准算法在遥感图像拼接配准方面面临的问题。根据空间相机TDICCD交错拼接的成像特点,提出了一种基于同名点轨迹追踪的成像拼接配准模型。通过建立辅助空间坐标系下的中心投影共线方程,将像点、摄像中心、景物点建立严谨的数学关系,可精确实现对同名像点在像面上的轨迹追踪。结合TDICCD在像面上的位置约束,计算图像上同名像对的纵向偏差像元数和横向偏差像元数。最后结合相机在轨所成条带图像和卫星辅助数据进行分析,选取多组像点进行配准,同名像对配准误差小于1 pixel,经验证模型算法可行。相比传统的遥感图像配准算法,该方法不需要已知图像内容,为一种严格的几何意义上的配准,具有很强的适应性和预测性,已应用在型号相机的地面复算,易移植应用于其他类型空间相机的图像配准与拼接。     

Accelerated Nonrigid Intensity-Based Image Registration Using Importance Sampling

Nonrigid image registration methods using intensity-based similarity metrics are becoming increasingly common tools to estimate many types of deformations. Nonrigid warps can be very flexible with a large number of parameters and gradient optimization schemes are widely used to estimate them. However, for large datasets, the computation of the gradient of the similarity metric with respect to these many parameters becomes very time consuming. Using a small random subset of image voxels to approximate the gradient can reduce computation time. This work focuses on the use of importance sampling to reduce the variance of this gradient approximation. The proposed importance sampling framework is based on an edge-dependent adaptive sampling distribution designed for use with intensity-based registration algorithms. We compare the performance of registration based on stochastic approximations with and without importance sampling to that using deterministic gradient descent. Empirical results, on simulated magnetic resonance brain data and real computed tomography inhale–exhale lung data from eight subjects, show that a combination of stochastic approximation methods and importance sampling accelerates the registration process while preserving accuracy.