Multimodal registration of visible,SWIR and LWIR images in a distributed smart camera system

We present a multimodal registration algorithm between images in the visible, short-wave infrared and long-wave infrared spectra. The algorithm works with two reference-objective image pairs and operates in two stages: (1) A calibration phase between static frames to estimate the transformation parameters using histogram of oriented gradients and the Chi-square distance; (2) a frame-by-frame mapping with these parameters using a projective transformation and a bilinear interpolation to map the objective video stream to the coordinate system of the reference video stream. We present a distributed heterogeneous architecture that combines a programmable processor core and a custom hardware accelerator for each node. The software performs the calibration phase, whereas the hardware computes the frame-by-frame mapping. We implemented our design using a Xilinx Zynq XC7Z020 system-on-a-chip for each node. The prototype uses 2.38W of power, 25% of the logic resources and 65% of the available on-chip memory per node. Running at 100MHz, the core can register 640  ×  512-pixel frames in 4ms after initial calibration, which allows our module to operate at up to 250 frames per second.     

基于Canny边缘SURF特征的红外与可见光图像配准算法

利用多源传感器之间获取信息的互补性,克服单传感器的缺陷,从而提高系统整体性能指标的思想已经在军事、医疗、卫星等领域获得了广泛的应用。可见光和红外图像相融合也能提高视觉应用场景中对目标的探测能力,降低目标警报的虚警率和漏警率,提升准确率和工作效益。对于红外与可见光图像配准过程中受不同传感器图像成像原理不同,成像结果图像灰度差异大、特征难以匹配的问题,可以利用红外和可见光图像的共有特征即边缘轮廓特征,采用Canny边缘提取算法提取出图像最基本、稳定的特征,然后在边缘图中使用SURF特征检测算法进行特征点提取与匹配,最后采用RANSAC进行精准匹配。由于边缘在红外和可见光图像中都是比较稳定的特征,而且在边缘轮廓图中进行特征提取将极大减少计算量和提高匹配率,因而最终能够获得较为准确的红外、可见光图像的变换关系。     

电力巡检场景下的红外与可见光图像配准方法

电力设备故障会导致停电事故,影响电网的安全稳定运行。根据电力设备运行时会产生热量的特点,提出一种电力设备的红外与可见光图像配准方法,便于进行异常发热故障检测。首先通过Sobel边缘检测算子提取电力设备的红外与可见光图像的边缘信息,得到边缘图像;然后通过SuperPoint算法检测2幅边缘图像的特征点并计算描述子,利用SuperGlue算法对特征点进行匹配;最后通过最小二乘法计算仿射变换模型参数,实现电力设备的红外与可见光图像配准。实验结果表明本文方法能够对电力设备的红外与可见光图像进行高精度的配准。     

基于VoxelMorph的可见光和红外遥感影像配准

由于可见光和红外的成像机理、成像波段不同,获取的遥感影像之间存在复杂的非线性辐射畸变,传统的配准方法难以实现两者的高精度配准。本文提出一种基于VoxelMorph的可见光和红外遥感影像配准方法,利用卷积神经网络对可见光和红外异源图像进行分步的精细化形变场计算,从而实现快速高精度配准。将可见光图像作为参考图像,利用U-Net网络计算待配准红外图像和参考（可见光）图像的形变场,实现全局对齐的仿射变换,然后通过空间转换网络进一步实现更高自由度变形。采用WHU-OPT-SAR数据集的实验结果表明,与基于尺度不变特征变换（SIFT）算法的传统配准方法相比,本文提出的基于VoxelMorph配准方法可以获得更好的配准效果,验证了基于VoxelMorph的配准方法在多源遥感影像领域的有效性。     

Efficient feedforward categorization of objects and human postures with address-event image sensors

This paper proposes an algorithm for feedforward categorization of objects and, in particular, human postures in real-time video sequences from address-event temporal-difference image sensors. The system employs an innovative combination of event based hardware and bio-inspired software architecture. An event-based temporal difference image sensor is used to provide input video sequences, while a software module extracts size and position invariant line features inspired by models of the primate visual cortex. The detected line features are organized into vectorial segments. After feature extraction, a modified line segment Hausdorff distance classifier combined with on-the-fly cluster-based size and position invariant categorization. The system can achieve about 90 percent average success rate in the categorization of human postures, while using only a small number of training samples. Compared to state-of-the-art bio-inspired categorization methods, the proposed algorithm requires less hardware resource, reduces the computation complexity by at least five times, and is an ideal candidate for hardware implementation with event-based circuits.     

基于嵌入式Ethernet技术的视频图像监控系统

阐述了嵌入式Ethrnmet技术的特点,并结合视频组态技术设计了一种新的图像监控系统。该系统设计开发出具有组态功能,面向数据监控、可见光视频监控、红外视频监测和数据采集功能。该系统功能特点是在红外或可见光摄像机后立即通过嵌入式模块将模拟信号转换为数字网络视频数据并进行适当地压缩与编码,利用以太网监控视频图像数据和控制信号,同时在监控端可以应用PC强大的处理能力对图像做进一步地分析与处理。     

基于边缘最优映射的红外和可见光图像自动配准算法

针对同一场景的红外和可见光图像间一致特征难以提取和匹配的难题, 提出了一种在多尺度空间中基于边缘最优映射的自动配准算法. 在由粗至细的尺度空间中, 算法分别采用仿射模型和投影模型作为参考图像和待配准图像间的空间变换模型. 在每个尺度层上, 首先基于相位一致性方法提取两幅图像的边缘结构, 并在相应的空间变换模型下将在待配准图像中提取的二值边缘映射到参考图像的边缘强度图上; 接着采用并行遗传算法寻找一组全局最优的模型参数, 使两幅图像间的结构相似度最大. 在各层的寻优结束之后, 使用Powell算法对全局寻优后的模型参数进行局部精化. 实验结果表明, 该算法能够充分利用图像间的视觉相似结构, 有效地实现红外和可见光图像的自动配准.     

基于S3C2410嵌入式无线视频监控系统的设计

介绍了一种基于S3C2410硬件平台和嵌入式Linux操作系统的视频监控系统的总体设计方案,在嵌入式Linux平台下利用软件实现视频数据的采集、MPEG-4编码和网络传输;阐述了系统的总体结构和各部分功能特点,选择xvidcore作为本系统中视频图像压缩模块中的核心算法;采用以嵌入式Linux系统为核心,基于其良好的网络功能,通过USB摄像头实时获取视频和CDMA模块完成无线网络及Internet的接入;控制终端利用Wi-Fi无线局域网技术通过AP接入Internet,真正实现视频监控的无线化,满足无线视频监控系统的要求。     

一种压缩域中的快速运动目标提取算法

论文提出了一种工作于MPEG压缩域的快速运动目标提取算法。算法以通过部分解码得到的运动向量和亮度分量的直流DCT系数作为输入,提取P帧的运动目标。首先采用鲁棒性回归分析估计全局运动,标记出与全局运动不一致的宏块,得到运动块的分布;然后将运动向量场插值作为时间域的特征,将重构的直流图像转换到LUV颜色空间作为空间域的特征,采用快速平均移聚类找到时间和空间特征具有相似性的区域,得到细化的区域边界;最后结合运动块分布和聚类分析的结果,通过基于马尔可夫随机场的统计标号方法进行背景分离,得到运动目标的掩模。实验结果表明该算法可以有效地消除运动向量噪声的影响,并有很高的处理速度,对于CIF格式的视频码流,每秒可以处理约50帧。     

基于卷积神经网络的单图像去雾模型硬件重构加速方法

针对基于卷积神经网络（CNN）的单图像去雾模型在移动/嵌入式端部署难,不易用做实时视频去雾的问题,提出一种基于Zynq片上系统（SoC）的去雾模型硬件重构加速方法。首先,提出量化-反量化算法,对两个代表去雾模型进行量化;其次,基于视频流存储器架构和软硬件协同、流水线等技术以及高级综合（HLS）工具,对量化后的去雾模型硬件重构并生成具有高性能扩展总线接口（AXI4）的硬件IP核。实验结果表明,在保证去雾效果的前提下,可以实现模型参数从float32到int5(5 bit)的量化,从而节省约84.4%的存储空间;所生成硬件IP核的最高像素时钟频率为182 Mpixel/s,能够实现1080P@60 frame/s的视频去雾;单帧640×480的雾图去雾仅需2.4 ms,而片上功耗仅为2.25 W。这种生成带有标准总线接口的硬件IP核也便于跨平台移植和部署,从而可以扩大这类去雾模型的应用范围。     

仿生眼视觉图像处理系统

为研制仿生眼球,构建了一个嵌入式的基于仿生控制的视觉图像处理系统。根据仿生眼对视觉系统的要求,设计了一个以TMS320DM642DSP为核心,以TVP5150、SAA7121H为编解码模块的视觉图像处理系统;实现了仿生眼视觉识别的一系列软件开发,包括实时图像采集、视频图像处理、视频输出及目标位置参数传递等功能;在CCS2.2的环境下,对视觉识别算法进行了探索,用经典的Prewitt算法以及改进的Prewitt算法对系统进行实验测试。     

RETRACTED ARTICLE: A novel registration and super-resolution jointed paradigm for medical images under internet of thing environment

This paper proposes a novel registration and super-resolution jointed paradigm for medical images under the Internet of thing environment. In the medical image processing, the matching issue is one catches wide attention with the domain of research. Image registration technique can be divided into similarity measure, optimization, geometric transformation, and interpolation, etc. As the first essential clue of our model, we propose the novel registration algorithm based on energy feature extraction. Generally, the matching energy function by the similarity measurement and a penalty constitution is called the external force and endogenic force separately. The matching is an external force and endogenic force mutual competition, eventually achieves the balanced process. Furtherly, we integrate the game analysis and area feature selection to achieve the better image super-resolution mode through the pretreatment of the image to change the initial value, so as to achieve the purpose of improving the performance. Besides the algorithm level innovation, we integrate the GPU and the IOT to construct the hardware based implementation of the proposed medical image processing system. The latency of registers to read and write data across a GPU’s entire storage system is minimal, it is private to each thread, and can only be accessed by its owning thread. For each thread, the local memory is also private and it is often used to deal with the problem of overflow register, reducing the buffer overflow caused by the entire application of a substantial decline in the possibility and shared memory is visible to all threads within the thread block. We then achieve the optimal integration of IOT and GPU. The experimental result proves the robustness of the method.

     

基于深度卷积特征提取的红外与可见光图像融合方法

目前多数红外与可见光图像融合算法在融合过程中通常需要对源图像进行分解,这样易导致融合图像细节模糊和显著性目标丢失.为解决该问题,提出一种基于深度卷积特征提取的红外与可见光图像融合方法.首先,利用迁移学习理论对EfficientNet特征提取性能进行分析,选择7个特征提取模块;然后,直接将源图像送入特征提取模块以实现显著性特征提取;接着,构造通道归一化和平均算子操作用于获取显著图,再使用Softmax与Up-sampling组合的融合规则来得到融合权重,将融合权重与源图像进行卷积,生成7幅候选融合图像;最后,将候选融合图像的像素最大值作为最终的重构融合图像.所有实验均在公共数据集上进行,并与经典的传统和深度学习方法比较,主客观实验结果均表明,所提出方法能够有效地融合红外与可见光图像中的重要信息,突显融合图像的细节纹理,具有更好的视觉效果和更少的图像伪影以及人工噪声.     

基于双谱序列图像的ATR系统设计与实现

根据可见光/红外两波段各自成像的优点,设计实现了一个基于双谱图像像素级融合的自动目标识别(ATR)系统。该系统包括图像融合、特征提取、目标识别三个主要部分,每部分有多种处理算法可供选择。系统基于多线程设计,可实时显示可见光/红外序列图像的融合、目标识别过程。实验验证了该系统的有效性。     

An iterative integrated framework for thermal–visible image registration,sensor fusion,and people tracking for video surveillance applications

In this work, we propose a new integrated framework that addresses the problems of thermal–visible video registration, sensor fusion, and people tracking for far-range videos. The video registration is based on a RANSAC trajectory-to-trajectory matching, which estimates an affine transformation matrix that maximizes the overlapping of thermal and visible foreground pixels. Sensor fusion uses the aligned images to compute sum-rule silhouettes, and then constructs thermal–visible object models. Finally, multiple object tracking uses blobs constructed in sensor fusion to output the trajectories. Results demonstrate the advantage of our proposed framework in obtaining better results for both image registration and tracking than separate image registration and tracking methods.     

The registration of CT image to the patient head by using an automated laser surface scanning system--a phantom study

In this paper, a practical methodology of surface-based registration supported by an automated laser surface scanning system to achieve good mapping performance is reported. The laser scanning system is used to digitize the facial feature of a phantom so as to mesh the physical space into triangular frame. The image space is established by translating the corresponding CT image into point set through using the medical image software tools. The image-to-physical registration task is carried out by a direct searching mechanism together with the objective function in an optimal fashion. The unconstrained nonlinear optimization algorithm performs the optimal searching iteration to find those parameters in the rigid-body transformation until the sum of the squared normal distances is minimized. Considering mapping the massive points in registration task would consume the computation time, there is only a suitable sample size to stand for the entire population with sufficient confidence and accuracy are extracted statistically from the CT point space to map to the laser scanning space. Registration results evaluated by gauging the position errors of the landmarks on phantom forehead show the proposed methodology has good ability to perform the image-to-physical registration.     

基于ARM的嵌入式网络摄像机的设计与实现

本文首先研究与设计了基于硬件压缩的网络视频监控系统各硬件结构,然后在此基础上完成了系统底层软件的设计,经过测试,达到了预期的效果。本文的研究工作,对于设计基于硬件压缩的网络视频监控系统提供了可以借鉴的设计思路,具有一定的应用价值。     

基于嵌入式系统的图像识别系统研究

信息技术的飞速发展,人们对多煤体图像处理的要求越来越迫切,文章通过借鉴嵌入式系统在信号处理、电子技术和计算机等领域的成功应用经验,在分析机器视觉系统特点的基础上,将机器视觉和嵌入式技术相结合,设计一种基于嵌入式处理器TMS320C6205的图像识别系统。介绍了系统工作原理和特点,给出了系统硬件结构图和软件的实现方案,实验结果显示,图像识别系统在光照比较好的条件下,能取得令人满意的效果。     

Design and implementation of a PC-based image-guided surgical system

In interactive, image-guided surgery, current physical space position in the operating room is displayed on various sets of medical images used for surgical navigation. We have developed a PC-based surgical guidance system (ORION) which synchronously displays surgical position on up to four image sets and updates them in real time. There are three essential components which must be developed for this system: (1) accurately tracked instruments; (2) accurate registration techniques to map physical space to image space; and (3) methods to display and update the image sets on a computer monitor. For each of these components, we have developed a set of dynamic link libraries in MS Visual C++ 6.0 supporting various hardware tools and software techniques. Surgical instruments are tracked in physical space using an active optical tracking system. Several of the different registration algorithms were developed with a library of robust math kernel functions, and the accuracy of all registration techniques was thoroughly investigated. Our display was developed using the Win32 API for windows management and tomographic visualization, a frame grabber for live video capture, and OpenGL for visualization of surface renderings. We have begun to use this current implementation of our system for several surgical procedures, including open and minimally invasive liver surgery.     

基于三参数模型的视频图像配准方法研究

视频图像配准是运动视频处理中的一项关键技术。提出了一种新的基于3参数模型的配准算法,该算法利用多尺度的角点检测方法从相邻两帧中抽取特征点,选用Hausdorff距离对特征点集进行匹配,计算3参数模型参数,从而实现图像配准。与传统的6参数仿射模型相比,模型参数的降低并没有显著降低配准效果,同时由于参数搜索空间的减少,该方法用更快的速度得到较准确的结果。