Stitched image quality assessment based on local measurement errors and global statistical properties

Image stitching is developed to generate wide-field images or panoramic images for virtual reality applications. However, the quality assessment of stitched images with respect to various stitching algorithms has been less studied. Effective stitched image quality assessment (SIQA) is advantageous to evaluate the performance of various stitching methods and optimize the design of stitching methods. In this paper, we propose a novel SIQA method by exploiting local measurement errors and global statistical properties for feature extraction. Comprehensive image attributes including ghosting, misalignment, structural distortion, geometric error, chromatic aberrations and blur are considered either locally or globally. The extracted local and global features are aggregated into an overall quality via regression. Experimental results on two benchmark databases demonstrate the superiority of the proposed metric over both the state-of-the-art quality models designed for natural images and stitched images.     

Seamless image stitching by minimizing false edges.

Various applications such as mosaicing and object insertion require stitching of image parts. The stitching quality is measured visually by the similarity of the stitched image to each of the input images, and by the visibility of the seam between the stitched images. In order to define and get the best possible stitching, we introduce several formal cost functions for the evaluation of the stitching quality. In these cost functions the similarity to the input images and the visibility of the seam are defined in the gradient domain, minimizing the disturbing edges along the seam. A good image stitching will optimize these cost functions, overcoming both photometric inconsistencies and geometric misalignments between the stitched images. We study the cost functions and compare their performance for different scenarios both theoretically and practically. Our approach is demonstrated in various applications including generation of panoramic images, object blending and removal of compression artifacts. Comparisons with existing methods show the benefits of optimizing the measures in the gradient domain.     

视频图像拼接和应用

论述实时处理由若干个摄像头采集的图像，即利用视频图像边界重叠的部分进行拼接，形成全景式的视频，然后实时显示拼接后的视频图像。     

航空面阵图像拼接的累积误差消除方法

针对航空面阵遥感图像拼接算法一般存在误差累积的问题，提出了航空面阵图像拼接的累积误差消除方法。首先，利用航空遥感平台位置信息与姿态信息完成每幅图像投影变换与粗拼接，确保了每幅投影变换后图像拼接误差的独立性，从而避免了误差累积现象；然后，采用SIFT算法提取粗拼接相邻图像重叠区域的同名像点对，并根据同名像点对的位置差异，按照误差均分的原则，逐张独立地进行坐标微调及坐标精调，直到满足一定精度为止，以提高图像拼接精度，同时避免坐标调整时产生的误差累积。最后，通过主观评价（对比校正后卫星图像和文中拼接效果图像的地理对应关系）和客观评价（SSIM算法）对图像拼接质量进行评价。理论分析与实验结果表明:该方法能有效规避图像拼接中的误差累积问题，相对当前主要拼接方法，能够很好地与地理空间二维坐标保持一致性，且具有很好的实用价值和应用前景。     

一种有效的全景图拼合预处理算法

全景图拼合的预处理主要包括对样本图像的特征提取以及相邻图像的特征匹配.本文首先改进Harris角点检测算子,以便准确提取样本图像的特征点并赋予特征描述符;提出一种基于小波系数的特征索引算法,实现了图像特征点对的快速搜索和两幅图像之间的匹配.实验结果表明,该算法得到的匹配点精确,搜索效率高,能够实现全景图的无缝拼接.     

面向机载分布式光电的图像拼接算法

随着航空无人系统技术的迅猛发展,分布式机载图像拼接技术已成为备受瞩目的研究领域。本文针对分布式机载图像拼接中存在的视差大和空间几何变换关系复杂等问题,在APAP图像拼接算法基础上提出了一种改进算法。该算法采用变形处理、线性化单应平滑外推至全局性变换以及网格划分法等方法,有效消除了模糊重影、减少边缘处投影失真,提高算法运行效率。在多个场景下的实验中,该算法表现出更小的对齐误差和更高的图像质量指标,包括均方根误差、峰值信噪比、结构相似度和图像熵等。在进行大规模图像拼接时,该改进算法能够实现154张图像的大规模拼接,得到10 k×10 k的高分辨率全景图像,拼接耗时为138 s。因此,该改进算法具有重要的实际应用价值,可用于分布式机载图像拼接的实际应用中。     

基于高阶奇异值分解和深度学习的光场图像拼接方法

光场图像拼接旨在提高光场图像的视场角。考虑到光场数据包含较多冗余,且传统拼接方法对于低纹理场景的光场图像鲁棒性不足,本文提出一种基于高阶奇异值分解(high-order singular value decomposition, HOSVD)和深度学习的光场图像拼接方法。首先,通过光流估计和HOSVD对光场图像进行降维,得到所有视角下一致空间信息的主基和不同视角下高频信息的其他基带。其次,提出注意力增强的无监督单应性估计网络来提高图像的配准精度。最后,将扭曲后的参考基带和目标基带进行光场重建与图像融合,得到最终的拼接光场。实验结果表明,该方法在拼接光场的主客观质量和角度一致性方面表现出较好的性能。     

Video image mosaic implement based on planar-mirror-based catadioptric system

Planar-mirror-based catadioptric method is one of the most hot topics in recent years. To overcome the disadvantages of the planar-mirror-based catadioptric panoramic camera, described by Nalwa (1996, 2001, 2000), such as the requirement for high-precision optical device designing and the stitching lines in the resulting images, we proposed a planar-mirror-based video image mosaic system with high precision for designing. Firstly, we designed a screw nut on our system, which can be adjusted to locate the viewpoints of the cameras’ mirror images at a single point approximately. It provides a method for those who have difficulties in their designing and manufacturing for high precision. Then, after the image distortion correction and cylinder projection transforms, we can stitch the images to get a wide field of view image by template matching algorithm. Finally, nonlinear weighting fusion is adopted to eliminate the stitching line effectively. The experimental results show that our system has good characteristics such as video rate capture, high resolution, no stitching line and without affection by the depth of field of view.     

基于改进RANSAC算法的全景图像拼接技术

现实场景中相机获取的图像视角范围往往是有限的,而实际需求又要求得到场景的全 景图,针对日常生活和工业生产中对全景图像的需求以及传统的RANSAC(random sample consensus)算法在图像配准环节因为迭代次数没有上限导致出现误匹配点对且配准 速度不高的缺陷,提出了一种改进RANSAC算法来提高全景图像拼接的效率。改进RANSAC 算法通过检测圆内的点来寻找一个最优数据检测模型,并通过粒子群算法不断更新迭代圆心 的坐标,最终得到一个最佳的匹配模型,消除特征点匹配环节出现的异常值,在提高特征 点配准的准确率的同时降低算法复杂度。在对多组图像进行拼接的实验表明,本文提出的改 进RANSAC算法相较于其他几种算法平均正确匹配率提高了9.057%, 同时算法的平均配准速率提高了5.173 s,实 现了较鲁棒的全景图像拼接效果。     

基于区域特征与灰度交叉相关的序列图像拼接

针对具有平移、旋转、缩放变换的序列图像连续拼接,提出一种将基于区域特征的配准算法和基于灰度交叉相关的配准算法相结合的拼接算法。该算法用迭代阈值分割算法提取区域．利用区域特征进行配准,建立初始的同名区域对;然后以同名区域对的质心点作为特征点,基于图像的灰度信息,选择交叉相关准则作为度量;最后得到图像间的精确变换关系,实现序列图像的拼接。实验结果表明,算法配准率高,鲁棒性强。     

基于离线标定的全景视频快速拼接算法设计

针对实际条码识别系统中全场景监控和传统图像拼接算法速度慢的问题,提出了一种基于离线标定的快速全景视频拼接算法。在实际应用中多台相机位置固定,采用离线标定计算出图像拼接的单应性矩阵,在实时拼接中直接加载该矩阵进行计算,从而省去了大量的特征提取和配准时间。为了提高图像特征的配准精度,设计了一种改进的SIFT(Scale-Invariant Feature Transform)特征匹配算法。实验结果表明,该算法对比传统算法显著减少了错误特征匹配的影响,提高了图像拼接速度,具有更好的特征匹配效果。     

全景图像自动拼接算法的优化设计

提出了一种结合空域和频域进行全景图像序列自动拼接融合的优化算法。首先利用改进的相位相关法对全景序列图像进行自动排序并确定重叠区域,根据重叠区域像素均值进行图像整体亮度差异自动调整以降低角点的误匹配率;然后使用改进的Harris 算子在空间域提取图像角点（无需人工设定阈值）,通过双向最大互相关系数匹配获得初始特征点对,并用RANSAC 算法实现精确匹配;最后采用非线性平滑算法对图像重叠区域进行融合处理。实验结果表明,该优化算法排序过程简单有效,特征点提取匹配过程的成功率和效率都较现有算法有很大提高,拼接的图像清晰度高,具有较高的稳健性和拼接精度。     

基于圆心定位的全景图像拼接与展开

针对特殊的全景采集镜头，经拼接及展开运算，得到人眼容易辨识的图像。系统镜头由两片像素阵列为4096×3072的CCD组成，拼接所形成的传感区域包含像素阵列为4096×4096，文中提出一种基于圆心定位的拼接算法，此算法通过采集到的两幅全景图像的圆心，以圆心为基准进行剪切定位，从而实现图像的拼接。并应用改进的分块查表法，将目标图像变为环带块，对拼接后的图像进行展开运算。仿真实验表明，本拼接算法可完成特殊镜头所采集的图像拼接工作、速度快、可靠性能强、符合实际应用要求。     

Fixed Homography–Based Real‐Time SW/HW Image Stitching Engine for Motor Vehicles

In this paper, we propose an efficient architecture for a real‐time image stitching engine for vision SoCs found in motor vehicles. To enlarge the obstacle‐detection distance and area for safety, we adopt panoramic images from multiple telegraphic cameras. We propose a stitching method based on a fixed homography that is educed from the initial frame of a video sequence and is used to warp all input images without regeneration. Because the fixed homography is generated only once at the initial state, we can calculate it using SW to reduce HW costs. The proposed warping HW engine is based on a linear transform of the pixel positions of warped images and can reduce the computational complexity by 90% or more as compared to a conventional method. A dual‐core SW/HW image stitching engine is applied to stitching input frames in parallel to improve the performance by 70% or more as compared to a single‐core engine operation. In addition, a dual‐core structure is used to detect a failure in state machines using rock‐step logic to satisfy the ISO26262 standard. The dual‐core SW/HW image stitching engine is fabricated in SoC with 254,968 gate counts using Global Foundry's 65 nm CMOS process. The single‐core engine can make panoramic images from three YCbCr 4:2:0 formatted VGA images at 44 frames per second and frequency of 200 MHz without an LCD display.     

旋转扫描序列图像的投影矩阵和拼接应用

利用8参数投影矩阵(单应性矩阵)拼接旋转扫描序列图像的方法,由于矩阵估计误差的影响,配准精度不高,易导致拼接图像中的景物歪斜.针对这一问题,通过研究相机拍摄旋转扫描序列图像的透视关系,推导了一种5参数的投影矩阵,减少了单应性矩阵中8个参数之间的相关性,从而降低了投影矩阵估计误差对全景图拼接的影响,提高了配准精度.图像拼接实验结果显示,应用该5参数投影矩阵得到的全景图,能够保证场景正直,配准精度较高,视觉效果良好.     

医学红外图像快速拼接方法研究

临床常用的红外影像设备仅能提供视野受局限的图像,因此需要通过图像拼接才能合成一幅完整的图像。本文提出了基于网格的拼接技术,采用由粗到细的配准过程,找到最佳匹配点。运用渐进渐出的方法来消除拼接缝隙,能够自动完成图像的拼接处理。实验证明,该方法配准速度快,融合效果好,能达到无缝拼接的效果。     

全景畸变正切改正的高光谱影像视场拼接方法

静止轨道全谱段高光谱成像仪需采用通道分离、视场和机械拼接的方式,并通过扫描镜摆扫获得宽视场、高空间分辨率的高光谱影像,若直接拼接,子视场间不一致的几何变形将使地物光谱信息失真。假设子视场间同名像点坐标的差异是由各子视场内部参数变化引起的,提出以全景畸变正切改正公式为数学模型的区域网平差方法,实现像方坐标系内的视场拼接。并在可见光近红外通道4个子视场仿真影像的基础上完成了视场拼接实验,达到了0.72 pixel的拼接精度。与基于共线条件方程区域网平差的视场拼接方法相比,该方法精度与其相当,且算法更简单,不需要控制点和DEM辅助,适用于初级影像生产。     

基于NSCT的图像拼接算法

为了提高图像拼接的速度和拼接图像的视觉效果,提出了一种基于非下采样轮廓波变换的图像拼接方法.首先利用角点检测算法获得待拼接图像的匹配点,选取以匹配点为中心的空间频率最大的一定大小的子图像为模板,利用流形上的优化算法求解变换参数;再将待拼接图像分解为不同尺度和不同方向的子频带图像,在变换域进行对应的拼接;最后利用重构算法将拼接的各个子频带图像进行反变换,得到最终的拼接图像.实验表明,此方法速度更快,视觉效果更好,拼接后的图像更加自然,过渡更平滑.     

Color image quality assessment based on sparse representation and reconstruction residual

Image quality assessment (IQA) is a fundamental problem in image processing. While in practice almost all images are represented in the color format, most of the current IQA metrics are designed in gray-scale domain. Color influences the perception of image quality, especially in the case where images are subject to color distortions. With this consideration, this paper presents a novel color image quality index based on Sparse Representation and Reconstruction Residual (SRRR). An overcomplete color dictionary is first trained using natural color images. Then both reference and distorted images are represented using the color dictionary, based on which two feature maps are constructed to measure structure and color distortions in a holistic manner. With the consideration that the feature maps are insensitive to image contrast change, the reconstruction residuals are computed and used as a complementary feature. Additionally, luminance similarity is also incorporated to produce the overall quality score for color images. Experiments on public databases demonstrate that the proposed method achieves promising performance in evaluating traditional distortions, and it outperforms the existing metrics when used for quality evaluation of color-distorted images.     

基于三维旋转模型的红外全景图像拼接方法

分布孔径红外系统(DAIRS)是目前各国在红外搜索跟踪系统领域的重点发展方向,DAIRS系统为了减少冗余,通常减少重叠区域面积以获得较宽视场覆盖,这使得基于特征的拼接方法提取特征数量不足。本文通过构建三维旋转模型,采用基于梯度下降的图像配准方法,并在时间维度上进行多帧迭代来进行三维重建,再采用改进的亮度调整算法和多分辨率过渡区融合消除拼接痕迹。实验结果表明,本文算法对于平面投影和柱面投影都实现了全景图像的无缝拼接,过渡区通过多分辨率融合有利于进步消除模糊现象。