基于模糊聚类的运动对象分割算法研究

为了从视频序列中分割出完整的、一致的运动视频对象,该文使用基于模糊聚类的分割算法获得组成对象边界的像素,从而提取对象。该算法首先使用了当前帧以及之前一些帧的图像信息计算其在小波域中不同子带的运动特征,并根据这些运动特征构造了低分辨率图像的运动特征矢量集;然后,使用模糊C-均值聚类算法分离出图像中发生显著变化的像素,以此代替帧间差图像,并利用传统的变化检测方法获得对象变化检测模型,从而提取对象;同时,使用相继两帧之间的平均绝对差值大小确定计算当前帧运动特征所需帧的数量,保证提取视频对象的精确性。实验结果证明该方法对于分割各种图像序列中的视频对象是有效的。     

Image processing algorithms for retinal montage synthesis, mapping,and real-time location determination

Although laser retinal surgery is the best available treatment for choroidal neovascularization, the current procedure has a low success rate (50%). Challenges, such as motion-compensated beam steering, ensuring complete coverage and minimizing incidental photodamage, can be overcome with improved instrumentation. This paper presents core image processing algorithms for (1) rapid identification of branching and crossover points of the retinal vasculature; (2) automatic montaging of video retinal angiograms; (3) real-time location determination and tracking using a combination of feature-tagged point-matching and dynamic-pixel templates. These algorithms tradeoff conflicting needs for accuracy, robustness to image variations (due to movements and the difficulty of providing steady illumination) and noise, and operational speed in the context of available hardware. The algorithm for locating vasculature landmarks performed robustly at a speed of 16-30 video image frames/s depending upon the field on a Silicon Graphics workstation. The montaging algorithm performed at a speed of 1.6-4 s for merging 5-12 frames. The tracking algorithm was validated by manually locating six landmark points on an image sequence with 180 frames, demonstrating a mean-squared error of 1.35 pixels. It successfully detected and rejected instances when the image dimmed, faded, lost contrast, or lost focus     

Markerless real-time 3-D target region tracking by motion backprojection from projection images

Accurate and fast localization of a predefined target region inside the patient is an important component of many image-guided therapy procedures. This problem is commonly solved by registration of intraoperative 2-D projection images to 3-D preoperative images. If the patient is not fixed during the intervention, the 2-D image acquisition is repeated several times during the procedure, and the registration problem can be cast instead as a 3-D tracking problem. To solve the 3-D problem, we propose in this paper to apply 2-D region tracking to first recover the components of the transformation that are in-plane to the projections. The 2-D motion estimates of all projections are backprojected into 3-D space, where they are then combined into a consistent estimate of the 3-D motion. We compare this method to intensity-based 2-D to 3-D registration and a combination of 2-D motion backprojection followed by a 2-D to 3-D registration stage. Using clinical data with a fiducial marker-based gold-standard transformation, we show that our method is capable of accurately tracking vertebral targets in 3-D from 2-D motion measured in X-ray projection images. Using a standard tracking algorithm (hyperplane tracking), tracking is achieved at video frame rates but fails relatively often (32% of all frames tracked with target registration error (TRE) better than 1.2 mm, 82% of all frames tracked with TRE better than 2.4 mm). With intensity-based 2-D to 2-D image registration using normalized mutual information (NMI) and pattern intensity (PI), accuracy and robustness are substantially improved. NMI tracked 82% of all frames in our data with TRE better than 1.2 mm and 96% of all frames with TRE better than 2.4 mm. This comes at the cost of a reduced frame rate, 1.7 s average processing time per frame and projection device. Results using PI were slightly more accurate, but required on average 5.4 s time per frame. These results are still substantially faster than 2-D to 3-D registration. We conclude that motion backprojection from 2-D motion tracking is an accurate and efficient method for tracking 3-D target motion, but tracking 2-D motion accurately and robustly remains a challenge.     

A new real-time retinal tracking system for image-guided laser treatment

A new system is proposed for tracking sensitive areas in the retina for computer-assisted laser treatment of choroidal neovascularization (CNV). The system consists of a fundus camera using red-free illumination mode interfaced to a computer that allows real-time capturing of video input. The first image acquired is used as the reference image and utilized by the treatment physician for treatment planning. A grid of seed contours over the whole image is initiated and allowed to deform by splitting and/or merging according to preset criteria until the whole vessel tree is demarcated. Then, the image is filtered using a one-dimensional Gaussian filter in two perpendicular directions to extract the core areas of such vessels. Faster segmentation can be obtained for subsequent images by automatic registration to compensate for eye movement and saccades. An efficient registration technique is developed whereby some landmarks are detected in the reference frame then tracked in the subsequent frames. Using the relation between these two sets of corresponding points, an optimal transformation can be obtained. The implementation details of proposed strategy are presented and the obtained results indicate that it is suitable for real-time location determination and tracking of treatment positions.     

Improved motion estimation time using a combination of dynamic reference frame selection and residue-based mode decision

Motion estimation using multiple reference frames is widely used as the basis for recent video coding standards (eg. H.264/AVC) to achieve increased coding efficiency. However, this increases the complexity of the encoding process. In this paper, a new technique for efficient motion estimation is proposed. A combination of multiple reference frame selection and image residue-based mode selection is used to improve motion estimation time. By dynamic selection of an initial reference frame in advance, the number of reference frames to be considered is reduced. In addition, from examination of the residue between the current block and reconstructed blocks in preceding frames, variable block size mode decisions are made. Modified initial motion vector estimation and early stop condition detection are also adopted to speed up the motion estimation procedure. Experimental results compare the performance of the proposed algorithm with a state of the art motion estimation algorithm and demonstrate significantly reduced motion estimation time while maintaining PSNR performance.     

基于优选特征轨迹的全分辨率视频稳定

该文提出一种基于优选特征轨迹的视频稳定算法。首先,采用改进的Harris角点检测算子提取特征点,通过K-Means聚类算法剔除前景特征点。然后,利用帧间特征点的空间运动一致性减少错误匹配和时间运动相似性实现长时间跟踪,从而获取有效特征轨迹。最后,建立同时包含特征轨迹平滑度与视频质量退化程度的目标函数计算视频序列的几何变换集以平滑特征轨迹获取稳定视频。针对图像扭曲产生的空白区,由当前帧定义区与参考帧的光流作引导来腐蚀,并通过图像拼接填充仍属于空白区的像素。经仿真验证,该文方法稳定的视频,空白区面积仅为Matsushita方法的33%左右,对动态复杂场景和多个大运动前景均具有较高的有效性并可生成内容完整的视频,既提高了视频的视觉效果,又减轻了费时的边界修复任务。     

基于视频序列的数字图像拼接技术研究进展

基于视频序列的数字图像拼接是指将具有重叠区的多帧视频通过数字配准和融合获得单幅宽视场静态全景图或动态全景图.基于视频序列的数字图像拼接技术主要包括全局快速配准算法、运动目标分割算法和无缝融合算法.首先分析理想数字图像拼接系统的特性,然后介绍近年来基于视频序列的数字图像拼接技术的研究进展,最后分析其研究动向.     

Multimodal registration of retinal images using self organizing maps

In this paper, an automatic method for registering multimodal retinal images is presented. The method consists of three steps: the vessel centerline detection and extraction of bifurcation points only in the reference image, the automatic correspondence of bifurcation points in the two images using a novel implementation of the self organizing maps and the extraction of the parameters of the affine transform using the previously obtained correspondences. The proposed registration algorithm was tested on 24 multimodal retinal pairs and the obtained results show an advantageous performance in terms of accuracy with respect to the manual registration.     

基于场景复杂度与不变特征的航拍视频实时配准算法

实时、鲁棒的图像配准是航拍视频电子稳像、全景图拼接和地面运动目标自动检测与跟踪的前提和关键技术.本文以航拍视频序列为处理对象,提出了一种新的基于场景复杂度与不变特征的实时配准算法,其主要特点包括：(1)在对航拍视频配准难点进行详细分析的基础上,有针对性的提出基于积分图的快速图像尺度空间构建、依据场景复杂度的检测特征点数量在线精确控制、基于描述子误差分布统计特性级的联分类器构造等新方法,使得算法配准性能不随场景的复杂度发生改变,能够在各种地貌条件下实时、稳定的进行图像配准;(2)将多尺度Harris角点和SIFT描述子相结合,并通过对帧间变换模型参数进行鲁棒估计,保证了算法具有良好的旋转、尺度、亮度不变性和配准精度.实验结果表明,算法可在场景变化、图像大幅度平移、尺度缩放和任意角度旋转等复杂条件下实时、精确的进行图像配准,对分辨率为320×240的航拍序列的平均处理速度达到20.7帧/秒.     

Predictive scheduling algorithms for real-time feature extraction and spatial referencing: application to retinal image sequences

Real-time spatial referencing is an important alternative to tracking for designing spatially aware ophthalmic instrumentation for procedures such as laser photocoagulation and perimetry. It requires independent, fast registration of each image frame from a digital video stream (1024 x 1024 pixels) to a spatial map of the retina. Recently, we have introduced a spatial referencing algorithm that works in three primary steps: 1) tracing the retinal vasculature to extract image feature (landmarks); 2) invariant indexing to generate hypothesized landmark correspondences and initial transformations; and 3) alignment and verification steps to robustly estimate a 12-parameter quadratic spatial transformation between the image frame and the map. The goal of this paper is to introduce techniques to minimize the amount of computation for successful spatial referencing. The fundamental driving idea is to make feature extraction subservient to registration and, therefore, only produce the information needed for verified, accurate transformations. To this end, the image is analyzed along one-dimensional, vertical and horizontal grid lines to produce a regular sampling of the vasculature, needed for step 3) and to initiate step 1). Tracing of the vascular is then prioritized hierarchically to quickly extract landmarks and groups (constellations) of landmarks for indexing. Finally, the tracing and spatial referencing computations are integrated so that landmark constellations found by tracing are tested immediately. The resulting implementation is an order-of-magnitude faster with the same success rate. The average total computation time is 31.2 ms per image on a 2.2-GHz Pentium Xeon processor.     

基于邻域运动一致性的运动矢量快速构建算法

在时移电视应用中,视频快进播放要求视频服务器能够通过跳帧抽取视频序列中的I帧和P帧的方式输出快速播放码流,当所跳帧中含有P帧时,必需对抽取出来的快进码流中的P帧进行重编码.为提高抽取后重编码效率,可利用码流中的原始运动矢量对跳帧后新参考帧条件下的运动矢量进行预测.在FDVS和ADVS算法基础上,提出了一种基于宏块及其相邻宏块运动一致性的算法来修正原始运动矢量,并通过实验将其与现有的FDVS和ADVS算法进行比较.     

基于时序多帧投影的空间目标检测算法

为了对星图中空间目标进行检测识别,对基于 SURF（Speed-Up Robust Featrues）算法的星图精确配准技术和美国 SBV（Space-Based Visual）计划采用的 MTI（Moving Target Indicati-or）在轨目标检测算法进行了深入研究,提出一种针对16 Bits 星图的多目标检测算法,具体包括：首先利用 SURF 算法提取序列星图的特征点,根据最小二乘法计算得到的全局运动参数对星图进行精确配准;然后利用一种改进的 MTI 算法对序列星图进行时序多帧投影以抑制背景,得到仅含有疑似目标的序列图像;最后经过目标初始运动状态的建立,速度滤波以及坐标插值得到目标的运动轨迹。利用实拍的20帧序列星图验证算法性能,经本文算法配准后,星像质心的均方误差（RMSE,Root Mean Square Error）最小达到0.3269 pixel,平均值为0.5441 pixel;序列图像中的3个运动目标均被检出,且无虚警。实验结果表明,本文配准算法的精度能够满足时序多帧投影的要求,且目标检测算法符合恒虚警原理。     

基于Kalman预测和Mean-shift算法的视频目标跟踪

提出一种用Kalman滤波理论和Mean-shift算法结合的视频目标跟踪方法,解决了目标变形、部分遮挡和运动速度过快问题。在初始帧中,确定跟踪目标并计算H分量直方图,将每帧图像转化为该直方图的概率投影图;在当前帧中,用Kalman滤波预测搜索窗口,并在搜索窗口中用自适应的Mean-shift算法精确匹配跟踪目标。实验结果表明,本文方法对刚体、非刚体和多目标的跟踪都具有良好的自适应性。     

Affine-structure-based facial image encoding

A real-time algorithm for affine-structure-based video compression for facial images is presented. The face undergoing motion is segmented and triangulated to yield a set of control points. The set of control points generated by triangulation are tracked across a few frames using an intensity-based correlation technique. For accurate motion and structure estimation a Kalman-filter-based algorithm is used to track features on the facial image. The structure information of the control points is transmitted only during the bootstrapping stage. After that only the motion information is transmitted to the decoder. This reduces the number of motion parameters associated with control points in each frame. The local motion of the eyes and lips is captured using local 2-D affine transformations. For real time implementation a quad-tree based search technique is adopted to solve local correlation. Any remaining reconstruction error is accounted for using predictive encoding. Results on real image sequences demonstrate the applicability of the method     

Semi-automatic 2D-to-3D video conversion based on background sprite generation

This paper presents a technique for semi-automatic 2D-to-3D stereo video conversion, which is known to provide user intervention in assigning foreground/background depths for key frames and then get depth maps for non-key frames via automatic depth propagation. Our algorithm treats foreground and background separately. For foregrounds, kernel pixels are identified and then used as the seeds for graph-cut segmentation for each non-key frame independently, resulting in results not limited by objects’ motion activity. For backgrounds, all video frames, after foregrounds being removed, are integrated into a common background sprite model (BSM) based on a relay-frame-based image registration algorithm. Users can then draw background depths for BSM in an integrated manner, thus reducing human efforts significantly. Experimental results show that our method is capable of retaining more faithful foreground depth boundaries (by 1.6–2.7 dB) and smoother background depths than prior works. This advantage is helpful for 3D display and 3D perception.     

基于雾线先验的时空关联约束视频去雾算法

现有视频去雾算法由于缺少对视频结构关联约束和帧间一致性分析,容易导致连续帧去雾结果在颜色和亮度上存在突变,同时去雾后的前景目标边缘区域也容易出现退化现象。针对上述问题,该文提出一种基于雾线先验的时空关联约束视频去雾算法,通过引入每帧图像在空间邻域中具有的结构关联性和时间邻域中具有的连续一致性,提高视频去雾算法的求解准确性和鲁棒性。算法首先使用暗通道先验估计每帧图像的大气光向量,并结合雾线先验求取初始透射率图。然后引入加权最小二乘边缘保持平滑滤波器对初始透射率图进行空间平滑,消除奇异点和噪声对估计结果的影响。进一步利用相机参数刻画连续帧间透射率图的时序变化规律,对独立求取的每帧透射率图进行时序关联修正。最后根据雾图模型获得最终的视频去雾结果。定性和定量的对比实验结果表明,该算法下视频去雾结果的帧间过渡更加自然,同时对每一帧图像的色彩还原更加准确,图像边缘的细节信息显示也更加丰富。     

基于Fourier-Mellin变换和Keren算法的改进运动估计算法

针对超分辨率图像重建中图像运动估计精度要求高,速度要求快的问题,对传统的基于Fourier-Mellin变换和Keren算法的运动估计方法做出以下改进:首先提取参考图像和待估计图像的边缘,从而避免了Fourier-Mellin变换的不足(对细节不明显的图像运动估计精度极差);由于只是用Fourier-Mellin变换进行粗估计,对角度估计精度要求不高,只需小于1°,因此在进行对数极坐标变换时,可以减少角度坐标和对数坐标的采样点数,大幅缩小了矩阵大小,提高了运动估计速度;由于先用Fourier-Mellin对待估计图像进行粗估计,Keren算法可以避开复杂的金字塔计算而只需一层估计,减少了运动估计时间。在VC++中的仿真实验表明,该方法有效地结合了Fourier-Mellin变换和Keren算法的优点,同时又提高了运动估计速度。经测试,用未改进的算法对328×500像素大小的两幅图像进行运算估计需要3.53s,而用改进的算法则只需要1.15s,大大提高了运动估计速度。     

Motion Compensated Subband Video Coding with Arbitrarily Shaped Region Adaptivity

The performance of Motion Compensated Discrete Cosine Transform (MC‐DCT) video coding is improved by using the region adaptive subband image coding [18]. On the assumption that the video is acquired from the camera on a moving platform and the distance between the camera and the scene is large enough, both the motion of camera and the motion of moving objects in a frame are compensated. For the compensation of camera motion, a feature matching algorithm is employed. Several feature points extracted using a Sobel operator are used to compensate the camera motion of translation, rotation, and zoom. The illumination change between frames is also compensated. Motion compensated frame differences are divided into three regions called stationary background, moving objects, and newly emerging areas each of which is arbitrarily shaped. Different quantizers are used for different regions. Compared to the conventional MC‐DCT video coding using block matching algorithm, our video coding scheme shows about 1.0‐dB improvements on average for the experimental video samples.     

An illumination invariant algorithm for subpixel accuracy image stabilization and its effect on MPEG-2 video compression

This paper offers a fast and simple near-closed-form solution for the least mean-squared-error (LMSE) estimation of the frame-to-frame global subpixel motion in an unsteady image sequence. The offered near-closed-form solution achieves unlimited subpixel accuracy by always employing a small and fixed number of computations, independent of the desired subpixel accuracy. The algorithm is designed so that it is insensitive to frame-to-frame intensity variations, which is a distinctive feature of the method. Experimental results demonstrate the superiority of the proposed method to the spatio-temporal differentiation and surface fitting algorithms under different illumination conditions. This paper furthermore discusses the effect of the proposed image stabilization algorithm on the performance of MPEG-2 video compression. We report that removal of global motion down to subpixel accuracy from an unsteady video improves MPEG-2 compression performance significantly (by at least 1 dB for all frame types), in spite of the fact that motion vectors are differentially encoded in MPEG-2. This result is supported via experimental results using the proposed subpixel registration algorithm and an analysis of the macroblock coding preferences accepted in MPEG-2.     

基于Open Pose的角色动作提取

文章利用基于OpenPose的人体姿态识别算法,将动画或者视频中的人物动作进行处理,实现对角色运动数据的捕获;采用基于运动捕捉数据的关键帧提取算法来获取角色运动的关键帧。实验结果表明,该方法具较高的识别准确率,并且能够提取出对原动画或视频具有概括力的关键帧。