Extracting mobile objects by sequential background detection on a video

This paper describes a technique for extracting moving objects from a video image sequence taken by a slowly moving camera as well as a fixed camera. The background subtraction method is effective for extracting moving objects from a video. But the latest background image should be employed for the subtraction in the mobile camera case and in order not to be influenced by the light intensity change. A temporal median technique is proposed in this paper which detects the background at every moment. The camera motion is estimated using a local correlation map and the temporal median filter is applied to the common image area among a set of successive image frames to extract the background. The technique was applied to the video images obtained at a junction from a hand-held camera and those taken at a pedestrians crossing by a camera fixed in a car and successfully detected pedestrians. This work was presented in part at the 13th International Symposium on Artificial Life and Robotics, Oita, Japan, January 31–February 2, 2008     

无人机侦察视频超分辨率重建方法

目的 无人机摄像资料的分辨率直接影响目标识别与信息获取,所以摄像分辨率的提高具有重大意义。为了改善无人机侦察视频质量,针对目前无人机摄像、照相数据的特点,提出一种无人机侦察视频超分辨率重建方法。方法 首先提出基于AGAST-Difference与Fast Retina Keypoint (FREAK)的特征匹配算法对视频目标帧与相邻帧之间配准,然后提出匹配区域搜索方法找到目标帧与航片的对应关系,利用航片对视频帧进行高频补偿,最后采用凸集投影方法对补偿后视频帧进行迭代优化。结果 基于AGAST-Difference与FREAK的特征匹配算法在尺度、旋转、视点等变化及运行速度上存在很大优势,匹配区域搜索方法使无人机视频的高频补偿连续性更好,凸集投影迭代优化提高了重建的边缘保持能力,与一种简单有效的视频序列超分辨率复原算法相比,本文算法重建质量提高约4 dB,运行速度提高约5倍。结论 提出了一种针对无人机的视频超分辨率重建方法,分析了无人机视频超分辨率问题的核心所在,并且提出基于AGAST-Difference与FREAK的特征匹配算法与匹配区域搜索方法来解决图像配准与高频补偿问题。实验结果表明,本文算法强化了重建图像的一致性与保真度,特别是对图像边缘细节部分等效果极为明显,且处理速度更快。     

Real-time detection of steam in video images

In this paper, we present a real-time image processing technique for the detection of steam in video images. The assumption made is that the presence of steam acts as a blurring process, which changes the local texture pattern of an image while reducing the amount of details. The problem of detecting steam is treated as a supervised pattern recognition problem. A statistical hidden Markov tree (HMT) model derived from the coefficients of the dual-tree complex wavelet transform (DT-CWT) in small 48×48 local regions of the image frames is used to characterize the steam texture pattern. The parameters of the HMT model are used as an input feature vector to a support vector machine (SVM) technique, specially tailored for this purpose. By detecting and determining the total area covered by steam in a video frame, a computerized image processing system can automatically decide if the frame can be used for further analysis. The proposed method was quantitatively evaluated by using a labelled image data set with video frames sampled from a real oil sand video stream. The classification results were 90% correct when compared to human labelled image frames. The technique is useful as a pre-processing step in automated image processing systems.     

Panoramic appearance-based recognition of video contents using matching graphs

This paper proposes a general scheme for recognizing the contents of a video using a set of panoramas recorded in a database. In essence, a panorama inherently records the appearances of an omni-directional scene from its central point to arbitrary viewing directions and, thus, can serve as a compact representation of an environment. In particular, this paper emphasizes the use of a sequence of successive frames in a video taken with a video camera, instead of a single frame, for visual recognition. The associated recognition task is formulated as a shortest-path searching problem, and a dynamic-programming technique is used to solve it. Experimental results show that our method can effectively recognize a video.     

结合光流法和卡尔曼滤波的视频稳像算法

针对手机拍摄过程中产生的视频抖动问题,提出了一种基于光流法和卡尔曼滤波的视频稳像算法。首先通过光流法预稳定抖动视频,对其生成的预稳定视频帧进行Shi-Tomasi角点检测,并采用LK算法跟踪角点,再利用RANSAC算法估计相邻帧间的仿射变换矩阵,由此计算得出原始相机路径;然后通过卡尔曼滤波器优化平滑相机路径,得到平滑相机路径;最后由原始相机路径与平滑路径的关系,计算相邻帧间的补偿矩阵,再利用补偿矩阵对视频帧逐一进行几何变换,由此得到稳定的视频输出。实验表明,该算法在处理6大类抖动视频时均有较好的效果,其中稳像后视频的PSNR值相比原始视频的PSNR值约提升了6.631 dB,视频帧间的结构相似性SSIM约提升了40%,平均曲率值约提升了8.3%。     

基于特征跟踪和网格路径运动的视频稳像算法

针对手持移动设备拍摄的抖动视频问题,提出了一种基于特征跟踪和网格路径运动的视频稳像算法。通过SIFT算法提取视频帧的特征点,采用KLT算法追踪特征点,利用RANSAC算法估计相邻帧间的仿射变换矩阵,将视频帧划分为均匀的网格,计算视频的运动轨迹,再通过极小化能量函数优化平滑多条网格路径。最后由原相机路径与平滑相机路径的关系,计算相邻帧间的补偿矩阵,利用补偿矩阵对每一帧进行几何变换,从而得到稳定的视频。实验表明,该算法在手持移动设备拍摄的抖动视频中有较好的结果,其中稳像后视频的PSNR平均值相比原抖动视频PSNR值大约提升了11.2 dB。与捆绑相机路径方法相比约提升了2.3 dB。图像间的结构相似性SSIM平均值大约提升了59%,与捆绑相机路径方法相比约提升了3.3%。     

Novel robust image watermarking based on subsampling and DWT

This paper presents a robust digital image watermarking scheme based on subsampling and DWT. Subsampling is firstly used to construct a subimage sequence as a video segment. Then, a random watermark sequence satisfied with Gaussian distribution is block-wised embedded into the DWT domain of these subimages repeatedly using the video watermarking technique. And watermark is detected through computing correlation between watermark and watermarked frames. The experiment results demonstrate that the proposed scheme achieves good robustness against JPEG compression, common image processing operation and geometric distortions. Furthermore, the proposed watermarking scheme is also robust against linear collusion and other video watermarking attacks.     

Fast Electronic Digital Image Stabilization for Off-Road Navigation

Image stabilization can be used as front-end system for many tasks that require dynamic image analysis, such as navigation and tracking of independently moving objects from a moving platform. We present a fast and robust electronic digital image stabilization system that can handle large image displacements based on a two-dimensional feature-based multi-resolution motion estimation technique. The method tracks a small set of features and estimates the movement of the camera between consecutive frames. Stabilization is achieved by combining all motion from a reference frame and warping the current frame back to the reference. The system has been implemented on parallel pipeline image processing hardware (a Datacube MaxVideo 200) connected to a SUN SPARCstation 20/612 via a VME bus adaptor. Experimental results using video sequences taken from a camera mounted on a vehicle moving on rough terrain show the robustness of the system while running at approximately 20 frames/s.     

Camera calibration with distortion models and accuracy evaluation

A camera model that accounts for major sources of camera distortion, namely, radial, decentering, and thin prism distortions is presented. The proposed calibration procedure consists of two steps: (1) the calibration parameters are estimated using a closed-form solution based on a distribution-free camera model; and (2) the parameters estimated in the first step are improved iteratively through a nonlinear optimization, taking into account camera distortions. According to minimum variance estimation, the objective function to be minimized is the mean-square discrepancy between the observed image points and their inferred image projections computed with the estimated calibration parameters. The authors introduce a type of measure that can be used to directly evaluate the performance of calibration and compare calibrations among different systems. The validity and performance of the calibration procedure are tested with both synthetic data and real images taken by tele- and wide-angle lenses     

基于增强现实的英语视听说移动教学软件设计与实现

由于当前大多数笔记本电脑不再配置光驱,但英语教材依然以光盘形式提供视频资料,影响学习效果,因此采用Unity3D集成Vuforia SDK设计实现基于增强现实技术的英语视听说移动教学软件VBook。系统首先构建识别图数据库存于云端,根据识别图名称命名对应的视频文件;然后利用Unity3D设计和渲染场景,设计ImageTarget对象的虚拟视频播放按钮,编写脚本代码实现对识别图数据库及其相应视频的访问;最后生成便于用户使用的移动端应用。用户只需将Camera镜头对准书本插图,即可呈现出虚实叠加的视觉效果,实现移动设备的英语教学视频播放。将增强现实技术应用于英语视频教学,能使用户享受到新颖的学习方法和虚实结合的交互体验。     

基于光流的快速人体姿态估计

针对目前深度学习领域人体姿态估计算法计算复杂度高的问题,提出了一种基于光流的快速人体姿态估计算法.在原算法的基础上,首先利用视频帧之间的时间相关性,将原始视频序列分为关键帧和非关键帧分别处理（相邻两关键帧之间的图像和前向关键帧组成一个视频帧组,同一视频帧组内的视频帧相似）,仅在关键帧上运用人体姿态估计算法,并通过轻量级光流场将关键帧识别结果传播到其他非关键帧.其次针对视频中运动场的动态特性,提出一种基于局部光流场的自适应关键帧检测算法,以根据视频的局部时域特性确定视频关键帧的位置.在OutdoorPose和HumanEvaI数据集上的实验结果表明,对于存在背景复杂、部件遮挡等问题的视频序列中,所提算法较原算法检测性能略有提升,检测速度平均可提升89.6%.     

基于二维透射变换的视频全景图拼接算法

视频序列全景图拼接技术是视频图像分析的关键技术之一。针对帧间运动模型为二维投影变换的常见视频序列,提出了一种全景图拼接算法.该算法首先采用改进的基于RANSAC的特征点匹配算法对相邻帧的运动参数进行估计,接着采用直接法对各帧与全景图间的运动参数进行精确调整,最后运用自适应中值滤波绘制出全景图。实验结果表明,该方法能够生成较高质量的全景图,基本能实现无缝连接。     

基于贝叶斯估计与区域划分遍历的四轴飞行器避障路径规划算法

为了提高利用图像处理技术进行四轴飞行器避障的实时性,提出一种基于贝叶斯估计与区域划分遍历的避障路径规划算法.首先,通过贝叶斯估计来对四轴飞行器采集到的视频图像进行预处理;其次,对采集到的图像进行障碍物概率分析以获取视频图像中的关键帧,最大限度地提高四轴飞行器的实时性;最后,对选取的图像帧进行背景差分实现障碍物识别,并通...     

Video text detection and segmentation for optical character recognition

In this paper, we present approaches to detecting and segmenting text in videos. The proposed video-text-detection technique is capable of adaptively applying appropriate operators for video frames of different modalities by classifying the background complexities. Effective operators such as the repeated shifting operations are applied for the noise removal of images with high edge density. Meanwhile, a text-enhancement technique is used to highlight the text regions of low-contrast images. A coarse-to-fine projection technique is then employed to extract text lines from video frames. Experimental results indicate that the proposed text-detection approach is superior to the machine-learning-based (such as SVM and neural network), multiresolution-based, and DCT-based approaches in terms of detection and false-alarm rates. Besides text detection, a technique for text segmentation is also proposed based on adaptive thresholding. A commercial OCR package is then used to recognize the segmented foreground text. A satisfactory character-recognition rate is reported in our experiments.Published online: 14 December 2004     

Omnidirectional Image Stabilization for Visual Object Recognition

In this paper, we present a pipeline for camera pose and trajectory estimation, and image stabilization and rectification for dense as well as wide baseline omnidirectional images. The proposed pipeline transforms a set of images taken by a single hand-held camera to a set of stabilized and rectified images augmented by the computed camera 3D trajectory and a reconstruction of feature points facilitating visual object recognition. The paper generalizes previous works on camera trajectory estimation done on perspective images to omnidirectional images and introduces a new technique for omnidirectional image rectification that is suited for recognizing people and cars in images. The performance of the pipeline is demonstrated on real image sequences acquired in urban as well as natural environments.     

Video frame interpolation via optical flow estimation with image inpainting

As we all know, video frame rate determines the quality of the video. The higher the frame rate, the smoother the movements in the picture, the clearer the information expressed, and the better the viewing experience for people. Video interpolation aims to increase the video frame rate by generating a new frame image using the relevant information between two consecutive frames, which is essential in the field of computer vision. The traditional motion compensation interpolation method will cause holes and overlaps in the reconstructed frame, and is easily affected by the quality of optical flow. Therefore, this paper proposes a video frame interpolation method via optical flow estimation with image inpainting. First, the optical flow between the input frames is estimated via combined local and global-total variation (CLG-TV) optical flow estimation model. Then, the intermediate frames are synthesized under the guidance of the optical flow. Finally, the nonlocal self-similarity between the video frames is used to solve the optimization problem, to fix the pixel loss area in the interpolated frame. Quantitative and qualitative experimental results show that this method can effectively improve the quality of optical flow estimation, generate realistic and smooth video frames, and effectively increase the video frame rate.     

摄像机旋转运动下的多帧图像对齐技术

针对传统旋转运动参数估计都是采用两帧图像对齐技术,提出了为多帧运动参数估计方法,即使用多帧子空间约束技术．证明了当摄像机参数不变时,多帧运动参数集合可嵌入一个低维线性子空间上;使用奇异值分解方法来降低线性子空间的秩,用最小二乘技术求解所有帧的运动参数．该方法不需要恢复任何3D信息;由于多帧参数估计法比两帧有更多的约束,因此取得更精确的图像对齐效果．该方法可用小图像进行参数估计．     

ZLBM: zero level binary mapping technique for video security

In recent days, providing security to data is a crucial and critical task in many image processing applications. Specifically, video security is an important and demanding concept. For this purpose, some of the embedding, encoding and decoding techniques are mentioned in existing works, but it has some drawbacks such as increased time complexity, computational complexity and memory consumption. Moreover, it does not provide high security during video transmission. To overcome all these issues, a new technique, namely, Zero Level Binary Mapping (ZLBM) is proposed in this paper for video embedding scheme. The motivation of this paper is to provide high security during video transformation by using the video steganography technique. At first, the cover and stego videos are given as the inputs and it will be converted into the video frames for further processing. Here, the Fuzzy Adaptive Median Filtering (FAMF) technique is employed to remove the impulse noise in the video frames. Then, the pixels in the filtered frames are grouped by using the block wise pixel grouping technique. After that, the frames are embedded with the help of ZLBM technique and encoded based on the patch wise code formation technique. On the receiver side, the inverse ZLBM and block wise pixel regrouping techniques are applied to get the original cover and stego videos. The novel concept of this paper is the use of ZLBM and patch wise code formation techniques for video embedding and compression. The main advantages of the proposed system are high security, good quality and reduced complexity. The experimental results evaluate the performance of the proposed video embedding technique in terms of Peak Signal-to-Noise Ratio (PSNR), Mean Squared Error (MSE), Compression Ratio (CR), Bits Per Pixel (BPP) and Signal-to-Noise Ratio (SNR).     

Effective and efficient human action recognition using dynamic frame skipping and trajectory rejection

Human action recognition (HAR) is a core technology for human–computer interaction and video understanding, attracting significant research and development attention in the field of computer vision. However, in uncontrolled environments, achieving effective HAR is still challenging, due to the widely varying nature of video content. In previous research efforts, trajectory-based video representations have been widely used for HAR. Although these approaches show state-of-the-art HAR performance for various datasets, issues like a high computational complexity and the presence of redundant trajectories still need to be addressed in order to solve the problem of real-world HAR. In this paper, we propose a novel method for HAR, integrating a technique for rejecting redundant trajectories that are mainly originating from camera movement, without degrading the effectiveness of HAR. Furthermore, in order to facilitate efficient optical flow estimation prior to trajectory extraction, we integrate a technique for dynamic frame skipping. As a result, we only make use of a small subset of the frames present in a video clip for optical flow estimation. Comparative experiments with five publicly available human action datasets show that the proposed method outperforms state-of-the-art HAR approaches in terms of effectiveness, while simultaneously mitigating the computational complexity.