Line-preserving video stitching for asymmetric cameras

Multimedia Tools and Applications - The narrow FOV on mobile phones is becoming unsatisfying with the development of video live broadcast. Fast and stable video stitching algorithm is urgently...     

Frame-level temporal calibration of video sequences from unsynchronized cameras

This paper describes a method for temporally calibrating video sequences from unsynchronized cameras by image processing operations, and presents two search algorithms to match and align trajectories across different camera views. Existing multi-camera systems assume that input video sequences are synchronized either by genlock or by time stamp information and a centralized server. Yet, hardware-based synchronization increases installation cost. Hence, using image information is necessary to align frames from the cameras whose clocks are not synchronized. The system built for temporal calibration is composed of three modules: object tracking module, calibration data extraction module, and the search module. A robust and efficient search algorithm is introduced that recovers the frame offset by matching the trajectories in different views, and finding the most reliable match. Thanks to information obtained from multiple trajectories, this algorithm is robust to possible errors in background subtraction and location extraction. Moreover, the algorithm can handle very large frame offsets. A RANdom SAmple Consensus (RANSAC) based version of this search algorithm is also introduced. Results obtained with different video sequences are presented, which show the robustness of the algorithms in recovering various range of frame offsets for video sequences with varying levels of object activity.     

Arbitrary viewpoint video synthesis from multiple uncalibrated cameras.

We propose a method for arbitrary view synthesis from uncalibrated multiple camera system, targeting large spaces such as soccer stadiums. In Projective Grid Space (PGS), which is a three-dimensional space defined by epipolar geometry between two basis cameras in the camera system, we reconstruct three-dimensional shape models from silhouette images. Using the three-dimensional shape models reconstructed in the PGS, we obtain a dense map of the point correspondence between reference images. The obtained correspondence can synthesize the image of arbitrary view between the reference images. We also propose a method for merging the synthesized images with the virtual background scene in the PGS. We apply the proposed methods to image sequences taken by a multiple camera system, which installed in a large concert hall. The synthesized image sequences of virtual camera have enough quality to demonstrate effectiveness of the proposed method.     

Real-time free viewpoint video from a range sensor and color cameras

We propose a method for computing a depth map at interactive rates from a set of closely spaced calibrated video cameras and a Time-of-Flight (ToF) camera. The objective is to synthesize free viewpoint videos in real-time. All computations are performed on the graphics processing unit, leaving the CPU available for other tasks. Depth information is computed from color camera data in textured regions and from ToF data in textureless ones. The trade-off between these two sources is determined locally based on the reliability of the depth estimates obtained from the color images. For this purpose, a confidence measure taking into account the shape of the photo-consistency score as a function of depth is used. The final depth map is computed by minimizing a cost function. This approach offers a significant time savings relative to other methods that apply denoising to the photo-consistency score maps, obtained at every depth, and importantly, still obtains acceptable quality of the rendered image.     

Personnel tracking on construction sites using video cameras

This paper discusses the possibility of- and need for-tracking workforce on construction jobsites using video cameras. An evaluation of algorithms and their associated results is presented. The principal objective of this paper is to test and demonstrate the feasibility of tracking workers from statically placed and dynamically moving cameras. This paper also reviews existing techniques to monitor workforce and describes areas where this work might be useful in engineering applications. The main difficulties associated with tracking on a construction site is the significant amount of visual clutter, the changing photometric visual content throughout the course of a day, and the presence of occluding and moving obstacles. The tracking of workers within the field of view of the camera will involve four tracking techniques, density mean-shift, Bayesian segmentation, active contours, and graph-cuts. Typical construction site video will be processed using the proposed algorithms and analyzed to determine the most appropriate tracking method for the video presented.     

Comparison of HDTV formats using objective video quality measures

In this paper we compare some of the objective quality measures with subjective, in several HDTV formats, to be able to grade the quality of the objective measures. Also, comparison of objective and subjective measures between progressive and interlaced video signal will be presented to determine which scanning emission format is better, even if it has different resolution format. Several objective quality measures will be tested, to examine the correlation with the subjective test, using various performance measures.     

Panoramic appearance-based recognition of video contents using matching graphs

This paper proposes a general scheme for recognizing the contents of a video using a set of panoramas recorded in a database. In essence, a panorama inherently records the appearances of an omni-directional scene from its central point to arbitrary viewing directions and, thus, can serve as a compact representation of an environment. In particular, this paper emphasizes the use of a sequence of successive frames in a video taken with a video camera, instead of a single frame, for visual recognition. The associated recognition task is formulated as a shortest-path searching problem, and a dynamic-programming technique is used to solve it. Experimental results show that our method can effectively recognize a video.     

基于特征点的视频流拼接帧选择优化算法

提出了一种具有公共视野的视频流拼接中视频帧的选择优化算法,算法计算候选帧与基准帧特征匹配对的特征向量差值的算术均值,选择均值最小的候选帧作为最佳拼接帧,与基准帧进行拼接.实验结果表明,基于该算法拼接的视频流全景视图效果较好,算法很好地解决了视频流拼接中最佳拼接帧的自动选择问题.     

基于SURF和动态ROI的实时视频拼接

为解决含有运动物体的视频拼接问题及解决不同景深下的目标拼接等此类问题,提出一种基于SURF和动态ROI方法的双目摄像机视频图像快速拼接方法.通过RANSAC方法与单应性矩阵平滑方法估计双目摄像头视频图像变换模型,融合成宽场景的视频图像.由于选取了动态ROI之后不进行重叠区域之外的SURF特征提取,提高了特征匹配的精准度,避免了ROI之外的误匹配特征点对,同时减少了程序的耗时.大量的实验结果表明,该系统能够自适应的对双目摄像头采集的图像进行视频拼接,获得清晰的宽场景视频.     

Coherent video generation for multiple hand-held cameras with dynamic foreground

Computational Visual Media - For many social events such as public performances, multiple hand-held cameras may capture the same event. This footage is often collected by amateur cinematographers...     

Transmission of correct gaze direction in video conferencing using screen-embedded cameras

In this paper, we propose a new video conferencing system that presents correct gaze directions of a remote user by switching among images obtained from multiple cameras embedded in a screen according to a local user’s position. Our proposed method reproduces a situation like that in which the remote user is in the same space as the local user. The position of the remote user to be displayed on the screen is determined so that the positional relationship between the users is reproduced. The system selects one of the embedded cameras whose viewing direction towards the remote user is the closest to the local user’s viewing direction to the remote user’s image on the screen. As a result of quantitative evaluation, we confirmed that, in comparison with the case using a single camera, the accuracy of gaze estimation was improved by switching among the cameras according to the position of the local user.

     

基于消失点检测的铁路环境视频自动拼接算法研究

提出了一种基于铁路环境视频的全景图自动拼接算法。根据铁路环境视频中的场景几何特性,检测场景中的线段及消失点,从而生成场景的空间布局关系;在此基础上,自动构建合适的采样区域,以达到无缝拼接的效果;最后实现全景图的自动拼接。实验结果表明,该方法实现了铁路环境视频的自动化拼接,且得到了更加令人满意的拼接效果。     

Contour-based object tracking with occlusion handling in video acquired using mobile cameras

We propose a tracking method which tracks the complete object regions, adapts to changing visual features, and handles occlusions. Tracking is achieved by evolving the contour from frame to frame by minimizing some energy functional evaluated in the contour vicinity defined by a band. Our approach has two major components related to the visual features and the object shape. Visual features (color, texture) are modeled by semiparametric models and are fused using independent opinion polling. Shape priors consist of shape level sets and are used to recover the missing object regions during occlusion. We demonstrate the performance of our method in real sequences with and without object occlusions.     

多目及单目环境下的人体朝向分析

提出了一种在智能空间中分别基于多目和单目环境下人体朝向分析的方法.在多目环境下,通过色度分析与灰度分析相结合的方法,分别计算每个摄像机拍到的图像中肤色区域占人体头部区域的比率,根据得到的几幅图像各自的比率值,就可以判断出人体的朝向.在单目环境下,可以在得到人体头部区域和人体头部肤色区域的情况下,计算出人体头部区域中各白色像素点横坐标的均值以及人体头部肤色区域中各白色像素点横坐标的均值,通过比较两均值的大小来判断人体的朝向.试验证明本文算法快速、可靠.     

Non-rigid metric reconstruction from perspective cameras

The metric reconstruction of a non-rigid object viewed by a generic camera poses new challenges since current approaches for Structure from Motion assume the rigidity constraint of a shape as an essential condition. In this work, we focus on the estimation of the 3-D Euclidean shape and motion of a non-rigid shape observed by a perspective camera. In such case deformation and perspective effects are difficult to decouple – the parametrization of the 3-D non-rigid body may mistakenly account for the perspective distortion. Our method relies on the fact that it is often a reasonable assumption that some of the points on the object’s surface are deforming throughout the sequence while others remain rigid. Thus, relying on the rigidity constraints of a subset of rigid points, we estimate the perspective to metric upgrade transformation. First, we use an automatic segmentation algorithm to identify the set of rigid points. These are then used to estimate the internal camera calibration parameters and the overall rigid motion. Finally, we formulate the problem of non-rigid shape and motion estimation as a non-linear optimization where the objective function to be minimized is the image reprojection error. The prior information that some of the points in the object are rigid can also be added as a constraint to the non-linear minimization scheme in order to avoid ambiguous configurations. We perform experiments on different synthetic and real data sets which show that even when using a minimal set of rigid points and when varying the intrinsic camera parameters it is possible to obtain reliable metric information.     

Correction to: Transmission of correct gaze direction in video conferencing using screen-embedded cameras

Multimedia Tools and Applications -     

Panoramic Video from Unstructured Camera Arrays

We describe an algorithm for generating panoramic video from unstructured camera arrays. Artifact‐free panorama stitching is impeded by parallax between input views. Common strategies such as multi‐level blending or minimum energy seams produce seamless results on quasi‐static input. However, on video input these approaches introduce noticeable visual artifacts due to lack of global temporal and spatial coherence. In this paper we extend the basic concept of local warping for parallax removal. Firstly, we introduce an error measure with increased sensitivity to stitching artifacts in regions with pronounced structure. Using this measure, our method efficiently finds an optimal ordering of pair‐wise warps for robust stitching with minimal parallax artifacts. Weighted extrapolation of warps in non‐overlap regions ensures temporal stability, while at the same time avoiding visual discontinuities around transitions between views. Remaining global deformation introduced by the warps is spread over the entire panorama domain using constrained relaxation, while staying as close as possible to the original input views. In combination, these contributions form the first system for spatiotemporally stable panoramic video stitching from unstructured camera array input.     

基于灰度塔评分的匹配模型构建在无人机网络视频拼接中的应用

对于复杂非配合情况下，视频拼接中特征匹配对的数目和特征匹配准确率无法同时达到后续稳像和拼接的要求这一问题，提出一种基于灰度塔对特征点进行评分后构建匹配模型来进行精准特征匹配的方法。首先，利用灰度级压缩后相近灰度级合并这一现象，建立灰度塔来实现对特征点的评分；而后，选取评分高的特征点建立基于位置信息的匹配模型；最后，依据匹配模型的定位进行区域分块匹配来避免全局特征点的干扰和大误差噪点匹配，选择误差最小的特征匹配对作为最终结果匹配对。另外，在运动的视频流中，可通过前后帧信息建立掩模进行区域特征提取，匹配模型也可选择性遗传给后帧以节约算法时间。实验结果表明，在运用了基于灰度塔评分的匹配模型后，特征匹配对准确率在95%左右。相同帧特征匹配对的数目相较于随机采样一致性有近10倍的提升，在兼顾匹配数目和匹配准确率的同时且无大误差匹配结果，对于环境和光照有较好的鲁棒性。     

相机运动条件下的行人检测与跟踪

行人检测与跟踪在司机辅助安全系统和视频监控等领域具有重要的地位.针对目前存在的关键问题,如人体运动,相机运动,背景及形状、角度等变化对检测及跟踪带来的干扰,提出了一种将运动信息与形状信息相结合的行人检测方法,准确检测运动摄像机拍摄的直立运动人体;使用了基于小面积目标的跟踪算法进行人体跟踪;利用实际拍摄的视频序列进行算法验证.实验结果表明,混合检测算法速度快,准确率高;基于小面积的跟踪算法能够鲁棒的跟踪检测到的运动人体.