Line-preserving video stitching for asymmetric cameras

Multimedia Tools and Applications - The narrow FOV on mobile phones is becoming unsatisfying with the development of video live broadcast. Fast and stable video stitching algorithm is urgently...     

Arbitrary viewpoint video synthesis from multiple uncalibrated cameras.

We propose a method for arbitrary view synthesis from uncalibrated multiple camera system, targeting large spaces such as soccer stadiums. In Projective Grid Space (PGS), which is a three-dimensional space defined by epipolar geometry between two basis cameras in the camera system, we reconstruct three-dimensional shape models from silhouette images. Using the three-dimensional shape models reconstructed in the PGS, we obtain a dense map of the point correspondence between reference images. The obtained correspondence can synthesize the image of arbitrary view between the reference images. We also propose a method for merging the synthesized images with the virtual background scene in the PGS. We apply the proposed methods to image sequences taken by a multiple camera system, which installed in a large concert hall. The synthesized image sequences of virtual camera have enough quality to demonstrate effectiveness of the proposed method.     

Frame-level temporal calibration of video sequences from unsynchronized cameras

This paper describes a method for temporally calibrating video sequences from unsynchronized cameras by image processing operations, and presents two search algorithms to match and align trajectories across different camera views. Existing multi-camera systems assume that input video sequences are synchronized either by genlock or by time stamp information and a centralized server. Yet, hardware-based synchronization increases installation cost. Hence, using image information is necessary to align frames from the cameras whose clocks are not synchronized. The system built for temporal calibration is composed of three modules: object tracking module, calibration data extraction module, and the search module. A robust and efficient search algorithm is introduced that recovers the frame offset by matching the trajectories in different views, and finding the most reliable match. Thanks to information obtained from multiple trajectories, this algorithm is robust to possible errors in background subtraction and location extraction. Moreover, the algorithm can handle very large frame offsets. A RANdom SAmple Consensus (RANSAC) based version of this search algorithm is also introduced. Results obtained with different video sequences are presented, which show the robustness of the algorithms in recovering various range of frame offsets for video sequences with varying levels of object activity.     

Real-time free viewpoint video from a range sensor and color cameras

We propose a method for computing a depth map at interactive rates from a set of closely spaced calibrated video cameras and a Time-of-Flight (ToF) camera. The objective is to synthesize free viewpoint videos in real-time. All computations are performed on the graphics processing unit, leaving the CPU available for other tasks. Depth information is computed from color camera data in textured regions and from ToF data in textureless ones. The trade-off between these two sources is determined locally based on the reliability of the depth estimates obtained from the color images. For this purpose, a confidence measure taking into account the shape of the photo-consistency score as a function of depth is used. The final depth map is computed by minimizing a cost function. This approach offers a significant time savings relative to other methods that apply denoising to the photo-consistency score maps, obtained at every depth, and importantly, still obtains acceptable quality of the rendered image.     

基于熵平衡的全景视频编码

高效的全景视频压缩对沉浸式、低延时的视觉体验十分重要。服务器端的全景视频通过投影到2D平面,经过编码、传输到客户端后,反投影到球面供用户观看,而全景视频投影过程中会在两极产生拉伸形变,由于过采样而产生像素冗余,破坏原有的信息熵分布。针对这一问题,提出一种改进的熵平衡算法,根据全景视频投影前后对应编码单元的熵不变原则,采用码率控制的R-λ模型根据投影面积变化比率来重新分配目标比特,以达到熵平衡,进而调整编码参数优化编码。实验结果表明,该算法相较于基准VTM14.0能达到1.87%的码率节省,编码时间下降10.07%,说明该算法在全景视频编码中能实现更高的压缩率和更短的编码时间。     

Real-time panorama video system using networked multiple cameras

Although panorama creation has been studied extensively, previous works still have limitations because they depend on customized hardware or post processing, resulting in high production cost and complexity. We propose and implement real-time panorama video system by using off-the-shelf embedded boards and camera modules. For real-time, we developed the performance improving methods such as reducing the homography computation load and employing multi thread structure. We conducted a series of experiments to investigate how different blending combinations and the number of threads affect the performance and image quality. We also analyzed the time consumption of threads and stitching steps.     

Personnel tracking on construction sites using video cameras

This paper discusses the possibility of- and need for-tracking workforce on construction jobsites using video cameras. An evaluation of algorithms and their associated results is presented. The principal objective of this paper is to test and demonstrate the feasibility of tracking workers from statically placed and dynamically moving cameras. This paper also reviews existing techniques to monitor workforce and describes areas where this work might be useful in engineering applications. The main difficulties associated with tracking on a construction site is the significant amount of visual clutter, the changing photometric visual content throughout the course of a day, and the presence of occluding and moving obstacles. The tracking of workers within the field of view of the camera will involve four tracking techniques, density mean-shift, Bayesian segmentation, active contours, and graph-cuts. Typical construction site video will be processed using the proposed algorithms and analyzed to determine the most appropriate tracking method for the video presented.     

Comparison of HDTV formats using objective video quality measures

In this paper we compare some of the objective quality measures with subjective, in several HDTV formats, to be able to grade the quality of the objective measures. Also, comparison of objective and subjective measures between progressive and interlaced video signal will be presented to determine which scanning emission format is better, even if it has different resolution format. Several objective quality measures will be tested, to examine the correlation with the subjective test, using various performance measures.     

Panoramic appearance-based recognition of video contents using matching graphs

This paper proposes a general scheme for recognizing the contents of a video using a set of panoramas recorded in a database. In essence, a panorama inherently records the appearances of an omni-directional scene from its central point to arbitrary viewing directions and, thus, can serve as a compact representation of an environment. In particular, this paper emphasizes the use of a sequence of successive frames in a video taken with a video camera, instead of a single frame, for visual recognition. The associated recognition task is formulated as a shortest-path searching problem, and a dynamic-programming technique is used to solve it. Experimental results show that our method can effectively recognize a video.     

基于圆形感兴趣区域多路视频实时拼接

针对视频拼接过程中面临的许多挑战,如实时性、有动态物体产生鬼影现象等,提出了一种基于圆形感兴趣区域（ROI）图像配准结合简化处理及图形处理器（GPU）加速的方法。首先,仅在ROI内提取特征点,提高了特征检测效率和匹配准确率。其次,为进一步降低时间开销,满足视频处理实时性需求,采用了两种策略：一方面,通过简化处理仅对首帧作图像配准,后续帧利用得到的单应性矩阵进行图像融合;另一方面,利用GPU多核实现并行化硬件加速。此外,当视场中有动态物体时,采用图形分割和多频带图像融合算法,有效地消除了鬼影。实验对两路640×480的视频进行拼接,该方法的处理速度可达27.8帧/秒。相对于基于加速鲁棒特征（SURF）算法的视频拼接方法,效率提高了26.27倍;相对于基于带方向的加速分段测试特征提取结合旋转的二进制鲁棒独立元素特征描述（ORB）算法的视频拼接方法,效率提高了11.57倍。实验结果表明,该方法可将多路视频实时地拼接为高质量的大场景视频。     

复杂背景下全景视频运动小目标检测算法

为解决复杂背景下全景视频中运动小目标检测精度低的问题,提出一种基于复杂背景下全景视频运动小目标检测算法.首先,为降低复杂背景信息的干扰,提高目标检测的精度,采用快速鲁棒性主成分分析(Fast RPCA)算法将全景视频图像的前景背景信息分离,并提取出前景信息作为有效的图像特征;然后,改进更快的基于区域的卷积神经网络(Faster R-CNN)中的区域生成网络(RPN)的候选框尺度大小,使之适应全景图像中的目标尺寸,再对前景特征图进行训练;最后,通过RPN网络和Fast R-CNN网络共享卷积层输出检测模型,实现对全景视频图像中小目标的精准检测.实验结果表明,所提出算法可以有效抑制复杂的背景信息对目标检测精度的影响,并对全景视频图像中的运动小目标具有较高的检测精度.     

基于特征点的视频流拼接帧选择优化算法

提出了一种具有公共视野的视频流拼接中视频帧的选择优化算法,算法计算候选帧与基准帧特征匹配对的特征向量差值的算术均值,选择均值最小的候选帧作为最佳拼接帧,与基准帧进行拼接.实验结果表明,基于该算法拼接的视频流全景视图效果较好,算法很好地解决了视频流拼接中最佳拼接帧的自动选择问题.     

基于SURF和动态ROI的实时视频拼接

为解决含有运动物体的视频拼接问题及解决不同景深下的目标拼接等此类问题,提出一种基于SURF和动态ROI方法的双目摄像机视频图像快速拼接方法.通过RANSAC方法与单应性矩阵平滑方法估计双目摄像头视频图像变换模型,融合成宽场景的视频图像.由于选取了动态ROI之后不进行重叠区域之外的SURF特征提取,提高了特征匹配的精准度,避免了ROI之外的误匹配特征点对,同时减少了程序的耗时.大量的实验结果表明,该系统能够自适应的对双目摄像头采集的图像进行视频拼接,获得清晰的宽场景视频.     

Coherent video generation for multiple hand-held cameras with dynamic foreground

Computational Visual Media - For many social events such as public performances, multiple hand-held cameras may capture the same event. This footage is often collected by amateur cinematographers...     

Vision-based estimation of the number of occupants using video cameras

Although occupancy information is critical to energy consumption of existing buildings, it still remains to be a major source of uncertainty. For reliable and accurate occupant modeling with minimal uncertainties, capturing precise occupant information on occupants is essential. This paper proposes a computer vision-based approach that utilizes deep learning architectures to estimate of the number of people in large, crowded spaces using multiple cameras. Various vision techniques (head detection, background elimination, head tracking) are implemented in three methods: (i) a method that instantaneously counts people in a scene, (ii) a method that incrementally counts people entering/exiting a room and (iii) a combination of the first two methods. These methods were applied in a classroom with heavy occlusions, and resulted in a high prediction capacity when compared to ground truth measurements. Future work in video-analytical approaches can address problems regarding lowering the computational cost of analysis, capturing occupancy data in complex room geometries and addressing concerns in privacy preservation.     

Applying image registration algorithm combined with CNN model to video image stitching

The Journal of Supercomputing - The purposes are to explore the video image stitching technique of Unmanned Aerial Vehicles (UAVs), expand the application of image registration algorithms and new...     

Transmission of correct gaze direction in video conferencing using screen-embedded cameras

In this paper, we propose a new video conferencing system that presents correct gaze directions of a remote user by switching among images obtained from multiple cameras embedded in a screen according to a local user’s position. Our proposed method reproduces a situation like that in which the remote user is in the same space as the local user. The position of the remote user to be displayed on the screen is determined so that the positional relationship between the users is reproduced. The system selects one of the embedded cameras whose viewing direction towards the remote user is the closest to the local user’s viewing direction to the remote user’s image on the screen. As a result of quantitative evaluation, we confirmed that, in comparison with the case using a single camera, the accuracy of gaze estimation was improved by switching among the cameras according to the position of the local user.

     

基于消失点检测的铁路环境视频自动拼接算法研究

提出了一种基于铁路环境视频的全景图自动拼接算法。根据铁路环境视频中的场景几何特性,检测场景中的线段及消失点,从而生成场景的空间布局关系;在此基础上,自动构建合适的采样区域,以达到无缝拼接的效果;最后实现全景图的自动拼接。实验结果表明,该方法实现了铁路环境视频的自动化拼接,且得到了更加令人满意的拼接效果。