固定多摄像头的视觉拼接技术

视频拼接技术是计算机图形学和计算机视觉的重要分支,它的发展基于静态图像的拼接技术,但由于视频信息的复杂性,视频拼接也有区别于图像拼接,针对实际运用中的实时拼接的需要,本文提出了一种基于控制帧的固定摄像头视频拼接方法。首先采集控制帧图像,对摄像头进行参数标定获得相机内参和光心坐标,再使用一种改进的畸变矫正方法去除摄像头畸变带来的成像失真。然后对控制帧图像进行SIFT特征提取并进行粗匹配,再用RANSAC的方法剔除误匹配点并拟合出图像变换单应阵。最后使用查表法将各摄像头的图像同步投影到大场景图片上,对重合区域进行光亮补偿和多带融合。最终实现速度可达25帧/秒的实时视频拼接。     

综采工作面视频监控系统优化设计

针对现有综采工作面视频监控系统占用网络带宽大、视频存储不完整、采煤机截割画面不突出、视频拼接画面参差不齐等问题,从系统硬件和软件、视频压缩和拼接算法等方面进行优化设计。硬件方面,引入硬盘录像机,以降低网络带宽占用率,解决视频传输卡顿问题;采用本地存储与远程存储相结合的方式,有效解决了视频存储丢失的问题。软件方面,以突出重点、局部放大为原则,采用实时视频与动画模拟结合的方式显示综采工作面视频画面与设备状态参数,解决了采煤机截割画面不突出的问题。算法方面,提出了基于深度学习的视频压缩方法,除压缩视频数据本身外,对帧间数据也进行压缩,有效降低了算法的码率;采用非线性损失真模型(NAM)矫正算法消除图像畸变,采用加速稳健特征(SURF)检测算法进行特征点检测,并通过双线性插值方法进行图像融合,从而实现全景视频拼接。探讨了综采工作面视频监控技术发展方向,包括摄像仪自清洁技术、智能识别技术、工作面全景视频拼接技术、5G与WiFi6融合通信技术、煤岩界面识别技术。     

一种基于CUDA的快速宽视频拼接的方法

在视频拼接过程中,需要进行大量的图像处理计算,高复杂度的拼接算法很难满足实时性要求。因此,为了实现多视频源快速拼接,不仅要求拼接算法具有较强的鲁棒性,同时还需要具有较低的复杂度。文中提出了一种新的快速宽视频拼接方法,此方法首先利用SURF算子提取图像特征点并进行匹配,接着使用多分辨率融合算法进行全景图融合,然后利用基于路径的方法结合光流实现低复杂度的视频插帧,同时整个宽视频拼接过程使用CUDA进行并行加速,最后用一个双目拼接系统对文中提出的方法进行可行性验证。     

基于SURF和动态ROI的实时视频拼接

为解决含有运动物体的视频拼接问题及解决不同景深下的目标拼接等此类问题,提出一种基于SURF和动态ROI方法的双目摄像机视频图像快速拼接方法.通过RANSAC方法与单应性矩阵平滑方法估计双目摄像头视频图像变换模型,融合成宽场景的视频图像.由于选取了动态ROI之后不进行重叠区域之外的SURF特征提取,提高了特征匹配的精准度,避免了ROI之外的误匹配特征点对,同时减少了程序的耗时.大量的实验结果表明,该系统能够自适应的对双目摄像头采集的图像进行视频拼接,获得清晰的宽场景视频.     

一种四路车载视频监控系统的设计与实现

针对目前汽车电子智能辅助系统的现状和市场需求,提出了一种基于多路视频解码复合技术和OSD显示技术的四路车载视频监控系统的设计方案;通过360度无死角四路鱼眼摄像头采集车身监控视频画面,实现多分割自由切换显示;阐述了视频监控系统的原理及软硬件模块的实现方法,重点讨论了视频分割、OSD显示模块的实现;最后,完成了整个系统的设计,并给出了实际运行结果。相比同类型的其他方案,成本下降了约三分之一。     

FPGA在视频拼接中的应用与实现

针对大场景视频拼接技术在汽车环视系统等领域的应用需求,并为了适应嵌入式系统快速发展的要求,提出了基于FPGA片上可编程系统(system on a programmable chip,SOPC)技术来实现多个摄像头视频数据的传输和拼接的大视场视频解决方案.系统的硬件平台的构建以Xilinx软核处理器Microblaze为核心,主要完成了视频数据的采集、存储、处理以及传输等工作,图像拼接部分采用频域相位相关算法,对待拼接图像进行配准,并通过融合算法得到具有360°全景视频信息的大视场图像.通过实验对系统进行测试,验证了系统的可行性.     

基于改进随机抽样一致算法的视频拼接算法

为了解决实时视频拼接中的鬼影和拼接缝问题,提出了一种基于改进随机抽样一致算法的实时视频拼接算法。首先,针对双目摄像头的重叠区域,采用加速鲁棒特征算法提取特征点,寻找特征匹配点对;然后利用改进的随机抽样一致算法去除误匹配点对,获得最优单应性矩阵;最后,对重叠区域像素进行动态融合处理。实验结果表明,采用本文算法可以有效地消除视频重叠区域的拼接缝和鬼影,同时可保证视频拼接系统的实时性。     

一种改进的摄像头视频实时拼接方法

针对普通摄像头水平视角较小的问题,通过同时采集具有一定重叠区域的摄像头视频帧图像,基于尺度不变特征变换（ScaleInvariantFeatureTransform,SIFT）特征点,用二分哈希搜索算法（DichotomyBasedHash,DBH）进行匹配,用随机采样一致（RandomSampleConsensus,RANSAC）算法消除误匹配,得到帧图像拼接映射关系。实验结果表明,该方法能有效地实现摄像头视频实时拼接,克服了既有方法在重叠区域小于20％时失效的不足,在重叠区域为10％左右时仍能取得有效的拼接。     

特定场景大视角视频拼接技术研究与实现

本文对视频拼接设计了一个简单的框架,主要是采用三路固定摄像进行实时拼接,在Uuntu16.04 系统安装 OpenCV开源库和CUDA8.0 软件包,并使用该系统搭建视频拼接框架系统。利用GPU编程技术,对视频拼接技术中最耗时 的特征匹配算法、细缝估计、多波段融合和扭曲映射算法进行加速。在计算资源足够的情况下,利用CUDA进行在线实时视 频流拼接。     

基于多协议融合的实时移动视频监控系统

提出一种支持多协议融合的实时视频监控系统的设计与实现方案.系统可通过 Hi3520嵌入式处理器平台和移动端摄像头进行视频采集,采用H.264/AVC压缩编码和参数优化,兼容RTP/RTSP和RTMP多种通信协议进行媒体流传输;设计的Android的移动客户端可以实现视频数据的实时解析、网络串流等.实验测试表明： PC和采集端显示的画面仅有1~1.5秒延迟,系统可以灵活实现不同应用场景下的高清、实时监控.     

Pose estimation from multiple cameras based on Sylvester’s equation

In this paper, we introduce a method to estimate the object’s pose from multiple cameras. We focus on direct estimation of the 3D object pose from 2D image sequences. Scale-Invariant Feature Transform (SIFT) is used to extract corresponding feature points from adjacent images in the video sequence. We first demonstrate that centralized pose estimation from the collection of corresponding feature points in the 2D images from all cameras can be obtained as a solution to a generalized Sylvester’s equation. We subsequently derive a distributed solution to pose estimation from multiple cameras and show that it is equivalent to the solution of the centralized pose estimation based on Sylvester’s equation. Specifically, we rely on collaboration among the multiple cameras to provide an iterative refinement of the independent solution to pose estimation obtained for each camera based on Sylvester’s equation. The proposed approach to pose estimation from multiple cameras relies on all of the information available from all cameras to obtain an estimate at each camera even when the image features are not visible to some of the cameras. The resulting pose estimation technique is therefore robust to occlusion and sensor errors from specific camera views. Moreover, the proposed approach does not require matching feature points among images from different camera views nor does it demand reconstruction of 3D points. Furthermore, the computational complexity of the proposed solution grows linearly with the number of cameras. Finally, computer simulation experiments demonstrate the accuracy and speed of our approach to pose estimation from multiple cameras.     

Feature fusion for basic behavior unit segmentation from video sequences

It has become increasingly popular to study animal behaviors with the assistance of video recordings. An automated video processing and behavior analysis system is desired to replace the traditional manual annotation. We propose a framework for automatic video based behavior analysis systems, which consists of four major modules: behavior modeling, feature extraction from video sequences, basic behavior unit (BBU) discovery and complex behavior recognition. BBU discovery is performed based on features extracted from video sequences, hence the fusion of multiple dimensional features is very important. In this paper, we explore the application of feature fusion techniques to BBU discovery with one and multiple cameras. We applied the vector fusion (SBP) method, a multi-variate vector visualization technique, in fusing the features obtained from a single camera. This technique reduces the multiple dimensional data into two dimensional (SBP) space, and the spatial and temporal analysis in SBP space can help discover the underlying data groups. Then we present a simple feature fusion technique for BBU discovery from multiple cameras with the affinity graph method. Finally, we present encouraging results on a physical system and a synthetic mouse-in-a-cage scenario from one, two, and three cameras. The feature fusion methods in this paper are simple yet effective.     

基于H．264的远程监控系统的设计

为摒弃传统监控终端各自为战的弊端,在简要介绍了H.264技术的基础上,描述了一种新型远程监控系统的设计与实现。采用软件协调多个监控终端动作的相互配合,大大减少了单位空间内监控终端的布局数量。并且该系统具有对运动目标进行实时、可靠智能跟踪的特点,通过用户的设置,系统可以达到无人值守的全自动、不间断监控的目的。同时,设计过程中涉及到的方法也可以对安防和自动跟踪等方面起到借鉴作用。     

Tri-view morphing

This paper presents an efficient image-based approach to navigate a scene based on only three wide-baseline uncalibrated images without the explicit use of a 3D model. After automatically recovering corresponding points between each pair of images, an accurate trifocal plane is extracted from the trifocal tensor of these three images. Next, based on a small number of feature marks using a friendly GUI, the correct dense disparity maps are obtained by using our trinocular-stereo algorithm. Employing the barycentric warping scheme with the computed disparity, we can generate an arbitrary novel view within a triangle spanned by three camera centers. Furthermore, after self-calibration of the cameras, 3D objects can be correctly augmented into the virtual environment synthesized by the tri-view morphing algorithm. Three applications of the tri-view morphing algorithm are demonstrated. The first one is 4D video synthesis, which can be used to fill in the gap between a few sparsely located video cameras to synthetically generate a video from a virtual moving camera. This synthetic camera can be used to view the dynamic scene from a novel view instead of the original static camera views. The second application is multiple view morphing, where we can seamlessly fly through the scene over a 2D space constructed by more than three cameras. The last one is dynamic scene synthesis using three still images, where several rigid objects may move in any orientation or direction. After segmenting three reference frames into several layers, the novel views in the dynamic scene can be generated by applying our algorithm. Finally, the experiments are presented to illustrate that a series of photo-realistic virtual views can be generated to fly through a virtual environment covered by several static cameras.     

Human segmentation by geometrically fusing visible-light and thermal imageries

From depth sensors to thermal cameras, the increased availability of camera sensors beyond the visible spectrum has created many exciting applications. Most of these applications require combining information from these hyperspectral cameras with a regular RGB camera. Information fusion from multiple heterogeneous cameras can be a very complex problem. They can be fused at different levels from pixel to voxel or even semantic objects, with large variations in accuracy, communication, and computation costs. In this paper, we propose a system for robust segmentation of human figures in video sequences by fusing visible-light and thermal imageries. Our system focuses on the geometric transformation between visual blobs corresponding to human figures observed at both cameras. This approach provides the most reliable fusion at the expense of high computation and communication costs. To reduce the computational complexity of the geometric fusion, an efficient calibration procedure is first applied to rectify the two camera views without the complex procedure of estimating the intrinsic parameters of the cameras. To geometrically register different blobs at the pixel level, a blob-to-blob homography in the rectified domain is then computed in real-time by estimating the disparity for each blob-pair. Precise segmentation is finally achieved using a two-tier tracking algorithm and a unified background model. Our experimental results show that our proposed system provides significant improvements over existing schemes under various conditions.     

一种无标记的身体与面部运动同步捕获方法

提供了一个无标记点的身体与面部运动同步捕获的方法.利用经过时间同步和空间标定的长焦彩色相机和Kinect相机来进行同步捕获.利用在环境中加入闪光来进行时间同步,使用张氏标定法进行空间标定,从而组成一组时间同步且空间对齐的混合相机（hybrid camera）.然后利用Kinect fusion扫描用户的人体模型并嵌入骨骼.最后利用时间和空间都对齐好的两个相机来进行同步采集.首先从深度图像中得到人脸的平移参考值,然后在平移参考值的帮助下根据彩色图像的2D特征点重建人脸.随后,把彩色图像中得到的头部姿态传递给身体捕获结果.结果对比实验和用户调研实验均表明所提出的运动捕获的结果要好于单个的运动捕获结果.     

被动自动跟踪监控系统的实现

介绍了一种数字硬盘录像(DVR)监控系统.它不同于以往的多个摄像机单独工作方式,而是采用普通摄像机与高速球摄像机协调工作的方式来进行全数字化监控和智能跟踪.可以有效地减少相同空间内摄像机的布局数量,还可以达到完全监控不遗漏的效果.系统的设计与实现具有全数字化、智能化,并能实现对运动目标进行实时、可靠智能跟踪的特点.系统通过用户的设置完全可以实现无人值守的全自动监控的目的.     

Surveillance camera scheduling: a virtual vision approach

We present a surveillance system, comprising wide field-of-view (FOV) passive cameras and pan/tilt/zoom (PTZ) active cameras, which automatically captures high-resolution videos of pedestrians as they move through a designated area. A wide-FOV static camera can track multiple pedestrians, while any PTZ active camera can capture high-quality videos of one pedestrian at a time. We formulate the multi-camera control strategy as an online scheduling problem and propose a solution that combines the information gathered by the wide-FOV cameras with weighted round-robin scheduling to guide the available PTZ cameras, such that each pedestrian is observed by at least one PTZ camera while in the designated area. A centerpiece of our work is the development and testing of experimental surveillance systems within a visually and behaviorally realistic virtual environment simulator. The simulator is valuable as our research would be more or less infeasible in the real world given the impediments to deploying and experimenting with appropriately complex camera sensor networks in large public spaces. In particular, we demonstrate our surveillance system in a virtual train station environment populated by autonomous, lifelike virtual pedestrians, wherein easily reconfigurable virtual cameras generate synthetic video feeds. The video streams emulate those generated by real surveillance cameras monitoring richly populated public spaces.A preliminary version of this paper appeared as [1].