基于逆向投影的全景泊车系统设计与实现

全景泊车系统利用安装在车身前后左右的4个广角摄像头实时采集周围的图像,并传送给图像处理设备进行校正、俯视变换和图像合成,最后在显示屏上形成一幅360°的全景俯视图。系统以处理器HI3520D和视频解码器NVP1918C为硬件核心,以鱼眼校正、投影变换和逆向投影算法为软件核心,最终实现实时显示无缝和清晰画面的360°全景泊车影像系统。     

环视鱼眼图像处理深度学习研究进展

视觉环境感知在自动驾驶汽车发展中起着关键作用,在智能后视镜、倒车雷达、360°全景、行车记录仪、碰撞预警、红绿灯识别、车道偏移、并线辅助和自动泊车等领域也有着广泛运用。传统的环境信息获取方式是窄角针孔摄像头,视野有限有盲区,解决这个问题的方法是环境信息感知使用鱼眼镜头,广角视图能够提供整个180°的半球视图,理论上仅需两个摄像头即可覆盖360°,为视觉感知提供更多信息。处理环视图像目前主要有两种途径：一是对图像先纠正,去失真,缺点是图像去失真会损害图像质量,并导致信息丢失;二是直接对形变的鱼眼图像进行建模,但目前还没有效果比较好的建模方法。此外,环视鱼眼图像数据集的缺乏也是制约相关研究的一大难题。针对上述挑战,本文总结了环视鱼眼图像的相关研究,包括环视鱼眼图像的校正处理、环视鱼眼图像中的目标检测、环视鱼眼图像中的语义分割、伪环视鱼眼图像数据集生成方法和其他鱼眼图像建模方法等,结合自动驾驶汽车的环境感知应用背景,分析了这些模型的效率和这些处理方法的优劣,并对目前公开的环视鱼眼图像通用数据集进行了详细介绍,对环视鱼眼图像中待解决的问题与未来研究方向做出预测和展望。     

基于DaVinci~(TM)的360°全景泊车影像系统的设计与实现

全景泊车影像系统通过安装在车身前后左右的4个超广角摄像头实时采集车辆四周的影像,经过图像处理器矫正变换和全景拼接后,形成一幅车辆四周的360°全景俯视图,并实时传送到中控台的显示设备上,驾驶员坐在车中即可看到车辆所处的位置以及车辆周围的障碍物,有效减少刮蹭、碰撞、陷落等事故的发生。系统采用TI公司的低成本高性能达芬奇DSP处理器TMS320DM6437和四路视频解码器TVP5158构成系统的硬件核心,以鱼眼矫正和鸟瞰变换算法以及全景拼接技术构成系统的软件核心,实现了具有高分辨率的画质和流畅处理速度的360°全景泊车影像系统。     

鱼眼图像校正和配准算法研究

提出一种对鱼眼图像进行校正和配准的算法。首先把鱼眼图像从相等距离投影模型映射到参数化球面投影模型,使得合成球面全景图成为一个平移求解问题。然后使用非线性优化算法进行图像的配准。在合成场景的球面全景图以后,通过重投影球面全景图到视平面来完成虚拟场景的实时漫游。     

基于图像配准的光学触控系统触点校正

针对目前光学多点触控系统存在的摄像头取景图像特性对触点校正精准度的限制,研究高性能的触点坐标校正形式,提出了一种通用的触点校准算法,使其扩展了对摄像头及安装环境的适应性.通过引进图像配准方法,利用特征点提取实现坐标系的高度统一;采用动态查找表法,提高图像配准效率.理论研究和实验结果验证了该方法的有效性.     

基于全景视觉的移动机器人地图创建与定位

提出一种高效的基于全景视觉的室内移动机器人地图构建和定位方法. 该方法充分利用全景视觉系统视野广阔、获取环境信息完整的特点, 根据全景图像生成环境描述子; 利用上述环境描述子描述环境, 创建拓扑地图, 将地图表示为环境描述子的集合. 在此基础上, 提出一种基于贝叶斯理论的定位方法, 根据当前全景摄像头的观测值, 利用已生成的地图完成状态跟踪, 全局定位和“绑架”定位. 最后通过实验验证了该方法的有效性, 并给出了计算成本分析.     

旋转不变的X线图像相关度拼接

针对现有X线图像拼接方法中分别存在的需要固定标志物、鲁棒性差及全景图像存在伪影等问题,提出一种旋转不变的图像自动拼接方法.基于特征点配准,结合改进的、旋转不变的相关度法进行特征匹配,对图像重合度要求较低.在图像融合中则采用2维动态权值和平衡曝光度等策略.经实验验证,本文配准算法在保证结果准确性的同时提高了鲁棒性;本文融合算法可有效地平衡曝光度差异,并避免拼接伪影,极大地提高了全景图像质量.     

FPGA在视频拼接中的应用与实现

针对大场景视频拼接技术在汽车环视系统等领域的应用需求,并为了适应嵌入式系统快速发展的要求,提出了基于FPGA片上可编程系统(system on a programmable chip,SOPC)技术来实现多个摄像头视频数据的传输和拼接的大视场视频解决方案.系统的硬件平台的构建以Xilinx软核处理器Microblaze为核心,主要完成了视频数据的采集、存储、处理以及传输等工作,图像拼接部分采用频域相位相关算法,对待拼接图像进行配准,并通过融合算法得到具有360°全景视频信息的大视场图像.通过实验对系统进行测试,验证了系统的可行性.     

基于物联网的配网一二次融合跨区协同全景监测技术

针对配网一二次融合跨区协同全景监测技术过度依赖网络环境,当网络发生堵塞时会造成监测存在死角的问题,提出基于物联网的配网一二次融合跨区协同全景监测技术。设计监测智能物理架构,支持多网多工下监测节点之间的横向交互和纵向交互,在此基础上,计算监控摄像头的旋转角度,采集监测环境及监测目标图像,通过全景图像拼接配准,实现对目标的全景监测。实验结果表明：设计的基于物联网的全景监测技术在通信组网内,全景监测角度精准,没有出现监测死角,实用性较好。     

一种柱面全景图像自动拼接算法

提出了一种基于特征块匹配的柱面全景图像拼接算法.首先将360度环绕拍摄的序列图像投影到柱面坐标系下;然后以一幅图像为基准图像,选择基准图像中边缘信息丰富的块作为基准块,利用特征块匹配法在待配准图像中找出与基准块匹配的配准块,进而实现两幅图像的配准;再根据配准结果计算出图像间的变换参数;最后采用平滑因子对两幅图像的重叠区域进行图像无缝拼接.实验证明,算法可以快速自动地生成柱面全景图像,具有良好的鲁棒性.     

A flexible vehicle surround view camera system by central-around coordinate mapping model

The surround view camera system is an emerging driving assistant technology that can assist drivers in parking by providing top-down view of surrounding situations. Such a system usually consists of four wide-angle or fish-eye cameras that mounted around the vehicle, and a bird-eye view is synthesized from images of these cameras. Commonly there are two fundamental problems for the surround view synthesis, geometric alignment and image synthesis. Geometric alignment performs fish-eye calibration and computes the image perspective transformation between the bird-eye view and images from the surrounding cameras. Image synthesis technique dedicates to seamless stitch between adjacent views and color balancing. In this paper, we propose a flexible central-around coordinate mapping (CACM) model for vehicle surround view synthesis. The CACM model calculates perspective transformation between a top-view central camera coordinate and the around camera coordinates by a marker point based method. With the transformation matrices, we could generate the pixel point mapping relationship between the bird-eye view and images of the surrounding cameras. After geometric alignment, an image fusion method based on distance weighting is adopted for seamless stitch, and an effective overlapping region brightness optimization method is proposed for color balancing. Both the seamless stitch and color balancing can be easily operated by using two types of weight coefficient under the framework of the CACM model. Experimental results show that the proposed approaches could provide a high-performance surround view camera system.

     

Evolutionary-based 3D reconstruction using an uncalibrated stereovision system: application of building a panoramic object view

In this paper, we propose an original evolutionary-based method for 3D panoramic reconstruction from an uncalibrated stereovision system (USS). The USS is composed of five cameras located on an arc of a circle around the object to be analyzed. The main originality of this work concerns the process of the calculation of the 3D information. Actually, with our method, 3D coordinates are directly obtained without any prior estimation of the fundamental matrix. The method operates in two steps. Firstly, points of interest are detected in pairs of images acquired by two consecutive cameras of the USS are matched. And secondly, using evolutionary algorithms, we jointly compute the transformed matrix between the two images and the respective depth of the points of interest. The accuracy of the proposed method is validated through a comparison with the depth values obtained using a traditional method. In order to perform 3D panoramic object reconstruction, the process is repeated for all the pairs of consecutive cameras. The 3D points thus obtained throughout the successive steps of the process which correspond to the different points of interest, are combined in order to obtain a set of 3D points all around the analyzed object.     

Measuring and modelling sewer pipes from video

This article presents a system for the automatic measurement and modelling of sewer pipes. The system recovers the interior shape of a sewer pipe from a video sequence which is acquired by a fish-eye lens camera moving inside the pipe. The approach is based on tracking interest points across successive video frames and posing the general structure-from-motion problem. It is shown that the tracked points can be reliably reconstructed despite the forward motion of the camera. This is achieved by utilizing a fish-eye lens with a wide field of view. The standard techniques for robust estimation of the two- and three-view geometry are modified so that they can be used for calibrated fish-eye lens cameras with a field of view less than 180°. The tubular arrangement of the reconstructed points allows pipe shape estimation by surface fitting. Hence, a method for modelling such surfaces with a locally cylindrical model is proposed. The system is demonstrated with a real sewer video and an error analysis for the recovered structure is presented.     

Omni-directional stereo

Omnidirectional views of an indoor environment at different locations are integrated into a global map. A single camera swiveling about the vertical axis takes consecutive images and arranges them into a panoramic representation, which provides rich information around the observation point: a precise omnidirectional view of the environment and coarse ranges to objects in it. Using the coarse map, the system autonomously plans consecutive observations at the intersections of lines connecting object points, where the directions of the imaging are estimated easily and precisely. From two panoramic views at the two planned locations, a modified binocular stereo method yields a more precise, but with direction-dependent uncertainties, local map. New observation points are selected to decrease the uncertainty, and another local map is yielded, which is then integrated into a more reliable global representation of the world with the adjacent local maps     

一种基于特征匹配的目标入侵检测方法

视频监控中常用云台摄像机监控视场较大的区域.对于云台摄像机跟随拍摄的情况,提出了一种基于特征匹配的目标入侵检测方法.通过提取的尺度不变特征变换(SIFT)特征点对,将当前图像和全景图像进行匹配,从而得到当前图像和全景图像投影关系,再将当前图像的坐标系变换到全景图像下,最后运用差分法,找到入侵目标.实验结果表明,即使当前图像与全景图像存在尺度、缩放、形变等差异,通过本方法也可正确地检测出入侵目标.     

Real-time panoramic video streaming system with overlaid interface concept for social media

In this paper, we propose a real-time panoramic video streaming system with overlaid interface concept for social media. The proposed system provides real-time panorama images for smart displays such as smart TVs, smart phones, and tablet PCs. Panorama images are collected at sporting events via panorama cameras. Contents thus collected are sent to servers and then provided to smart devices via live sports video streams. Users select a panorama camera and choose their viewing angle and zooming factor for the selected panorama camera. The proposed system provides immersive and realistic views of live sporting events on users’ displays. Furthermore, an overlaid panoramic interface concept is proposed for immersive live baseball watching combined with tightly integrated social media experience.     

A hemispherical imaging camera

Most of the existing panoramic cameras have cylindrical field of view (FOV) which has a 360° width in azimuth, but a limited height in elevation. However, a hemispherical (or near spherical) FOV is highly desirable in many applications such as robot navigation and surveillance. This paper presents a planar-mirror-based panoramic camera (HEM) which is capable of acquiring nearly hemispherical FOV panoramic images at high and substantially uniform resolution, from a single viewpoint, at video rates. The camera consists of multiple imaging sensors and a hexagonal prism made of planar mirror faces. The sensors are positioned in such a way that they image different parts of the scene from the same virtual viewpoint, either directly or after reflections off the prism. A panoramic image is constructed by concatenating the images taken by different sensors. The proposed system is designed such that the resolution across entire FOV has the same level of uniformity as delivered by a conventional, non-panoramic camera over a small FOV, and the sensor area utilization is maximized. We also present a novel camera placement technique that helps co-locate the optic points of all seven sensors at a single viewpoint, and a method to compensate for artifacts near mirror boundaries caused by finite size of lens aperture. An implementation of the proposed system as well as resulting hemispherical panoramic images are included.     

Structure from motion with wide circular field of view cameras

This paper presents a method for fully automatic and robust estimation of two-view geometry, autocalibration, and 3D metric reconstruction from point correspondences in images taken by cameras with wide circular field of view. We focus on cameras which have more than 180/spl deg/ field of view and for which the standard perspective camera model is not sufficient, e.g., the cameras equipped with circular fish-eye lenses Nikon FC-E8 (183/spl deg/), Sigma 8 mm-f4-EX (180/spl deg/), or with curved conical mirrors. We assume a circular field of view and axially symmetric image projection to autocalibrate the cameras. Many wide field of view cameras can still be modeled by the central projection followed by a nonlinear image mapping. Examples are the above-mentioned fish-eye lenses and properly assembled catadioptric cameras with conical mirrors. We show that epipolar geometry of these cameras can be estimated from a small number of correspondences by solving a polynomial eigenvalue problem. This allows the use of efficient RANSAC robust estimation to find the image projection model, the epipolar geometry, and the selection of true point correspondences from tentative correspondences contaminated by mismatches. Real catadioptric cameras are often slightly noncentral. We show that the proposed autocalibration with approximate central models is usually good enough to get correct point correspondences which can be used with accurate noncentral models in a bundle adjustment to obtain accurate 3D scene reconstruction. Noncentral camera models are dealt with and results are shown for catadioptric cameras with parabolic and spherical mirrors.     

基于MSCR与CS-LBP的鱼眼图像特征区域匹配方法

相比普通镜头,鱼眼镜头拥有更大的视场角,甚至可以直接获取半球域的图像信息,在立体视觉领域,应用鱼眼镜头来采集全景图像可减少镜头及图像采集模块数目,简化系统、提高运算速度、降低成本。但同时鱼眼镜头图像也存在一定程度的畸变,越靠近边缘畸变越严重。因此,在光轴正交或是角度更大的立体视觉系统中,进行相关图像的特征点匹配存在困难,直接影响立体视觉系统的应用效果。然而采用一种具有仿射不变性的图像匹配算法即可解决这个问题,首先提取原始图像的MSCR特征区域,其次引进CS-LBP算子对各个MSCR区域进行特征描述,应用特征权重的卡方距离比较法进行唯一匹配,最后进行椭圆拟合及连线标记使得匹配结果可视化。且通过实验验证了此方法的稳定一致性,可应用于大旋转角度的鱼眼图像的特征匹配。