Registration and interactive planar segmentation for stereo images of polyhedral scenes

We introduce a two-step iterative segmentation and registration method to find coplanar surfaces among stereo images of a polyhedral environment. The novelties of this paper are: (i) to propose a user-defined initialization easing the image matching and segmentation, (ii) to incorporate color appearance and planar projection information into a Bayesian segmentation scheme, and (iii) to add consistency to the projective transformations related to the polyhedral structure of the scenes. The method utilizes an assisted Bayesian color segmentation scheme. The initial user-assisted segmentation is used to define search regions for planar homography image registration. The two reliable methods cooperate to obtain probabilities for coplanar regions with similar color information that are used to get a new segmentation by means of quadratic Markov measure fields (QMMF). We search for the best regions by iterating both steps: registration and segmentation.     

注意力机制的曲面沉浸式投影系统补偿

目的 沉浸式投影系统已广泛运用于虚拟现实系统之中,然而沉浸式投影系统中的互反射现象严重影响着虚拟现实系统的落地使用。沉浸式投影系统的互反射是指由于投影机光线和屏幕反射光线相互叠加造成的亮度冗余现象,严重影响了投影系统的成像质量和人眼的视觉感受。为此,本文提出一种新的基于互反射通道(inter-reflection channel,IRC)先验和注意力机制的神经网络。方法 IRC先验基于这样一个事实,即大多数受到互反射影响的投影图像都包含一些亮度较高的区域。高亮度区域往往受互反射影响更为严重,而低亮度区域受互反射影响程度较低。根据这一规律,采用IRC先验作为注意力图的监督样本,获取补偿图像的亮度区域信息。同时,为了对投影图像不同区域按影响程度进行差异化补偿,提出一种新的由两个相同子网络构成的补偿网络结构Pair-Net。结果 实验对比了4种现有方法,Pair-Net在ROI(region of interesting)指标分析上取得了明显优势,在人眼感受上有显著的效果提升。结论 本文提出的基于注意力机制的网络模型能够针对不同区域进行差异化补偿,很大程度上消除了互反射影响,提升了沉浸式投影系统的成像质量。     

A system for real-time panorama generation and display in tele-immersive applications

Wide field-of-view (FOV) is necessary for many industrial applications, such as air traffic control, large vehicle driving and navigation. Unfortunately, the supporting structure/frame in most systems usually blocks part of the view, results in "blind spot" and raises the risk to the pilot. In this paper, we introduce a video-based tele-immersive system, called the immersive cockpit. It captures live videos from the working site and recreates an immersive environment at the remote site where the pilot situates. It immerses the pilot at the remote site with a panoramic view of the environment, and hence improves interactivity and safety. The design goals of our system are real-time, live, low-cost, and scalable. We stitch multiple video streams captured from ordinary charged couple device cameras to generate a panoramic video. To avoid being blocked by the supporting frame, we allow a flexible placement of cameras. This approach trades the accuracy of the generated panoramic image for a larger FOV. To reduce the computation, parameters for stitching are determined once during the system initialization. The panoramic video is presented on an immersive display which covers the FOV of the viewer. We discuss how to correctly present the panoramic video on this nonplanar immersive display screen by sweet spot relocation. We also present the result and the performance evaluation of the system.     

基于图像分割的无人机遥感影像目标提取技术

为全面增强遥感影像上地物波谱的反射特性能力,实现对无人机目标的无误提取,提出基于图像分割的无人机遥感影像目标提取技术;在类哈尔滤波器结构的支持下,按照区域环境中地物目标的颜色特征,完成低层影像特征的量化处理,实现基于图像分割技术的无人机遥感深度图获取;定义与无人机遥感影像相关的基本名词,通过原始特征选择的方式,判定地物波谱的平均反射特性水平,得到准确的特征元素相关性测度数值,完成无人机遥感影像的目标融合处理;在此基础上,分割多分辨率条件下的影像目标,在定义图像纹理与尺度条件的同时,得到最终的地物提取结果,实现基于图像分割无人机遥感影像目标提取技术的应用;对比实验结果表明,在初始采集相位条件及亚像素条件下,目标地物的波谱宽度均超过7.0μm,遥感影像的反射特性能力大幅提升,满足对无人机目标无误提取的实用需求。     

Stereo vision system for moving object detecting and locating based on CMOS image sensor and DSP chip

Currently, most of the stereo vision systems are constructed on PC-based or multi-CPU combination structures with two CCD cameras. It is difficult to be applied in movable plants for stand-alone requirement. Due to electronic technology development, the complementary metal-oxide semiconductor (CMOS) image sensor has been widely used in a lot of electronic commercial products and the digital signal processor (DSP) operation speed and capacity are good enough for stereo vision system requirement. Here, a new stereo vision platform is designed with TMS320C6416 DSK board integrated with two CMOS color image sensors for detecting and locating moving objects. The data communication interface, system monitoring timing flow, and image pre-processing software programs are developed, too. This system can be used to detect and track any moving object without object color and shape limitations of previous study. Experimental results are used to evaluate this system’s dynamic performance. This low cost stereo vision system can be employed in movable platform for stand-alone application, i.e., mobile robot.     

On multiuser perspectives in passive stereographic virtual environments

The use of stereographic systems is spreading out in modern society, from the revolution of cinematography to its adoption in high‐tech products such as portable gaming devices or photo cameras. However, the fruition of immersive stereographic systems by more than one person at a time is still a research issue. In more detail, the class of passive stereo systems presents technological limitations of displaying correct multiuser perspectives. In fact, the stereo image projected onto the screen is usually rendered according to a unique point of view (PoV). Nevertheless, in multiuser systems, the selection of an appropriate PoV can minimize both optical discomfort and perspective distortion. This paper aims to evaluate which among existing PoV calculation methods provides the best performances in terms of projection realism, optical comfort and overall system usability in multiuser passive stereo systems. The performances have been evaluated in three different “distance” scenarios to take into account also the effects of binocular disparity in the PoV calculation. To accomplish this objective, we administered a questionnaire to nine couple subjects, evaluating each of the investigated PoV calculation methods for each of the three distance scenarios. Copyright © 2013 John Wiley & Sons, Ltd.     

A setup for panoramic stereo imaging

In this paper, based on the proposed basic implementation of circular projection, a multi-camera setup which can be used for panoramic stereo imaging is presented. Firstly, we get multiple stereo pairs instantly by fixing multiple cameras on the proposed setup, which is an approximate implementation of circular projection. We then prove that the influence on the resultant stereo pairs generated by this approximate setup is slight based on similar triangles and structural similarity (SSIM). With these stereo pairs, the automatic panoramic image stitching algorithm is applied to generate panoramas for both eyes. In addition, with reasonable distance between cameras and the scene, physical dimensions of this setup are given. Finally, experimental results demonstrate that our scheme can be used to generate cylindrical stereo panoramic image with proper scene depth, which can provide viewers with distinguishing stereoscopic experience. The device can be installed on the mobile equipment and captures panoramic stereo image in a moment without stopping. In the future, the proposed setup can be used in the field of network navigation, video monitor and virtual reality.     

An algorithm for the visualization of stereo images simultaneously captured with different exposures

The visualization of stereo images obtained from two eyepiece cameras of a stereo microscope with different exposures is studied. This problem is solved to improve the quality of resulting images in the case when one image is not sufficient for capturing an object with the desired color reproduction accuracy and high level of detail both in dark and light regions. An approach to solving this problem in which differently exposed images are split between two views is considered. This approach allows us to significantly reduce the capturing time and to enhance the quality of capturing moving objects. The algorithm described in [1] is used as the basic algorithm; the main steps of this algorithm are the stereo matching of two input images and the construction of high dynamic range images. Modifications of the basic algorithm that use different stereo matching techniques are proposed. The application of the algorithm described in [2] for the visualization of stereo images without constructing high dynamic range images is discussed. A database of images captured with different exposures by a stereo microscope is created. The quality of algorithms applied to the images from this database is evaluated in the HDR-VDP-2.2 metric [3].     

立体图像对的生成

获取同一场景的立体图像对是实现双目立体成像的一个关键问题。提出了一种在三维场景已经建好的情况下生成立体图像对的方法。该方法根据双目立体视觉的原理，利用3DS MAX中的摄像机对象对场景中的物体进行坐标变换和透视投影变换，分别生成左眼视图和右眼视图。实验结果表明，两个目标摄像机与三维模型的位置关系以及基线长度是影响立体效果的重要因素，改变目标摄像机与三维模型的位置，可以分别生成正视差、负视差的立体图像对，当AB与CO的比例参数为0.05时，生成的立体图像对的立体效果较佳。     

Aerial full spherical HDR imaging and display

This paper describes a framework for aerial imaging of high dynamic range (HDR) scenes for use in virtual reality applications, such as immersive panorama applications and photorealistic superimposition of virtual objects using image-based lighting. We propose a complete and practical system to acquire full spherical HDR images from the sky, using two omnidirectional cameras mounted above and below an unmanned aircraft. The HDR images are generated by combining multiple omnidirectional images captured with different exposures controlled automatically. Our system consists of methods for image completion, alignment, and color correction, as well as a novel approach for automatic exposure control, which selects optimal exposure so as to avoid banding artifacts. Experimental results indicated that our system generated better spherical images compared to an ordinary spherical image completion system in terms of naturalness and accuracy. In addition to proposing an imaging method, we have carried out an experiment about display methods for aerial HDR immersive panoramas utilizing spherical images acquired by the proposed system. The experiment demonstrated HDR imaging is beneficial to immersive panorama using an HMD, in addition to ordinary uses of HDR images.     

基于立体视觉运动速度检测系统的研究及实现

介绍了一种基于立体视觉的运动速度检测系统的结构框架。系统在采集卡自带的软件开发包的基础上进行二次软件开发,加入了摄像机标定和图像处理分析部分。整个检测系统主要包括图像采集和运动检测两大环节;涵盖摄像机标定、图像获取及预处理、目标分割、3D定位、速度求取5个功能模块,并具有友好的操作界面,方便用户使用。文章重点针对目标分割和定位中所采用的技术方法展开了讨论。该系统可以具体应用在车辆和游乐设施等的运动速度检测上,在设备的安全运行以及交通监管方面有重要意义。     

Combining color and depth for enhanced image segmentation and retargeting

As depth cameras become more popular, pixel depth information becomes easier to obtain. This information can clearly enhance many image processing applications. However, combining depth and color information is not straightforward as these two signals can have different noise characteristics, differences in resolution, and their boundaries do not generally agree. We present a technique that combines depth and color image information from real devices in synergy. In particular, we focus on combining them to improve image segmentation. We use color information to fill and clean depth and use depth to enhance color image segmentation. We demonstrate the utility of the combined segmentation for extracting layers and present a novel image retargeting algorithm for layered images.     

五自由度立体视觉平台--目标物体的识别与定位

从立体视觉与机器人控制集成的角度出发,建立了一个主动立体视觉跟踪和定位系统,用于柔性装配线中装配零件的运动跟踪和装配工位的精确定位。讨论了使用立体视觉进行快速目标识别和定位的方法,该方法将颜色分割与基于形状特征相结合,提出采用基于局部图像的HSV阈值分割和形状识别相结合的图像处理方法,该方法经过HSV阈值和形状判据,准确地识别出物体,得到物体的边界和质心。能快速准确地识别出物体并进行定位,满足柔性生产线小规模定制产品装配的要求。最后给出的实验结果验证了该方法的有效性。     

一种基于Snakes模型的中医舌象分割改进算法

舌象分割是中医舌诊客观化研究中的一个重要步骤,其结果好坏直接影响到舌象自动分析结果.传统的snakes舌体分割算法的颜色依赖性太大,当舌象的采集环境发生变化时,该算法对所采集的舌象分割结果往往不能令人满意.本文提出的基于snakes模型的改进算法将舌象转换到多种颜色空间,两次应用Snakes动态轮廓模型,最大化地提取了...     

面向复杂背景中彩色立体图像的障碍物检测

目标识别技术作为图像理解领域的核心研究内容之一,一直以来是文献中讨论的热门话题.随着计算机科学的发展以及彩色成像设备的普及,越来越多的彩色立体图像在实际生产、生活中得到应用,面向彩色立体图像的目标识别技术逐渐成为吸引学者注意力的新方向.本文基于立体视觉信息,对安装在行进车辆上的两个摄像机在一段时间内获取的立体视频信息进行分析,提取车辆前方可能对行车安全造成威胁的障碍物.在V视差理论的基础上引入两个改进算法首先对原始立体图像进行数据净化和前景/背景分割,使后续算法不受复杂、多变背景的干扰,同时减少数据处理的冗余.然后使用绝对值差(SAD)[9]法直接计算彩色立体图像的浓密视差图,而无需彩色到灰度图像的转换.实验证明,本文提出的改进算法针对实际立体图像可以准确、快速地提取障碍物信息.     

SVM-based image partitioning for vision recognition of AGV guide paths under complex illumination conditions

Applying computer vision to mobile robot navigation has been studied more than two decades. For the commercial off-the-shelf (COTS) automated guided vehicles (AGV) products, the cameras are still not widely used for the acquisition of guidance information from the environment. One of the most challenging problems for a vision guidance system of AGVs lies in the complex illumination conditions. Compared to the applications of computer vision where on-machine cameras are fixed in place, it is difficult to structure the illumination circumstance for an AGV that needs to travel through a large work space. In order to distinguish the original color features of path images from their illumination artifacts, an illumination-adaptive image partitioning approach is proposed based on the support vector machine (SVM) classifier with the slack constraint and the kernel function, which is utilized to divide a path image to low-, normal-, and high-illumination regions automatically. Moreover, an intelligent path recognition method is developed to carry out guide color enhancement and adaptive threshold segmentation in different regions. Experimental results show that the SVM-based classifier has the satisfactory generalization ability, and the illumination-adaptive path recognition approach has the high adaptability to the complex illumination conditions, when recognizing the path pixels in the field of view with both high-reflective and dark-shadow regions. The 98% average rate of path recognition will significantly facilitate the subsequent operation of path fitting for vision guidance of AGVs.     

双目视觉测量传感器研究

研究了按照激光三角测量原理由 1个线激光器及 2个CCD摄像机组成的双目视觉测量传感器。探讨了 2个CCD摄像机的摆放位置及其对测量精度的影响,确定了其几何参数。在此基础上,论述了其工作过程,包括其标定方法、标定过程、扫描图像采集、特征提取、特征匹配及三维数据生成。该双目视觉测量传感器配合扫描机构(如三坐标测量机 )即可对物体进行非接触式三维激光扫描测量,其测量速度快,精度比较高,可以应用于逆向工程领域。     

基于双目立体视觉避障的四旋翼飞行器设计

将双目立体视觉技术融入四旋翼飞行器设计,通过两个在相对水平位置上的单目相机模拟人的眼睛,在双目立体视觉测距系统上经过图像采集、双目矫正、双目立体匹配等步骤得到视差结果,计算出深度信息并进行三维重构,实现四旋翼飞行器一键起飞、定高、定点巡航、悬停及自主避障。实验结果表明,所设计的飞行器响应速度快,跟随性能好,定位精度较高。     

融合深度和颜色信息的图像物体分割算法

讨论立体图对的图像分割问题,提出一种基于深度和颜色信息的图像物体分割算法。该算法首先利用基于聚类的Mean-shift分割算法对目标图像进行适度的过分割,同时借助双目立体视觉算法获取立体图对的稠密深度图,并依据深度不连续性从过分割结果中选取用于继续进行“精致”分割的种子点集,接着对未分配种子标签的区域用图割算法分配标签,并对彼此之间没有深度不连续边界但具有不同标签的相邻区域进行融合。相比于传统图像分割算法,该算法可有效克服过分割和欠分割问题,获取具有一定语义的图像分割结果。相关的对比实验结果验证了该算法的有效性。     

一种基于双目立体视觉的图像校正算法设计

立体校正作为立体匹配的先决条件,能消除图像对对应像素的垂直视差,有效地提高立体匹配的精确度.针对校正带来的图像失真问题,研究了一种新的非标定相机的立体校正方法.该方法首先利用SIFT算法提取匹配点,再结合极线校正准则和雅可比行列式来构建能量函数,通过最小化能量函数求得基础矩阵和校正投影变换矩阵,并在像素级上优化了投影变换矩阵.研究结果表明:所设计的校正方法有效地减小了校正误差,更好地保持了原图像的重采样,降低了图像的失真度.