Fast topology estimation for image mosaicing using adaptive information thresholding

Over the past decade, several image mosaicing methods have been proposed in robotic mapping and remote sensing applications. Owing to rapid developments in obtaining optical data from areas beyond human reach, there is a high demand from different science fields for creating large-area image mosaics, often using images as the only source of information. One of the most important steps in the mosaicing process is motion estimation between overlapping images to obtain the topology, i.e., the spatial relationships between images.In this paper, we propose a generic framework for feature-based image mosaicing capable of obtaining the topology with a reduced number of matching attempts and of getting the best possible trajectory estimation. Innovative aspects include the use of a fast image similarity criterion combined with a Minimum Spanning Tree (MST) solution, to obtain a tentative topology and information theory principles to decide when to update trajectory estimation. Unlike previous approaches for large-area mosaicing, our framework is able to naturally deal with the cases where time-consecutive images cannot be matched successfully, such as completely unordered sets. This characteristic also makes our approach robust to sensor failure. The performance of the method is illustrated with experimental results obtained from different challenging underwater image sequences.     

Video motion stitching using trajectory and position similarities

Stitching motions in multiple videos into a single video scene is a challenging task in current video fusion and mosaicing research and film production. In this paper, we present a novel method of video motion stitching based on the similarities of trajectory and position of foreground objects. First, multiple video sequences are registered in a common reference frame, whereby we estimate the static and dynamic backgrounds, with the former responsible for distinguishing the foreground from the background and the static region from the dynamic region, and the latter functioning in mosaicing the warped input video sequences into a panoramic video. Accordingly, the motion similarity is calculated by reference to trajectory and position similarity, whereby the corresponding motion parts are extracted from multiple video sequences. Finally, using the corresponding motion parts, the foregrounds of different videos and dynamic backgrounds are fused into a single video scene through Poisson editing, with the motions involved being stitched together. Our major contributions are a framework of multiple video mosaicing based on motion similarity and a method of calculating motion similarity from the trajectory similarity and the position similarity. Experiments on everyday videos show that the agreement of trajectory and position similarities with the real motion similarity plays a decisive role in determining whether two motions can be stitched. We acquire satisfactory results for motion stitching and video mosaicing.     

Manifold Modeling and Its Application to Tubular Scene Manifold Mosaicing Algorithm

Manifold mosaicing is arguably the most important class of image mosaicing methods. The existing manifold mosaicing methods work reasonably well only for scenes with simple plane structure and for images taken in a usual way such that the camera??s motion direction is perpendicular to its optical axis. A novel manifold modeling theoretical framework is presented to unify the current image mosaicing methods. Based on this framework, an effective mosaicing algorithm is also proposed to mosaic tubular scenes which have failed most existing methods. We adopt a two-step shaping strategy to parallelize the optical flows and change the topological genus of the image strips by M?bius mappings and circular ring extending. We performed computational experiments via image sequences acquired from tubular scenes and obtained excellent panoramas. The theoretical framework and algorithms in this paper have immediate applications to many practical tubular scene mosaicing problems in medical imaging, industrial inspection, gaming, virtual reality and robotics.     

一种基于运动特征的快速有效的视频镜头全拼图方法

由于全拼图能比关键帧提供更多的视觉信息,因此它已经成为视觉计算中一个重要的分析工具。为了提高全拼图的质量和拼图速度,提出了一种基于运动特征的快速有效的全拼图生成方法。该方法首先给定一个视频镜头,并基于运动相位熵的分析方法决定该镜头内容是否适合生成全拼图;然后,对于适合生成全拼图的镜头,通过构造全局运动路径的方法,仅需要挑选全部视频帧的一个子集用来生成高质量的全拼图。实验结果表明,与传统的全拼图方法相比,该新方法在提高全拼图的视觉质量的同时,显著地降低了计算时间。     

Mosaicing on adaptive manifolds

Image mosaicing is commonly used to increase the visual field of view by pasting together many images or video frames. Existing mosaicing methods are based on projecting all images onto a predetermined single manifold: A plane is commonly used for a camera translating sideways, a cylinder is used for a panning camera, and a sphere is used for a camera which is both panning and tilting. While different mosaicing methods should therefore be used for different types of camera motion, more general types of camera motion, such as forward motion, are practically impossible for traditional mosaicing. A new methodology to allow image mosaicing in more general cases of camera motion is presented. Mosaicing is performed by projecting thin strips from the images onto manifolds which are adapted to the camera motion. While the limitations of existing mosaicing techniques are a result of using predetermined manifolds, the use of more general manifolds overcomes these limitations.     

一种水下运动物体三维轨迹视觉测量方法

针对海洋工程中采用的设备深海悬垂法安装过程,采用多摄像头视频运动分析方 法计算水下三维运动轨迹可用于指导海洋工程的结构安装和分析设备水下运动特征。水下视频 和图像的处理获取面临着诸多挑战,首先由于水下环境悬浮物和颗粒较多,光在水下发生了散 射,使水下图像发生了退化;其次水下视频运动分析遇到的一个主要障碍是光线的折射引起的 图像误差。由于光在水、玻璃、空气不同介质间发生折射,光路发生弯曲,陆地上的摄像机成 像模型在水中不再适用,需要提出新的水下摄像机成像模型。本文引入带光线折射的水下摄像 机成像模型,研究水下摄像机的内参数和外参数标定方法,利用固定布置的 3 个水下摄像机拍 摄的目标水下运动视频来计算水下目标的轨迹。该方法适用于水池环境下水下物体大范围运动, 可以得到较为精确的轨迹,并得到了实验验证。     

基于块匹配的鲁棒视频图像镶嵌算法

为了获取宽视野的场景表示,提出了一种基于块匹配的视频图像镶嵌算法,该算法首先采用基于相位相关的块匹配方法估计出视频图像间的运动矢量场,并剔除其中由于图像噪声或运动物体的遮挡而导致外点运动矢量,然后根据图像的运动矢量场确定出图像子块之间的对应点对,进而利用得到的对应点对迭代求解图像间的变换模型参数以实现视频图像的自动镶嵌.针对真实场景的视频图像序列进行实验,获得了较好的镶嵌结果,表明了该算法的有效性.     

Minimal Aspect Distortion (MAD) Mosaicing of Long Scenes

Long scenes can be imaged by mosaicing multiple images from cameras scanning the scene. We address the case of a video camera scanning a scene while moving in a long path, e.g. scanning a city street from a driving car, or scanning a terrain from a low flying aircraft. A robust approach to this task is presented, which is applied successfully to sequences having thousands of frames even when using a hand-held camera. Examples are given on a few challenging sequences. The proposed system consists of two components: (i) Motion and depth computation. (ii) Mosaic rendering. In the first part a “direct” method is presented for computing motion and dense depth. Robustness of motion computation has been increased by limiting the motion model for the scanning camera. An iterative graph-cuts approach, with planar labels and a flexible similarity measure, allows the computation of a dense depth for the entire sequence. In the second part a new minimal aspect distortion (MAD) mosaicing uses depth to minimize the geometrical distortions of long panoramic images. In addition to MAD mosaicing, interactive visualization using X-Slits is also demonstrated. This research was supported by the Israel Science Foundation. Video examples and high resolution images can be viewed in .     

基于对数极坐标映射的图像拼接方法

图像拼接在基于图像的绘制、视频检索以及景物匹配等领域有着广泛的应用，为了获取大画面宽视场的场景表示，针对存在旋转及缩放变化的图像，提出了一种基于对数极坐标映射的图像拼接方法，该方法先将图像从笛卡儿坐标空间转换到对数极坐标空间，使得笛卡儿坐标空间中图像的旋转和缩放转换为对数极坐标空间中图像的二维平移，这样可直接利用相位相关法来估算出图像间的旋转角度和缩放因子，然后以此作为初值，再采用非线性最小化优化算法进一步求精图像间的运动参数来实现图像的配准，最后通过图像融合来实现图像的拼接。实验结果表明，该方法是有效的。     

彩色图像的单应矩阵估计算法

图像间单应矩阵估计是图像配准与图像拼接中的核心问题,传统的估计方法是针对灰度图像的算法。本文以分层运动估计为基础提出了彩色图像的平面单应矩阵的估计算法。此算法采用色度与饱和度不变为约束条件得到彩色图像的光流方程,显著改善了亮度不变约束的不足之处;采用最优导数计算滤波器计算图像导数,提高了算法的精度与稳健性;采用尺度总体最小二乘方法代替最小二乘或总体最小二乘方法来估计模型参数,提高了算法对于图像噪声的适应性。实验结果表明,该算法稳健性好、精度高,而且可以得到稠密的匹配点。     

Drift-Free Real-Time Sequential Mosaicing

We present a sequential mosaicing algorithm for a calibrated rotating camera which can for the first time build drift-free, consistent spherical mosaics in real-time, automatically and seamlessly even when previously viewed parts of the scene are re-visited. Our mosaic is composed of elastic triangular tiles attached to a backbone map of feature directions over the unit sphere built using a sequential EKF SLAM (Extend Kalman Filter Simultaneous Localization And Mapping) approach. This method represents a significant advance on previous mosaicing techniques which either require off-line optimization or which work in real-time but use local alignment of nearby images and ultimately drift. We demonstrate the system’s real-time performance with real-time mosaicing results from sequences with 360 degrees pan. The system shows good global mosaicing ability despite the challenging conditions: hand-held simple low-resolution webcam, varying natural outdoor illumination, and people moving in the scene.     

Robust Image Mosaicing of Soccer Videos using Self-Calibration and Line Tracking

In this paper we propose an accurate and robust image mosaicing method of soccer video taken from a rotating and zooming camera using line tracking and self-calibration. The mosaicing of soccer videos is not easy, because their playing fields are low textured and moving players are included in the fields. Our approach is to track line features on the playing fields. The line features are detected and tracked using a self-calibration technique for a rotating and zooming camera. To track line features efficiently, we propose a new line tracking algorithm, called camera parameter guided line tracking, which works even when the camera motion undergoes sudden changes. Since we do not need to know any model for scenes beforehand, the proposed algorithm can be easily extended to other video sources, as well as other sports videos. Experimental results show the accuracy and robustness of the algorithm. An application of mosaicing is also presented.     

水下滑翔机器人运动调节机构设计与运动性能分析

描述了水下滑翔机器人3个运动调节机构的设计，即浮力调节机构、俯仰调节机构和横滚调节机构，分析了运动调节机构与运动之间的关系．提出了采用CFX水动力计算软件分析水下滑翔机器人运动性能的方法．根据CFX计算结果，用最小二乘法参数辨识方法辨识出定常滑翔运动的水动力参数．简化了空间螺旋回转运动过程，通过CFX水动力计算方法进行回转特性分析，估算回转半径．     

Displacement Following of Hidden Objects in a Video Sequence

In a video sequence, computing the motion of an object requires the continuity of the apparent velocity field. This property does not hold when the object is hidden by an occlusion during its motion. The minimization of an energy functional leads to a simple algorithm which allows the recovery of the most likely trajectory of the occluded object from optical flow data at the border of the occlusion. Optical flow used for developing our method is an improvement on any variational technique of computing it. This improvement is based on a multichannel segmentation.     

Rectified Surface Mosaics

We approach mosaicing as a camera tracking problem within a known parameterized surface. From a video of a camera moving within a surface, we compute a mosaic representing the texture of that surface, flattened onto a planar image. Our approach works by defining a warp between images as a function of surface geometry and camera pose. Globally optimizing this warp to maximize alignment across all frames determines the camera trajectory, and the corresponding flattened mosaic image. In contrast to previous mosaicing methods which assume planar or distant scenes, or controlled camera motion, our approach enables mosaicing in cases where the camera moves unpredictably through proximal surfaces, such as in medical endoscopy applications.     

基于最佳邻域重构指数的水下高光谱目标检测

水下机器人仅通过传统光学相机获取图像很难在复杂水下环境中或目标物具有保护色的情况下检测到目标,而通过高光谱技术进行水下目标检测可以改善这一情况;由于直接运用传统高光谱检测方法难以满足水下机器人对水下目标检测的要求,提出了一种基于最佳邻域重构指数(ONRIF)的高光谱目标检测方法,该方法通过线性重构的思想进行邻域寻优,选出信息量高且波段相关性低的波段组合,并使用所选波段的融合图像进行目标检测;结果表明,与直接对原始水下海产品高光谱图像进行检测相比,该方法在保证检测效果的前提下,大量减少了检测时间和数据冗余程度;还提出了一种在相同环境下对同类目标物的单波段快速采集检测方法,大大提高了采集数据的速度,可以满足水下机器人对海产品检测的需求.     

基于复值小波分解的图象拼合

图象拼合是一种基于图象的场景编码方法，它被很多基于图象的绘制IBR（image based rendering）系统采用，用来建立复杂的虚拟场景表示（例如，360°球面和柱面全景图、环境映照及高分辨率图象等）.基于复值小波多分辨率分解(Complex Wavelet Multiresolution Decomposition)提出了一种有效的图象拼合方法，它能同时地、逐步求精地对图象进行匹配和整合.首先，采用复值小波变换不仅可以保证全局优化的结果，还能够满足图象整合的规模不变和平移不变性.其次，基于多分辨     

Design and performance evaluation of a biomimetic microrobot for the father–son underwater intervention robotic system

Underwater intervention is a favorite and difficult task for AUVs. To realize the underwater manipulation for the small size spherical underwater robot SUR-II, a father–son underwater intervention robotic system (FUIRS) is proposed in our group. The FUIRS employs a novel biomimetic microrobot to realize an underwater manipulation task. This paper describes the biomimetic microrobot which is inspired by an octopus. The son robot can realize basic underwater motion, i.e. grasping motion, object detection and swimming motion. To enhance the payload, a novel buoyancy force adjustment method was proposed which can provides 11.8 mN additional buoyancy force to overcome the weight of the object in water. Finally, three underwater manipulation experiments are carried out to verify the performance of the son robot. One is carried by swimming motion and buoyancy adjustment; the other two are only carried by buoyancy adjustment. And the experimental results show that the son robot can realize the underwater manipulation of different shape and size objects successfully. The swimming motion can reduce the time cost of underwater manipulation remarkably.     

结合平衡控制的动作过渡算法

为了有效地利用运动捕获数据来驱动虚拟人物,研究了动作过渡中的约束求解问题,该运动学约束求解可由解析的IK算法快速完成,但对每帧附加约束,以保证解的连续性,同时,为了使生成的动作更加真实,提出了一种在动作过渡的关键点求取平衡姿势的方法,并通过构造偏移映射,将其叠加到已生成的动作过渡序列图象上,从而较好地模拟了动态平衡过程。