主动目标几何建模研究方法综述

目的 目标建模是机器视觉领域的主要研究方向之一,主动目标建模是在保证建模完整度的情况下,通过有计划地调节相机的位姿参数,以更少的视点和更短的运动路径实现目标建模的智能感知方法。为了反映主动目标建模的研究现状和最新进展,梳理分析了2004年以来的相关文献,对国内外研究方法做出概括性总结。方法 以重构模型类型和规划视点所用信息作为划分依据,将无模型的主动目标建模方法分为基于表面的主动目标建模方法、基于搜索的目标建模方法和两者相结合的方法3大类,重点对前两类方法进行综述,首先解释了每类方法的基本思想,总结每类方法涉及的问题,然后对相关问题的主要研究方法进行归纳和分析,最后将各个问题的解决方法进行合理的搭配组合,形成不同的主动目标建模方法,并对各类方法的优势和局限性进行了总结。结果 各类主动目标建模算法在适用场景范围、计算复杂度等方面存在差异,但相对于传统的被动目标建模方法,当前的主动目标建模算法已经能够极大程度地提高建模任务的质量和降低建模所需代价。结论 基于表面的主动目标建模方法思想相对简单,但仅适用于表面简单的目标建模。基于搜索的目标建模方法能够量化地评价每一个候选视点,适用广泛且涉及的问题相对于基于表面的方法有更大的解决空间,有更多的研究成果产生。将二者涉及问题的不同研究方法相搭配,可以构成不同的主动目标建模方法子类。     

Toward coherent object detection and scene layout understanding

Detecting objects in complex scenes while recovering the scene layout is a critical functionality in many vision-based applications. In this work, we advocate the importance of geometric contextual reasoning for object recognition. We start from the intuition that objects' location and pose in the 3D space are not arbitrarily distributed but rather constrained by the fact that objects must lie on one or multiple supporting surfaces. We model such supporting surfaces by means of hidden parameters (i.e. not explicitly observed) and formulate the problem of joint scene reconstruction and object recognition as the one of finding the set of parameters that maximizes the joint probability of having a number of detected objects on K supporting planes given the observations. As a key ingredient for solving this optimization problem, we have demonstrated a novel relationship between object location and pose in the image, and the scene layout parameters (i.e. normal of one or more supporting planes in 3D and camera pose, location and focal length). Using a novel probabilistic formulation and the above relationship our method has the unique ability to jointly: i) reduce false alarm and false negative object detection rate; ii) recover object location and supporting planes within the 3D camera reference system; iii) infer camera parameters (view point and the focal length) from just one single uncalibrated image. Quantitative and qualitative experimental evaluation on two datasets (desk-top dataset [1] and LabelMe [2]) demonstrates our theoretical claims.     

Aroundplot: Focus+context interface for off-screen objects in 3D environments

In exploring 3D environments from a first-person viewpoint, the narrow field-of-view makes it difficult to search for an off-screen object, a task that becomes even harder if the user is looking through the small screen of a mobile phone. This paper presents Aroundplot, a novel focus+context interface for providing multiple location cues for off-screen objects in an immersive 3D environment. One part of this technique is a mapping method from 3D spherical coordinates to 2D orthogonal fisheye, which tackles the problems of existing 3D location cue displays such as occlusion among the cues and discordance with the human frame of reference. The other part is a dynamic magnification method that magnifies the context in the direction the view is moving to alleviate the distortion of the orthogonal fisheye and thus to support precise movement. In an evaluation, the participants could find the target object for a given location cue faster and more accurately with Aroundplot than with a top-down 2D radar. They were more accurate with Aroundplot than with a 3D arrow cluster when the number of objects was large; however, accuracy with a small number of objects and the search speed with any number of objects were not significantly different.     

基于立体成像几何特性的动态有限搜索匹配法

提出了一种基于立体成像几何特性的动态有限搜索匹配法。该法根据被测物体与摄像机标定参照物在空间上的相互关系及被测物体形态，确定被测物体特征点视差匹配大致范围。利用立体成像几何特性，采用动态有限搜索法，以求出的视差为参考，根据左(右)图像中特征点与所得匹配对应点的水平视差值，确定在右(左)图像中的搜索方向和搜索范围，采用灰度区域相关计算，确定下一个匹配特征点，并依此类推。由于特征点灰度不稳定，且随视点的不同而有很大差异，为能得到正确匹配，需结合几何相似性约束。通过对一已知三维坐标标准件的计算机仿真实验，证实了该方法的有效性。     

基于目标检测的机器人手眼标定方法

智能协作机器人依赖视觉系统感知未知环境中的动态工作空间定位目标,实现机械臂对目标对象的自主抓取回收作业。RGB-D相机可采集场景中的彩色图和深度图,获取视野内任意目标三维点云,辅助智能协作机器人感知周围环境。为获取抓取机器人与RGB-D相机坐标系之间的转换关系,提出基于yolov3目标检测神经网络的机器人手眼标定方法。将3D打印球作为标靶球夹持在机械手末端,使用改进的yolov3目标检测神经网络实时定位标定球的球心,计算机械手末端中心在相机坐标系下的3D位置,同时运用奇异值分解方法求解机器人与相机坐标系转换矩阵的最小二乘解。在6自由度UR5机械臂和Intel RealSense D415深度相机上的实验结果表明,该标定方法无需辅助设备,转换后的空间点位置误差在2 mm以内,能较好满足一般视觉伺服智能机器人的抓取作业要求。     

基于自适应几何分析的高效交互式导航

三维场景的虚拟导航需要同时保证相机视角的光滑性和智能性,在算法设计上极具挑战。文章研发了一套实时自动生成光滑连续且高效的相机路径导航系统。首先通过线下模型几何分析和语义分析,包含分析场景中建筑物模型、道路模型以及非建筑的重要模型等,求得模型重要性值;其次是自适应地获取路线采样点以及高效存储采样点的视角图片;最后是利用动态规划的方式求出每个采样点的最佳视角并光滑地连接这些采样点形成相机运行轨迹。4 个不同三维场景的实验结果表明了所提算法的高效性和智能性,同时文章所进行的用户调查也充分反映了所提方法具有的明显优势。     

Effectiveness of a Camera as a UAV Mounted Search Sensor for Target Detection: An Experimental Investigation

In this paper, we consider a problem of autonomous search using single or multiple Unmanned Ariel Vehicles (UAVs) mounted with downward-facing cameras. A model of the effectiveness of the search sensor, camera, in this case, is essential for developing strategies for optimal deployment and path planning of UAVs for efficient search. The probability of detection of a target of interest as a function of its distance from the point directly below the camera is used to model the search effectiveness. We carried out experiments and obtained a search effectiveness model for a camera in the laboratory environment using ArUco markers and triangular shapes as targets.

     

Recognizing large isolated 3-D objects through next view planning using inner camera invariants.

Most model-based three-dimensional (3-D) object recognition systems use information from a single view of an object. However, a single view may not contain sufficient features to recognize it unambiguously. Further, two objects may have all views in common with respect to a given feature set, and may be distinguished only through a sequence of views. A further complication arises when in an image, we do not have a complete view of an object. This paper presents a new online scheme for the recognition and pose estimation of a large isolated 3-D object, which may not entirely fit in a camera's field of view. We consider an uncalibrated projective camera, and consider the case when the internal parameters of the camera may be varied either unintentionally, or on purpose. The scheme uses a probabilistic reasoning framework for recognition and next-view planning. We show results of successful recognition and pose estimation even in cases of a high degree of interpretation ambiguity associated with the initial view.     

Self-calibrated visual servoing with respect to axial-symmetric 3D objects

A self-calibrated approach to visual servoing with respect to non-planar targets modeled through a pair of coaxial circles plus one point is discussed. Full calibration data (fixed internal parameters) are obtained from two views, and used to recover the Euclidean structure of an auxiliary virtual plane associated to the target, together with the relative pose of the camera. Pose disambiguation is achieved without requiring any real third view of the target. The approach benefits of an off-line planning strategy by which the camera follows a 3D helicoidal path around an arbitrarily chosen axis. A convenient choice for the helicoidal axis is found to be that of the target axis itself. Simulation results demonstrate that the approach is robust with respect to noise both in the off-line and on-line control phases.     

On view likelihood and stability

We define two measures on views: view likelihood and view stability. View likelihood measures the probability that a certain view of a given 3D object is observed; it may be used to identify typical, or “characteristic” views. View stability measures how little the-image changes as the viewpoint is slightly perturbed; it may be used to identify “generic” views. Both definitions are shown to be identical up to the prior probability of camera orientations, and determined by the 2D metric used to compare images. We analytically derive the stability and likelihood measures for two feature-based 2D metrics, where the most stable and most likely view is shown to be the flattest view of the 3D shape. Incorporating view likelihood or stability in 3D object recognition and 3D reconstruction increases the chance of robust performance. In particular, we propose to use these measures to enhance 3D object recognition and 3D reconstruction algorithms, by adding a second step where the most likely solution is selected among all feasible solutions. These applications are demonstrated using simulated and real images