Localization and Geometric Reconstruction of Mobile Robots Using a Camera Ring

In this paper, a system capable of obtaining the 3-D pose of a mobile robot using a ring of calibrated cameras attached to the environment is proposed. The system robustly tracks point fiducials in the image plane of the set of cameras generated by the robot's rigid shape in motion. Each fiducial is identified with a point belonging to a sparse 3-D geometrical model of the robot's structure. Such a model allows direct pose estimation from image measurements, and it can easily be enriched at each iteration with new points as the robot motion evolves. The process is divided into an initialization step, where the structure of the robot is obtained, and an online step, which is solved using sequential Bayesian inference. The approach allows the proper modeling of uncertainty in measurements and estimations, and at the same time, it serves as a regularization step in pose estimation. The proposed system is verified using simulated and real data.      

Skeleton body pose tracking from efficient three-dimensional motion estimation and volumetric reconstruction

We address the problem of body pose tracking in a scenario of multiple camera setup with the aim of recovering body motion robustly and accurately. The tracking is performed on three-dimensional (3D) space using 3D data, including colored volume and 3D optical flow, which are reconstructed at each time step. We introduce strategies to compute multiple camera-based 3D optical flow and have attained efficient and robust 3D motion estimation. Body pose estimation starts with a prediction using 3D optical flow and then is changed to a lower-dimensional global optimization problem. Our method utilizes a voxel subject-specific body model, exploits multiple 3D image cues, and incorporates physical constraints into a stochastic particle-based search initialized from the deterministic prediction and stochastic sampling. It leads to a robust 3D pose tracker. Experiments on publicly available sequences show the robustness and accuracy of our approach.     

三维点云的两步校准法及其应用研究

针对具有平面特征的工件的位姿识别,提出了一种三维点云两步校准法。首先,对三维点云进行多次随机抽样,每次抽取3个点并计算其所构成平面的单位法向量;经多次抽样后,得到一个平面单位法向量集合,采用密度聚类算法提取主平面的单位法向量,并根据主平面单位法向量与目标向量的轴角变换关系实现点云的倾斜度校正。然后,先将点云投影至主平面,再将各点的高度映射为主平面内的灰度,得到点云灰度图;再利用图像模板匹配算法获取点云在主平面内投影的平移量和旋转角,经平移和旋转后实现点云位姿的最终校准。将所提出的三维点云两步校准法应用于高压输电线塔塔座自动焊接系统,以识别塔座的位姿信息,并与基于采样一致性法识别的塔座位姿信息进行对比。结果表明,相比于采样一致性法,所提出的两步校准法既提高了工件位姿的识别精度,又缩短了约76%的耗时。研究结果可为具有平面特征的工件的位姿快速校准及识别提供参考。三维点云两步校准法在先进制造业中具有广阔的应用前景。     

Cooperative Tracking Using Multiple PTZ Thermal Imagers

A cooperative object tracking framework is proposed which shifts the priority of tracking by pose estimation based on registration between fields of view (FOVs) of different pan?Ctilt?Czoom (PTZ) thermal imagers, avoiding transferring the local features from one imager to another. When an object is selected for tracking, the related PTZ thermal imager tracks it using an improved particle filtering method, and estimates the pose of the imager simultaneously. Once the object enters an overlapping FOV of two imagers, the handoff thermal imager is activated immediately according to the spatial relationship built by pose estimation, and turns to tacking and pose estimation status. Meanwhile, the previous camera returns to waiting status for reactivation. Experiments are conducted to show the proposed framework is applicable to cooperative object tracking for PTZ thermal infrared imagers.     

Residual-Based robust estimation and image-motion analysis

We explain that the task of multiple rigid motion segmentation and estimation from image feature point correspondence demands an estimator of high robustness. We show that a heuristics-based partial modeling approach can be used to develop a highly robust estimator called the MF estimator for general regression, where “MF” represents an abbreviation of Model Fitting. Finally, we provide experimental results in estimating single rigid motion from a mixture of 2D-2D (or image-image), 3D-2D (or range-image) and 3D-3D (or range-range) corresponding point data by using the proposed MF estimator. As will be seen, only four well matched corresponding point pairs are needed to get a good estimate of motion parameters no matter how many mismatched corresponding point pairs (or outliers) occur. The article represents an initial effort towards robust image-motion analysis.     

Grasp Planning and Visual Servoing for an Outdoors Aerial Dual Manipulator

This paper describes a system for grasping known objects with unmanned aerial vehicles (UAVs) provided with dual manipulators using an RGB-D camera. Aerial manipulation remains a very challenging task. This paper covers three principal aspects for this task: object detection and pose estimation, grasp planning, and in-flight grasp execution. First, an artificial neural network (ANN) is used to obtain clues regarding the object’s position. Next, an alignment algorithm is used to obtain the object’s six-dimensional (6D) pose, which is filtered with an extended Kalman filter. A three-dimensional (3D) model of the object is then used to estimate an arranged list of good grasps for the aerial manipulator. The results from the detection algorithm—that is, the object’s pose—are used to update the trajectories of the arms toward the object. If the target poses are not reachable due to the UAV’s oscillations, the algorithm switches to the next feasible grasp. This paper introduces the overall methodology, and provides the experimental results of both simulation and real experiments for each module, in addition to a video showing the results.     

虚拟遥操作系统中灵巧手动作控制研究

介绍了在机械灵巧手虚拟遥操作系统中，关于采用立体视觉方法检测计算操作人员手势姿态及手指各关节参数方面的研究。多指灵巧机械手“复制”操作人员手的动作姿态，目的是进行不规则形状物体的抓取方法及可选择性抓取动作的直接控制研究。     

运动背景中结合特征位移矢量场模糊分割与 OTSU法的运动检测

运动背景中的运动检测难度较大,背景运动补偿后差分以及分割光流场可实现动目标和背景的分离,差分前需进行鲁棒的背景估计,且差分后易出现空洞,而光流估计在噪声以及目标运动速度较大时并不准确,尤其在光照变化时,两种方法均易失效。本文提出一种特征点位移矢量场模糊分割与图像自适应阈值化相结合的运动检测方法,实现在无任何关于运动目标或者运动背景先验信息条件下的动目标检测。通过改进的 SIFT匹配方法生成鲁棒的特征位移矢量场,采用模糊 C均值聚类算法对 SIFT位移矢量场进行无监督分类,实现动目标与背景特征的自适应分离。 OTSU法和形态学操作实现图像的自适应分割,用以修正特征点凸包,最终分割出动目标区域。与鲁棒的背景运动补偿后差分以及光流估计的对比实验表明,在目标运动速度较大、光照变化以及噪声情况下,本文方法均能够检测出运动目标,且在光照变化下的优势明显。     

双工业机器人协调复杂作业实验研究

就双机器人搬运刚体之外的复杂作业提出了面向操作物的运动描述与规划方法，以及适应于工业机器人系统的基于关节位置控制的协调控制策略。在两工业机器人协调系统上实现了相关多轴孔装配，复杂工件边缘跟踪，弹性体操作与装配作业实验。     

摄像机旋转运动下的快速目标检测算法

论文提出了一种摄像机旋转运动下的快速目标检测算法。首先为图像的全局运动建立旋转参数模型,然后基于运动预测在相邻帧之间建立SIFT特征点对,利用RANSAC去除外点的影响,结合最小二乘法求解全局运动参数进行运动补偿,基于残差图像的更新策略实时更新特征点集,以适应背景的变化,最后使用帧差法获得运动目标。该算法不仅保持了SIFT本身的优越性能,而且极大地提高了检测速度。实验结果表明该算法可以实时准确的检测出运动目标。     

面向板料精准堆垛的线特征位姿标定方法

目的 板料堆垛是现代流程工业生产中常见的生产环节,其堆垛质量影响着包装和运输安全。针对多边形板料自动堆垛中机械手抓取和对齐问题,提出一种基于网格筛选的线特征相机位姿标定方法,给出空间特征点的选择标准,解决因板料纹理不足带来的点特征精度下降的问题。方法 首先通过检测特征点来表示待匹配线,然后利用基于三角网格的线特征构型筛选算法,获得精确的匹配线;再利用单应性矩阵求解获得转换矩阵,最后通过转换在线板料图像,获得实际抓取点和转换角度。结果 通过桌面板料抓取实验和现场验证,重投影误差不超过2像素,1 m板料的堆垛误差为0.5 mm,结论 证明了此方法在板料自动堆垛过程中针对特征检测和位姿估计问题的有效性。     

基于视觉引导和超声测距的运动目标跟踪和抓取

提出了装配机器人系统中一种基于视觉引导和超声测距的运动目标跟踪和抓取方法。介绍了运动目标的跟踪原理 ,采用图像雅可比矩阵进行机器人运动控制 ,跟踪运动的目标。利用视觉引导技术 ,获取目标在图像平面中的位置与方位 ,进行平面跟踪 ,然后引导超声波测距装置测取目标深度信息。实验结果验证了本文提出的这种方法的可行性和有效性     

曲面复合材料缝合机器人自动路径规划技术

为提高复合材料预制件缝合机器人的灵活性与加工柔性,针对缝合机器人自动路径规划问题,研究了基于三维光栅扫描技术的曲面复合材料缝合机器人视觉接缝提取技术.针对三维扫描摄像机视场固定的特点,对摄像机进行标定实验,改进传统的手眼标定方法,提出一种用于求解手眼关系矩阵的四点标定新方法,建立了机器人坐标系与摄像机坐标系的位姿映射关系.经过手眼关系标定,三维光栅扫描仪采样得到的缝合预制件点云数据可转化为机器人坐标系下的点云数据坐标.进一步对采集的预制件点云数据进行接缝中心线提取,提出了一种缝合轨迹及姿态规划算法,采用三次多项式对接缝中心线进行空间曲线拟合,采用最小二乘法对缝合微切平面进行平面拟合,完成机器人操作空间前进矢量及接近矢量的计算.最后,将规划结果应用于缝合机器人的离线编程仿真与实验中.实验结果表明,该系统精度高,线迹成型良好,可以满足机器人缝合系统的要求.     

Face verification through tracking facial features.

We propose an algorithm for face verification through tracking facial features by using sequential importance sampling. Specifically, we first formulate tracking as a Bayesian inference problem and propose to use Markov chain Monte Carlo techniques for obtaining an empirical solution. A reparameterization is introduced under parametric motion assumption, which facilitates the empirical estimation and also allows verification to be addressed along with tracking. The facial features to be tracked are defined on a grid with Gabor attributes (jets). The motion of facial feature points is modeled as a global two-dimensional (2-D) affine transformation (accounting for head motion) plus a local deformation (accounting for residual motion that is due to inaccuracies in 2-D affine modeling and other factors such as facial expression). Motion of both types is processed simultaneously by the tracker: The global motion is estimated by importance sampling, and the residual motion is handled by incorporating local deformation into the measurement likelihood in computing the weight of a sample. Experiments with a real database of face image sequences are presented.     

Three-dimensional motion measurements using feature tracking

Feature tracking was developed to efficiently compute motion measurements from volumetric ultrasound images. Prior studies have demonstrated the motion magnitude accuracy and computation speed of feature tracking. However, the previous feature tracking implementations were limited by performance of their calculations in rectilinear coordinates. Also, the previous feature tracking approaches did not fully explore the three dimensional (3- D) nature of volumetric image analysis or utilize the 3-D directional information from the tracking calculations. This study presents an improved feature tracking method which achieves further computation speed gains by performing all calculations in the native spherical coordinates of the 3-D ultrasound image. The novel method utilizes a statistical analysis of tracked directions of motion to achieve better rejection of false tracking matches. Results from in vitro tracking of a speckle target show that the new feature tracking method is significantly faster than correlation search and can accurately determine target motion magnitude and 3-D direction.     

全口义齿排牙机器人虚拟仿真系统

以全口义齿排牙机器人为研究对象,采用3个手指9个自由度的灵巧手结构设计.利用Simmechanics建立了一种全新的机电一体化建模和仿真系统,可实现机械系统模型与控制系统模型在同一框架下的仿真.应用D-H坐标理论推导手物系统的坐标变换关系,分析排牙机器人的运动学和动力学问题,建立各关节变量和适于抓取的手指末端的位置与姿态表达式,研究智能抓取算法和最优轨迹规划,最终的仿真结果令人满意.     

仿人手掌的机器人变掌机构设计与分析

针对大小不同、形状各异的物体，机器人多指手的抓取规划和控制问题目前尚未得到完善的解决。为此，提出利用仿人手掌来提高机器人多指手的灵巧度以及扩大其工作空间，以降低抓取规划和控制的难度。首先，分析了人体手掌的运动特点，根据人体手掌的骨骼结构，研发了一种仿人手掌的机器人变掌机构，该变掌机构具有6个自由度，通过构型变换，可使安装于变掌机构的手指改变工作空间以及抓取点位置。其次，通过建立变掌机构的数学模型，分析已知抓取点个数和抓取点位置时变掌机构的6个自由度。然后，利用三维软件对变掌机构进行建模，并对其抓取动作进行仿真。最后，通过实验平台以及实物样机进行抓取实验，分别利用刚性手指、柔性手指对13种不同的物体进行抓取，验证了变掌机构的抓取能力以及适应性。该仿人手掌的机器人变掌机构能够简单快速地实现自适应构型，可以对尺寸、质量、形状差异较大的物体进行抓取，适用于二指、三指、四指的抓取，并且可连接刚性或柔性手指，既提高了机器人多指手的抓取质量以及适应性，又降低了其抓取规划和控制的难度。研究结果可对机器人多指手的普遍应用提供借鉴意义。     

Estimating three-dimensional vehicle motion in an outdoor scene using stereo image sequences

We propose a motion estimation system that uses stereo image pairs as the input data. To perform experimental work, we also obtain a sequence of outdoor stereo images taken by two metric cameras. The system consists of four main stages, which are (1) determination of point correspondences on the stereo images, (2) correction of distortions in image coordinates, (3) derivation of 3D point coordinates from 2D correspondences, and (4) estimation of motion parameters based on 3D point correspondences. For the first stage of the system, we use a four-way matching algorithm to obtain matched point on two stereo image pairs at two consecutive time instants (t_i and t_{i + 1}). Since the input data are stereo images taken by cameras, it has two types of distortions, which are (i) film distortion and (ii) lens distortion. These two distortions must be corrected before any process can be applied on the matched points. To accomplish this goal, we use (i) bilinear transform for film distortion correction and (ii) lens formulas for lens distortion correction. After correcting the distortions, the results are 2D coordinates of each matched point that can be used to derive 3D coordinates. However, due to data noise, the calculated 3D coordinates to not usually represent a consistent rigid structure that is suitable for motion estimation; therefore, we suggest a procedure to select good 3D point sets as the input for motion estimation. The procedure exploits two constraints, rigidity between different time instants and uniform point distribution across the object on the image. For the last stage, we use an algorithm to estimate the motion parameters. We also wish to know what is the effect of quantization error on the estimated results; therefore an error analysis based on quantization error is performed on the estimated motion parameters. In order to test our system, eight sets of stereo image pairs are extracted from an outdoor stereo image sequence and used as the input data. The experimental results indicate that the proposed system does provide reasonable estimated motion parameters.     

PSF dedicated to estimation of displacement vectors for tissue elasticity imaging with ultrasound

This paper investigates a new approach devoted to displacement vector estimation in ultrasound imaging. The main idea is to adapt the image formation to a given displacement estimation method to increase the precision of the estimation. The displacement is identified as the zero crossing of the phase of the complex cross-correlation between signals extracted from the lateral direction of the ultrasound RF image. For precise displacement estimation, a linearity of the phase slope is needed as well as a high phase slope. Consequently, a particular point spread function (PSF) dedicated to this estimator is designed. This PSF, showing oscillations in the lateral direction, leads to synthesis of lateral RF signals. The estimation is included in a 2-D displacement vector estimation method. The improvement of this approach is evaluated quantitatively by simulation studies. A comparison with a speckle tracking technique is also presented. The lateral oscillations improve both the speckle tracking estimation and our 2-D estimation method. Using our dedicated images, the precision of the estimation is improved by reducing the standard deviation of the lateral displacement error by a factor of 2 for speckle tracking and more than 3 with our method compared to using conventional images. Our method performs 7 times better than speckle tracking. Experimentally, the improvement in the case of a pure lateral translation reaches a factor of 7. Finally, the experimental feasibility of the 2-D displacement vector estimation is demonstrated on data acquired from a Cryogel phantom.     

Probabilistic 3D object recognition and pose estimation using multiple interpretations generation

This paper presents a probabilistic object recognition and pose estimation method using multiple interpretation generation in cluttered indoor environments. How to handle pose ambiguity and uncertainty is the main challenge in most recognition systems. In order to solve this problem, we approach it in a probabilistic manner. First, given a three-dimensional (3D) polyhedral object model, the parallel and perpendicular line pairs, which are detected from stereo images and 3D point clouds, generate pose hypotheses as multiple interpretations, with ambiguity from partial occlusion and fragmentation of 3D lines especially taken into account. Different from the previous methods, each pose interpretation is represented as a region instead of a point in pose space reflecting the measurement uncertainty. Then, for each pose interpretation, more features around the estimated pose are further utilized as additional evidence for computing the probability using the Bayesian principle in terms of likelihood and unlikelihood. Finally, fusion strategy is applied to the top ranked interpretations with high probabilities, which are further verified and refined to give a more accurate pose estimation in real time. The experimental results show the performance and potential of the proposed approach in real cluttered domestic environments.