一种机器人导航中自然路标的匹配与跟踪方法

提出了一种基于SIFT和KLT算法的自然路标匹配与跟踪方法。该方法利用SIFT算子提取图像中自然路标的特征点集作为模板,然后将机器人采集图像中的SIFT特征点集与模板特征点集进行匹配,获取二者之间仿射关系,并解算自然路标在视野中的位置,为机器人自定位提供参考信息。机器人在运行过程中,将KLT算法与SIFT算法相结合对成功匹配的自然路标进行跟踪,较好地解决了SIFT算法效率低下的问题。实验结果表明该方法对自然路标具有较好的匹配和跟踪效果。     

Visual Navigation in Natural Environments: From Range and Color Data to a Landmark-Based Model

This paper concerns the exploration of a natural environment by a mobile robot equipped with both a video color camera and a stereo-vision system. We focus on the interest of such a multi-sensory system to deal with the navigation of a robot in an a priori unknown environment, including (1) the incremental construction of a landmark-based model, and the use of these landmarks for (2) the 3-D localization of the mobile robot and for (3) a sensor-based navigation mode.For robot localization, a slow process and a fast one are simultaneously executed during the robot motions. In the modeling process (currently 0.1 Hz), the global landmark-based model is incrementally built and the robot situation can be estimated from discriminant landmarks selected amongst the detected objects in the range data. In the tracking process (currently 4 Hz), selected landmarks are tracked in the visual data; the tracking results are used to simplify the matching between landmarks in the modeling process.Finally, a sensor-based visual navigation mode, based on the same landmark selection and tracking, is also presented; in order to navigate during a long robot motion, different landmarks (targets) can be selected as a sequence of sub-goals that the robot must successively reach.     

姿态和表情变化下的三维人脸标志点定位

三维人脸标志点定位在人脸识别、人脸跟踪、人脸建模、表情分析等方面具有非常重要的作用. 然而, 在姿态和表情变化很大的条件下进行标志点定位, 这仍然是一个很具挑战性的课题. 本文提出一种对姿态和表情不敏感的三维人脸标志点定位方法, 利用HK曲率分析检测出候选标志点, 根据对面部形状的先验知识, 提出一种基于人脸几何结构的分类策略对候选标志点进一步细分, 通过把候选标志点与人脸标志点模型进行匹配, 实现标志点的精确定位. 首先在CASIA数据集对该方法的标志点定位精度进行测试, 然后在UND/FRGC v2.0数据集对该方法与其他方法进行比较. 实验结果表明该方法在姿态和表情变化很大的条件下具有高精度和高鲁棒性.     

Visual Recognition of Workspace Landmarks for Topological Navigation

In this work, robot navigation is approached using visual landmarks. Landmarks are not preselected or otherwise defined a priori; they are extracted automatically during a learning phase. To facilitate this, a saliency map is constructed on the basis of which potential landmarks are highlighted. This is used in conjunction with a model-driven segregation of the workspace to further delineate search areas for landmarks in the environment. For the sake of robustness, no semantic information is attached to the landmarks; they are stored as raw patterns, along with information readily available from the workspace segregation. This subsequently facilitates their accurate recognition during a navigation session, when similar steps are employed to locate landmarks, as in the learning phase. The stored information is used to transform a previously learned landmark pattern, according to the current position of the observer, thus achieving accurate landmark recognition. Results obtained using this approach demonstrate its validity and applicability in indoor workspaces.     

How important is location information in saliency detection of natural images

Location information, i.e., the position of content in image plane, is considered as an important supplement in saliency detection. The effect of location information is usually evaluated by integrating it with the selected saliency detection methods and measuring the improvement, which is highly influenced by the selection of saliency methods. In this paper, we provide direct and quantitative analysis of the importance of location information for saliency detection in natural images. We firstly analyze the relationship between content location and saliency distribution on four public image datasets, and validate the distribution by simply treating location based Gaussian distribution as saliency map. To further validate the effectiveness of location information, we propose a location based saliency detection approach, which completely initializes saliency maps with location information and propagate saliency among patches based on color similarity, and discuss the robustness of location information’s effect. The experimental results show that location information plays a positive role in saliency detection, and the proposed method can outperform most state-of-the-art saliency detection methods and handle natural images with different object positions and multiple salient objects.     

基于改进的颜色和SURF特征的粒子滤波目标跟踪

针对视频中运动目标的准确跟踪问题,提出了一种改进的颜色直方图特征和SURF特征的粒子滤波跟踪算法。采用SURF算法提取特征点,利用分层迭代的KLT算法对特征点进行稳定跟踪。将SURF特征与改进的视觉显著性颜色特征进行乘性融合,作为粒子滤波的观测概率。针对跟踪过程中SURF匹配数下降和不稳定的现象,设计了SURF特征模板集的更新策略。与传统特征的跟踪进行多组对比实验,其结果证明了该方法对光照和遮挡具有很好的鲁棒性,对目标跟踪的准确率更高。     

一种基于RANSAC的点云柱状化轴线特征表示法

现有的三维地图构建算法多强调对地图构建的精确性,导致成图效率低、成本高。 为了提高建立地图的效率,提出了一种对地标性物体进行圆柱体识别与提取并以其轴线特征作 为地标构建简化地图的改进算法。基于随机采样一致算法（RANSAC）对点云模型中的待提取 主体模型生成待估计圆柱模型并进行匹配,通过对单应性矩阵及其误差函数的计算得到迭代过 程中的最佳阈值,以得到最佳匹配圆柱模型并提高提取效率,然后用所提取的圆柱轴线描述地 标的空间位置,圆柱半径描述地标的空间几何信息。通过与传统 RANSAC 方法的仿真实验对比, 证明该方法可以有效的精简地图,为后续识别地标路径规划奠定基础。     

区域损失函数的孪生网络目标跟踪

针对预训练卷积神经网络提取的深度特征空间分辨率低,快速运动造成运动目标空间细节信息丢失等问题,提出用区域损失函数构建孪生网络的目标跟踪,进一步降低深度特征通道之间的冗余性,并减少高层信息丢失。利用线下预训练的VGG-16卷积神经网络提取深度特征,构成初始深度特征空间。通过区域损失函数构建特征和尺度选择网络,根据反向传播的梯度大小进行特征选择。对筛选后的特征进行拼接,融入到孪生网络中匹配跟踪。在OTB-2013、OTB-2015、VOT2016、TempleColor数据集上与其他算法对比。实验结果表明,该算法在快速运动、低分辨率等场景中表现出较好的跟踪精度和鲁棒性。     

Appearance‐based landmark selection for visual localization

Visual localization in outdoor environments is subject to varying appearance conditions rendering it difficult to match current camera images against a previously recorded map. Although it is possible to extend the respective maps to allow precise localization across a wide range of differing appearance conditions, these maps quickly grow in size and become impractical to handle on a mobile robotic platform. To address this problem, we present a landmark selection algorithm that exploits appearance co‐observability for efficient visual localization in outdoor environments. Based on the appearance condition inferred from recently observed landmarks, a small fraction of landmarks useful under the current appearance condition is selected and used for localization. This allows to greatly reduce the bandwidth consumption between the mobile platform and a map backend in a shared‐map scenario, and significantly lowers the demands on the computational resources on said mobile platform. We derive a landmark ranking function that exhibits high performance under vastly changing appearance conditions and is agnostic to the distribution of landmarks across the different map sessions. Furthermore, we relate and compare our proposed appearance‐based landmark ranking function to popular ranking schemes from information retrieval, and validate our results on the challenging University of Michigan North Campus long‐term vision and LIDAR data sets (NCLT), including an evaluation of the localization accuracy using ground‐truth poses. In addition to that, we investigate the computational and bandwidth resource demands. Our results show that by selecting 20–30% of landmarks using our proposed approach, a similar localization performance as the baseline strategy using all landmarks is achieved.     

Autonomous Salient Feature Detection through Salient Cues in an HSV Color Space for Visual Indoor Simultaneous Localization and Mapping

For successful simultaneous localization and mapping (SLAM), perception of the environment is important. This paper proposes a scheme to autonomously detect visual features that can be used as natural landmarks for indoor SLAM. First, features are roughly selected from the camera image through entropy maps that measure the level of randomness of pixel information. Then, the saliency of each pixel is computed by measuring the level of similarity between the selected features and the given image. In the saliency map, it is possible to distinguish the salient features from the background. The robot estimates its pose by using the detected features and builds a grid map of the unknown environment by using a range sensor. The feature positions are stored in the grid map. Experimental results show that the feature detection method proposed in this paper can autonomously detect features in unknown environments reasonably well.     

Autonomous sub-image matching for two-dimensional electrophoresis gels using MaxRST algorithm

Matching two-dimensional electrophoresis (2-DE) gel images typically generates a bottleneck in the automated protein analysis, and image distortion and experimental variation, which reduce the matching accuracy. However, conventional matching schemes only compare two complete images, and landmark selection and registration procedures are rather time-consuming. This work presents a novel and robust Maximum Relation Spanning Tree (MaxRST) algorithm, in which an autonomous sub-image matching method does not require registering or manual selection of landmarks. The 2D gel images are represented graphically. Image features are then quantitatively extracted regardless of image size. Similarity between a sub-image and large image is then determined based on Gaussian similarity measurement inspired by fuzzy method, thereby increasing the accuracy of fractional matching. The proposed autonomous matching algorithm achieves an accuracy of up to 97.29% when matching 627 2-DE gel test images. In addition to accommodating image rotation, reversals, shape deformation and intensity changes, the proposed algorithm effectively addresses the sub-image mapping problem and was analyzed thoroughly using a large dataset containing 4629 images. The contributions of this work are twofold. First, this work presents a novel MaxRST strategy and autonomous matching method that does not require manual landmark selection. Second, the proposed method, which extends 2-DE gel matching to query sub-image and a database containing large sets of images, can be adopted for mapping and locating, and to compare small gel images with large gel images with robustness and efficiency.     

ARCosmetics: a real-time augmented reality cosmetics try-on system

A virtual cosmetics try-on system provides a realistic try-on experience for consumers and helps them efficiently choose suitable cosmetics. In this article, we propose a real-time augmented reality virtual cosmetics try-on system for smartphones (ARCosmetics), taking speed, accuracy, and stability into consideration at each step to ensure a better user experience. A novel and very fast face tracking method utilizes the face detection box and the average position of facial landmarks to estimate the faces in continuous frames. A dynamic weight Wing loss is introduced to assign a dynamic weight to every landmark by the estimated error during training. It balances the attention between small, medium, and large range error and thus increases the accuracy and robustness. We also designed a weighted average method to utilize the information of the adjacent frame for landmark refinement, guaranteeing the stability of the generated landmarks. Extensive experiments conducted on a large 106-point facial landmark dataset and the 300-VW dataset demonstrate the superior performance of the proposed method compared to other state-of-the-art methods. We also conducted user satisfaction studies further to verify the efficiency and effectiveness of our ARCosmetics system.     

Robust outdoor stereo vision SLAM for heavy machine rotation sensing

The paper presents a robust outdoor stereo vision simultaneous localization and mapping (SLAM) algorithm. It estimates camera pose reliably in outdoor environments with directional sunlight illumination causing shadows and non-uniform scene lighting. The algorithm has been developed to measure a mining rope shovel’s rotation angle about its vertical axis (“swing” axis). A stereo camera is mounted externally to the shovel house (upper revolvable portion of the shovel), with a clear view of the shovel’s lower carbody. As the shovel house swings, the camera revolves with the shovel house in a planar circular orbit, seeing differing views of the carbody top. During the swing, the SLAM algorithm builds a map of observed 3D features on the carbody and simultaneously using these landmarks to estimate the camera position. This estimated camera position is then used to compute the shovel swing angle. Two novel techniques are employed to improve the SLAM algorithm’s robustness in outdoor environments. First, a “Locally Maximal” feature selection technique for Harris corners is used to select features more consistently in non-uniformly illuminated scenes. Another novel technique is the use of 3D “Feature Clusters” as SLAM landmarks rather than individual single features. The Feature Cluster landmarks improve the robustness of the landmark matching and allow significant reduction of the SLAM filter computational cost. This approach of estimating the shovel swing angle has a maximum error of ±1° upon SLAM map convergence. Results demonstrate the improvements of using the novel techniques compared to previous methods.     

Exploiting contrast cues for salient region detection

Visual saliency detection is an important cue used in human visual system, which can offer efficient solutions for both biological and artificial vision systems. Although there are many saliency detection models that can achieve good results on public datasets, the accuracy and reliability of salient object detection models still remains a challenge. For this reason, a novel effective salient region detection model is presented in this paper. Based on the principle that a combination of global statistics and surrounding contrast saliency operators can yield even better results than just using either alone, we use a histogram-based contrast method to calculate the global saliency values in an opponent color space. At the same time, we partition the input image into a set of regions, and the regional saliency is detected by considering the color isolation with spatial information and textural distinctness simultaneously. The final saliency is obtained based on a weighted fusion of the two saliency results. The experimental results from three widely used databases validate the efficacy of the proposed method in comparison with fourteen state-of-the-art existing methods.     

Learning to select distinctive landmarks for mobile robot navigation

In landmark-based navigation systems for mobile robots, sensory perceptions (e.g., laser or sonar scans) are used to identify the robot’s current location or to construct internal representations, maps, of the robot’s environment. Being based on an external frame of reference (which is not subject to incorrigible drift errors such as those occurring in odometry-based systems), landmark-based robot navigation systems are now widely used in mobile robot applications.The problem that has attracted most attention to date in landmark-based navigation research is the question of how to deal with perceptual aliasing, i.e., perceptual ambiguities. In contrast, what constitutes a good landmark, or how to select landmarks for mapping, is still an open research topic. The usual method of landmark selection is to map perceptions at regular intervals, which has the drawback of being inefficient and possibly missing ‘good’ landmarks that lie between sampling points.In this paper, we present an automatic landmark selection algorithm that allows a mobile robot to select conspicuous landmarks from a continuous stream of sensory perceptions, without any pre-installed knowledge or human intervention during the selection process. This algorithm can be used to make mapping mechanisms more efficient and reliable. Experimental results obtained with two different mobile robots in a range of environments are presented and analysed.     

基于视觉显著性的均值漂移跟踪算法

为解决突变运动下的目标跟踪问题,提出了一种基于视觉显著性的均值漂移跟踪算法,将视觉注意机制运用到均值漂移跟踪框架中,利用时空显著性算法对视频序列进行检测,生成视觉显著图,从视觉显著图对应的显著性区域中建立目标的颜色特征表示模型来实现运动目标跟踪.实验结果表明:该算法在摄像机摇晃等动态场景下可以较准确检测出时空均显著的目标,有效克服了在运动目标发生丢失和遮挡等情况下跟踪不稳定的问题,具有较强的鲁棒性,从而实现复杂场景下目标较准确的跟踪.     

大规模图数据边受限制的最短距离查询算法

计算两点之间的最短距离是标记图的基本操作之一。对于大图,根据路标节点估算两点之间最短距离的方法来提高查询效率。现有的路标节点选择策略不能在中心性和计算量小两方面同时满足,路标节点存储到其他节点的距离信息,存储量仍然很大。对于大规模有向图来说,路标节点选取策略保证中心性的同时减少了计算量,使用了DBSCAN聚类思想将节点划分成不同的类,选择具有联通性的向前和向后核心节点作为向前和向后路标节点;存储类内路标节点与普通节点之间的距离信息以及类间路标节点之间的距离信息来减少存储量;源节点通过向后路标节点和向前路标节点到达目标节点,采用上界和下界的最小均值作为估计值。理论证明算法策略在时间复杂度和空间复杂度方面与传统方法相比降低了。实验证明对于大图在平均相对误差方面与传统方法误差数量级相同。     

Recursive scan-matching SLAM

This paper presents Scan-SLAM, a new generalization of simultaneous localization and mapping (SLAM). SLAM implementations based on extended Kalman filter (EKF) data fusion have traditionally relied on simple geometric models for defining landmarks. This limits EKF-SLAM to environments suited to such models and tends to discard much potentially useful data. The approach presented in this paper is a marriage of EKF-SLAM and scan correlation. Landmarks are no longer defined by analytical models; instead they are defined by templates composed of raw sensed data. These templates can be augmented as more data become available so that the landmark definition improves with time. A new generic observation model is derived that is generated by scan correlation, and this permits stochastic location estimation for landmarks with arbitrary shape within the Kalman filter framework. The statistical advantages of an EKF representation are augmented with the general applicability of scan matching. Scan matching also serves to enhance data association reliability by providing a shape metric for landmark disambiguation. Experimental results in an outdoor environment are presented which validate the algorithm.     

Vision-based localization algorithm based on landmark matching, triangulation, reconstruction, and comparison

Many generic position-estimation algorithms are vulnerable to ambiguity introduced by nonunique landmarks. Also, the available high-dimensional image data is not fully used when these techniques are extended to vision-based localization. This paper presents the landmark matching, triangulation, reconstruction, and comparison (LTRQ global localization algorithm, which is reasonably immune to ambiguous landmark matches. It extracts natural landmarks for the (rough) matching stage before generating the list of possible position estimates through triangulation. Reconstruction and comparison then rank the possible estimates. The LTRC algorithm has been implemented using an interpreted language, onto a robot equipped with a panoramic vision system. Empirical data shows remarkable improvement in accuracy when compared with the established random sample consensus method. LTRC is also robust against inaccurate map data.