On the Repeatability and Quality of Keypoints for Local Feature-based 3D Object Retrieval from Cluttered Scenes

3D object recognition from local features is robust to occlusions and clutter. However, local features must be extracted from a small set of feature rich keypoints to avoid computational complexity and ambiguous features. We present an algorithm for the detection of such keypoints on 3D models and partial views of objects. The keypoints are highly repeatable between partial views of an object and its complete 3D model. We also propose a quality measure to rank the keypoints and select the best ones for extracting local features. Keypoints are identified at locations where a unique local 3D coordinate basis can be derived from the underlying surface in order to extract invariant features. We also propose an automatic scale selection technique for extracting multi-scale and scale invariant features to match objects at different unknown scales. Features are projected to a PCA subspace and matched to find correspondences between a database and query object. Each pair of matching features gives a transformation that aligns the query and database object. These transformations are clustered and the biggest cluster is used to identify the query object. Experiments on a public database revealed that the proposed quality measure relates correctly to the repeatability of keypoints and the multi-scale features have a recognition rate of over 95% for up to 80% occluded objects.     

改进仿射尺度不变特征变换算法的图像配准

为了更好地处理匹配效率、重复纹理匹配和仿射不变性匹配等问题,对完全仿射不变特征变换（ASIFT）算法进行两方面改进。匹配框架中特征提取的改进提高了ASIFT算法的匹配效率;利用优化随机采样算法（ORSA）结合以单应矩阵为几何线性约束模型的随机抽样一致性（RANSAC）改进匹配算法,提高了匹配精度和重复纹理结构的适应能力。实验结果表明,提出的改进算法能较好地匹配高度相似纹理,计算量小,计算速度快且精度高。     

Loop Closure Detection via Locality Preserving Matching With Global Consensus

A critical component of visual simultaneous localization and mapping is loop closure detection (LCD), an operation judging whether a robot has come to a pre-visited area. Concretely, given a query image (i.e., the latest view observed by the robot), it proceeds by first exploring images with similar semantic information, followed by solving the relative relationship between candidate pairs in the 3D space. In this work, a novel appearance-based LCD system is proposed. Specifically, candidate frame selection is conducted via the combination of Super-features and aggregated selective match kernel (ASMK). We incorporate an incremental strategy into the vanilla ASMK to make it applied in the LCD task. It is demonstrated that this setting is memory-wise efficient and can achieve remarkable performance. To dig up consistent geometry between image pairs during loop closure verification, we propose a simple yet surprisingly effective feature matching algorithm, termed locality preserving matching with global consensus (LPM-GC). The major objective of LPM-GC is to retain the local neighborhood information of true feature correspondences between candidate pairs, where a global constraint is further designed to effectively remove false correspondences in challenging sceneries, e.g., containing numerous repetitive structures. Meanwhile, we derive a closed-form solution that enables our approach to provide reliable correspondences within only a few milliseconds. The performance of the proposed approach has been experimentally evaluated on ten publicly available and challenging datasets. Results show that our method can achieve better performance over the state-of-the-art in both feature matching and LCD tasks. We have released our code of LPM-GC at https://github.com/jiayi-ma/LPM-GC.      

基于局部显著特征的快速图像配准方法

针对SIFT算法在进行图像配准时存在提取特征点数目大、无法精确控制、运算速度慢、配准点精度不高的问题,提出一种基于局部显著特征的快速图像配准方法。该方法首先对原始图像和待配准图像进行降采样,对降采样图像分别提取SIFT特征点,并对特征点运用改进的K-means聚类算法进行聚类;然后利用聚类结果筛选聚类区域,在各聚类区域提取显著特征点进行粗匹配;最后利用显著特征点在原始图像中定位显著区域,对所得显著区域进行精配准。实验结果表明,该方法减少了图像匹配时间,控制了特征点数量,在保证匹配准确度的同时,有效地提高了特征匹配的效率。     

基于显著图的SIFT特征检测与匹配

基于尺度不变特征变换(SIFT)特征的图像匹配存在特征点数量大、运算时间长等问题。为此,引入视觉注意机制,提出一种基于显著图的SIFT特征检测与匹配方法。比较常用的显著图计算模型,选择谱残差方法提取图片的显著图。对显著图进行二值化和形态学等处理,得到规则合理的显著区域。在显著区域内提取SIFT特征,生成特征向量,进行图像匹配。实验结果表明,该方法能提高运算效率,并且得到的SIFT特征更加稳定。     

基于视觉约束能量最小化的特征点匹配算法

为了解决海上目标航空遥感图像重复特征较多导致的匹配不一致问题,并简化匹配过程,文中提出了基于SIFT视觉约束能量最小化的匹配算法(CEM-SIFT)。该算法将约束能量最小化模型应用于特征点的匹配,通过构造有限脉冲响应线性滤波器,采用视觉信息计算其能量值,使得待匹配的点集经过滤波之后的平均输出能量在一定约束下达到最小值,最终实现含重复信息的特征精确匹配。采用10组航空遥感海冰图像对算法进行测试,结果表明,相对于采用SIFT欧氏距离(ED-SIFT),在匹配重复特征比较多、点集规模比较大的图像时,CEM-SIFT算法的匹配精度更高,能够达到100%。     

Towards reliable matching of images containing repetitive patterns

This paper aims to solve the problem of matching images containing repetitive patterns. Although repetitive patterns widely exist in real world images, these images are difficult to be matched due to local ambiguities even if the viewpoint changes are not very large. It is still an open and challenging problem. To solve the problem, this paper proposes to match pairs of interest points and then obtain point correspondences from the matched pairs of interest points based on the low distortion constraint, which is meant that the distortions of point groups should be small across images. By matching pairs of interest points, local ambiguities induced by repetitive patterns can be reduced to some extent since information in a much larger region is used. Meanwhile, owing to our newly defined compatibility measure between one correspondence and a set of point correspondences, the obtained point correspondences are very reliable. Experiments have demonstrated the effectiveness of our method and its superiority to the existing methods.     

Sequential Monte Carlo for Bayesian matching of objects with occlusions

We consider the problem of locating instances of a known object in a novel scene by matching the fiducial features of the object. The appearance of the features and the shape of the object are modeled separately and combined in a Bayesian framework. In this paper, we present a novel matching scheme based on sequential Monte Carlo, in which the features are matched sequentially, utilizing the information about the locations of previously matched features to constrain the task. The particle representation of hypotheses about the object position allow matching in multimodal and cluttered environments, where batch algorithms may have convergence difficulties. The proposed method requires no initialization or predetermined matching order, as the sequence can be started from any feature. We also utilize a Bayesian model to deal with features that are not detected due to occlusions or abnormal appearance. In our experiments, the proposed matching system shows promising results, with performance equal to batch approaches when the target distribution is unimodal, while surpassing traditional methods under multimodal conditions. Using the occlusion model, the object can be localized from only a few visible features, with the nonvisible parts predicted from the conditional prior model.     

基于全卷积网络的中小目标检索方法(201909撤稿)

基于预训练卷积神经网络(convolutional neural networks, CNNs)的图像表示已成为图像检索任务中一种新的方法,但是这种图像表示方法均是对图像的整体特征表示,无法适用于目标仅占被检索图像的部分区域的检索.为了解决该问题,提出一种基于全卷积网络的中小目标检索方法,该方法将预训练全卷积网络应用于目标较小、仅占被检索图像部分区域的检索.1)利用全卷积网络对输入图像大小不受限制的优势,给定被检索图像,经过全卷积网络得到该图像的特征矩阵表示;2)给定查询目标图像,利用全卷积神经网络,得到目标图像的特征表示;3)将目标特征,与被检索图像的特征矩阵的每一个特征进行相似性比对,得到相似值和匹配最优位置.进一步引入多尺度、多比例变换以适用不同大小的实例目标.在标准数据集Oxford5K上的实验表明:该算法的检索性能优于现有算法.另外,在搜集的Logo数据集,该算法得到了不错的检索效果,进一步验证了算法的普适性和有效性.     

Real-time SLAM relocalization with online learning of binary feature indexing

A visual simultaneous localization and mapping (SLAM) system usually contains a relocalization module to recover the camera pose after tracking failure. The core of this module is to establish correspondences between map points and key points in the image, which is typically achieved by local image feature matching. Since recently emerged binary features have orders of magnitudes higher extraction speed than traditional features such as scale invariant feature transform, they can be applied to develop a real-time relocalization module once an efficient method of binary feature matching is provided. In this paper, we propose such a method by indexing binary features with hashing. Being different from the popular locality sensitive hashing, the proposed method constructs the hash keys by an online learning process instead of pure randomness. Specifically, the hash keys are trained with the aim of attaining uniform hash buckets and high collision rates of matched feature pairs, which makes the method more efficient on approximate nearest neighbor search. By distributing the online learning into the simultaneous localization and mapping process, we successfully apply the method to SLAM relocalization. Experiments show that camera poses can be recovered in real time even when there are tens of thousands of landmarks in the map.     

基于多尺度残差网络的单应估计方法

单应估计是许多计算机视觉任务中一个基础且重要的步骤。传统单应估计方法基于特征点匹配,难以在弱纹理图像中工作。深度学习已经应用于单应估计以提高其鲁棒性,但现有方法均未考虑到由于物体尺度差异导致的多尺度问题,所以精度受限。针对上述问题,提出了一种用于单应估计的多尺度残差网络。该网络能够提取图像的多尺度特征信息,并使用多尺度特征融合模块对特征进行有效融合,此外还通过估计四角点归一化偏移进一步降低了网络优化难度。实验表明,在MS-COCO数据集上,该方法平均角点误差仅为0.788个像素,达到了亚像素级的精度,并且在99%情况下能够保持较高的精度。由于综合利用了多尺度特征信息且更容易优化,该方法精度显著提高,并具有更强的鲁棒性。     

基于语义引导特征聚合的显著性目标检测网络

在显著性目标检测网络的设计中, U型结构使用广泛. 但是在U型结构显著性检测方法中, 普遍存在空间位置细节丢失和边缘难以细化的问题, 针对这些问题, 提出一种基于语义信息引导特征聚合的显著性目标检测网络, 通过高效的特征聚合来获得精细的显著性图. 该网络由混合注意力模块(Mixing attention module, MAM)、增大感受野模块(Enlarged receptive field module, ERFM)和多层次聚合模块(Multi-level aggregation module, MLAM)三个部分组成. 首先, 利用增大感受野模块处理特征提取网络提取出的低层特征, 使其在保留原有边缘细节的同时增大感受野, 以获得更加丰富的空间上/下文信息; 然后, 利用混合注意力模块处理特征提取网络的最后一层特征, 以增强其表征力, 并作为解码过程中的语义指导, 不断指导特征聚合; 最后, 多层次聚合模块对来自不同层次的特征进行有效聚合, 得到最终精细的显著性图. 在6个基准数据集上进行了实验, 结果验证了该方法能够有效地定位显著特征, 并且对边缘细节的细化也很有效.     

鲁棒且快速的特征点匹配算法

针对大部分基于特征的立体匹配速度过慢的问题,提出一种在频域下提取特征点坐标、空间域下提取特征描 述子的算法。首先,研究了图像的有效编码理论;其次,确定图像的显著性特征点坐标及其尺度;最后,构造适应特征 点尺度的模板提取图像的特征,用最近部法则进行特征点的匹配。实验结果表明,该算法效率高、计算快,同时也具有 较强的尺度及仿射变换鲁棒性,在速度与性能上达到了一个很好的平衡点。     

多尺度Transformer与层次化边界引导的显著性目标检测

针对显著性目标检测算法中全局和局部信息难以联合表征和目标边界难以细化的问题,提出了一种多尺度Transformer与层次化边界引导的显著性目标检测算法。首先,构建Transformer模型提取全局信息,同时通过自注意力机制获取有判别性的浅层局部特征,对全局和局部信息进行联合表征。然后,引入Tokens-to-Token方法提取多尺度特征,使模型实现尺度变换平滑的编解码。进一步,提出了一种层次化的边界学习策略,引导模型在每个解码特征层提取精细化的显著性目标边界特征,提升显著性目标边界的预测准确性。实验结果表明,提出的算法在四个公开显著性目标检测数据集上均优于八种主流的显著性目标检测算法,并且通过消融实验验证了提出模型和边界学习策略的有效性。     

单应性矩阵自适应估计方法

如何从初始匹配点集中估计出精确的单应性矩阵,有效地剔除误匹配,一直以来都是视觉领域研究的重点和难点,也是实际相关技术应用中最为关键的一步。通过将特征点对相似度概念应用于LMedS的样本选取过程,提出了一种新的单应性矩阵自适应的估计方法。区别于传统LMeds方法从无序匹配点集中随机选取样本的过程,该方法首先以点对间的相似度对整个初始匹配点进行降序排列,然后从前往后依次选取样本。实验结果表明,与LMedS相比,该方法估计出的单应性矩阵更精确、鲁棒,效率更高（得到最佳模型所需的迭代次数仅约为LMedS的1/5）,同时弥补了RANSAC及其改进方法需预先设置距离偏差阈值的不足。     

数据更新中要素变化检测与匹配方法

在要素类之间缺乏同名实体关联关系的情况下,通过空间分析自动识别出当前要素的同名实体及它们之间的变化信息。在查询当前要素的候选匹配集时,设计了一种基于自定义空间拓扑关系的空间查询方法,缩小了的空间查询范围并减少了查询次数,提高了空间分析的效率;在确定当前要素的同名实体时,提出了基于权重的空间要素相似性计算模型,基于该模型有效地对复杂空间关系下的要素进行了匹配,提高了匹配的准确性。     

基于随机抽样一致性的多平面区域检测算法

在随机抽样一致性(RANSAC)的基础上,提出了一种对多个平面区域同时进行检测的算法.该算法假设对同一场景的一对未定标图像已经进行了特征点提取和匹配,首先利用对极几何约束计算出一对极点,然后随机抽取多组3对而非4对特征点定义多个待确定单应性矩阵模型,对图像对中的多个平面区域同时进行检测.模拟实验和真实实验都证明该算法具有运算量小、准确性高、鲁棒性好等优点.     

Depth incorporating with color improves salient object detection

Detecting salient objects in challenging images attracts increasing attention as many applications require more robust method to deal with complex images from the Internet. Prior methods produce poor saliency maps in challenging cases mainly due to the complex patterns in the background and internal color edges in the foreground. The former problem may introduce noises into saliency maps and the later forms the difficulty in determining object boundaries. Observing that depth map can supply layering information and more reliable boundary, we improve salient object detection by integrating two features: color information and depth information which are calculated from stereo images. The two features collaborate in a two-stage framework. In the object location stage, depth mainly helps to produce a noise-filtered salient patch, which indicates the location of the object. In the object boundary inference stage, boundary information is encoded in a graph using both depth and color information, and then we employ the random walk to infer more reliable boundaries and obtain the final saliency map. We also build a data set containing 100+ stereo pairs to test the effectiveness of our method. Experiments show that our depth-plus-color based method significantly improves salient object detection compared with previous color-based methods.