Cross-media retrieval with collective deep semantic learning

Multimedia Tools and Applications - Cross-media retrieval is becoming a new trend of information retrieval technique. It has been received great attentions from both academia and industry. In this...     

Unrestricted deep metric learning using neural networks interaction

Pattern Analysis and Applications - In many machine learning applications and algorithms, the algorithm performance and accuracy are highly dependent on the metric used to measure the distance...     

结合深度度量学习的血缘关系识别

目的 人脸图像分析是计算机视觉和模式识别领域的重要研究方向之一,基于人脸图像的血缘关系识别是对给定的一对或一组人脸图像,判断其是否存在某种血缘关系。人脸血缘关系识别不仅在生物特征识别领域有着重要研究价值,而且在社交媒体挖掘、失散家庭成员寻找等社会生活领域中有重要的应用价值。针对当前大多数算法都是基于传统机器学习方法,提出一种采用深度度量学习进行人脸图像血缘关系研究的新方法。方法 目前深度学习算法能很好地理解单张人脸图像,但是多个主体间的关系探究仍然是计算机视觉领域富有挑战性的问题之一。为此,提出一种基于深度度量学习的父母与子女的血缘关系识别方法。首先使用超过5 000 000张人脸图像的样本集训练一个深度卷积神经网络FaceCNN并提取父母与子女的人脸图像深度特征,之后引入判别性度量学习方法,使得具有血缘关系的特征尽可能地靠近,反之则尽可能地远离。然后对特征进行分层非线性变换使其具有更强判别特性。最后根据余弦相似度分别计算父亲、母亲和孩子的相似度并利用相似概率值得到双亲和孩子的综合相似度得分。结果 算法在TSKinFace数据集上验证了FaceCNN提取特征与深度度量学习结合进行血缘关系识别的有效性,最终在该数据集上父母与儿子和女儿的血缘关系识别准确率分别达到87.71%和89.18%,同时算法在进行血缘度量学习和双亲相似度计算仅需要3.616 s。结论 提出的血缘关系识别方法,充分利用深度学习网络良好的表征和学习能力,不仅耗时少,而且有效地提高了识别准确率。     

Semi-supervised clustering with deep metric learning and graph embedding

World Wide Web - As a common technology in social network, clustering has attracted lots of research interest due to its high performance, and many clustering methods have been presented. The most...     

Full-scaled deep metric learning for pedestrian re-identification

Multimedia Tools and Applications - The pedestrian re-identification problem (i.e., re-id) is essential and pre-requisite in multi-camera video surveillance studies, provided the fact that...     

深度度量学习综述

深度度量学习已成为近年来机器学习最具吸引力的研究领域之一,如何有效的度量物体间的相似性成为问题的关键。现有的依赖成对或成三元组的损失函数,由于正负样本可组合的数量极多,因此一种合理的解决方案是仅对训练有意义的正负样本采样,也称为“难例挖掘”。为减轻挖掘有意义样本时的计算复杂度,代理损失设置了数量远远小于样本集合的代理点集。该综述按照时间顺序,总结了深度度量学习领域比较有代表性的算法,并探讨了其与softmax分类的联系,发现两条看似平行的研究思路,实则背后有着一致的思想。进而文章探索了许多致力于提升softmax判别性能的改进算法,并将其引入到度量学习中,从而进一步缩小类内距离、扩大类间距,提高算法的判别性能。     

Robust deep multi-view subspace clustering networks with a correntropy-induced metric

Since multi-view subspace clustering combines the advantages of deep learning to capture the nonlinear nature of data, deep multi-view subspace clustering methods have demonstrated superior ability to shallow multi-view subspace clustering methods. Most existing methods assume that sample reconstruction errors incurred by noise conform to the prior distribution of the corresponding norm, allowing for simplification of the problem and focus on designing specific regularization on self-representation matrices to exploit consistent and diverse information among different views. However, the noise distributions in different views are always very complex, and in practice the noise distributions do not necessarily conform to this hypothesis. Furthermore, the commonly used diversity regularization based on value-awareness to enhance diversity among different view representations is not sufficiently accurate. To alleviate the above deficiencies, we propose novel robust deep multi-view subspace clustering networks with a correntropy-induced metric (RDMSCNet). (1) A correntropy-induced metric (CIM) is utilized to flexibly handle various complex noise distributions in a data-driven manner to improve the robustness of the model. (2) A position-aware diversity regularization based on the exclusivity definition is employed to enforce the diversity of the different view representations for modelling the consistency and diversity simultaneously. Extensive experiments show that RDMSCNet can deliver enhanced performance over state-of-the-art approaches.

     

MetricFusion: Generalized metric swarm learning for similarity measure

Learning distance metrics for measuring the similarity between two data points in unsupervised and supervised pattern recognition has been widely studied in unconstrained face verification tasks. Motivated by the fact that enforcing single distance metric learning for verification via an empirical score threshold is not robust in uncontrolled experimental conditions, we therefore propose to obtain a metric swarm by learning local patches alike sub-metrics simultaneously that naturally formulates a generalized metric swarm learning (GMSL) model with a joint similarity score function solved by an efficient alternative optimization algorithm. Further, each sample pair is represented as a similarity vector via the well-learned metric swarm, such that the face verification task becomes a generalized SVM-alike classification problem. Therefore, the verification can be enforced in the represented metric swarm space that can well improve the robustness of verification under irregular data structure. Experiments are preliminarily conducted using several UCI benchmark datasets for solving general classification problem. Further, the face verification experiments on real-world LFW and PubFig datasets demonstrate that our proposed model outperforms several state-of-the-art metric learning methods.     

Learning deep compact similarity metric for kinship verification from face images

Recent advances in kinship verification have shown that learning an appropriate kinship similarity metric on human faces plays a critical role in this problem. However, most of existing distance metric learning (DML) based solutions rely on linearity assumption of the kinship metric model, and the domain knowledge of large cross-generation discrepancy (e.g., large age span and gender difference between parent and child images) has not been considered in metric learning, leading to degraded performance for genetic similarity measure on human faces. To address these limitations, we propose in this work a new kinship metric learning (KML) method with a coupled deep neural network (DNN) model. KML explicitly models the cross-generation discrepancy inherent on parent-child pairs, and learns a coupled deep similarity metric such that the image pairs with kinship relation are pulled close, while those without kinship relation (but with high appearance similarity) are pushed as far away as possible. Moreover, by imposing the intra-connection diversity and inter-connection consistency over the coupled DNN, we introduce the property of hierarchical compactness into the coupled network to facilitate deep metric learning with limited amount of kinship training data. Empirically, we evaluate our algorithm on several kinship benchmarks against the state-of-the-art DML alternatives, and the results demonstrate the superiority of our method.     

Self-supervised deep metric learning for ancient papyrus fragments retrieval

International Journal on Document Analysis and Recognition (IJDAR) - This work focuses on document fragments association using deep metric learning methods. More precisely, we are interested in...     

Cell tracking using deep neural networks with multi-task learning

Cell tracking plays crucial role in biomedical and computer vision areas. As cells generally have frequent deformation activities and small sizes in microscope image, tracking the non-rigid and non-significant cells is quite difficult in practice. Traditional visual tracking methods have good performances on tracking rigid and significant visual objects, however, they are not suitable for cell tracking problem. In this paper, a novel cell tracking method is proposed by using Convolutional Neural Networks (CNNs) as well as multi-task learning (MTL) techniques. The CNNs learn robust cell features and MTL improves the generalization performance of the tracking. The proposed cell tracking method consists of a particle filter motion model, a multi-task learning observation model, and an optimized model update strategy. In the training procedure, the cell tracking is divided into an online tracking task and an accompanying classification task using the MTL technique. The observation model is trained by building a CNN to learn robust cell features. The tracking procedure is started by assigning the cell position in the first frame of a microscope image sequence. Then, the particle filter model is applied to produce a set of candidate bounding boxes in the subsequent frames. The trained observation model provides the confidence probabilities corresponding to all of the candidates and selects the candidate with the highest probability as the final prediction. Finally, an optimized model update strategy is proposed to enable the multi-task observation model for the variation of the tracked cell over the entire tracking procedure. The performance and robustness of the proposed method are analyzed by comparing with other commonly-used methods. Experimental results demonstrate that the proposed method has good performance to the cell tracking problem.     

Siamese transformer network-based similarity metric learning for cross-source remote sensing image retrieval

Neural Computing and Applications - As a fundamental technique for mining and analysis of remote sensing (RS) big data, content-based remote sensing image retrieval (CBRSIR) has received a lot of...     

On the similarity metric and the distance metric

Similarity and dissimilarity measures are widely used in many research areas and applications. When a dissimilarity measure is used, it is normally required to be a distance metric. However, when a similarity measure is used, there is no formal requirement. In this article, we have three contributions. First, we give a formal definition of similarity metric. Second, we show the relationship between similarity metric and distance metric. Third, we present general solutions to normalize a given similarity metric or distance metric.     

CSD: A multi-user similarity metric for community recommendation in online social networks

Communities are basic components in networks. As a promising social application, community recommendation selects a few items (e.g., movies and books) to recommend to a group of users. It usually achieves higher recommendation precision if the users share more interests; whereas, in plenty of communities (e.g., families, work groups), the users often share few. With billions of communities in online social networks, quickly selecting the communities where the members are similar in interests is a prerequisite for community recommendation. To this end, we propose an easy-to-compute metric, Community Similarity Degree (CSD), to estimate the degree of interest similarity among multiple users in a community. Based on 3460 emulated Facebook communities, we conduct extensive empirical studies to reveal the characteristics of CSD and validate the effectiveness of CSD. In particular, we demonstrate that selecting communities with larger CSD can achieve higher recommendation precision. In addition, we verify the computation efficiency of CSD: it costs less than 1 hour to calculate CSD for over 1 million of communities. Finally, we draw insights about feasible extensions to the definition of CSD, and point out the practical uses of CSD in a variety of applications other than community recommendation.     

基于字典学习的跨媒体检索技术

在研究跨媒体信息检索时,对于不同模态数据的异构性提出了挑战,针对如何更好的克服异构问题以提高多模态数据之间的检索精度,提出了一种基于字典学习的新跨媒体检索技术。首先,通过字典学习方法学习两个不同模态数据之间的稀疏系数,然后,通过特征映射方案由两个不同的投影矩阵分别把它们投入共同的特征子空间,最后,通过标签对齐同一类来增强不同模态之间的相关性。实验结果表明,与传统的同构子空间学习方法相比,基于字典的算法分类性能优越,该实验方法在两个数据集上优于几种最先进的方法。     

Locally adaptive subspace and similarity metric learning for visual data clustering and retrieval

Subspace and similarity metric learning are important issues for image and video analysis in the scenarios of both computer vision and multimedia fields. Many real-world applications, such as image clustering/labeling and video indexing/retrieval, involve feature space dimensionality reduction as well as feature matching metric learning. However, the loss of information from dimensionality reduction may degrade the accuracy of similarity matching. In practice, such basic conflicting requirements for both feature representation efficiency and similarity matching accuracy need to be appropriately addressed. In the style of “Thinking Globally and Fitting Locally”, we develop Locally Embedded Analysis (LEA) based solutions for visual data clustering and retrieval. LEA reveals the essential low-dimensional manifold structure of the data by preserving the local nearest neighbor affinity, and allowing a linear subspace embedding through solving a graph embedded eigenvalue decomposition problem. A visual data clustering algorithm, called Locally Embedded Clustering (LEC), and a local similarity metric learning algorithm for robust video retrieval, called Locally Adaptive Retrieval (LAR), are both designed upon the LEA approach, with variations in local affinity graph modeling. For large size database applications, instead of learning a global metric, we localize the metric learning space with kd-tree partition to localities identified by the indexing process. Simulation results demonstrate the effective performance of proposed solutions in both accuracy and speed aspects.     

三元组深度哈希学习的司法案例相似匹配方法

在数量庞大的司法案例文书中进行相似案例匹配可以有效地提升司法部门的工作效率。但司法案件文本不仅长,而且文本自身还具有一定程度的结构复杂性,因此司法案例文本匹配与传统自然语言处理任务相比,具有较高的难度。为解决上述问题,本文基于三元组深度哈希学习模型提出了一种司法案例相似匹配方法,首先使用预训练的BERT中文模型分组提取文书的特征;再利用文书三元组相似性关系,训练深度神经网络模型,用于生成文书的哈希码表示;最后,基于文书哈希码的海明距离来判断是否为相似案例。实验结果表明,本文采用哈希学习方法极大地降低了文书特征表示的存储开销,提高了相似案例匹配的速度。     

Image interpolation using convolutional neural networks with deep recursive residual learning

Multimedia Tools and Applications - Recent developments of image super-resolution often utilize the deep convolutional neural network (CNN) and residual learning to relate the observed...