Cross-media retrieval with collective deep semantic learning

Multimedia Tools and Applications - Cross-media retrieval is becoming a new trend of information retrieval technique. It has been received great attentions from both academia and industry. In this...     

Unrestricted deep metric learning using neural networks interaction

Pattern Analysis and Applications - In many machine learning applications and algorithms, the algorithm performance and accuracy are highly dependent on the metric used to measure the distance...     

Semi-supervised clustering with deep metric learning and graph embedding

World Wide Web - As a common technology in social network, clustering has attracted lots of research interest due to its high performance, and many clustering methods have been presented. The most...     

Full-scaled deep metric learning for pedestrian re-identification

Multimedia Tools and Applications - The pedestrian re-identification problem (i.e., re-id) is essential and pre-requisite in multi-camera video surveillance studies, provided the fact that...     

Robust deep multi-view subspace clustering networks with a correntropy-induced metric

Since multi-view subspace clustering combines the advantages of deep learning to capture the nonlinear nature of data, deep multi-view subspace clustering methods have demonstrated superior ability to shallow multi-view subspace clustering methods. Most existing methods assume that sample reconstruction errors incurred by noise conform to the prior distribution of the corresponding norm, allowing for simplification of the problem and focus on designing specific regularization on self-representation matrices to exploit consistent and diverse information among different views. However, the noise distributions in different views are always very complex, and in practice the noise distributions do not necessarily conform to this hypothesis. Furthermore, the commonly used diversity regularization based on value-awareness to enhance diversity among different view representations is not sufficiently accurate. To alleviate the above deficiencies, we propose novel robust deep multi-view subspace clustering networks with a correntropy-induced metric (RDMSCNet). (1) A correntropy-induced metric (CIM) is utilized to flexibly handle various complex noise distributions in a data-driven manner to improve the robustness of the model. (2) A position-aware diversity regularization based on the exclusivity definition is employed to enforce the diversity of the different view representations for modelling the consistency and diversity simultaneously. Extensive experiments show that RDMSCNet can deliver enhanced performance over state-of-the-art approaches.

     

Self-supervised deep metric learning for ancient papyrus fragments retrieval

International Journal on Document Analysis and Recognition (IJDAR) - This work focuses on document fragments association using deep metric learning methods. More precisely, we are interested in...     

Siamese transformer network-based similarity metric learning for cross-source remote sensing image retrieval

Neural Computing and Applications - As a fundamental technique for mining and analysis of remote sensing (RS) big data, content-based remote sensing image retrieval (CBRSIR) has received a lot of...     

On the similarity metric and the distance metric

Similarity and dissimilarity measures are widely used in many research areas and applications. When a dissimilarity measure is used, it is normally required to be a distance metric. However, when a similarity measure is used, there is no formal requirement. In this article, we have three contributions. First, we give a formal definition of similarity metric. Second, we show the relationship between similarity metric and distance metric. Third, we present general solutions to normalize a given similarity metric or distance metric.     

CSD: A multi-user similarity metric for community recommendation in online social networks

Communities are basic components in networks. As a promising social application, community recommendation selects a few items (e.g., movies and books) to recommend to a group of users. It usually achieves higher recommendation precision if the users share more interests; whereas, in plenty of communities (e.g., families, work groups), the users often share few. With billions of communities in online social networks, quickly selecting the communities where the members are similar in interests is a prerequisite for community recommendation. To this end, we propose an easy-to-compute metric, Community Similarity Degree (CSD), to estimate the degree of interest similarity among multiple users in a community. Based on 3460 emulated Facebook communities, we conduct extensive empirical studies to reveal the characteristics of CSD and validate the effectiveness of CSD. In particular, we demonstrate that selecting communities with larger CSD can achieve higher recommendation precision. In addition, we verify the computation efficiency of CSD: it costs less than 1 hour to calculate CSD for over 1 million of communities. Finally, we draw insights about feasible extensions to the definition of CSD, and point out the practical uses of CSD in a variety of applications other than community recommendation.     

Locally adaptive subspace and similarity metric learning for visual data clustering and retrieval

Subspace and similarity metric learning are important issues for image and video analysis in the scenarios of both computer vision and multimedia fields. Many real-world applications, such as image clustering/labeling and video indexing/retrieval, involve feature space dimensionality reduction as well as feature matching metric learning. However, the loss of information from dimensionality reduction may degrade the accuracy of similarity matching. In practice, such basic conflicting requirements for both feature representation efficiency and similarity matching accuracy need to be appropriately addressed. In the style of “Thinking Globally and Fitting Locally”, we develop Locally Embedded Analysis (LEA) based solutions for visual data clustering and retrieval. LEA reveals the essential low-dimensional manifold structure of the data by preserving the local nearest neighbor affinity, and allowing a linear subspace embedding through solving a graph embedded eigenvalue decomposition problem. A visual data clustering algorithm, called Locally Embedded Clustering (LEC), and a local similarity metric learning algorithm for robust video retrieval, called Locally Adaptive Retrieval (LAR), are both designed upon the LEA approach, with variations in local affinity graph modeling. For large size database applications, instead of learning a global metric, we localize the metric learning space with kd-tree partition to localities identified by the indexing process. Simulation results demonstrate the effective performance of proposed solutions in both accuracy and speed aspects.     

Image interpolation using convolutional neural networks with deep recursive residual learning

Multimedia Tools and Applications - Recent developments of image super-resolution often utilize the deep convolutional neural network (CNN) and residual learning to relate the observed...     

Enhancing remote sensing image retrieval using a triplet deep metric learning network

ABSTRACT

With the rapid growing of remotely sensed imagery data, there is a high demand for effective and efficient image retrieval tools to manage and exploit such data. In this letter, we present a novel content-based remote sensing image retrieval (RSIR) method based on Triplet deep metric learning convolutional neural network (CNN). By constructing a Triplet network with metric learning objective function, we extract the representative features of the images in a semantic space in which images from the same class are close to each other while those from different classes are far apart. In such a semantic space, simple metric measures such as Euclidean distance can be used directly to compare the similarity of images and effectively retrieve images of the same class. We also investigate a supervised and an unsupervised learning methods for reducing the dimensionality of the learned semantic features. We present comprehensive experimental results on two public RSIR datasets and show that our method significantly outperforms state-of-the-art.     

Cross-media retrieval based on semi-supervised regularization and correlation learning

Multimedia Tools and Applications - As large scale multimedia data in heterogeneous spaces is flooding into the Internet, cross-media retrieval is becoming increasingly significant. In cross-media...     

A similarity metric for edge images

The performance of several discrepancy measures for the comparison of edge images is analyzed and a novel similarity metric aimed at overcoming their problems is proposed. The algorithm finds an optimal matching of the pixels between the images and estimates the error produced by this matching. The resulting Pixel Correspondence Metric (PCM) can take into account edge strength as well as the displacement of edge pixel positions in the estimation of similarity. A series of experimental tests shows the new metric to be a robust and effective tool in the comparison of edge images when a small localization error of the detected edges is allowed.     

Non-linear metric learning using pairwise similarity and dissimilarity constraints and the geometrical structure of data

The problem of clustering with side information has received much recent attention and metric learning has been considered as a powerful approach to this problem. Until now, various metric learning methods have been proposed for semi-supervised clustering. Although some of the existing methods can use both positive (must-link) and negative (cannot-link) constraints, they are usually limited to learning a linear transformation (i.e., finding a global Mahalanobis metric). In this paper, we propose a framework for learning linear and non-linear transformations efficiently. We use both positive and negative constraints and also the intrinsic topological structure of data. We formulate our metric learning method as an appropriate optimization problem and find the global optimum of this problem. The proposed non-linear method can be considered as an efficient kernel learning method that yields an explicit non-linear transformation and thus shows out-of-sample generalization ability. Experimental results on synthetic and real-world data sets show the effectiveness of our metric learning method for semi-supervised clustering tasks.     

Enhancing multimodal disaster tweet classification using state-of-the-art deep learning networks

During disasters, multimedia content on social media sites offers vital information. Reports of injured or deceased people, infrastructure destruction, and missing or found people are among the types of information exchanged. While several studies have demonstrated the importance of both text and picture content for disaster response, previous research has primarily concentrated on the text modality and not so much success with multi-modality. Latest research in multi-modal classification in disaster related tweets uses comparatively primitive models such as KIMCNN and VGG16. In this research work we have taken this further and utilized state-of-the-art models in both text and image classification to try and improve multi-modal classification of disaster related tweets. The research was conducted on two different classification tasks, first to detect if a tweet is informative or not, second to understand the response needed. The process of multimodal analysis is broken down by incorporating different methods of feature extraction from the textual data corpus and pre-processing the corresponding image corpus, then we use several classification models to train and predict the output and compare their performances while tweaking the parameters to improve the results. Models such as XLNet, BERT and RoBERTa in text classification and ResNet, ResNeXt and DenseNet in image classification were trained and analyzed. Results show that the proposed multimodal architecture outperforms models trained using a single modality (text or image alone). Also, it proves that the newer state-of-the-art models outperform the baseline models by a reasonable margin for both the classification tasks.

     

A unified group technology implementation using the backpropagation learning rule of neural networks

Two engineering problems in implementing Group Technology are part family formation and part classification. Regardless of the approach adopted for the formation and classification, a critical problem is how to maintain consistency. The consistency problem can be addressed most effectively if the formation and classification is a single procedure rather than two separate procedures. A feedforward neural network using the Backpropagation learning rule is adopted to automatically generate part families during the part classification process. The spontaneous generalization capability of the neural network is utilized in classifying the parts into the families and creating new families if necessary. A heuristic algorithm using the neural network is described with an illustrative example.     

Fast similarity metric for real-time template-matching applications

In this study, a visual similarity metric based on precision–recall graphs is presented as an alternative to the widely used Hausdorff distance (HD). Such metric, called maximum cardinality similarity metric, is computed between a reference shape and a test template, each one represented by a set of edge points. We address this problem using a bipartite graph representation of the relationship between the sets. The matching problem is solved using the Hopcroft–Karp algorithm, taking advantage of its low computational complexity. We present a comparison between our results and those obtained from applying the partial Hausdorff distance (PHD) to the same test sets. Similar results were found using both approaches for standard template-matching applications. Nevertheless, the proposed methodology is more accurate at determining the completeness of partial shapes under noise conditions. Furthermore, the processing time required by our methodology is lower than that required to compute the PHD, for a large set of points.