Document Image Retrieval through Word Shape Coding

This paper presents a document retrieval technique that is capable of searching document images without OCR (optical character recognition). The proposed technique retrieves document images by a new word shape coding scheme, which captures the document content through annotating each word image by a word shape code. In particular, we annotate word images by using a set of topological shape features including character ascenders/descenders, character holes, and character water reservoirs. With the annotated word shape codes, document images can be retrieved by either query keywords or a query document image. Experimental results show that the proposed document image retrieval technique is fast, efficient, and tolerant to various types of document degradation.     

Retrieval of machine-printed Latin documents through Word Shape Coding

This paper reports a document retrieval technique that retrieves machine-printed Latin-based document images through word shape coding. Adopting the idea of image annotation, a word shape coding scheme is proposed, which converts each word image into a word shape code by using a few shape features. The text contents of imaged documents are thus captured by a document vector constructed with the converted word shape code and word frequency information. Similarities between different document images are then gauged based on the constructed document vectors. We divide the retrieval process into two stages. Based on the observation that documents of the same language share a large number of high-frequency language-specific stop words, the first stage retrieves documents with the same underlying language as that of the query document. The second stage then re-ranks the documents retrieved in the first stage based on the topic similarity. Experiments show that document images of different languages and topics can be retrieved properly by using the proposed word shape coding scheme.     

Word spotting in historical printed documents using shape and sequence comparisons

Information spotting in scanned historical document images is a very challenging task. The joint use of the mechanical press and of human controlled inking introduced great variability in ink level within a book or even within a page. Consequently characters are often broken or merged together and thus become difficult to segment and recognize. The limitations of commercial OCR engines for information retrieval in historical document images have inspired alternative means of identification of given words in such documents. We present a word spotting method for scanned documents in order to find the word images that are similar to a query word, without assuming a correct segmentation of the words into characters. The connected components are first processed to transform a word pattern into a sequence of sub-patterns. Each sub-pattern is represented by a sequence of feature vectors. A modified Edit distance is proposed to perform a segmentation-driven string matching and to compute the Segmentation Driven Edit (SDE) distance between the words to be compared. The set of SDE operations is defined to obtain the word segmentations that are the most appropriate to evaluate their similarity. These operations are efficient to cope with broken and touching characters in words. The distortion of character shapes is handled by coupling the string matching process with local shape comparisons that are achieved by Dynamic Time Warping (DTW). The costs of the SDE operations are provided by the DTW distances. A sub-optimal version of the SDE string matching is also proposed to reduce the computation time, nevertheless it did not lead to a great decrease in performance. It is possible to enter a query by example or a textual query entered with the keyboard. Textual queries can be used to directly spot the word without the need to synthesize its image, as far as character prototype images are available. Results are presented for different documents and compared with other methods, showing the efficiency of our method.     

Probability based document clustering and image clustering using content-based image retrieval

Clustering of related or similar objects has long been regarded as a potentially useful contribution of helping users to navigate an information space such as a document collection. Many clustering algorithms and techniques have been developed and implemented but as the sizes of document collections have grown these techniques have not been scaled to large collections because of their computational overhead. To solve this problem, the proposed system concentrates on an interactive text clustering methodology, probability based topic oriented and semi-supervised document clustering. Recently, as web and various documents contain both text and large number of images, the proposed system concentrates on content-based image retrieval (CBIR) for image clustering to give additional effect to the document clustering approach. It suggests two kinds of indexing keys, major colour sets (MCS) and distribution block signature (DBS) to prune away the irrelevant images to given query image. Major colour sets are related with colour information while distribution block signatures are related with spatial information. After successively applying these filters to a large database, only small amount of high potential candidates that are somewhat similar to that of query image are identified. Then, the system uses quad modelling method (QM) to set the initial weight of two-dimensional cells in query image according to each major colour and retrieve more similar images through similarity association function associated with the weights. The proposed system evaluates the system efficiency by implementing and testing the clustering results with Dbscan and K-means clustering algorithms. Experiment shows that the proposed document clustering algorithm performs with an average efficiency of 94.4% for various document categories.     

Word spotting for historical documents

Searching and indexing historical handwritten collections are a very challenging problem. We describe an approach called word spotting which involves grouping word images into clusters of similar words by using image matching to find similarity. By annotating “interesting” clusters, an index that links words to the locations where they occur can be built automatically. Image similarities computed using a number of different techniques including dynamic time warping are compared. The word similarities are then used for clustering using both K-means and agglomerative clustering techniques. It is shown in a subset of the George Washington collection that such a word spotting technique can outperform a Hidden Markov Model word-based recognition technique in terms of word error rates. An erratum to this article can be found at     

基于层级匹配的维吾尔文关键词文档图像检索

为实现基于关键词的维吾尔文文档图像检索,提出一种基于由粗到细层级匹配的关键词文档图像检索方法。使用改进的投影切分法将经过预处理的文档图像切分成单词图像库,使用模板匹配对关键词进行粗匹配;在粗匹配的基础上,提取单词图像的方向梯度直方图(HOG)特征向量;通过支持向量机(SVM)分类器学习特征向量,实现关键词图像检索。在包含108张文档图像的数据库中进行实验,实验结果表明,检索准确率平均值为91.14%,召回率平均值为79.31%,该方法能有效实现基于关键词的维吾尔文文档图像检索。     

A segmentation-free word spotting method for historical printed documents

In this paper, a two-step segmentation-free word spotting method for historical printed documents is presented. The first step involves a minimum distance matching between a query keyword image and a document page image using keypoint correspondences. In the second step of the method, the matched keypoints on the document image serve as indicators for creating candidate image areas. The query keyword image is matched against the candidate image areas in order to properly estimate the bounding boxes of the detected word instances. The method is evaluated using two datasets of different languages and is compared against segmentation-free state-of-the-art methods. The experimental results show that the proposed method outperforms significantly the competitive approaches.     

基于视觉词典的单目视觉闭环检测算法

针对移动机器人单目视觉同步定位与地图构建中的闭环检测问题,文中设计一种基于视觉词典的闭环检测算法。算法对采集的每帧图像通过SURF进行特征提取,应用模糊K均值算法对检测的视觉特征向量进行分类,在线构建表征图像的视觉词典。为精确表征局部视觉特征与视觉单词间的相似关联,利用混合高斯模型建立视觉词典中的每一视觉单词的概率模型,实现图像基于视觉词典的概率向量表示,通过向量的内积来计算图像间的相似度。为保证闭环检测的成功率,应用贝叶斯滤波融合历史闭环检测与相似度信息来计算闭环假设的后验概率分布。另外,引入浅层记忆与深度记忆两种内存管理机制来保证算法执行的快速性。实验结果证明该方法的有效性。     

Document retrieval from compressed images

With the emergence of digital libraries, more and more documents are stored and transmitted through the Internet in the format of compressed images. It is of significant meaning to develop a system which is capable of retrieving documents from these compressed document images. Aiming at the popular compression standard-CCITT Group 4 which is widely used for compressing document images, we present an approach to retrieve the documents from CCITT Group 4 compressed document images in this paper. The black and white changing elements are extracted directly from the compressed document images to act as the feature pixels, and the connected components are detected simultaneously. Then the word boxes are bounded based on the merging of the connected components. Weighted Hausdorff distance is proposed to assign all of the word objects from both the query document and the document from database to corresponding classes by an unsupervised classifier, whereas the possible stop words are excluded. Document vectors are built by the occurrence frequency of the word object classes, and the pair-wise similarity of two document images is represented by the scalar product of the document vectors. Nine groups of articles pertaining to different domains are used to test the validity of the presented approach. Preliminary experimental results with the document images captured from students’ theses show that the proposed approach has achieved a promising performance.     

Spoken query based word spotting in digitized Tamil documents

This paper presents an integrated approach to spot the spoken keywords in digitized Tamil documents by combining word image matching and spoken word recognition techniques. The work involves the segmentation of document images into words, creation of an index of keywords, and construction of word image hidden Markov model (HMM) and speech HMM for each keyword. The word image HMMs are constructed using seven dimensional profile and statistical moment features and used to recognize a segmented word image for possible inclusion of the keyword in the index. The spoken query word is recognized using the most likelihood of the speech HMMs using the 39 dimensional mel frequency cepstral coefficients derived from the speech samples of the keywords. The positional details of the search keyword obtained from the automatically updated index retrieve the relevant portion of text from the document during word spotting. The performance measures such as recall, precision, and F-measure are calculated for 40 test words from the four groups of literary documents to illustrate the ability of the proposed scheme and highlight its worthiness in the emerging multilingual information retrieval scenario.     

Keyword-guided word spotting in historical printed documents using synthetic data and user feedback

In this paper, we propose a novel technique for word spotting in historical printed documents combining synthetic data and user feedback. Our aim is to search for keywords typed by the user in a large collection of digitized printed historical documents. The proposed method consists of the following stages: (1) creation of synthetic image words; (2) word segmentation using dynamic parameters; (3) efficient feature extraction for each word image and (4) a retrieval procedure that is optimized by user feedback. Experimental results prove the efficiency of the proposed approach.     

基于Word Spotting技术的蒙古文古籍图像检索中的特征选择

设计了一个基于word spotting技术的蒙古文《甘珠尔经》图像检索的系统框架。在充分分析了蒙古文《甘珠尔经》中手写单词图像特点的基础上,提出了采用轮廓特征、投影特征和笔划穿越数目来表示单词图像。在由5500个单词图像构成的数据集上进行对比实验,确定了最佳的特征组合,平均准确率（MAP）能达到78.79%,R-Precision能达到73.01%。实验结果表明,所选择的特征是合理的、有效的。     

Word searching in unconstrained layout using character pair coding

Word searching in non-structural layout such as graphical documents is a difficult task due to arbitrary orientations of text words and the presence of graphical symbols. This paper presents an efficient approach for word searching in documents of non-structural layout using an efficient indexing and retrieval approach. The proposed indexing scheme stores spatial information of text characters of a document using a character spatial feature table (CSFT). The spatial feature of text component is derived from the neighbor component information. The character labeling of a multi-scaled and multi-oriented component is performed using support vector machines. For searching purpose, the positional information of characters is obtained from the query string by splitting it into possible combinations of character pairs. Each of these character pairs searches the position of corresponding text in document with the help of CSFT. Next, the searched text components are joined and formed into sequence by spatial information matching. String matching algorithm is performed to match the query word with the character pair sequence in documents. The experimental results are presented on two different datasets of graphical documents: maps dataset and seal/logo image dataset. The results show that the method is efficient to search query word from unconstrained document layouts of arbitrary orientation.     

基于两方向动态时间规整的无分割手写汉字检测

中文文本布局复杂,汉字种类多,书写随意性大,因而手写汉字检测是一个很有挑战的问题。本文提出了一种无分割的手写中文文档字符检测的方法。该方法用SIFT定位文本中候选关键点,然后基于关键点位置和待查询汉字大小来确定候选字符的位置,最后用两个方向动态时间规整(Dynamic Time Warping, DTW)算法来筛选候选字符。实验结果表明,该方法能够在无需将文本分割为字符的情况下准确找到待查询的汉字,并且优于传统的基于DTW字符检测方法。     

Information retrieval in document image databases

With the rising popularity and importance of document images as an information source, information retrieval in document image databases has become a growing and challenging problem. In this paper, we propose an approach with the capability of matching partial word images to address two issues in document image retrieval: word spotting and similarity measurement between documents. First, each word image is represented by a primitive string. Then, an inexact string matching technique is utilized to measure the similarity between the two primitive strings generated from two word images. Based on the similarity, we can estimate how a word image is relevant to the other and, thereby, decide whether one is a portion of the other. To deal with various character fonts, we use a primitive string which is tolerant to serif and font differences to represent a word image. Using this technique of inexact string matching, our method is able to successfully handle the problem of heavily touching characters. Experimental results on a variety of document image databases confirm the feasibility, validity, and efficiency of our proposed approach in document image retrieval.     

基于神经网络的交互式图像检索

提出了基于神经网络的交互式图像检索方法,系统根据用户对检索结果的评价,动态构造神经网络,描述图像之间的相似性;图像间的这种相似性以及本次检索结果可以作为以后检索的历史信息保存在神经网络中,从而提高下一次检索的效率。实验表明,该方法嵌入到典型的图像检索系统中,改善了图像检索性能。     

带后缀印刷体维吾尔文映射关系检索

目的 维吾尔文属于黏着性语言，其组成方式是在词干上添加词缀来实现不同的语义，在添加词缀的过程中词干的尾部会发生一定的形态变化，而且词干添加词缀的时候也可能会发生弱化、脱落、增音等音变现象导致进一步的形态变化，所以利用目前的图像文字检索（word spotting）技术只能检索到某一具体的维吾尔文词汇，却不能以某一词干为检索词，检索出其对应的带后缀的词语。为此，提出了基于映射关系的带后缀印刷体维吾尔文词语检索技术。方法 首先利用局部特征对维吾尔文词图像进行特征提取，其次将获得的特征用快速最近邻搜索（fast library for approximate nearest neighbors，FLANN）双向匹配来获得特征匹配集，最后将特征匹配集进行单应性变换和透视变换到待检索维吾尔文词图像上，把特征匹配集转化为空间关系，经过映射匹配对特征匹配集的空间关系进行后缀词检索，从而实现印刷体维吾尔文图像带后缀词检索的需求。结果 实验数据选取190幅维吾尔文印刷体文本图像中的17 648幅切割词图像，并对其中30幅词图像的167幅后缀词图像进行后缀检索，采用不同的局部特征算法进行后缀检索对比，结果表明，尺度不变特征变换（scale-invariant feature transform，SIFT）算法的后缀检索效果优于SURF（speeded up robust features）算法，精确率和召回率分别达到了94.23%和88.02%，在印刷体文档图像中，可以高效地检索到词干组成的后缀词，能够满足用户的不同检索需求，具有普适性。在弱化、脱落、增音和多种音变同时出现以及词干尾部发生变化的不同情况下进行后缀检索对比实验，实验结果表明在弱化和词干尾部变化而导致的形态变化中，检索效率最佳。结论 本文提出的基于映射关系进行后缀词图像检索的方法，是第一次对维吾尔文带后缀词检索方式的一种实现，利用匹配集之间的空间关系，对维吾尔文带后缀词图像实现了高效检索的目的。