An adaptive local binarization method for document images based on a novel thresholding method and dynamic windows

Binary image representation is essential format for document analysis. In general, different available binarization techniques are implemented for different types of binarization problems. The majority of binarization techniques are complex and are compounded from filters and existing operations. However, the few simple thresholding methods available cannot be applied to many binarization problems. In this paper, we propose a local binarization method based on a simple, novel thresholding method with dynamic and flexible windows. The proposed method is tested on selected samples called the DIBCO 2009 benchmark dataset using specialized evaluation techniques for binarization processes. To evaluate the performance of our proposed method, we compared it with the Niblack, Sauvola and NICK methods. The results of the experiments show that the proposed method adapts well to all types of binarization challenges, can deal with higher numbers of binarization problems and boosts the overall performance of the binarization.     

Extracting hot spots of topics from time-stamped documents

Identifying time periods with a burst of activities related to a topic has been an important problem in analyzing time-stamped documents. In this paper, we propose an approach to extract a hot spot of a given topic in a time-stamped document set. Topics can be basic, containing a simple list of keywords, or complex. Logical relationships such as and, or, and not are used to build complex topics from basic topics. A concept of presence measure of a topic based on fuzzy set theory is introduced to compute the amount of information related to the topic in the document set. Each interval in the time period of the document set is associated with a numeric value which we call the discrepancy score. A high discrepancy score indicates that the documents in the time interval are more focused on the topic than those outside of the time interval. A hot spot of a given topic is defined as a time interval with the highest discrepancy score. We first describe a naive implementation for extracting hot spots. We then construct an algorithm called EHE (Efficient Hot Spot Extraction) using several efficient strategies to improve performance. We also introduce the notion of a topic DAG to facilitate an efficient computation of presence measures of complex topics. The proposed approach is illustrated by several experiments on a subset of the TDT-Pilot Corpus and DBLP conference data set. The experiments show that the proposed EHE algorithm significantly outperforms the naive one, and the extracted hot spots of given topics are meaningful.     

A quick tour on suffix arrays and compressed suffix arrays

Suffix arrays are a key data structure for solving a run of problems on texts and sequences, from data compression and information retrieval to biological sequence analysis and pattern discovery. In their simplest version, they can just be seen as a permutation of the elements in {1,2,…,n}, encoding the sorted sequence of suffixes from a given text of length n, under the lexicographic order. Yet, they are on a par with ubiquitous and sophisticated suffix trees. Over the years, many interesting combinatorial properties have been devised for this special class of permutations: for instance, they can implicitly encode extra information, and they are a well characterized subset of the n! permutations. This paper gives a short tutorial on suffix arrays and their compressed version to explore and review some of their algorithmic features, discussing the space issues related to their usage in text indexing, combinatorial pattern matching, and data compression.     

基于粗糙集理论的文本分类算法研究

文本分类是中文信息处理的重要研究领域。给文本分配一个或多个不同的类别,可提高文本检索和存储的处理效率。粗糙集是一种不需要任何先验信息的分类方法,通过对文本分词、过滤掉停用词之后把剩余的词语作为特征项,然后把文本用向量空间模型表示出来,将文本集转化成不带决策属性的信息系统,用粗糙集理论中核心内容属性约简实现对文本的分类。实验表明,该方法的查准率和查全率都有所提高。     

中英文文本分类系统异同因素的探讨

论文介绍了文本分类技术的研究进展及发展方向,给出了中英文文本分类系统构建的一般模型和过程,分析总结了中英文文本分类过程中的异同,并介绍了针对这些异同因素常用的处理方法。     

基于概念扩充的中文文本过滤模型

1 前言今天,以因特网为主体的信息高速公路仍在不断普及和发展,因特网上蕴涵的海量信息远远超过人们的想象,面对这样的信息汪洋大海,人们往往感到束手无策,无所适从,出现所谓的“信息过载”问题。如何帮助人们有效地选择和利用所感兴趣的信息,同时保证人们在信息选择方面的个人隐私权利?这已成为学术界和企业界所十分关注的焦点。因此,信息过滤技术应     

基于概念的中文文本可视化表示机制

为了浏览因特网上日益增多的在线中文文本,本文给出了基于概念的中文文本可视化表示机制,以直观的方式组织和表示文本及文本集,其基本思想是：首先在概念扩充的基础上,进行文本分类,然后,利用本文提出的提出的文本特征抽取方法和摘要方法,获取广西类别、广西、广西正文的标记的信息,通过类别,文本、有选择地浏览文本。     

一个多Agent系统模型

一个Ａｇｅｎｔ的知识和问题求解能力是有限的,为更好的服务于用户,必须与其它Ａｇｅｎｔ全作,Ａｇｅｔｎｔ寻找合作伙伴是问题求解的关键。本文结构Ｎ．Ｒ．Ｊｅｎｎｉｎｇｓ等人的工作给出了一种系统模型,任务减少了通开锁,自适应单元增强了Ａｅｇｎｔ对外界变化的自适应能力,Ａｇｅｎｔ个性倾向性使模型更合理,排除了冲突因素,提高了２求解的效率。     

Web文本特征选择算法的研究

以向量空间模型作为Web文本的表示方法,结合Web文本的结构特征对向量空间模型中的特征选择算法进行了分析并加以改进。在改进的算法中,体现出了特征词在Web文档结构中的位置信息;引入了信息论中熵的概念,用词的熵函数对权值进行调整,从而更加准确地选取有效的特征词。实验验证了改进算法的可行性和有效性。