基于自动规约规则的HTML文档元数据提取

利用HTML文档的元数据,可以为Web检索提供多样化的检索手段.本文提出了一种从HTML文档自动提取文档元数据的方法,对其中提取规则的设计、规约算法及其复杂度分析做出了重点介绍.该方法的提取规则在语法形式上和文档片断接近,更适合自动生成,通过自动规约生成规则无需人工分析,适应Web文档特点.文章最后给出了实验结果并进行了分析.     

基于C4.5的论文元数据抽取算法研究

基于特征相似度的论文元数据抽取算法与传统的DOM树方法相比,提高了论文元数据的抽取成功率.但论文元数据的抽取效率却不高,而论文元数据的抽取与蜘蛛的论文爬取是同时进行的,对元数据的抽取有实时性要求,因此影响了论文数据量的增加.经过比较研究,利用决策树中的C4.5算法对特征相似度算法进行改进,论文元数据的抽取成功率提高了2%,抽取效率提高了62%.     

一种新的基于Ontology的信息抽取方法*

把语法分析和Ontology 结合起来,先利用领域Ontology里的概念、关系、关键字自动生成标注规则(Rule),然后对文章、句子的语法结构进行分析,再利用语法分析的结果和先前生成的标注规则一起对文档进行信息标注与抽取,最后把信息抽取的结果以记录的形式输出。     

一种面向专利摘要的领域术语抽取方法

专利领域中术语抽取结果的好坏决定了本体构建的质量。提出一种自动生成过滤词典并结合词汇密集度等影响因子的术语抽取方法。首先在分词和词性标注的基础上,对文献匹配词性规则算法生成的模板得到候选长术语和单词型短术语集合,然后利用文档一致度生成的过滤词典过滤部分候选长术语集,最后针对长术语的构成特点,将词汇密集度、文档差比、文档一致度三个术语因子加权平均作为整个长术语的术语权重值,并按值高低排序。在8000篇专利摘要文献的基准语料上进行实验,随机选取五组实验数据,平均准确率达到86%。结果表明该方法在领域术语抽取方面是行之有效的。     

一个基于关联规则的多层文档聚类算法

提出了一种新的基于关联规则的多层文档聚类算法,该算法利用新的文档特征抽取方法构造了文档的主题和关键字特征向量。首先在主题特征向量空间中利用频集快速算法对文档进行初始聚类,然后在基于主题关键字的新的特征向量空间中利用类间距和连接度对初始文档类进行求精,从而得到最终聚类。由于使用了两层聚类方法,使算法的效率和精度都大大提高;使用新的文档特征抽取方法还解决了由于文档关键字过多而导致文档特征向量的维数过高的问题。     

一种基于启发式搜索的论文元数据提取算法

随着大量的科研论文以电子文档的形式出现, 为了高效地检索这些科研文献资料,从中精确地抽取这些论文头部的元数据信息显得十分有必要.在条件随机场模型的基础上,提出了一种启发式搜索算法来对论文头部的元数据信息进行提取.该算法首先利用文本分块技术和特征提取规则来对文本进行预处理,然后结合条件随机场概率模型利用启发式搜索来进行元数据的提取.实验结果表明,在相同概率模型的基础上,该算法有着较好的性能,在精确度和召回率方面都有了明显的提高.     

数据库元数据的自动语义标注

语义异构是异构数据库信息集成中要解决的关键问题.为了使关系数据库的表和字段具有语义信息,将数据库元数据自动标注成语义元数据成为研究的热点.基于概念名和概念结构的语义相似度计算,提出了一种数据库元数据自动语义标注方法.首先从关系数据库的元数据中提取隐含的语义信息,并据此创建领域本体,然后通过计算元数据与本体实体间的语义相似度对提取的元数据进行自动语义标注,提出的相似度算法综合考虑了概念名称和结构的相似性,并采取了必要的优化措施进行改进.经实验测试证明,该方法具有较高的标注正确率,是一种行之有效的语义标注方法.     

Web信息抽取系统的设计

为了获取分散Web页面中隐含信息,设计了Web信息抽取系统。该系统首先使用一种改进的HITS主题精选算法进行信息采集;然后对Web页面的HTML结构进行文档的数据预处理;最后,基于DOM树的XPath绝对路径生成算法来获取被标注结点的XPath表达式,并使用XPath语言结合XSLT技术来编写抽取规则,从而得到结构化的数据库或XML文件,实现了Web信息的定位和抽取。通过一个购物网站的抽取实验证明,该系统的抽取效果良好,可以实现相似Web页面的批量抽取。     

基于类别的特征选择算法的文本分类系统

目前的索引词选择算法大多是基于词频的,没有利用训练样本中的类别信息,为此提出了一种新的基于类别的特征选择算法。该算法根据某个词是否存在于文档中导致该类文档相似度的区别,来确定该词区分不同文档的分辨力,以此分辨力作为选取关键词的重要度。以该算法为基础,设计了一个英文文本自动分类系统,并对该系统进行了测试和结果分析。     

统计与规则相融合的领域术语抽取算法

中文领域术语自动抽取是中文信息处理中的一项基础性课题,并在自然语言生成、信息检索、文本摘要等领域中有广泛的应用。针对领域术语抽取问题,采用基于规则和多种统计策略相融合的方法,从词语度和领域度两个角度出发,提出一种领域术语的抽取算法并构建出相应的抽取系统。系统流程包括基于左右信息熵扩展的候选领域术语获取、基于词性搭配规则与边界信息出现概率知识库相结合的词语度筛选策略以及基于词频-逆文档频率(TF?IDF)的领域度筛选策略。运用此算法不但能抽取出领域的常见用词,还可以挖掘出领域新词。实验结果显示,基于如上方法构建的领域术语抽取系统的准确率为84.33%,所提方法能够有效支持中文领域术语的自动抽取。     

语义桌面环境下一种基于概念图的文档检索方法

语义桌面作为语义Web的一个重要分支,可以为个人计算机用户提供丰富的元数据,用以记录桌面文档的各种特征。这些特征包括文件的常规属性和与用户行为相关的属性,它们为桌面文档检索提供帮助。受到"概念图"理论的启发,本文提出了一种语义桌面环境下的文档检索算法。该算法能有效地利用语义桌面提供的元数据建立一种便于快速查找的文档索引结构,迅速地确定用户查询与桌面文档之间的投影算子。实验表明,该算法的时间效率比以往的基于匹配推理的投影算法有很大的提高,可以在很大程度上满足用户对桌面文档进行快速检索的需求。     

面向机器学习的流式文档逻辑结构标注方法研究

针对采用机器学习方法识别流式文档结构时语料库稀少、语料标注复杂的问题,该文在研究文档的逻辑结构和编辑语义特征的基础上,确立流式文档逻辑结构标注体系,并提出一种三段式的半自动文档逻辑结构标注方法: 第一阶段通过机助人工实现文档元数据的分离式标注,第二阶段自动重建逻辑结构,第三阶段自动填充特征向量。实验结果表明,该文提出的文档逻辑结构标注方法能够节省人工成本、提高机器学习算法对文档结构识别的准确率与召回率,F值达到97.5%。     

Methodologies and tools for audio digital archives

In response to the proposal of digitizing the entire back-run of several European audio archives, many research projects have been carried out in order to discover the technical issues involved in making prestigious audio documents digitally available, which are related to the A/D transfer process and supervised metadata extraction. This article gives an innovative approach to metadata extraction from such a complex source material. This article also describes the protocols defined, the processes undertaken, the results ascertained from several audio documents preservation projects and the techniques used. In addition, a number of recommendations are given for the re-recording process, aimed at minimizing the information loss and to automatically measure the unintentional alterations introduced by the A/D equipment.     

Towards the semantic and context-aware management of mobile multimedia

Users of mobile devices can nowadays easily create large quantities of mobile multimedia documents tracing significant events attended, places visited or, simply, moments of their everyday life. However, they face the challenge of organizing these documents in order to facilitate searching through them at a later time and sharing them with other users. We propose using context awareness and semantic technologies in order to improve and facilitate the organization, annotation, retrieval and sharing of personal mobile multimedia documents. Our approach combines metadata extracted and enriched automatically from the users’ context with annotations provided manually by the users and with annotations inferred by applying user-defined rules to context features. These new contextual metadata are integrated into the processes of annotation, sharing and keyword-based retrieval.     

实例驱动的自适应本体学习

针对知识管理中本体构建存在的问题,将聚类算法与ODP（Open Directory Project）目录有机结合,给出了一种基于知识资源元数据的自适应本体学习方法。根据元数据对文档进行聚类形成本体概念,将生成的概念分别映射到ODP中确定概念间的层次关系,生成初始本体;根据内聚性和相关性的变化进行自适应本体学习,实现本体更新和概念丰富,以及时跟踪知识的变化。提出的自适应本体学习方法能够很好地反映研究领域的演变过程和发展趋势,满足知识型组织进行知识管理和研究人员共享知识的需求。实验结果表明了方法的有效性。     

基于规则归纳的信息抽取系统实现

面对Web信息的迅猛增长,信息抽取技术非常适合于从大量的文档中抽取需要的事实数据。通过文档对象模型（DOM）解析以及检索、抽取、映射等规则的定义,设计并实现了一种具有规则归纳能力的信息抽取系统,用于Web信息的自动检索。在用于抽取规则归纳的框架下,还重点对用于生成抽取模式的WHISK学习算法进行了实验对比分析,结果表明系统对于单槽和多槽数据都具有不错的归纳学习能力。     

Automatic metadata mining from multilingual enterprise content

Personalization is increasingly vital especially for enterprises to be able to reach their customers. The key challenge in supporting personalization is the need for rich metadata, such as metadata about structural relationships, subject/concept relations between documents and cognitive metadata about documents (e.g. difficulty of a document). Manual annotation of large knowledge bases with such rich metadata is not scalable. As well as, automatic mining of cognitive metadata is challenging since it is very difficult to understand underlying intellectual knowledge about document automatically. On the other hand, the Web content is increasing becoming multilingual since growing amount of data generated on the Web is non-English. Current metadata extraction systems are generally based on English content and this requires to be revolutionized in order to adapt to the changing dynamics of the Web. To alleviate these problems, we introduce a novel automatic metadata extraction framework, which is based on a novel fuzzy based method for automatic cognitive metadata generation and uses different document parsing algorithms to extract rich metadata from multilingual enterprise content using the newly developed DocBook, Resource Type and Topic ontologies. Since the metadata generation process is based upon DocBook structured enterprise content, our framework is focused on enterprise documents and content which is loosely based on the DocBook type of formatting. DocBook is a common documentation formatting to formally produce corporate data and it is adopted by many enterprises. The proposed framework is illustrated and evaluated on English, German and French versions of the Symantec Norton 360 knowledge bases. The user study showed that the proposed fuzzy-based method generates reasonably accurate values with an average precision of 89.39% on the metadata values of document difficulty, document interactivity level and document interactivity type. The proposed fuzzy inference system achieves improved results compared to a rule-based reasoner for difficulty metadata extraction (∼11% enhancement). In addition, user perceived metadata quality scores (mean of 5.57 out of 6) found to be high and automated metadata analysis showed that the extracted metadata is high quality and can be suitable for personalized information retrieval.     

Machine Learning for Intelligent Processing of Printed Documents

A paper document processing system is an information system component which transforms information on printed or handwritten documents into a computer-revisable form. In intelligent systems for paper document processing this information capture process is based on knowledge of the specific layout and logical structures of the documents. This article proposes the application of machine learning techniques to acquire the specific knowledge required by an intelligent document processing system, named WISDOM++, that manages printed documents, such as letters and journals. Knowledge is represented by means of decision trees and first-order rules automatically generated from a set of training documents. In particular, an incremental decision tree learning system is applied for the acquisition of decision trees used for the classification of segmented blocks, while a first-order learning system is applied for the induction of rules used for the layout-based classification and understanding of documents. Issues concerning the incremental induction of decision trees and the handling of both numeric and symbolic data in first-order rule learning are discussed, and the validity of the proposed solutions is empirically evaluated by processing a set of real printed documents.