表格信息抽取引擎的设计与实现

讨论针对Web表格的信息抽取，分析并给出了表格信息抽取引擎的系统结构，以及实现该系统所涉及的关键技术和数据模型，为用户提供一种以Web表格为信息抽取对象的、支持抽取方式选择的Web表格信息抽取工具。     

Web表格信息抽取的研究

Web表格信息抽取是信息抽取在Web表格上的一种应用，是当今的一个研究热点。本文首先分析了Web表格信息抽取的过程，包括表格识别、结构识别以及“属性-值”对的提取；然后对当前国内外在基于特定域和独立城两种表格信息抽取研究方法上的动态及成果追行了比较和分析。在此基础上，提出了表格抽取的关键技术——表格结构识别上的一些想法；最后展望了Web表格信息抽取技术的发展趋势。     

一种自动抽取Web信息方法的设计与实现

针对目前Web信息抽取技术实现复杂、维护困难以及抽取速度慢的问题,本文根据Web页面的特点,提出一种新的Web抽取策略.此策略在处理Web页面时降低了处理Web页面的结构的复杂性,提高了Web信息抽取的速度.并根据策略建立了该Web信息自动抽取方法的模型,此模型首先分析页面的结构,根据结构快速生成抽取规则,构建规则库;并对页面抽取的内容进行分析,构建资源库.基于此模型的方法能自主学习,实现自动抽取.这在很大程度上减少了人工参与,并能获得比较好的抽取结果.     

基于扩展标记图的Web信息抽取器

介绍了一种新的Web信息抽取器,该抽取器基于扩展标记图模型,实观了数据和模式的分离,应用于Web检索系统中,能够有效地支持标记级实时信息检索、抽取和重组。还介绍了其在Web信息检索系统PowerSearcher中的实际应用。     

利用表格特征信息的Web文本分类研究与实现

构建了关于Web表格特征信息知识的领域本体,提出并设计了一种用于Web文本分类的二次分类模型。该模型使用支持向量机方法对测试样本进行第一次分类;由于设定了较高的分类阈值,一次分类后部分测试样本未确定所属类别,对于这些测试样本,抽取样本中的Web表格特征信息,与基于领域本体的分类模板进行相似度匹配,进行第二次分类。最后通过实验验证了该方法的可行性。     

一种鲁棒性的结构未知表格分析方法

模型未知表格的分析是表格识别中文本分析阶段的一个重要且具有挑战性的问题.目前的一般方法仅能容忍表格线的微小断线.文章提出一种基于抽取表格线的分析结构未知表格的策略.利用抽取的表格线的特征知识和局部约束可以选择一些有效边.在扫描水平和垂直表格线时,如果环绕边都有效,则产生一个矩形块,引入迭代可以更好地利用全局信息并使抽取结果满足约束关系.这种矩形块的抽取可以容忍表格线大的断线或不合适的分割,可以处理诸如嵌入矩形块的复杂结构.矩形块被抽取后,表格的其他部件可以通过搜索剩余的部分来抽取.表格测试实验证明,该方法     

基于页面分类的Web信息抽取方法研究

通过对现有Web信息抽取方法和当前Web网页特点的分析,发现现有抽取技术存在抽取页面类型固定和抽取结果不准确的问题,为了弥补以上两个不足,文中提出了一种基于页面分类的Web信息抽取方法,此方法能够完成对互联网上主流信息的提取。通过对页面进行分类和对页面主体的提取,分别克服传统方法抽取页面类型固定和抽取结果不够准确的问题。文中设计了一个完整的Web信息抽取模型,并给出了各功能模块的实现方法。该模型包含页面主体提取、页面分类和信息抽取等模块,并利用正则表达式自动生成抽取规则,提高了抽取方法的通用性和准确性。最后用实验证实了文中方法的有效性与正确性。     

基于表格语义的Web信息抽取方法的研究

在Web页面常用到表格这种元素。本文提出一种根据表格语义来进行信息抽取方法。首先提出了一种短语语义相似度的度量方法,然后利用短语语义的相似度确定表格标题行(列),并对表格行(列)与抽取字段的对应关系进行计算,最后计算表格的整体语义,度量该表格与所要抽取的内容有多大相关度。     

基于Web的表格信息抽取研究

如今,Web成为了网络信息的主要平台。根据研究发现,表格在Web文本中被经常使用。正因为表格形式简洁并且含有丰富的信息,自动理解表格在知识管理、信息检索、Web挖掘等应用中有着广泛的用途,所以研究Web表格信息抽取有着重要的现实意义。互联网上有大量信息采用HTML表格表示,由于HTML不描述数据的内容,机器不能理解和查询。论文首先将HTML文档转换为XML文档,结合本体形成启发式规则,对表格定位、表格结构识别两个关键技术进行了分析。在此基础上,利用HTML表格属性,将HTML表格标准化,从而适用于复杂表格的信息抽取。     

基于Web的表格信息抽取研究

如今,Web成为了网络信息的主要平台。根据研究发现,表格在Web文本中被经常使用。正因为表格形式简洁并且含有丰富的信息,自动理解表格在知识管理、信息检索、Web挖掘等应用中有着广泛的用途,所以研究Web表格信息抽取有着重要的现实意义。互联网上有大量信息采用HTML表格表示,由于HTML不描述数据的内容,机器不能理解和查询。论文首先将HTML文档转换为XML文档,结合本体形成启发式规则,对表格定位、表格结构识别两个关键技术进行了分析。在此基础上,利用HTML表格属性,将HTML表格标准化,从而适用于复杂表格的信息抽取。     

Transforming arbitrary tables into logical form with TARTAR

The tremendous success of the World Wide Web is countervailed by efforts needed to search and find relevant information. For tabular structures embedded in HTML documents, typical keyword or link-analysis based search fails. The Semantic Web relies on annotating resources such as documents by means of ontologies and aims to overcome the bottleneck of finding relevant information. Turning the current Web into a Semantic Web requires automatic approaches for annotation since manual approaches will not scale in general. Most efforts have been devoted to automatic generation of ontologies from text, but with quite limited success. However, tabular structures require additional efforts, mainly because understanding of table contents requires the comprehension of the logical structure of the table on the one hand, as well as its semantic interpretation on the other. The focus of this paper is on the automatic transformation and generation of semantic (F-Logic) frames from table-like structures. The presented work consists of a methodology, an accompanying implementation (called TARTAR) and a thorough evaluation. It is based on a grounded cognitive table model which is stepwise instantiated by the methodology. A typical application scenario is the automatic population of ontologies to enable query answering over arbitrary tables (e.g. HTML tables).     

Detecting tables in Web documents

The TABLE tags in HTML (Hypertext Markup Language) documents are widely used for formatting layout of Web documents as well as for describing genuine tables with relational information. As a prerequisite for information extraction from the Web, this paper presents an efficient method for sophisticated table detection. The proposed method consists of two phases: preprocessing and attribute–value relations extraction. During preprocessing, a part of genuine or non-genuine tables are filtered out using a set of rules, which are devised based on careful examination of general characteristics of various HTML tables. The remaining tables are detected at the attribute–value relations extraction phase. Specifically, a value area is extracted and checked out whether there is syntactic coherency. Furthermore, the method looks for semantic coherency between an attribute area and a value area of a table. Experimental results with 11,477 TABLE tags from 1393 HTML documents show that the method has performed better compared with previous works, resulting in a precision of 97.54% and a recall of 99.22%.     

Web table discrimination with composition of rich structural and content information

A table is a well-organized and summarized knowledge expression for a domain. Therefore, it is of great importance to extract information from tables. However, many tables in Web pages are used not to transfer information but to decorate pages. One of the most critical tasks in Web table mining is thus to discriminate meaningful tables from decorative ones. The main obstacle of this task comes from the difficulty of generating relevant features for discrimination. This paper proposes a novel discrimination method using a composite kernel which combines parse tree kernels and a linear kernel. Because a Web table is represented as a parse tree by an HTML parser, it is natural to represent the structural information of a table as a parse tree. In this paper, two types of parse trees are used to represent structural information within and around a table. These two trees define the structure kernel that handles the structural information of tables. The contents of a Web table are manipulated by a linear kernel with content features. Support vector machines with the composite kernel distinguish meaningful tables from decorative ones with high accuracy. A series of experiments show that the proposed method achieves state-of-the-art performance.     

Extracting logical structures from HTML tables

While HTML is mainly designed for the visual rendering of Web documents, XML is widely accepted as a standard format to process and manage information. In particular, it can embed the information of logical structures. However, in order to utilize XML, the logical structures of HTML tables should first be extracted and transformed into XML representations. This paper presents an efficient method for the process, which consists of two phases: area segmentation and structure analysis. The area segmentation cleans up tables and segments them into attribute and value areas by checking visual and semantic coherency. The hierarchical structure between attribute and value areas is then analyzed and transformed into an XML representation using a proposed table model. Experimental results with 1180 HTML tables show that the proposed method performs better than conventional methods, resulting in an average accuracy of 86.7%.     

基于等价压缩快速聚类的Web表格知识抽取

Web表格知识抽取是一种重要的获取高质量知识的途径,在知识图谱、网页挖掘等方面具有广泛的研究意义与应用价值。传统的Web表格知识抽取方法主要依赖于良好的表格结构和足够的先验知识,但在复杂的表格结构以及先验知识不足等情形下难以奏效。针对这类方法的问题,该文通过充分利用表格自身的结构特点,提出了一套可面向大规模数据的基于等价压缩快速聚类的Web表格知识抽取方法,以无监督的聚类方式获得相似形式结构的表格,从而推测其语义结构以抽取知识。实验结果表明,基于等价压缩的快速聚类算法在保持同水平的聚类准确率的前提下,在时间性能上相比传统方法有大幅度的提升,5 000个表格的聚类时间由72小时缩短为20分钟,且在表格聚类后利用表格模板所抽取的知识三元组的准确率也达到了令人满意的结果。     

网上表格数据到XML的自动转换

互联网上有大量信息采用HTML表格表示,由于HTML不描述数据的内容,机器不能理解和查询。论文利用HTML表格属性,在表格中插入冗余单元,使HTML表格规范化;对没有标志表头的HTML表格,采用格式化的信息的量化值识别网上表格的表头。在此基础上,提出了通过获取表格属性与值对应的语义层次,自动转换HTML表格数据为XML文挡的新方法。     

Table detection in online ink notes

In documents, tables are important structured objects that present statistical and relational information. In this paper, we present a robust system which is capable of detecting tables from free style online ink notes and extracting their structure so that they can be further edited in multiple ways. First, the primitive structure of tables, i.e., candidates for ruling lines and table bounding boxes, are detected among drawing strokes. Second, the logical structure of tables is determined by normalizing the table skeletons, identifying the skeleton structure, and extracting the cell contents. The detection process is similar to a decision tree so that invalid candidates can be ruled out quickly. Experimental results suggest that our system is robust and accurate in dealing with tables having complex structure or drawn under complex situations.     

Detecting and representing relevant Web deltas in WHOWEDA

In this paper, we present a mechanism for detecting and representing changes, given the old and new versions of a set of interlinked Web documents, retrieved in response to a user's query. In particular, we show how to detect and represent Web deltas, i.e., changes in the Web documents that are relevant to a user's query in the context of our Web warehousing system called WHOWEDA (Warehouse of Web Data). In WHOWEDA, Web information is materialized views stored in Web tables in the form of Web tuples. These Web tuples, represented as directed graphs, can be manipulated using a set of Web algebraic operators. In this paper, we present a mechanism to detect relevant Web deltas using Web algebraic operators such as the Web join and the outer Web join. Web join is used to detect identical documents residing in two Web tables, whereas, outer Web join, a derivative of Web join, is used to identify dangling Web tuples. We show how to represent these changes using delta Web tables. We develop formal algorithms for the generation of delta Web tables identifying Web documents which have been added, deleted, or modified since the last query.     

识别和抽取Web列表中的关系信息

有大量的关系信息存在于各种各样的Web列表中，但使用目前的搜索引擎却难以找到它们。本文提出了一种基于语义和数据特征的方法，用于识别和抽取Web列表中的关系信息。我们首先建立一个模型，描述所要的关系信息，然后寻找Web上的列表并估计它们是否包含所要的关系信息，当估计值足够大时．则从中抽取所要的关系信息。     

A scalable hybrid approach for extracting head components from Web tables

We have established a preprocessing method for determining the meaningfulness of a table to allow for information extraction from tables on the Internet. A table offers a preeminent clue in text mining because it contains meaningful data displayed in rows and columns. However, tables are used on the Internet for both knowledge structuring and document design. Therefore, we were interested in determining whether or not a table has meaningfulness that is related to the structural information provided at the abstraction level of the table head. Accordingly, we: 1) investigated the types of tables present in HTML documents, 2) established the features that distinguished meaningful tables from others, 3) constructed a training data set using the established features after having filtered any obvious decorative tables, and 4) constructed a classification model using a decision tree. Based on these features, we set up heuristics for table head extraction from meaningful tables, and obtained an F-measure of 95.0 percent in distinguishing meaningful tables from decorative tables and an accuracy of 82.1 percent in extracting the table head from the meaningful tables.