Clustering Web pages based on their structure

Several techniques have been recently proposed to automatically generate Web wrappers, i.e., programs that extract data from HTML pages, and transform them into a more structured format, typically in XML. These techniques automatically induce a wrapper from a set of sample pages that share a common HTML template. An open issue, however, is how to collect suitable classes of sample pages to feed the wrapper inducer. Presently, the pages are chosen manually. In this paper, we tackle the problem of automatically discovering the main classes of pages offered by a site by exploring only a small yet representative portion of it. We propose a model to describe abstract structural features of HTML pages. Based on this model, we have developed an algorithm that accepts the URL of an entry point to a target Web site, visits a limited yet representative number of pages, and produces an accurate clustering of pages based on their structure. We have developed a prototype, which has been used to perform experiments on real-life Web sites.     

图像转换代理服务器及其在Internet中的应用

在Web服务器和客户机之间设置代理服务器，可以使不同性能的客户机浏览Internet的内容。通过分析图像在网页源文件中的表示方法，可以推断出图像在网页中的意义。根据图像在网页中的意义、客户机的性能、网络带宽和用户设置，代理服务器采用不同的转换方法和策略进行图像的转换，对图像重新编码，使转换后的图像适应于不同的客户机。     

XML raises concerns as it gains prominence

XML's growing implementation has raised several key concerns. First, because they provide considerable information about a document's content, XML files include much more data than HTML files. This can burden a company's network, processor, and storage infrastructures. In addition to extensively describing a document's contents, XML files typically appear in plain text. These factors create security concerns. And many developers may not be cooperating with the effort to transition the Web from HTML to XML via XHTML (Extensible HTML).     

Webizing mobile augmented reality content

This paper presents a content structure for building mobile augmented reality (AR) applications in HTML5 to achieve a clean separation of the mobile AR content and the application logic for scaling as on the Web. We propose that the content structure contains the physical world as well as virtual assets for mobile AR applications as document object model (DOM) elements and that their behaviour and user interactions are controlled through DOM events by representing objects and places with a uniform resource identifier. Our content structure enables mobile AR applications to be seamlessly developed as normal HTML documents under the current Web eco-system.     

基于嵌入式服务器在监测控制中的应用

针对嵌入式Web技术能监测和控制各种嵌入式设备进行网络互联的特性,结合HTML和Web技术,重点论述了嵌入式服务器软硬件部分的设计思想和体系架构.试验结果表明,嵌入式Web技术可实现对设备状态的实时监测和控制.     

用动态HTML制作动态Web页面

本文通过介绍动态ＨＴＭＬ，介绍了一些制作动态Ｗｅｂ页面的实例与方法。     

基于缩进轮廓的HTML文档重复模式挖掘方法

HTML文档重复模式挖掘是找到Web页面编码模版的关键,是Web数据自动抽取和Web内容挖掘的基础。传统的基于字符串匹配和树匹配的重复模式挖掘方法虽然具有较高的精确度,但是其性能对于处理海量的Web页面来说仍然是一个挑战。为了提高性能,提出了一种基于缩进轮廓的HTML文档重复模式挖掘方法。该方法首先定义了缩进轮廓模型,是一种由HTML文档每行代码的缩进值及行首的HTML标签构成的数据结构,它是HTML文档的一种简化抽象;该方法通过检测缩进轮廓中的串联重复波段,间接地挖掘HTML文档中的重复模式。实验表明,该方法不但具有较高的精确度,而且较明显地提升了性能。     

基于HTML批注技术研究与实现

随着信息化推进与电子政务的发展,Web模式下批注技术的研究越来越迫切。对Web批注作出研究,提出一种基于HTML的批注技术。它在不采用插件技术与不改变原始文件的基本内容基础上,实现随处对Web任何一段HTML元素进行独立附加批注,并且支持多用户操作。批注浏览时,通过原始文件与批注文件合成,达到每段HTML元素按需显示对应批注用户的批注内容。     

Utilizing the multiple facets of WWW contents

Current query languages for the Web (e.g., W3QL, WebLog and WebSQL) explore the structure of the Web. However, usually, the structure of the Web has little to do with the semantics of the data. Therefore, it is practically difficult to pose database queries over the Web. We introduce a new type of tags for denoting the semantics of data stored in HTML pages. These semantic tags (implemented as HTML comments) superimpose on HTML pages semistructured objects in the style of the OEM model. The paper discusses two implemented tools for fully utilizing the semantics. The first is a visualization tool for displaying both the HTML reading of Web pages and the OEM reading of Web pages. The second tool is a query language, similar to LOREL, that can query the HTML structure and/or the OEM reading. The above formalism and tools provide data-modeling capabilities for the Web that fit its heterogeneous nature. Real database queries, taking the OEM point of view, can be formulated, including queries about the schema as well as queries about the HTML structure of Web pages. Therefore, the query language is not restricted to portions of the Web in which semantic tags are used.     

Do HTML tags flag semantic content

HTML tags are designed to support only the display of Web page content, but the study quantifies their feasibility as proxies for semantic content as well. More specifically, the author examined a number of Web pages to try to quantify whether, consciously or otherwise, authors use headings or the highlighted text accompanying a link to indicate the subject matter of the corresponding page. The investigations focused on empirically evaluating the feasibility of using key Web page elements as proxies to indicate page contents. Additionally, the author compared text retrieval by keyword. The author describes the methods he used and summarizes results. The empirical results suggest that text in HTML headings and in anchor texts is useful for indicating Web page content for logic programming     

On Finding Templates on Web Collections

Templates are pieces of HTML code common to a set of web pages usually adopted by content providers to enhance the uniformity of layout and navigation of theirs Web sites. They are usually generated using authoring/publishing tools or by programs that build HTML pages to publish content from a database. In spite of their usefulness, the content of templates can negatively affect the quality of results produced by systems that automatically process information available in web sites, such as search engines, clustering and automatic categorization programs. Further, the information available in templates is redundant and thus processing and storing such information just once for a set of pages may save computational resources. In this paper, we present and evaluate methods for detecting templates considering a scenario where multiple templates can be found in a collection of Web pages. Most of previous work have studied template detection algorithms in a scenario where the collection has just a single template. The scenario with multiple templates is more realistic and, as it is discussed here, it raises important questions that may require extensions and adjustments in previously proposed template detection algorithms. We show how to apply and evaluate two template detection algorithms in this scenario, creating solutions for detecting multiple templates. The methods studied partitions the input collection into clusters that contain common HTML paths and share a high number of HTML nodes and then apply a single-template detection procedure over each cluster. We also propose a new algorithm for single template detection based on a restricted form of bottom-up tree-mapping that requires only small set of pages to correctly identify a template and which has a worst-case linear complexity. Our experimental results over a representative set of Web pages show that our approach is efficient and scalable while obtaining accurate results.     

灵活结构网页的正文提取

在Web数据挖掘中,由于网页大多都含有指向其他页面的超链接等噪音信息,为了减少噪音信息对Web数据挖掘效果的影响,有必要对网页进行净化处理,提取其中的正文,同时,现实中很多网页的代码结构不是特别规范,对此,提出一种对灵活结构网页适用的正文抽取算法。将网页用HTML标签分割成节点形式,找出其中含有正文内容的一个节点,以此节点为基础向前和向后进行余下正文内容的抽取。实验结果表明,本算法的适用性强、正确率较高。     

Automatic discovery of Web Query Interfaces using machine learning techniques

The amount of information contained in databases available on the Web has grown explosively in the last years. This information, known as the Deep Web, is heterogeneous and dynamically generated by querying these back-end (relational) databases through Web Query Interfaces (WQIs) that are a special type of HTML forms. The problem of accessing to the information of Deep Web is a great challenge because the information existing usually is not indexed by general-purpose search engines. Therefore, it is necessary to create efficient mechanisms to access, extract and integrate information contained in the Deep Web. Since WQIs are the only means to access to the Deep Web, the automatic identification of WQIs plays an important role. It facilitates traditional search engines to increase the coverage and the access to interesting information not available on the indexable Web. The accurate identification of Deep Web data sources are key issues in the information retrieval process. In this paper we propose a new strategy for automatic discovery of WQIs. This novel proposal makes an adequate selection of HTML elements extracted from HTML forms, which are used in a set of heuristic rules that help to identify WQIs. The proposed strategy uses machine learning algorithms for classification of searchable (WQIs) and non-searchable (non-WQI) HTML forms using a prototypes selection algorithm that allows to remove irrelevant or redundant data in the training set. The internal content of Web Query Interfaces was analyzed with the objective of identifying only those HTML elements that are frequently appearing provide relevant information for the WQIs identification. For testing, we use three groups of datasets, two available at the UIUC repository and a new dataset that we created using a generic crawler supported by human experts that includes advanced and simple query interfaces. The experimental results show that the proposed strategy outperforms others previously reported works.     

一种新的Web数据模型-EOEM

随着XML的普及，开发具有同时查询XML文档和HTML文档能力的新一代Web查询语言已是当务之急．但现有的Web数据模型存在种种局限．本文提出了一种新的Web数据模型，能够同时支持HTML和XML等半结构化文档的查询．     

Web文档清洗系统中HTML解析器的开发

对于组建一个面向Web的信息系统来说 ,去除掉脚本、广告链接以及导航链接等无用数据 ,将提高信息存储和检索的效率 ;同时 ,基于语义对Web文档进行合并和分割也会有助于信息的管理 ,这些都是Web文档清洗系统的任务。在Web文档清洗中 ,无论是脱机的规则学习还是联机的文档清洗 ,都需要建立在对Web文档的结构和内容进行分析的基础之上。从HTML解析的一般概念入手 ,结合Web文档清洗系统的需求 ,描述了一个自主开发的HTML解析器的结构 ,并对其组成部分 :词典、词法分析器和语法分析器的设计作了详细的讨论     

WAP页面转换代理系统原理及其实现

为了解决WAP应用层与Web应用层之间的语言差别,文章提出了一个WAP页面转换代理系统的方案,设计并实现了从HTML到WML网页转换算法,并提出利用知识库构造基于Web内容的自适应页面转换代理的处理方法。实验证明,该方法能够很好地解决移动计算设备上网存在的语言差异问题,另一方面,它也能够较好地解决普通Web页面无线设备上显示时可能出现的布局失调问题。     

SVG——一种支持可缩放矢量图形的Web浏览语言规范

该文介绍并探讨了在目前国际上一种最新的基于Ｗｅｂ浏览的图象图形规范,并从技术的角度分析了该规范的特点及其优点,同时给出了一个典型示例。     

XML技术在化学深层网数据提取中的应用

Internet上的化学数据库是宝贵的化学信息资源,如何有效地利用这些数据是化学深层网所要解决的问题。本文总结了化学深层网的特点,基于XML技术实现从数据库检索返回的半结构化HTML页面中提取数据的目标,使之成为可供程序直接调用做进一步计算的数据。在数据提取过程中,先采用JTidy规范化HTML,得到格式上完整、内容无误的XHTML文档,利用包含着XPath路径语言的XSLT数据转换模板实现数据转换和提取。其中XPath表达式的优劣决定了XSLT数据转换模板能否长久有效地提取化学数据,文中着重介绍了如何编辑健壮的XPath表达式,强调了XPath表达式应利用内容和属性特征实现对源树中数据的定位,并尽可能地降低表达式之间的耦合度,前瞻性地预测化学站点可能出现的变化并在XSLT数据转换模板中采取相应的措施以提高表达式的长期有效性。为创建化学深层网数据提取的XSLT数据提取模板提供方法指导。