基于DOM树的网页去噪技术

在网页文本信息自动采集的实际应用中,网页检索采集到的资源存在着大量与网页主题内容无关的"噪音"信息,本文主要内容是研究基于DOM树的网页去噪技术,通过对网页结构的转化,将网页转化成DOM树,在此基础上过滤掉网页噪音信息,提取网页文本的正文内容。     

面向终端用户的网页过滤模板动态生成技术研究

为了满足网络终端用户对网页噪音的过滤需求,提出一种面向终端用户的动态模板网页过滤系统模型,它基于模板并利用用户反馈自动进化过滤系统。设计了模板生成算法,模拟实验验证了该算法的有效性。     

面向网页结构特征的Hopfield 算法

针对目前互联网信息资源广泛、网页结构复杂、噪音信息较多的现状,主题爬虫获取有效信息过程中精确度低、耗费时间长等问题。结合经典的Hopfield算法,提出了针对网页结构特征进行分块的主题爬行改进算法,实验证明该改进算法在一定程度上能有效地解决目前信息获取过程中所面临的问题。     

面向移动终端的屏幕自适应网页设计

分析了屏幕自适应网页设计的基本原理,系统阐述了综合运用媒体查询技术与流式布局设计屏幕自适应网页的方法,探讨了使用PC浏览器测试网页屏幕自适应性的简便方法。     

基于网页分块的科技信息采集系统的设计与实现

在互联网海量的信息处理上,采用定向追踪的方法,对网络科技信息进行采集、设计、纳入系统框架等工作,是互联网科技信息采集整理系统的主要功能,本文对基于网页分块的科技信息采集系统的设计与实现展开论述,详细论证网页分块、数据消重等关键技术的实现策略,论证这个系统的优势以及推广后能够带给科技研究领域的便捷.     

基于分块的新闻网页信息抽取算法

为了更彻底地清洗网页噪音,减少网页噪音对新闻内容抽取准确率的影响,提出基于模板页的相同噪音块清洗方法和基于class属性的同类噪音块和特殊噪音块清洗方法;在此基础上,利用新闻网页在内容布局结构上的特征,提出基于起始块和终止块的新闻内容抽取方法。实验结果表明,与已有的算法相比,提出的方法抽取准确率更高,能够同时适应正文内容存放在单块和多块的情形,并且有效地解决了正文内容较短时的抽取问题。     

基于网页结构的网页去噪算法设计

本文对网页去噪的定义和分类、经典方法以及实验方法等进行了研究,在重定义启发式规则的基础上,针对文本类网页,设计新的算法,并用代码进行实现,在对比结果中验证,该算法能很好的解决文本类网页噪音问题。     

基于分块的网页信息解析器的研究与设计

详细介绍了网页信息解析的基本技术手段,在综合权衡优缺点的基础上,提出了针对新 闻网站复杂结构页面较为有效的分块算法,并结合实际的项目需求,设计实现了网页信息解析器 TVPS,实验结果表明,该解析器具有良好的性能,满足实际的需求。     

基于网页框架和规则的网页噪音去除方法

提出了一种基于网页框架和规则的网页去除噪音的新方法，该方法根据网页中HTML标签将网页分成若干部分，对各个table的长宽比属性进行比较，去掉长宽比很大的部分，并对其余table中的内容进行分析，根据内部是否存在和段落文字有关的标签

或
等来区分主题内容和噪音内容，在此基础上去除噪音内容。对来自CWT200G语料的132 559个网页进行测试后的结果表明，该方法可以有效地去除网页噪音，使索引文件减少约75%，大大地提高了检索速度，准确度也得到一定提高。     

10.

基于移动搜索的网页设计优化研究     

夏忠球 《软件工程师》2015,(4):14-16

本文介绍了移动搜索的特点、面向移动设备的网站优化规则和策略,从样式表、网页标签、页面结构、网页导航、屏幕分辨率优化等方面,阐述了移动端网页优化的具体方法。     

11.

Web页面清洗技术的研究与实现   总被引：15，自引：1，他引：15  

周源远  王继成  郑刚  张福炎 《计算机工程》2002,28(9):48-50,197

文章提出了Ｗeb页面清洗的概念,给出了一种基于规则的Ｗeb页面清洗方案,并实现了基于此方案的系统。文中的页面清洗建立在Ｗeb页面的ＤＯＭ树结构上,并通过人工判定的方法进行了实验和评估,实验结果表明该方案切实可行,清洗方法具有较快的速度和准确性。     

12.

PCA-based web page watermarking     

Qijun Zhao 《Pattern recognition》2007,40(4):1334-1341

The tamper-proof of web pages is of great importance. Some watermarking schemes have been reported to solve this problem. However, both these watermarking schemes and the traditional hash methods have a problem of increasing file size. In this paper, we propose a novel watermarking scheme for the tamper-proof of web pages, which is free of this embarrassment. For a web page, the proposed scheme generates watermarks based on the principal component analysis (PCA) technique. The watermarks are then embedded into the web page through the upper and lower cases of letters in HTML tags. When a watermarked web page is tampered, the extracted watermarks can detect the modifications to the web page, thus we can keep the tampered one from being published. Extensive experiments are performed on the proposed scheme and the results show that the proposed scheme can be a feasible and efficient tool for the tamper-proof of web pages.     

13.

Web页面主题相关性排序算法的研究   总被引：3，自引：0，他引：3       下载免费PDF全文

何国斌  赵晶璐 《计算机工程与应用》2009,45(23):149-151

分析了Web页面主题的分布的特点,对经典的页面排序算法进行了探讨,提出了一种基于内容和超链接分析并结合用户点击行为的相关性排序算法。该算法考虑了超文本标记、锚文本、文本内容等对相关性的影响,引入动态比较矩阵来计算相应的权重系数,能够客观分析网页所包含的主题信息,使检索结果排序更合理。实验表明,该算法能有效提高查准率,较好地解决了主题的漂移现象,且具有较好的性能。     

14.

Device-aware desktop web page transformation for rendering on handhelds   总被引：1，自引：0，他引：1  

Hassan A. Artail  Mackram Raydan 《Personal and Ubiquitous Computing》2005,9(6):368-380

This paper illustrates a new approach to automatic re-authoring of web pages for rendering on small-screen devices. The approach is based on automatic detection of the device type and screen size from the HTTP request header to render a desktop web page or a transformed one for display on small screen devices, for example, PDAs. Known algorithms (transforms) are employed to reduce the size of page elements, to hide parts of the text, and to transform tables into text while preserving the structural format of the web page. The system comprises a preprocessor that works offline and a just-in-time handler that responds to HTTP requests. The preprocessor employs Cascading Style Sheets (CSS) to set default attributes for the page and prepares it for the handler. The latter is responsible for downsizing graphical elements in the page, converting tables to text, and inserting visibility attributes and JavaScript code to allow the user of the client device to interact with the page and cause parts of the text to disappear or reappear. A system was developed that implements the approach and was used it to collect performance results and conduct usability testing. The importance of the approach lies in its ability to display hidden parts of the web page without having to revisit the server, thus reducing user wait times considerably, saving battery power, and cutting down on wireless network traffic.     

15.

基于改进DOM树的网页去噪声方法     

丁宝琼  谢远平  吴琼 《计算机应用》2009,29(Z1)

一个普通的Web网页可以被分成信息块和噪声块两个部分,因此快速准确地去除网页中的噪声块是网页信息采集处理的关键技术之一.利用网页模板的相似性,提出一种基于改进DOM的网页噪声去除算法,并将通过此算法处理的网页用于SVM分类系统,结果显示经过处理的网页对分类结果的准确性有了一定的改进.     

16.

Web Service并发系统的设计与研究   总被引：2，自引：1，他引：2  

孙斐  邱锦伦 《计算机工程与设计》2008,29(23)

由于Internet的发展和大规模应用需求的不断涌现,单个甚至多个WebServices也往往不能很好地满足一些复杂的应用.提出了WebService并发系统的设想.在实现技术上,通过划分小粒度服务,对小粒度服务的调用和整合,最后通过WebService并发总线控制实现WebService的并发系统.从而提高服务的速度和效率,充分利用资源并保证负载平衡.该模型对于改造和开发企业级Web Service有一定的适应性.     

17.

基于网页标识的图像内容过滤防火墙的研究     

王少鹏  孙敏 《计算机工程与应用》2008,44(1):119-122

提出一种新的图像内容过滤防火墙,针对网页标识（Logo）进行图像内容分析,综合应用多种算法比较各个标识之间的相似度,由此来判定是否过滤网页。由于这类图像体积小,处理速度快,将其应用在防火墙中很好地改善了当前防火墙在图像内容过滤方面的不足,提高防火墙的过滤能力和过滤速度。     

18.

面向移动终端的Web页面重组技术综述*     

史晶  吴庆波  杨沙洲 《计算机应用研究》2011,28(12):4405-4408

在移动终端上浏览传统Web页面,存在着页面布局不合理、屏幕适应性差、噪声信息多等问题,严重影响页面的显示效果.Web页面重组技术通过对页面信息进行提取、组合,能够有效地解决上述问题,能够满足移动用户丰富多彩的页面体验效果.首先从页面提取和组合等方面对页面重组技术进行了论述,同时分析了相关技术的适用范围以及其复杂性,最后对当前领域研究的重点问题进行了总结.     

19.

A fast HTML web page change detection approach based on hashing and reducing the number of similarity computations     

Hassan  Kassem   《Data & Knowledge Engineering》2008,66(2):326-337

This paper describes a fast HTML web page detection approach that saves computation time by limiting the similarity computations between two versions of a web page to nodes having the same HTML tag type, and by hashing the web page in order to provide direct access to node information. This efficient approach is suitable as a client application and for implementing server applications that could serve the needs of users in monitoring modifications to HTML web pages made over time, and that allow for reporting and visualizing changes and trends in order to gain insight about the significance and types of such changes. The detection of changes across two versions of a page is accomplished by performing similarity computations after transforming the web page into an XML-like structure in which a node corresponds to an open–close HTML tag. Performance and detection reliability results were obtained, and showed speed improvements when compared to the results of a previous approach.     

基于移动搜索的网页设计优化研究

本文介绍了移动搜索的特点、面向移动设备的网站优化规则和策略,从样式表、网页标签、页面结构、网页导航、屏幕分辨率优化等方面,阐述了移动端网页优化的具体方法。     

Web页面清洗技术的研究与实现

文章提出了Ｗeb页面清洗的概念,给出了一种基于规则的Ｗeb页面清洗方案,并实现了基于此方案的系统。文中的页面清洗建立在Ｗeb页面的ＤＯＭ树结构上,并通过人工判定的方法进行了实验和评估,实验结果表明该方案切实可行,清洗方法具有较快的速度和准确性。     

PCA-based web page watermarking

The tamper-proof of web pages is of great importance. Some watermarking schemes have been reported to solve this problem. However, both these watermarking schemes and the traditional hash methods have a problem of increasing file size. In this paper, we propose a novel watermarking scheme for the tamper-proof of web pages, which is free of this embarrassment. For a web page, the proposed scheme generates watermarks based on the principal component analysis (PCA) technique. The watermarks are then embedded into the web page through the upper and lower cases of letters in HTML tags. When a watermarked web page is tampered, the extracted watermarks can detect the modifications to the web page, thus we can keep the tampered one from being published. Extensive experiments are performed on the proposed scheme and the results show that the proposed scheme can be a feasible and efficient tool for the tamper-proof of web pages.     

Web页面主题相关性排序算法的研究

分析了Web页面主题的分布的特点,对经典的页面排序算法进行了探讨,提出了一种基于内容和超链接分析并结合用户点击行为的相关性排序算法。该算法考虑了超文本标记、锚文本、文本内容等对相关性的影响,引入动态比较矩阵来计算相应的权重系数,能够客观分析网页所包含的主题信息,使检索结果排序更合理。实验表明,该算法能有效提高查准率,较好地解决了主题的漂移现象,且具有较好的性能。     

Device-aware desktop web page transformation for rendering on handhelds

This paper illustrates a new approach to automatic re-authoring of web pages for rendering on small-screen devices. The approach is based on automatic detection of the device type and screen size from the HTTP request header to render a desktop web page or a transformed one for display on small screen devices, for example, PDAs. Known algorithms (transforms) are employed to reduce the size of page elements, to hide parts of the text, and to transform tables into text while preserving the structural format of the web page. The system comprises a preprocessor that works offline and a just-in-time handler that responds to HTTP requests. The preprocessor employs Cascading Style Sheets (CSS) to set default attributes for the page and prepares it for the handler. The latter is responsible for downsizing graphical elements in the page, converting tables to text, and inserting visibility attributes and JavaScript code to allow the user of the client device to interact with the page and cause parts of the text to disappear or reappear. A system was developed that implements the approach and was used it to collect performance results and conduct usability testing. The importance of the approach lies in its ability to display hidden parts of the web page without having to revisit the server, thus reducing user wait times considerably, saving battery power, and cutting down on wireless network traffic.     

基于改进DOM树的网页去噪声方法

一个普通的Web网页可以被分成信息块和噪声块两个部分,因此快速准确地去除网页中的噪声块是网页信息采集处理的关键技术之一.利用网页模板的相似性,提出一种基于改进DOM的网页噪声去除算法,并将通过此算法处理的网页用于SVM分类系统,结果显示经过处理的网页对分类结果的准确性有了一定的改进.     

Web Service并发系统的设计与研究

由于Internet的发展和大规模应用需求的不断涌现,单个甚至多个WebServices也往往不能很好地满足一些复杂的应用.提出了WebService并发系统的设想.在实现技术上,通过划分小粒度服务,对小粒度服务的调用和整合,最后通过WebService并发总线控制实现WebService的并发系统.从而提高服务的速度和效率,充分利用资源并保证负载平衡.该模型对于改造和开发企业级Web Service有一定的适应性.     

基于网页标识的图像内容过滤防火墙的研究

提出一种新的图像内容过滤防火墙,针对网页标识（Logo）进行图像内容分析,综合应用多种算法比较各个标识之间的相似度,由此来判定是否过滤网页。由于这类图像体积小,处理速度快,将其应用在防火墙中很好地改善了当前防火墙在图像内容过滤方面的不足,提高防火墙的过滤能力和过滤速度。     

面向移动终端的Web页面重组技术综述*

在移动终端上浏览传统Web页面,存在着页面布局不合理、屏幕适应性差、噪声信息多等问题,严重影响页面的显示效果.Web页面重组技术通过对页面信息进行提取、组合,能够有效地解决上述问题,能够满足移动用户丰富多彩的页面体验效果.首先从页面提取和组合等方面对页面重组技术进行了论述,同时分析了相关技术的适用范围以及其复杂性,最后对当前领域研究的重点问题进行了总结.     

A fast HTML web page change detection approach based on hashing and reducing the number of similarity computations

This paper describes a fast HTML web page detection approach that saves computation time by limiting the similarity computations between two versions of a web page to nodes having the same HTML tag type, and by hashing the web page in order to provide direct access to node information. This efficient approach is suitable as a client application and for implementing server applications that could serve the needs of users in monitoring modifications to HTML web pages made over time, and that allow for reporting and visualizing changes and trends in order to gain insight about the significance and types of such changes. The detection of changes across two versions of a page is accomplished by performing similarity computations after transforming the web page into an XML-like structure in which a node corresponds to an open–close HTML tag. Performance and detection reliability results were obtained, and showed speed improvements when compared to the results of a previous approach.