Web缓存流量特征模型研究和应用

根据Web缓存流量访问特征建立数学模型，设计实现了Web缓存流量特征模拟生成器（WebSimGen）。利用两层代理缓存结构、基于ADF（Aggregation、Disaggregation和Filtering）模型对Web缓存流量的访问特征和性能进行测试，实验表明模拟日志具有和真实日志类似的访问特性。Web生成器具有较大的灵活性，能够克服真实日志的一些缺点，为进一步提高Web缓存性能和预取技术提供了重要依据。     

Web预取模型分析

WWW的快速增长导致网络拥塞和服务器超载.缓存技术被认为是减轻服务器负载、减少网络拥塞、降低客户访问延迟的有效途径之一,但作用有限.为进一步提高WWW性能,引入了预取技术.文中首先介绍了Web预取技术的基本思想及其研究可行性,然后分析了现有Web预取模型,最后给出了一个Web预取模型应具有的关键属性.     

Web预取模型分析

WWW的快速增长导致网络拥塞和服务器超载。缓存技术被认为是减轻服务器负载、减少网络拥塞、降低客户访问延迟的有效途径之一，但作用有限。为进一步提高WWW性能，引入了预取技术。文中首先介绍了Web预取技术的基本思想及其研究可行性，然后分析了现有Web预取模型，最后给出了一个Web预取模型应具有的关键属性。     

Web预取技术的研究

预取技术是提高缓存命中率和解决Web访问延迟问题的主要方案,本文研究了网页预取技术,将数据挖掘应用于Web预取中,设计了一个为用户提供个性化服务的Web预取模型;详细介绍了对Web日志进行预处理的方法;提出了新的预取替换算法。     

Web对象访问特征模拟器的设计与实现

Web缓存是一个提高Web性能非常有效的方法，它可以位于网络的不同位置：客户端，代理服务器端，服务器端。研究表明Web缓存命中率可以达到30％-50％。Web缓存在应用中最大的问题就是Web缓存管理，研究Web访问特征是有效进行Web缓存管理的基础。Web日志生成模拟器对于研究Web缓存系统有很大地帮助，目前有两种方法模拟生成Web访问日志：日志驱动方法，数学模拟方法。日志驱动方法利用对历史日志进行变换来模拟生成新的日志，数学模拟方法在充分研究Ⅵ协对象访问特征的基础上，通过建立数学模型来模拟生成Web日志。该文通过分析Web对象访问特征，采用数学模拟方法分别模拟了Web对象高频区及低频区流行度特征，Web对象大小重尾分布特征，Web访问的时间局部性特征；设计并实现了一个Web日志模拟生成器WEBSIM。该模拟器不仅可以模拟生成Web对象访问日志，而且具有较大的灵活性，为进一步研究Web缓存技术和预取技术提供依据。     

无线Web访问的缓存和预取机制

缓存和预取在提高无线环境下的Web访问性能方面发挥着重要作用。文章研究针对无线局域网的Web缓存和预取机制,分别基于数据挖掘和信息论提出了采用序列挖掘和延迟更新的预测算法,设计了上下文感知的预取算法和获益驱动的缓存替换机制,上述算法已在Web缓存系统OnceEasyCache中实现。性能评估实验结果表明,上述算法的集成能有效地提高缓存命中率和延迟节省率。     

基于AOP的智能Web缓存框架

通过引入面向方面编程技术,提出一种新的智能Web缓存框架。描述该框架的组成结构与工作原理,对缓存设计时需要解决的透明性、一致性、替换算法和预取策略等主要问题进行讨论并给出性能测试和分析。实验结果表明,该缓存框架可较大程度提高Web应用系统的性能。     

一种智能的预取算法

网络延迟问题是用户QoS的主要问题之一,它依赖诸多因素如网络带宽、传输延迟、排队延迟和客户机及服务器的处理速度。目前主要采用缓存和预取技术来减少网络延迟,但缓存技术所提高的缓存代理服务器的命中率是有限的。该文系统地阐述了目前预取算法的基本思想并把它们分成四类:基于流行度、基于交互、基于访问概率和基于数据挖掘的预取算法。在对它们进行分析比较的基础上,提出了一种智能的预取方案。该方案使用模糊匹配来计算用户对页面的访问概率,同时要控制预取的量和预取的时刻,以避免对网络的性能产生负面影响。     

Web访问特征模型建模

Web访问特征模型建模是进行有效Web缓存管理的基础。该文根据Web访问的四个典型特征建立综合的数学模型,实现了Web访问特征建模生成器(WebGenM),实验表明模拟器能较好地模拟网络访问流的特征,而且易于使用,具有较大的灵活性,其为进一步的Web缓存和预取技术的研究提供了重要依据。     

基于主观贝叶斯方法的数据预取技术

随着信息技术的迅速发展,网络规模随着用户数呈指数级增长,也越来越庞大,要提高用户的访问速度,主要采用缓存和预取技术来减少网络延迟,提出了一种智能的预取方案.该方案使用模糊匹配来计算用户对页面的访问概率,用主观贝叶斯方法计算网页的重要性,同时控制预取的量和预取的时刻,以避免对网络的性能产生负面影响.     

ART2 Clustering of Multiple Web Objects for Qualitative Web Prefetching

The web resources in the World Wide Web are rising, to large extent due to the services and applications provided by it. Because web traffic is large, gaining access to these resources incurs user-perceived latency. Although the latency can never be avoided, it can be minimized to a larger extent. Web prefetching is identified as a technique that anticipates the user’s future requests and fetches them into the cache prior to an explicit request made. Because web objects are of various types, a new algorithm is proposed that concentrates on prefetching embedded objects, including audio and video files. Further, clustering is employed using adaptive resonance theory (ART)2 in order to prefetch embedded objects as clusters. For comparative study, the web objects are clustered using ART2, ART1, and other statistical techniques. The clustering results confirm the supremacy of ART2 and, thereby, prefetching web objects in clusters is observed to produce a high hit rate.     

Performance evaluation of Web proxy cache replacement policies

The continued growth of the World-Wide Web and the emergence of new end-user technologies such as cable modems necessitate the use of proxy caches to reduce latency, network traffic and Web server loads. In this paper we analyze the importance of different Web proxy workload characteristics in making good cache replacement decisions. We evaluate workload characteristics such as object size, recency of reference, frequency of reference, and turnover in the active set of objects. Trace-driven simulation is used to evaluate the effectiveness of various replacement policies for Web proxy caches. The extended duration of the trace (117 million requests collected over 5 months) allows long term side effects of replacement policies to be identified and quantified.

Our results indicate that higher cache hit rates are achieved using size-based replacement policies. These policies store a large number of small objects in the cache, thus increasing the probability of an object being in the cache when requested. To achieve higher byte hit rates a few larger files must be retained in the cache. We found frequency-based policies to work best for this metric, as they keep the most popular files, regardless of size, in the cache. With either approach it is important that inactive objects be removed from the cache to prevent performance degradation due to pollution.     

Predictive Prefetching on the Web and Its Potential Impact in the Wide Area

The rapid increase of World Wide Web users and the development of services with high bandwidth requirements have caused the substantial increase of response times for users on the Internet. Web latency would be significantly reduced, if browser, proxy or Web server software could make predictions about the pages that a user is most likely to request next, while the user is viewing the current page, and prefetch their content.In this paper we study Predictive Prefetching on a totally new Web system architecture. This is a system that provides two levels of caching before information reaches the clients. This work analyses prefetching on a Wide Area Network with the above mentioned characteristics. We first provide a structured overview of predictive prefetching and show its wide applicability to various computer systems. The WAN that we refer to is the GRNET academic network in Greece. We rely on log files collected at the network's Transparent cache (primary caching point), located at GRNET's edge connection to the Internet. We present the parameters that are most important for prefetching on GRNET's architecture and provide preliminary results of an experimental study, quantifying the benefits of prefetching on the WAN. Our experimental study includes the evaluation of two prediction algorithms: an n most popular document algorithm and a variation of the PPM (Prediction by Partial Matching) prediction algorithm. Our analysis clearly shows that Predictive prefetching can improve Web response times inside the GRNET WAN without substantial increase in network traffic due to prefetching.     

Web object-based storage management in proxy caches

Proxy caches are essential to improve the performance of the World Wide Web and to enhance user perceived latency. Appropriate cache management strategies are crucial to achieve these goals. In our previous work, we have introduced Web object-based caching policies. A Web object consists of the main HTML page and all of its constituent embedded files. Our studies have shown that these policies improve proxy cache performance substantially.In this paper, we propose a new Web object-based policy to manage the storage system of a proxy cache. We propose two techniques to improve the storage system performance. The first technique is concerned with prefetching the related files belonging to a Web object, from the disk to main memory. This prefetching improves performance as most of the files can be provided from the main memory rather than from the proxy disk. The second technique stores the Web object members in contiguous disk blocks in order to reduce the disk access time. We used trace-driven simulations to study the performance improvements one can obtain with these two techniques. Our results show that the first technique by itself provides up to 50% reduction in hit latency, which is the delay involved in providing a hit document by the proxy. An additional 5% improvement can be obtained by incorporating the second technique.     

HTTP代理服务器的设计与实现

实现了一个代理服务器系统，设计了代理的模型，给出了代理各个模块的设计结构图。该系统使用IP地址和用户口令对用户进行限制，使系统具备了一定的安全访问功能；缓存和预取的使用，提高了客户的响应速度，降低了网络流量。     

A Data Cube Model for Prediction-Based Web Prefetching

Reducing the web latency is one of the primary concerns of Internet research. Web caching and web prefetching are two effective techniques to latency reduction. A primary method for intelligent prefetching is to rank potential web documents based on prediction models that are trained on the past web server and proxy server log data, and to prefetch the highly ranked objects. For this method to work well, the prediction model must be updated constantly, and different queries must be answered efficiently. In this paper we present a data-cube model to represent Web access sessions for data mining for supporting the prediction model construction. The cube model organizes session data into three dimensions. With the data cube in place, we apply efficient data mining algorithms for clustering and correlation analysis. As a result of the analysis, the web page clusters can then be used to guide the prefetching system. In this paper, we propose an integrated web-caching and web-prefetching model, where the issues of prefetching aggressiveness, replacement policy and increased network traffic are addressed together in an integrated framework. The core of our integrated solution is a prediction model based on statistical correlation between web objects. This model can be frequently updated by querying the data cube of web server logs. This integrated data cube and prediction based prefetching framework represents a first such effort in our knowledge.     

Performance analysis of a client-side caching/prefetching system for Web traffic

Network congestion remains one of the main barriers to the continuing success of the Internet. For Web users, congestion manifests itself in unacceptably long response times. One possible remedy to the latency problem is to use caching at the client, at the proxy server, or within the Internet. However, Web documents are becoming increasingly dynamic (i.e., have short lifetimes), which limits the potential benefit of caching. The performance of a Web caching system can be dramatically increased by integrating document prefetching (a.k.a. “proactive caching”) into its design. Although prefetching reduces the response time of a requested document, it also increases the network load, as some documents will be unnecessarily prefetched (due to the imprecision in the prediction algorithm). In this study, we analyze the confluence of the two effects through a tractable mathematical model that enables us to establish the conditions under which prefetching reduces the average response time of a requested document. The model accommodates both passive client and proxy caching along with prefetching. Our analysis is used to dynamically compute the “optimal” number of documents to prefetch in the subsequent client’s idle (think) period. In general, this optimal number is determined through a simple numerical procedure. Closed-form expressions for this optimal number are obtained for special yet important cases. We discuss how our analytical results can be used to optimally adapt the parameters of an actual prefetching system. Simulations are used to validate our analysis and study the interactions among various system parameters.