集中管理式Web缓存系统及性能分析

共享缓存文件是减少网络通信量和服务器负载的重要方法，本文在介绍Web Caching技术及流行的Web缓存通信协议ICP的基础上，提出了一种集中管理式Web缓存系统，该系统通过将用户的HTTP请求，按照一定的算法分发到系统中某一合适的缓存服务器上，从而消除了缓存系统内部服务器之间庞大的通信开销及缓存处理负担，减少了缓存内容的冗余度．通过分析，证明了集中管理式Web缓存系统比基于ICP的简单缓存系统具有缓存效率高、处理开销低、延迟小等优点，并且该系统具有良好的可扩展性．     

一种新的基于连续哈希函数的合作式Web缓存系统

提出了一种新的基于连续哈希函数的合作式缓存系统。针对传统合作式缓存系统中多级转发造成的高时延和多重哈希计算问题，设计了一种高效的Web对象定位和路由模式，保证任意Web请求只需计算一次哈希且至多经过一次转发就可到达目标节点。采用失效-触发的策略来解决路由表一致性维护的问题，减少了网络开销，提高了系统的可扩展性和可靠性。仿真实验表明，该系统性能优于基于互联网缓存协议和缓存阵列路由协议的系统。     

一种域分布式合作Web缓存系统

在分析Internet缓存协议（ICP）和基于集中管理的协作式Web缓存系统（CMCS）的基础上，提出了一种域分布合作Web缓存系统模型（DDCCS），该模型克服了ICP的缓存内容冗余问题和CMCS集中管理的不健壮问题，并能充分考虑各代理缓存器的处理能力和缓存能力，采用区间散化方法和加权区间片分配算法，使得各代理的负担更加均匀合理，在性能上较ICP和CMCS有较大的改进。     

Web缓存层次模型与代价分析

Web缓存分层结构在避免单点失效、提高缓存性能方面具有重要作用.论文研究了Web缓存层次模型,提出请求分发的三种模式,并利用代价函数分析探讨了缓存模型性能.根据Web访问共同特征,实验采用数学建模方法生成模拟日志,模拟不同层采用不同替换算法(LRU、LFU、GDS)时的缓存性能.结果表明,模拟日志的高频区、低频区流行度访问特征分别服从齐普夫第一定律、第二定律,具有真实日志的特性,能够模拟用户请求评价Web缓存层次模型性能;当低层代理缓存采用LFU或LRU替换算法,高层代理缓存采用GDS替换算法时,两层缓存模型在命中率、字节命中率方面有较好的性能表现.     

集群缓存系统中代理缓存技术的研究

针对集群缓存系统的特点,对其内部各代理缓存进行了详细的研究.在缓存副本的存储方面,采用了面向网站的存储方案,更好的实现个性化服务的功能;在缓存副本的管理方面,采用了将替换和一致性结合在一起的RCA算法,使得整个系统有较高的命中陈旧率;在代理协作方面,通过重定向的方法将请求从未命中的代理转发给缓存了请求的代理,避免了代理之间复制对象,有效地减少了缓存对象的冗余度.通过理论分析推导,证明了它能大大减少访问延迟、有效地减轻了Web服务器的负担.实验证明,本协作方案有较高的命中率.     

一个基于集中管理的协作式Web缓存系统

共享不同代理的缓存Web文档是减少通信量和减轻网络瓶颈的重要方法。在分析原有Web缓存通信协议（ICP）的基础上,提出了一种新的协作Web缓存系统（CMCS）并作了分析比较。通过将HTTP请求均匀分散到系统各个代理,消除了代理之间庞大的通信开销以及由此带来的处理负担。在动态变化的网络环境下,有效地将各个代理组织起来,处理来自服务器的文档。另外也克服了以往每个代理里有大量冗余内容,造成各个代理内容趋向的情形。     

一种新型分布式代理缓存系统—双层集群缓存

为了提高代理系统的整体性能,基于内部网络用户访问时间的局部性和相似性,并结合现有的分布式缓存系统,本文提出了一种新型的分布式代理缓存系统——双层缓存集群.双层缓存集群系统分为网内集群缓存层和代理集群缓存层,采用双层代理缓存结构,充分利用现有内部网络资源,分散了代理的负担.降低了代理之间的通信开销,还增强了缓存资源的利用率,提高了用户请求命中率,降低了代理系统的整体资源消耗.     

基于树扩展朴素贝叶斯分类器的Web代理服务器缓存优化

Web代理服务器缓存能在一定程度上减少网络拥塞现象和用户的访问延迟,减轻服务器负载。然而Web代理缓存的缓存命中率和字节命中率较低,并不能很好地起到加速网络请求响应的效果。为此,研究监督学习方法,使用树扩展朴素贝叶斯分类器对Web日志数据进行分类,进而预测可能会再次访问到的Web对象,并结合最近最少使用(LRU)算法,提出一种新的缓存策略。实验结果表明,树扩展的贝叶斯分类器在精度和召回率指标上优于朴素贝叶斯和BP神经网络等分类器,通过树扩展的贝叶斯分类器优化后的缓存策略与普通LRU算法相比,不仅可以提高缓存的效率,而且可有效提高Web代理缓存的请求命中率和字节命中率。     

基于标记的缓存协作分布式Web服务器系统

介绍了提高Web服务器性能的前沿技术--分布式Web服务器系统,讨论了现有各种方案的优缺点,在此基础上提出一种新的分布式Web服务器系统.该系统使用基于标记的缓存协作用户请求分发方法(tag based cache cooperative Web requests distribution,简称TB-CCRD),通过前端机把系统中各个Web服务器的缓存组织成一个大的虚拟缓存系统,提高系统的整体缓存命中率,缩短了请求的响应时间;通过分布式处理TCP连接转交来消除前端机的性能瓶颈;利用标记通告URL在缓存中的位置,避免了额外的系统内部通信.从而得到了一个可扩展的高性能分布式Web服务器系统.     

利用基于内容的缓存代理提高Web服务器性能

分布式Web服务器系统(DWSS)是提高网络服务性能的一种前沿方式.文中分析了当前几种在客户端、主机端提升Web服务性能的方法,提出了一种基于内容的缓存代理协作分布式Web服务器系统模型.该模型可以有效减少用户获得响应的时间,避免了服务器之间不必要的通信,提高了Web服务器整体的性能.     

Exploiting Regularities in Web Traffic Patterns for Cache Replacement

Abstract. Caching web pages at proxies and in web servers' memories can greatly enhance performance. Proxy caching is known to reduce network load and both proxy and server caching can significantly decrease latency. Web caching problems have different properties than traditional operating systems caching, and cache replacement can benefit by recognizing and exploiting these differences. We address two aspects of the predictability of traffic patterns: the overall load experienced by large proxy and web servers, and the distinct access patterns of individual pages. We formalize the notion of ``cache load' under various replacement policies, including LRU and LFU, and demonstrate that the trace of a large proxy server exhibits regular load. Predictable load allows for improved design, analysis, and experimental evaluation of replacement policies. We provide a simple and (near) optimal replacement policy when each page request has an associated distribution function on the next request time of the page. Without the predictable load assumption, no such online policy is possible and it is known that even obtaining an offline optimum is hard. For experiments, predictable load enables comparing and evaluating cache replacement policies using partial traces , containing requests made to only a subset of the pages. Our results are based on considering a simpler caching model which we call the interval caching model . We relate traditional and interval caching policies under predictable load, and derive (near)-optimal replacement policies from their optimal interval caching counterparts.     

一种基于多维特征分析的网页代理服务发现方法

网页代理提供了一种快捷的中继服务,与其它类型的代理服务相比,如隐匿网络/VPN服务/Socks代理等,用户可以不需要安装任何软件就免费使用。因此,网页代理在绕过访问限制、隐藏身份等方面的便利性上有其不可比拟的优势。然而,网页代理在获取个人隐私信息、推送垃圾广告、隐匿行踪等方面也给人们的网络生活带来严重的安全威胁。所以,如何快速有效地将它们与大量正常网页区分开来成为网络空间安全面临的一个重要挑战。针对这一问题,本文提出了一种基于多维特征分析的网页代理发现方法——ProxyMiner。在主动发现方面,引入了网页代理特有的结构特征和内容特征,通过机器学习的方法进行预测发现。在被动发现方面,基于用户访问网页代理特有的访问模式,通过构建二分图对代理用户进行谱聚类分析,获取代理用户群体访问的顶级域名,从而发现网页代理。此方法仅基于客户端IP地址和目标URL,不需要任何有关HTTP头（经常会被恶意修改）或数据包（通常是加密的或不可用的）的信息。实验结果表明,在相同数据集上,相比于传统检测方法,ProxyMiner可以显著提高网页代理检测效果,降低平均检测时间。     

基于网络性能的智能Web加速技术——缓存与预取

Web业务在网络业务中占有很大比重,在无法扩大网络带宽时,需要采取一定技术合理利用带宽,改善网络性能。研究了基于RTT（round trip time)等网络性能指标的Web智能加速技术,在对Web代理服务器上的业务进行分析和对网络RTT进行测量分析的基础上,提出了智能预取控制技术及新的缓存（cache）替换方法。对新算法的仿真研究表明,该方法提高了缓存的命中率。研究表明预取技术在不明显增加网络负荷的前提下,提高了业务的响应速度,有效地改进了Web访问性能。     

PTC: Proxies that Transcode and Cache in Heterogeneous Web Client Environments

Advances in computing and communication technologies have resulted in a wide variety of networked mobile devices that access data over the Internet. In this paper, we argue that servers by themselves may not be able to handle this diversity in client characteristics and so intermediaries, such as proxies, should be employed to handle the mismatch between the server-supplied data and the client capabilities. Since existing proxies are primarily designed to handle traditional wired hosts, such proxy architectures will need to be enhanced to handle mobile devices. We propose such an enhanced proxy architecture that is capable of handling the heterogeneity in client needs—specifically the variations in client bandwidth and display capabilities. Our architecture combines transcoding (which is used to match the fidelity of the requested object to client capabilities) and caching (which is used to reduce the latency for accessing popular objects). Proxies that Transcode and Cache, PTCs, intelligently adapt to prevailing system conditions using learning techniques to decide whether to transcode locally or fetch an appropriate version from the server. Our experimental results indicate that the use of PTCs produces significant improvements in the client response times. We show that such results hold true for a variety of data content types like images and video data. Further, we find that even simple learning techniques can lead to significant performance improvements.     

富客户端技术使用前景分析

早期的WEB应用主要是传输文本数据（比如HTML页面）,而它们传输的是静态的页面,客户端只有和服务器端进行交互,页面的内容才会改变。客户端和服务器端的交互会导致页面的整体切换,RIA（富客户端）技术则克服了HTML的限制,它将页面的切换限制在更小的局部,只有需要切换的内容才会进行更新,这样不但减轻了服务器端的负担,而且传输的内容会更少,更利于数据传输。会获得更好的用户体验。     

Enforcing direct communications between clients and Web servers to improve proxy performance and security

The amount of dynamic Web contents and secured e‐commerce transactions has been dramatically increasing on the Internet, where proxy servers between clients and Web servers are commonly used for the purpose of sharing commonly accessed data and reducing Internet traffic. A significant and unnecessary Web access delay is caused by the overhead in proxy servers to process two types of accesses, namely dynamic Web contents and secured transactions, not only increasing response time, but also raising some security concerns. Conducting experiments on Squid proxy 2.3STABLE4, we have quantified the unnecessary processing overhead to show its significant impact on increased client access response times. We have also analyzed the technical difficulties in eliminating or reducing the processing overhead and the security loopholes based on the existing proxy structure. In order to address these performance and security concerns, we propose a simple but effective technique from the client side that adds a detector interfacing with a browser. With this detector, a standard browser, such as the Netscape/Mozilla, will have simple detective and scheduling functions, called a detective browser. Upon an Internet request from a user, the detective browser can immediately determine whether the requested content is dynamic or secured. If so, the browser will bypass the proxy and forward the request directly to the Web server; otherwise, the request will be processed through the proxy. We implemented a detective browser prototype in Mozilla version 0.9.7, and tested its functionality and effectiveness. Since we have simply moved the necessary detective functions from a proxy server to a browser, the detective browser introduces little overhead to Internet accessing, and our software can be patched to existing browsers easily. Copyright © 2004 John Wiley & Sons, Ltd.     

Adaptive hash routing for a cluster of client-side web proxies

Hash routing reduces cache misses for a cluster of web proxies by eliminating the duplication of cache contents. In this paper, we investigate the optimization of hash routing performance by dynamically adapting object and DNS allocations to the traffic pattern. An analytical model is developed for hash routing that takes into consideration the original request distribution, the object allocation strategy, the speeds of the proxies and the cache hit ratios. Based on this model, the optimal hash routing problem is studied. The analytical results are applied to the design of two adaptive hash routing schemes: ADA-OBJ optimizes object allocation under static client configuration, and ADA-OBJ/DNS optimizes both object and DNS allocations under dynamic client configuration. Trace-driven simulation experiments have been conducted to evaluate the performance of the proposed schemes. The results show that they significantly outperform the intuitive static hash routing scheme based only on the speeds of the proxies.