Analyzing the feasibility of building a new mass storage system on distributed resources

The average PC now contains a large and increasing amount of storage with an ever greater amount left unused. We believe there is an opportunity for organizations to harness the vast unused storage capacity on their PCs to create a very large, low‐cost, shared storage system. What is needed is the proper storage system architecture and software to exploit and manage the unused portions of existing PC storage devices across an organization and make it reliably accessible to users and applications. We call our vision of such a storage system Storage@desk (SD). This paper describes our first step towards the realization of SD—a study of machine and storage characteristics and usage in a model organization. We studied 729 PCs in an academic institution for 91 days, monitoring the configuration, load and usage of the major machine subsystems, i.e. disk, memory, CPU and network. To further analyze the availability characteristics of storage in an SD system, we performed a trace‐driven simulation of some basic storage allocation strategies. This paper presents the results of our data collection efforts, our analysis of the data, our simulation results and our conclusion that an SD system is indeed feasible and holds promise as a cost‐effective way to create massive storage systems. Copyright © 2007 John Wiley & Sons, Ltd.     

An Integrated Approach to Security on Demand in ATM Networks

Abstract

With the increasing use of Web services, many new challenges concerning data security are becoming critical. Especially in mobile services, where clients are generally thin in terms of computation power and storage space, a remote server can be outsourced for the computation or can act as a data store. Unfortunately, such a data store may not always be trustworthy, and clients with sensitive data and queries may want protection from malicious attacks. This article presents a technique to hide tree-structured data from potentially malicious data stores, while allowing clients to traverse the data to locate an object of interest without leaking information to the data store. The two motivating applications for this approach are hiding (1) tree-like XML data as well as XML queries that are in the form of tree-paths, and (2) tree-structured indexes and queries executed on such data structures. We show that this task is achievable through a one-server protocol that introduces only a limited and adjustable communication overhead. This is especially essential in low-bandwidth (such as wireless) distributed environments. The proposed protocol has desirable communication and concurrency performance, as demonstrated by the experiments we have conducted.     

MidCloud: an agent‐based middleware for effective utilization of replicated Cloud services

The Cloud relies heavily on resource replication to support the demands of the clients efficiently. Replicated Cloud services are distributed across large geographic areas and are accessible via the Internet. This paper describes MidCloud; an agent‐based middleware that provides Cloud clients with dynamic load balancing and fault tolerance mechanisms for effective utilization of replicated Cloud services and resources. MidCloud can be used to connect clients with multiple replicated Cloud services and provide fast and reliable service delivery from multiple replicas. Several approaches for load balancing and fault tolerance in distributed systems were introduced; however, they require prior knowledge of the environment's operating conditions and/or constant monitoring of these conditions at run time that allows the applications to adjust the load and redistribute the tasks when operational conditions change and when failures occur. These techniques work well when there is no high communication delay. Yet, this is not true in the Cloud, where data storage and computation servers are scattered all over the world and communication delays are usually very high. MidCloud deploys approaches to reduce the negative impact of high and dynamic delays on the Cloud servers and the Internet. The experimental results show the positive effects of using MidCloud to provide efficient load balancing and fault tolerance. Copyright © 2013 John Wiley & Sons, Ltd.     

Zero knowledge based client side deduplication for encrypted files of secure cloud storage in smart cities

As typical applications in the field of the cloud computing, cloud storage services are popular in the development of smart cities for their low costs and huge storage capacity. Proofs-of-ownership (PoW) is an important cryptographic primitive in cloud storage to ensure that a client holds the whole file rather than part of it in secure client side data deduplication. The previous PoW schemes worked well when the file is in plaintext. However, the privacy of the clients’ data may be vulnerable to honest-but-curious attacks. To deal with this issue, the clients tend to encrypt files before outsourcing them to the cloud, which makes the existing PoW schemes inapplicable any more. In this paper, we first propose a secure zero-knowledge based client side deduplication scheme over encrypted files. We prove that the proposed scheme is sound, complete and zero-knowledge. The scheme can achieve a high detection probability of the clients’ misbehavior. Then we introduced a proxy re-encryption based key distribution scheme. This scheme ensures that the server knows nothing about the encryption key even though it acts as a proxy to help distributing the file encryption key. It also enables the clients who have gained the ownership of a file to share the file with the encryption key generated without establishing secure channels among them. It is proved that the clients’ private key cannot be recovered by the server or clients collusion attacks during the key distribution phase. Our performance evaluation shows that the proposed scheme is much more efficient than the existing client side deduplication schemes.     

一种面向数据可用性和存储可靠性动态要求的自适应纠删码存储策略设计

为了满足指数级增长的大数据存储需求,现代的分布式存储系统需要提供大容量的存储空间以及快速的存储服务.因此在主流的分布式存储系统中,均应用了纠删码技术以节约数据中心的磁盘成本,保证数据的可靠性,并且满足应用程序和客户端的快速存储需求.在实际应用中数据往往重要程度并不相同,对数据可用性要求不一,且不同磁盘的故障率和可靠性动态不一的特点,对于传统RAID存储方式包括基于纠删码的存储系统提出了新的挑战.本文提出了一种面向数据可用性和磁盘可靠性动态要求的灵活自适应纠删码存储设计On-demand ARECS(On-demand Availability and Reliability Oriented Adaptive Erasure Coded Storage System),根据存储后端数据可用性和磁盘可靠性的多个维度进行设计,综合确定纠删码编码策略和存储节点选择,从而减少存储冗余度和存储延迟,同时提高数据可用性和存储可靠性.我们在Tahoe-LAFS开源分布式文件系统中进行了实验,实验结果验证了我们的理论分析,在保证具有多样性要求的数据可用性和磁盘可靠性的前提下,明显减少了数据冗余度和存储延迟.     

DOLAR: virtualizing heterogeneous information spaces to support their expansion

Users expect applications to successfully cope with the expansion of information as necessitated by the continuous inclusion of novel types of content. Given that such content may originate from ‘not‐seen thus far’ data collections and/or data sources, the challenging issue is to achieve the return of investment on existing services, adapting to new information without changing existing business‐logic implementation. To address this need, we introduce DOLAR (Data Object Language And Runtime), a service‐neutral framework which virtualizes the information space to avoid invasive, time‐consuming, and expensive source‐code extensions that frequently break applications. Specifically, DOLAR automates the introduction of new business‐logic objects in terms of the proposed virtual ‘content objects’. Such user‐specified virtual objects align to storage artifacts and help realize uniform ‘store‐to‐user’ data flows atop heterogeneous sources, while offering the reverse ‘user‐to‐store’ flows with identical effectiveness and ease of use. In addition, the suggested virtual object composition schemes help decouple business logic from any content origin, storage and/or structural details, allowing applications to support novel types of items without modifying their service provisions. We expect that content‐rich applications will benefit from our approach and demonstrate how DOLAR has assisted in the cost‐effective development and gradual expansion of a production‐quality digital library. Experimentation shows that our approach imposes minimal overheads and DOLAR‐based applications scale as well as any underlying datastore(s). Copyright © 2011 John Wiley & Sons, Ltd.     

A data access framework for service-oriented rich clients

Facilitated by the SOA and new Web technologies, Service-Oriented Rich Clients (SORCs) compose various Web-delivered services in Web browser to create new applications. The SORCs support client-side data storage and manipulation and provide more features than traditional thin clients. However, the SORCs might suffer from data access issues, mainly due to both client-side incompatible data sources and server-side improper or even undesirable cache strategies. Addressing the data access issues, this paper proposes a data access framework for SORCs. The main contributions of this paper are as follows. First, the framework makes the SORCs accommodate heterogeneous local storage solutions and diverse Web browsers properly. The framework abstracts the underlying details of different local storages and selects the most proper data sources for current SORC in use. Secondly, the framework provides a cache mechanism, which supports client-side customized cache strategies. An adaptive technique for the strategies is also proposed to adjust cache strategies based on users?? historical actions to achieve better performance.     

Configuring large‐scale storage using a middleware with machine learning

The proliferation of cloud services and other forms of service‐oriented computing continues to accelerate. Alongside this development is an ever‐increasing need for storage within the data centres that host these services. Management applications used by cloud providers to configure their infrastructure should ideally operate in terms of high‐level policy goals, and not burden administrators with the details presented by particular instances of storage systems. One common technology used by cloud providers is the Storage Area Network (SAN). Support for seamless scalability is engineered into SAN devices. However, SAN infrastructure has a very large parameter space: their optimal deployment is a difficult challenge, and subsequent management in cloud storage continues to be difficult. parindent = 10pt In this article, we discuss our work in SAN configuration middleware, which aims to provide users of large‐scale storage infrastructure such as cloud providers with tools to assist them in their management and evolution of heterogeneous SAN environments. We propose a middleware rather than a stand‐alone tool so that the middleware can be a proxy for interacting with, and informing, a central repository of SAN configurations. Storage system users can have their SAN configurations validated against a knowledge base of best practices that are contained within the central repository. Desensitized information is exported from local management applications to the repository, and the local middleware can subscribe to updates that proactively notify storage users should particular configurations be updated to be considered as sub‐optimal, or unsafe. Copyright © 2011 John Wiley & Sons, Ltd.     

SchedSP: a Grid‐based application service provider of scheduling solutions

In this paper, SchedSP, a middleware framework for providing scheduling solutions as services over the Internet, is presented. Emphasis is given on creating a reusable framework that facilitates the development of specialized clients for the input, output and control interfaces of the various scheduling applications. SchedSP manages the task of preparing and running the required processes and allows the application interface developer to focus on the functionality and efficiency of the interface. The Internet‐based scheduling applications created are competitive in all aspects with traditional locally executed applications. In this paper, detailed architecture and implementation details of the SchedSP framework prototype are presented. In addition, the methodology for creating specific case studies based on the SchedSP middleware framework is presented. Copyright © 2005 John Wiley & Sons, Ltd.     

From broadcast television to Internet audio/video: techniques and tools for VCR‐style interactivity

One of the new applications evolving in the Internet is streaming audio/video. A major reason for its growing popularity is interest in the compelling new services that become possible. Prototype services are being developed which are new to the Internet but offer the same look, feel, and functionality that have traditionally only been found in services delivered via other communication medium, e.g. broadcast television. In addition, the Internet is evolving to offer ‘value‐added’ services, like streaming audio/video with VCR‐style interactivity and embedded hyperlinks. We are poised both on seeing the development of new paradigms for interacting with audio/video, and on seeing the merging of broadcast television and Internet‐based broadcasts. Before this process can be considered successful, a number of technical challenges, derived from the various ways in which content is physically delivered, must be solved. In this paper, we focus on the value‐added service of VCR interactivity. VCR interactivity has long been a challenge for both broadcast television and streamed Internet audio/video. The challenge is how to provide individualized playout for content being streamed to a large group of users using one‐to‐many delivery. While some new companies are starting to offer devices which provide this kind of service for broadcast television, there are still numerous technical challenges for the Internet‐based version of a similar service. This paper has a three‐fold objective. First, we describe the types of services available in the traditional broadcast infrastructure and compare these to the types of services that are deployed or possible in Internet‐based services. Second, we describe our attempts to implement some of the more challenging and novel service types. In particular, we examine client‐based control of programs streamed over the Internet to tens, thousands, or even millions of users. Finally, we discuss the impact of these services on the protocols and applications used to support Internet‐based, multi‐party conferencing. Copyright © 2001 John Wiley & Sons, Ltd.     

A cost‐efficient resource provisioning algorithm for DHT‐based cloud storage systems

Personal cloud storage provides users with convenient data access services. Service providers build distributed storage systems by utilizing cloud resources with distributed hash table (DHT), so as to enhance system scalability. Efficient resource provisioning could not only guarantee service performance, but help providers to save cost. However, the interactions among servers in a DHT‐based cloud storage system depend on the routing process, which makes its execution logic more complicated than traditional multi‐tier applications. In addition, production data centers often comprise heterogeneous machines with different capacities. Few studies have fully considered the heterogeneity of cloud resources, which brings new challenges to resource provisioning. To address these challenges, this paper presents a novel resource provisioning model for service providers. The model utilizes queuing network for analysis of both service performance and cost estimation. Then, the problem is defined as a cost optimization with performance constraints. We propose a cost‐efficient algorithm to decompose the original problem into a sub‐optimization one. Furthermore, we implement a prototype system on top of an infrastructure platform built with OpenStack. It has been deployed in our campus network. Based on real‐world traces collected from our system and Dropbox, we validate the efficiency of our proposed algorithms by extensive experiments. Copyright © 2016 John Wiley & Sons, Ltd.     

基于动态云的智慧农业架构研究

云计算是所有数据汇集到云中心进行统一处理,用户不再需要部署计算能力很强的客户端,而是直接从“云冶里（服务器端）获得计算能力。流行的云计算架构采用强数据中心（服务端）、弱客户端模式。而建设云计算中心的强服务端,需要硬件高配置,需要良好的网络环境,造成建设云中心资金投入过大。对于智慧农业云架构,亦是如此。目前智慧农业的各网络客户端硬件配置优良。文中提出基于动态云的智慧农业架构,该架构通过将具有一定存储能力、数据处理能力、能够提供一定网络服务能力的客户端,根据需要,结合到云数据中心,使该中心的规模动态地扩大或者缩小。实际上,就是让某客户端为云服务中的其他客户端提供服务,减轻实际云数据中心的数据存储、数据处理、资源配置等压力。基于该动态云的智慧农业架构部分已经实现,并且在南京和安徽地区的部分农产品的各环节中得到稳定应用,证明了该架构的实用性。     

Customizing the swarm storage system using agents

Swarm is a scalable, modular storage system that uses agents to customize low‐level storage functions to meet the needs of high‐level services. Agents influence low‐level storage functions such as data layout, metadata management, and crash recovery. An agent is a program that is attached to data in the storage system and invoked when events occur during the data's lifetime. For example, before Swarm writes data to disk, agents attached to the data are invoked to determine a layout policy. Agents are typically persistent, remaining attached to the data they manage until the data are deleted; this allows agents to continue to affect how the data are handled long after the application or storage service that created the data has terminated. In this paper, we present Swarm's agent architecture, describe the types of agents that Swarm supports and the infrastructure used to support them, and discuss their performance overhead and security implications. We describe how several storage services and applications use agents, and the benefits they derive from doing so. Copyright © 2005 John Wiley & Sons, Ltd.     

Dynamic proxy-assisted scalable broadcasting of videos for heterogeneous environments

Periodic broadcasting (PB) is a scalable technique for providing video-on-demand services. It significantly reduces server I/O and backbone network bandwidth requirements at the expense of high storage space and high network bandwidth requirements for clients. Traditional protocols assume homogeneous clients with identical resources. Unfortunately, in practice clients have very different bandwidths, and these are usually insufficient to provide video-on-demand (VoD) service from a PB server. Existing work on heterogeneous clients has focused on devising broadcast schedules to cater to low-bandwidth clients, which inevitably requires an extra backbone network bandwidth between the server and the clients. In our previous work, we proposed to use proxies residing at the edge of backbone network to accommodate low bandwidth clients for PB-based VoD services. The server broadcasts a video using a PB protocol while the proxy receives and stores the data in its local buffer and broadcasts the stored data to the clients in its local network. It significantly reduces the waiting time of low-bandwidth clients without requiring any extra backbone bandwidth by using a proxy buffer and channels. However, although lots of PB protocols have been proposed, the scheme can be applied only to some old PB protocols based on a pyramid protocol. In this paper, we propose a proxy-assisted PB system that can be generally applied to almost all the existing PB protocols, by dynamically managing buffer space and channels in proxy servers. Thus, with our proposed system, PB VoD system can be optimized in terms of the resource usages in backbone networks, proxy servers, and clients, by adopting more suitable PB protocols.     

Improving the I/O performance of intermediate multimedia storage nodes

The data retrieval operations in servers and proxies for Media-on-Demand applications represent a severe bottleneck, because a potentially (very) high number of users concurrently retrieve data with high data rates. In the Intermediate Storage Node Concept (INSTANCE) project, we have developed a new architecture for Media-on-Demand storage nodes that maximizes the number of concurrent clients a single node can support. We avoid the traditional bottlenecks, like copy operations, multiple copies of the same data element in main memory, and checksum calculation in communication protocols, by designing, implementing, and tightly integrating three orthogonal techniques: a zero-copy-one-copy memory architecture, network level framing, and integrated error management. In this paper, we describe the INSTANCE storage node, and present an evaluation of our mechanisms. Our experimental performance results show that the integration of these three techniques in NetBSD at least doubles the number of concurrent clients that a single storage node can serve in our testbed.     

基于Overlay Networks的区分服务模型

如何为基于重叠网的各种业务提供端到端,QoS是一个崭新的研究领域。提出一个基于重叠网的区分服务模型——ONDS。该模型利用数据源累积分层、服务聚集和服务管道等应用层技术,并针对ONDS的业务需求设计一种接纳控制算法。具有良好的可扩展性和灵活性。模拟实验结果表明ONDS模型能够为端用户提供服务区分和良好的QoS服务。     

PANGAEA—an information system for environmental sciences

PANGAEA is an information system for processing, long-term storage, and publication of georeferenced data related to earth science fields. Essential services supplied by PANGAEA are project data management and the distribution of visualization and analysis software. Organization of data management includes quality control and publication of data and the dissemination of metadata according to international standards. Data managers are responsible for acquisition and maintenance of data. The data model used reflect the information processing steps in the earth science fields and can handle any related analytical data. The basic technical structure corresponds to a three tiered client/server architecture with a number of comprehensive clients and middleware components controlling the information flow and quality. On the server side a relational database management system (RDBMS) is used for information storage. The web-based clients include a simple search engine (PangaVista) and a data mining tool (ART). The client used for maintenance of information contents is optimized for data management purposes. Analysis and visualization of metainformation and analytical data is supported by a number of software tools, which can either be used as ‘plug-ins’ of the PANGAEA clients or as standalone applications, distributed as freeware from the PANGAEA website. Established and well-documented software tools are the mini-GIS PanMap, the plotting tool PanPlot, and Ocean Data View (ODV) for the exploration of oceanographic data. PANGAEA operates on a long-term basis. The available resources are sufficient not only for the acquisition of new data and the maintenance of the system but also for further technical and organizational developments.     

Scheduling time‐critical requests for multiple data objects in on‐demand broadcast

On‐demand broadcast is an effective data dissemination approach in mobile computing environments. Most of the recent studies on on‐demand data broadcast assume that clients request only a single‐data‐object at a time. This assumption may not be practical for the increasingly sophisticated mobile applications. In this paper, we investigate the scheduling problem of time‐critical requests for multiple data objects in on‐demand broadcast environments and observe that existing scheduling algorithms designed for single‐data‐object requests perform unsatisfactorily in this new setting. Based on our analysis, we propose new algorithms to improve the system performance. Copyright © 2010 John Wiley & Sons, Ltd.     

ProcessAtlas: A scalable and extensible platform for business process analytics

In today's knowledge‐, service‐, and cloud‐based economy, an overwhelming amount of business‐related data are being generated at a fast rate daily from a wide range of sources. These data increasingly show all the typical properties of big data: wide physical distribution, diversity of formats, nonstandard data models, and independently managed and heterogeneous semantics. In this context, there is a need for new scalable and process‐aware services for querying, exploration, and analysis of process data in the enterprise because (1) process data analysis services should be capable of processing and querying large amount of data effectively and efficiently and, therefore, have to be able to scale well with the infrastructure's scale and (2) the querying services need to enable users to express their data analysis and querying needs using process‐aware abstractions rather than other lower‐level abstractions. In this paper, we introduce ProcessAtlas, ie, an extensible large‐scale process data querying and analysis platform for analyzing process data in the enterprise. The ProcessAtlas platform offers an extensible architecture by adopting a service‐based model so that new analytical services can be plugged into the platform. In ProcessAtlas, we present a domain‐specific model for representing process knowledge, ie, process‐level entities, abstractions, and the relationships among them modeled as graphs. We provide services for discovering, extracting, and analyzing process data. We provide efficient mapping and execution of process‐level queries into graph‐level queries by using scalable process query services to deal with the process data size growth and with the infrastructure's scale. We have implemented ProcessAtlas as a MapReduce‐based prototype and report on experiments performed on both synthetic and real‐world datasets.