企业内部分布式存储高效扩展机制研究

在研究文件存储基础设备与技术发展的基础上,提出一种可高效扩展的分布式存储机制,将企业中已有的服务器和存储设备作为分布式存储的存储单元,将文件作为存储对象,建立一种高效率甚至是零等待时间的可以随时扩展或减少存储单元的分布式存储机制。企业中存在的服务器和存储设备较多时,利用该分布式存储机制后存储性能会得到较大的提升,管理人员文件管理的劳动强度将大大降低。重点是能够帮助企业充分利用原有的服务器和存储设备,减少更新换代的一次性大量投入。     

基于SAN的磁盘数据备份和管理软件设计

存储区域网络是一个连接服务器和存储设备的网络,是存储领域最流行的技术。它采用光纤通道技术实现服务器和存储设备之间的互联,为服务器和存储设备之间提供高速的数据传输通道。这里讨论了SAN带来的优势,分析了数据备份技术,并对它们各自的优缺点做了比较。最后,给出了一个基于SAN的数据备份软件的设计方案。     

基于随机森林的高能物理数据放置策略

随着LHAASO高海拔宇宙线等高能物理实验规模的不断扩大,每年需要存储PB级的海量物理数据。高能物理海量存储系统一般采用随机的数据放置策略,没有考虑数据访问场景和服务器节点、存储设备的差异性。针对以上问题,提出一种异构存储环境下基于随机森林算法的数据放置策略,根据存储设备性能差异划分快慢存储池,同时对后期文件的读写访问场景进行预测和识别,综合考虑当前设备负载为数据找到最佳的放置位置。使用真实物理实验数据验证了算法的有效性。     

一个网络文件存储系统TNS关键技术

基于多服务器架构、为多用户服务的网络文件存储系统普遍存在资源分配不均,重复文件多,存储空间浪费严重的问题。设计并实现了TNS网络文件存储系统,该系统基于多服务器存储架构,分别由用户服务器、索引服务器、数据服务器、共享服务器、管理服务器和登录服务器组成,为多用户服务,采用一致性Hash实现负载均衡,支持在客户端进行文件粒度的重复数据删除。经过实际生产环境运行测试,具有良好的负载均衡能力和重复数据删除功能,可以有效节省存储空间,提高存储设备利用率。     

空间数据存储对象的元数据可伸缩性管理*

基于空间数据面向对象存储思想和云存储可扩展架构,将控制信息集中在元数据服务器集群中管理,而实际的空间数据基于对象存储分布到存储设备集群中,实现控制信息路径与数据传输路径的分离,并缓存热点空间数据对象接口,以减少元数据访问次数和降低其服务器负载;基于对象存储设备的并行性和CDMI标准对元数据进行自上而下的功能分层管理,增...     

基于IP的SAN存储区域网络技术

SAN是一个连接服务器和存储设备的网络,是存储区域中最流行的技术。详细阐述和分析了基于iSCSI协议的IP SAN环境。通过IP SAN与InfiniBand服务器结构的整合方式,表明IP SAN具有高性能、低成本和数据共享的优势。     

IPoIB体系结构及其应用

InfiniRand (IB) is a high speed, channel based interconnecting technology between systems and devices. It will remove the bottleneck limiting the performance of current servers and storage divices. This article proposes an IP over IrdiniBand (IPoIB) architecture and its application. It will function as a bridge between IP and IB architecture insystems IP and IB coexists. It enables the applications to take advantages of IB without changes in upper layers IP and other protocols. We also introduce the IPoIB protocol and show some experiment results in this paper. The experi-ment results show that IPoIB is suitable to be used in the front-end servers and interface devices for the IrdiniBand-based network servers.     

Highly available and heterogeneous continuous media storage systems

A number of recent technological trends have made data intensive applications such as continuous media (audio and video) servers a reality. These servers store and retrieve large volumes of data using magnetic disks. Servers consisting of multiple nodes and large arrays of heterogeneous disk drives have become a fact of life for several reasons. First, magnetic disks might fail. Failed disks are almost always replaced with newer disk models because the current technological trend for these devices is one of annual increase in both performance and storage capacity. Second, storage requirements are ever increasing, forcing servers to be scaled up progressively. In this study, we present a framework to enable parity-based data protection for heterogeneous storage systems and to compute their mean lifetime. We describe the tradeoffs associated with three alternative techniques: independent subservers, dependent subservers, and disk merging. The disk merging approach provides a solution for systems that require highly available secondary storage in environments that also necessitate maximum flexibility.     

Exploring and Modelling IoT Offloading Policies in Edge Cloud Environments

The Internet of Things (IoT) has recently become a popular technology that can play increasingly important roles in every aspect of our daily life. For collaboration between IoT devices and edge cloud servers, edge server nodes provide the computation and storage capabilities for IoT devices through the task offloading process for accelerating tasks with large resource requests. However, the quantitative impact of different offloading architectures and policies on IoT applications’ performance remains far from clear, especially with a dynamic and unpredictable range of connected physical and virtual devices. To this end, this work models the performance impact by exploiting a potential latency that exhibits within the environment of edge cloud. Also, it investigates and compares the effects of loosely-coupled (LC) and orchestrator-enabled (OE) architecture. The LC scheme can smoothly address task redistribution with less time consumption for the offloading sceneries with small scale and small task requests. Moreover, the OE scheme not only outperforms the LC scheme in the large-scale tasks requests and offloading occurs but also reduces the overall time by 28.19%. Finally, to achieve optimized solutions for optimal offloading placement with different constraints, orchestration is important.     

SAN存储技术

目前,存储应用的体系结构主要有DAS、NAS和SAN 3种模式。3种模式从体系架构的逻辑上看,有明显的区别。SAN是通过专用高速网将一个或多个网络存储设备和服务器连接起来的专用存储系统。可采用光纤通道、IP/Ethernet、InfiniBand互联技术分别组建FC-SAN、iSCSI-SAN和InfiniBand-SAN。     

Network and device-level impacts: performance and reliability of active I/O storage systems

Distributed active storage architectures are designed to offload user-level processing to the peripheral from the host servers. In this paper, we report preliminary investigation on performance and fault recovery designs, as impacted by emerging storage interconnect protocols and state-of-the-art storage devices. Empirical results obtained using validated device-level and interconnect data demonstrate the significance of the said parameters on the overall system performance and reliability.     

跨域客户间口令认证的密钥交换协议

为实现跨域客户与客户之间安全的通信,并提高该过程的效率和实用性,提出一种使用普通存储设备或移动智能设备实现基于口令认证的密钥交换协议.该协议使用普通存储设备和随机数取代昂贵的智能卡和时钟同步系统,使得它在私钥环境下可以抵抗各种已知的攻击,达到相应的安全性要求.保留了智能卡的优点,无需保存客户的口令表或者验证表,消除智能卡在实际应用中受到的各种限制.普通存储设备和随机数的引入增强了协议的安全性、提高了协议执行的效率和实用性.     

Online Bicriteria Load Balancing Using Object Reallocation

We study the bicriteria load balancing problem on two independent parameters under the allowance of object reallocation. The scenario is a system of $M$ distributed file servers located in a cluster, and we propose three online approximate algorithms for balancing their loads and required storage spaces during document placement. The first algorithm is for heterogeneous servers. Each server has its individual tradeoff of load and storage space under the same rule of selection. The other two algorithms are for homogeneous servers. The second algorithm combines the idea of the first one and the best existing solution for homogeneous servers. Using document reallocation, we obtain a smooth tradeoff curve of the upper bounds of load and storage space. The last one bounds the load and storage space of each server by less than three times of their trivial lower bounds, respectively; and more importantly, for each server, the value of at least one parameter is far from its worst case. The time complexities of these three algorithms are $O(log M)$ plus the cost of document reallocation.     

CPicker: Leveraging Performance-Equivalent Configurations to Improve Data Center Energy Efficiency

The poor energy proportionality of server is seen as the principal source for low energy efficiency of modern data centers. We find that different resource configurations of an application lead to similar performance, but have distinct energy consumption. We call this phenomenon as “performance-equivalent resource configurations (PERC)”, and its performance range is called equivalent region (ER). Based on PERC, one basic idea for improving energy efficiency is to select the most efficient configuration from PERC for each application. However, it cannot support every application to obtain optimal solution when thousands of applications are run simultaneously on resource-bounded servers. Here we propose a heuristic scheme, CPicker, based on genetic programming to improve energy efficiency of servers. To speed up convergence, CPicker initializes a high quality population by first choosing configurations from regions that have high energy variation. Experiments show that CPicker obtains above 17% energy efficiency improvement compared with the greedy approach, and less than 4% efficiency loss compared with the oracle case.     

Multi-task learning neural networks for breath sound detection and classification in pervasive healthcare

With the emergence of many grave Chronic obstructive pulmonary diseases (COPDs) and the COVID-19 pandemic, there is a need for timely detection of abnormal respiratory sounds, such as deep and heavy breaths. Although numerous efficient pervasive healthcare systems have been proposed for tracking patients, few studies have focused on these breaths. This paper presents a method that supports physicians in monitoring in-hospital and at-home patients by monitoring their breath. The proposed method is based on three deep neural networks in audio analysis: RNNoise for noise suppression, SincNet - Convolutional Neural Network, and Residual Bidirectional Long Short-Term Memory for breath sound analysis at edge devices and centralized servers, respectively. We also developed a pervasive system with two configurations: (i) an edge architecture for in-hospital patients; and (ii) a central architecture for at-home ones. Furthermore, a dataset, named BreathSet, was collected from 27 COPD patients being treated at three hospitals in Vietnam to verify our proposed method. The experimental results demonstrated that our system efficiently detected and classified breath sounds with F1-scores of 90% and 91% for the tiny model version on low-cost edge devices, and 90% and 95% for the full model version on central servers, respectively. The proposed system was successfully implemented at hospitals to help physicians in monitoring respiratory patients in real time¹.     

Data migration on parallel disks: Algorithms and evaluation

Our work is motivated by the problem of managing data on storage devices, typically a set of disks. Such storage servers are used as web servers or multimedia servers, for handling high demand for data. As the system is running, to exhibit good performance, it needs to respond dynamically to changes in demand for different data items. There are known algorithms for mapping demand to a layout. When the demand changes, a new layout can be computed. In this work we study thedata migration problem, which arises when we need to change one layout to another quickly. This problem has been studied earlier where for each disk a new layout has been prescribed. However, to apply these algorithms effectively, we identify another problem that we refer to as the correspondence problem, whose solution has a significant impact on the overall solution for the data migration problem. We study algorithms for the data migration problem in more detail and identify variations of the basic algorithm that seem to improve performance in practice, even though some of the variations have poor worst-case behavior. This research was supported by the NSF Awards CCR-0113192 and EIA-0091474 as well as the Okawa Research Award. This work made use of Integrated Media Systems Center Shared Facilities supported by the National Science Foundation under Cooperative Agreement No. EEC-9529152; any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect those of the National Science Foundation. This work was done while Svetlana Shargorodskaya was at the University of Maryland.     

SAN存储技术研究

SAN是通过专用高速网将一个或多个网络存储设备和服务器连接起来的专用存储系统，未来的信息存储将以SAN存储方式为主。可采用光纤通道、IP／Ethernet、InfiniBand互联技术分别组建FC-SAN、iSCSI-SAN和InfiniBand-SAN。对SAN这一最具发展潜力的网络存储技术进行了较为全面的介绍，重点研究了iSCSI-SAN和InfiniBand-SAN这两种新的存储技术。     

Design of a large scale multimedia storage server

Large scale multimedia storage servers will be an integral part of the emerging distributed multimedia computing infrastructure. However, given the modest rate of improvements in storage transfer rates, designing servers that meet the demands of multimedia applications is a challenging task that needs significant architectural innovation.Our research project, called Massively-parallel And Real-time Storage (mars) architecture, is aimed at the design and prototype implementation of a large scale multimedia storage server. It uses some of the well-known techniques in parallel I/O, such as data striping and Redundant Arrays of Inexpensive Disks (raid) and an innovative atm based interconnect inside the server to achieve a scalable architecture that transparently connects storage devices to an atm-based broadband network. The atm interconnect within the server uses a custom asic called ATM Port Interconnect Controller (apic) currently being developed as a part of an arpa sponsored gigabit local atm testbed. Our architecture relies on innovative data striping and real-time scheduling to allow a large number of guaranteed concurrent accesses, and uses separation of metadata from real data to achieve a direct flow of the media streams between the storage devices and the network. This paper presents our system architecture; one that is scalable in terms of the number of supported users and the throughput.     

Dynamic information‐based scalable hashing on a cluster of web cache servers

Caching web pages is an important part of web infrastructures. Medium to large‐scale infrastructures deploy a cluster of servers to solve the scalability and storage problems inherent in caching. In this paper we present dynamic information‐based scalable hashing that evenly hashes client requests to a cluster of cache servers, resulting in performance scalability. Runtime information is used to determine when and how to cache pages. Cached pages are stored and retrieved mutually exclusively to/from all the servers to minimize the use of storage, resulting in storage scalability. We set up an experimental environment consisting of various machines, including client servers, a cluster of 16 cache servers, and a load balancer. We demonstrate through experimental results that dynamic information‐based scalable hashing maximizes both performance scalability and storage scalability while the existing approaches do only either one of the two. Copyright © 2011 John Wiley & Sons, Ltd.     

Design and evaluation of a generic software architecture foron-demand video servers

Introduces the design, implementation and evaluation of a generic software architecture for on-demand video servers. We describe different key components for controlling the storage and network devices within the server. The interactive collaborations between these software components are also illustrated. The experimental results indicate a very promising direction in exploring the right combinations of these software components. The server is thus able to increase the number of concurrent video accesses with the same hardware configuration. For instance, with the right combinations, the system achieved about 80% of the storage system bandwidth of four disks, about 70% of the storage system bandwidth of six disks, and generally reached the maximal achieved SCSI bandwidth when eight disks are used over two SCSI buses. Our research and experimental results are based on video servers currently under construction across a variety of hardware platforms, including SMP, DMP and clusters of PCs or workstations. The most advanced prototype server is based on an SGI shared-memory multiprocessor with a mass storage system consisting of RAID-3 disk arrays. With all the enabling/management schemes, we were able to further investigate interesting research issues by considering the user's access profiles for taking advantage of popular video titles. The results were significant, with a range of 60% improvement given a 512 kByte block size. In addition to the experimental results, theoretical performance models were also developed that closely match to our collected experimental results