Two Emerging Serial Storage Interfaces for Supporting Digital Libraries: Serial Storage Architecture (SSA) and Fiber Channel-Arbitrated Loop (FC-AL)

Digital libraries require not only high storage space capacity but also high performance storage systems which provide the fast accesses to the data. These requirements can not be efficiently supported with the traditional SCSI interfaces. Several serial storage interfaces have been proposed for constructing storage systems with high transfer bandwidth, large storage capacity, and fault tolerance feature. Among them, Serial Storage Architecture (SSA) and Fibre Channel-Arbitrated Loop (FC-AL) are considered as the next generation storage interfaces with broad industry support. Both technologies support simple cabling, long transmission distance, high data bandwidth, large capacity, fault tolerance, and fair sharing of link bandwidth. In this paper, a tutorial and a comparison of these two technologies are presented. The tutorial examines their interface specifications, transport protocols, fairness algorithms, and capabilities of fault tolerance. The comparison focuses on their protocol overhead, flow control, fairness algorithms, and fault tolerance. The paper also summarizes the recently proposed Aaron Proposal which incorporates features from both SSA and FC-AL and targets at merging these two technologies.     

Design and evaluation of a generic software architecture foron-demand video servers

Introduces the design, implementation and evaluation of a generic software architecture for on-demand video servers. We describe different key components for controlling the storage and network devices within the server. The interactive collaborations between these software components are also illustrated. The experimental results indicate a very promising direction in exploring the right combinations of these software components. The server is thus able to increase the number of concurrent video accesses with the same hardware configuration. For instance, with the right combinations, the system achieved about 80% of the storage system bandwidth of four disks, about 70% of the storage system bandwidth of six disks, and generally reached the maximal achieved SCSI bandwidth when eight disks are used over two SCSI buses. Our research and experimental results are based on video servers currently under construction across a variety of hardware platforms, including SMP, DMP and clusters of PCs or workstations. The most advanced prototype server is based on an SGI shared-memory multiprocessor with a mass storage system consisting of RAID-3 disk arrays. With all the enabling/management schemes, we were able to further investigate interesting research issues by considering the user's access profiles for taking advantage of popular video titles. The results were significant, with a range of 60% improvement given a 512 kByte block size. In addition to the experimental results, theoretical performance models were also developed that closely match to our collected experimental results     

Have storage area networks come of age?

Organizations are processing and storing an exploding volume of information. Because of this, they are looking for new ways to access their data more quickly and reliably across their networks. This has led organizations to begin using storage area networks (SANs) based on fibre channel technology. SANs link storage devices (such as disks, disk arrays, and tape drives) to create a pool of storage that users can access directly. In essence, SANs uncouple storage devices from the LAN and put them on their own network. This approach relieves bandwidth congestion on the LAN and permits much faster data access than today's LAN-based storage streams. SANs also permit better storage management and more fault tolerance. SANs have traditionally used Small Computer Systems Interface (SCSI) technology, but now fibre channel technology is allowing SANs to transmit data at higher speeds over greater distances, which should accelerate the adoption of the storage networks. Fibre channel is also more scalable than SCSI. Fibre-channel-based SAN technology has so many advantages that it will become increasingly popular, particularly as researchers correct its problems and develop ways for it to transmit data over even longer distances     

I/O issues in a multimedia system

In future computer system design, I/O systems will have to support continuous media such as video and audio, whose system demands are different from those of data such as text. Multimedia computing requires us to focus on designing I/O systems that can handle real-time demands. Video- and audio-stream playback and teleconferencing are real-time applications with different I/O demands. We primarily consider playback applications which require guaranteed real-time I/O throughput. In a multimedia server, different service phases of a real-time request are disk, small computer systems interface (SCSI) bus, and processor scheduling. Additional service might be needed if the request must be satisfied across a local area network. We restrict ourselves to the support provided at the server, with special emphasis on two service phases: disk scheduling and SCSI bus contention. When requests have to be satisfied within deadlines, traditional real-time systems use scheduling algorithms such as earliest deadline first (EDF) and least slack time first. However, EDF makes the assumption that disks are preemptable, and the seek-time overheads of its strict real-time scheduling result in poor disk utilization. We can provide the constant data rate necessary for real-time requests in various ways that require trade-offs. We analyze how trade-offs that involve buffer space affect the performance of scheduling policies. We also show that deferred deadlines, which increase buffer requirements, improve system performance significantly     

Design of a large scale multimedia storage server

Large scale multimedia storage servers will be an integral part of the emerging distributed multimedia computing infrastructure. However, given the modest rate of improvements in storage transfer rates, designing servers that meet the demands of multimedia applications is a challenging task that needs significant architectural innovation.Our research project, called Massively-parallel And Real-time Storage (mars) architecture, is aimed at the design and prototype implementation of a large scale multimedia storage server. It uses some of the well-known techniques in parallel I/O, such as data striping and Redundant Arrays of Inexpensive Disks (raid) and an innovative atm based interconnect inside the server to achieve a scalable architecture that transparently connects storage devices to an atm-based broadband network. The atm interconnect within the server uses a custom asic called ATM Port Interconnect Controller (apic) currently being developed as a part of an arpa sponsored gigabit local atm testbed. Our architecture relies on innovative data striping and real-time scheduling to allow a large number of guaranteed concurrent accesses, and uses separation of metadata from real data to achieve a direct flow of the media streams between the storage devices and the network. This paper presents our system architecture; one that is scalable in terms of the number of supported users and the throughput.     

Lossless data transmission for Internet of things application over wireless multimedia sensor networks using enhanced and optimal path scheduling approach to maximizing the quality of service

The congestion of packet forwarding between a source and destination is challenging on downlink transmission in the entire file (ex. Audio and Video). Whenever file is been uploaded to the server, a user requests for file where server transmits it without knowledge of user's bandwidth, which is a major, cause of packet loss or time duration in the receiver end. To accumulate the better solution, Enhanced and Optimal Path Scheduling Approach (EOPSA) designs to find optimal path scheduling for multimedia data transmission in multimedia sensor network over cloud server using IoT devices. EOPSA studied the multisource video-on-demand streaming in multimedia sensor networks. The method introduced a heuristic distributed protocol to find optimal route for multimedia data transmissions. Efficient way to identify the bandwidth before the transmission ensures link establishment between sender and receiver. Here, the capture of bandwidth helps to check user's system capability to forward requested media data. Based on experiment evaluation, EOPSA improves 0.20 packet delivery ratio, 130 throughput, 0.20 second average delay and 14 communication overhead for 15, 25, 50, 75, and 100 nodes compared than conventional methods.     

一种构建超大规模数据中心的模块化网络结构

模块化数据中心网络的模块间互联结构和路由负责模块的有效组织,以及不同模块服务器间的高效通信.为此,如何设计具有高带宽、高容错和高可扩展能力的互联结构以支持大规模、超大规模数据中心的构建成为模块化数据中心网络需要解决的首要问题.提出了一种构建超大规模模块化数据中心的模块间互联结构MDKautz,该结构通过模块内大量未被使用的交换机预留高速端口将模块以Kautz图互连,在无需额外增加任何高端交换设备的前提下,构造出具有高带宽、高容错和灵活可持续扩展性的超大规模数据中心网络.对MDKautz的构建方法、路由策略以及扩展方法进行了分析,数学分析和模拟实验结果证明了该新型网络结构具有良好的拓扑特性和通信性能,可有效支持数据中心高带宽、高容错的典型应用.     

ChattyGraph:面向异构多协处理器的高可扩展图计算系统

现阶段,随着数据规模扩大化和结构多样化的趋势日益凸现,如何利用现代链路内链的异构多协处理器为大规模数据处理提供实时、可靠的并行运行时环境,已经成为高性能以及数据库领域的研究热点.利用多协处理器(GPU)设备的现代服务器(multi-GPU server)硬件架构环境,已经成为分析大规模、非规则性图数据的首选高性能平台.现有研究工作基于Multi-GPU服务器架构设计的图计算系统和算法(如广度优先遍历和最短路径算法),整体性能已显著优于多核CPU计算环境.然而,这类图计算系统中,多GPU协处理器间的图分块数据传输性能受限于PCI-E总线带宽和局部延迟,导致通过增加GPU设备数量无法达到整体系统性能的类线性增长趋势,甚至会出现严重的时延抖动,进而已无法满足大规模图并行计算系统的高可扩展性要求.经过一系列基准实验验证发现,现有系统存在如下两类缺陷:(1)现代GPU设备间数据通路的硬件架构发展日益更新(如NVLink-V1,NVLink-V2),其链路带宽和延迟得到大幅改进,然而现有系统受限于PCI-E总线进行数据分块通信,无法充分利用现代GPU链路资源(包括链路拓扑、连通性和路由);(2)在...     

面向Android系统的手机电梯预约APP

本文针对电梯存在的预约地点固定、预约方式不灵活的问题,设计了一款手机App电梯预约系统.本系统以电梯运行数据采集、数据采集控制终端与服务器接入作为关键技术,实现了远程客户端与现场设备的数据共享和远程监视.现场设备采用RS485总线,通过嵌入式电梯控制器建立与服务器的连接链路.在移动客户端,根据电梯运行特性开发了手机应用,完成客户端数据实时更新.本设计基于Android系统,具有电梯位置显示、电梯人数监视、远程呼梯及电梯到站提醒等功能,方便用户根据电梯使用情况,合理规划乘梯时刻.     

A unified framework for designing high performance in-memory and hybrid memory file systems

The emerging non-volatile memory technologies provide a new choice for storing persistent data in memory. Therefore, file system structure needs re-studying and re-designing. Our goal is to design a framework that gives high-performance in-memory file accesses and allows a file whose data can be stored across memory and block device. This paper presents a novel unified framework for in-memory and hybrid memory file systems based on the concept that each file has a contiguous “File Virtual Address Space”. Within this framework, the file access for in-memory data can be efficiently handled by address translation hardware and the virtual address space of file. The file accesses for data in block devices are handled by a dedicated page fault handler for file system. A file system called Hybrid Memory File System (HMFS) is implemented based on this framework. Experimental results show that the throughput of HMFS approaches the memory bus bandwidth in best cases. Compared with in-memory file systems, HMFS reaches 5 times, 2.1 times, and 1.6 times faster than EXT4 on Ramdisk, RAMFS, and PMFS, respectively. Compared with EXT4 on SSD and EXT4 using page cache, HMFS also achieves 100 times and tens of times performance improvement, respectively.     

面向数据中心网络的链路故障实时检测即服务

在大规模数据中心网络中,链路故障检测是保障网络连通性,确保线上业务正常运转的重要手段.当前链路故障检测功能一般由中间盒设备来提供或被直接整合到交换设备中.随着软件定义网络和网络功能虚拟化(network function virtualization, NFV)技术的发展,各项网络功能正逐渐从专用设备中分离出来,以服务的形式部署在云端为用户提供解决方案.然而,当前链路故障检测方法面临着单次探测用时过长、网络带宽占用率过高以及服务器负载过重等严峻挑战,并不适用于构建实时性需求较高的云服务.为此,需要对已有链路故障检测工作中存在的问题进行分析,提出探测矩阵的概念,以及基于探测矩阵优化的链路故障检测方法,并设计一个链路故障检测控制器与SDN控制器协同的服务架构,以此实现云端的链路故障实时检测即服务.最后,通过仿真实验的方式验证了该实时检测方法在单次探测用时、网络带宽占用以及端点负载3方面同之前工作相比具有显著优势,且优化探测矩阵所带来的开销是可容忍的.     

iSCSI目标器的实现

SCSI是一种支持块级别访问的通用磁盘技术,但主要缺点是SCSI总线的距离限制.iSCSI协议使用以太网传输SCSI命令,因而可以有效地解决SCSI的扩展性.研究iSCSI的协议,并提出iSCSI目标器的实现模型.     

Disk placement for arbitrary-rate playback in an interactive video server

Multimedia data, especially continuous media including video and audio objects, represent a rich and natural stimulus for humans, but require large amount of storage capacity and real-time processing. In this paper, we describe how to organize video data efficiently on multiple disks in order to support arbitrary-rate playback requested by different users independently. Our approach is to segment and decluster video objects and to place the segments in multiple disks using a restricted round-robin scheme, called prime round-robin (PRR). Its placement scheme provides uniform load balance of disks for arbitrary retrieval rate as well as normal playback, since it eliminates hot spots. Moreover, it does not require any additional disk bandwidth to support VCR-like operations such as fast-forward and rewind. We have studied the various effects of placement and retrieval schemes in a storage server by simulation. The results show that PRR offers even disk accesses, and the failure in reading segment by deadline occurs only at the beginning of new operations. In addition, the number of users admitted is not decreased, regardless of arbitrary-rate playback requests.     

基于知识库与全景图像匹配算法的母线保护装置故障三维分级检测方法

研究了基于知识库与全景图像匹配算法的母线保护装置故障三维分级检测方法,合理检测母线保护装置故障,保障母线保护装置安全稳定运行。将失败案例、故障排除手册等信息作为知识库构建层级的理论依据,设计关系型知识;通过全景图像采集、特征提取与匹配等操作设计三维模型索引知识,构建母线保护装置知识库;在推理机中引入模糊假言推理算法,获取相应的故障诊断结果;在三维可视化层级中使用PHP语言,搭建Web服务器,设计三维可视化后端程序,呈现故障检测的三维可视化结果。实验结果表明：该方法可有效实现母线保护装置故障的三维分级检测,故障检测结果与实际故障状况一致,可更好地满足实际工作需要。     

基于2D Cache结构的H.264运动补偿访存带宽优化方法

H.264/AVC的运动补偿处理环节需要消耗大量的内存访问带宽,这成为制约其性能的关键因素.分析表明,如此巨大的带宽消耗具体来自5个方面:像素数据的重复读取、地址对齐、突发访问、SDRAM页切换和内存竞争冲突.提出一种基于2D Cache结构的运动补偿带宽优化方法,充分利用像素的重用以减少数据的重复读取.同时通过结合数据在SDRAM中映射方式的优化,将众多短而随机的访问整合为地址对齐的突发访问,并减少了访问过程中页切换的次数.此外还提出了访存的组突发访问模式,以解决SDRAM竞争冲突所引入的开销.实验结果表明采用上述优化设计后,运动补偿的访存带宽降低了82.9~87.6%,同现存优化效率较高的方法相比,带宽进一步减少了64%~87%.在达到相同带宽减少幅度的前提下,所提出的新方法比传统Cache结构电路面积减少91%.该方法目前已在一款多媒体SoC芯片设计中实际应用.     

Modeling and performance comparison of reliability strategies fordistributed video servers

Large scale video servers are typically based on disk arrays that comprise multiple nodes and many hard disks. Due to the large number of components, disk arrays are susceptible to disk and node failures that can affect the server reliability. Therefore, fault tolerance must be already addressed in the design of the video server. For fault tolerance, we consider parity-based as well as mirroring-based techniques with various distribution granularities of the redundant data. We identify several reliability schemes and compare them in terms of the server reliability and per stream cost. To compute the server reliability, we use continuous time Markov chains that are evaluated using the SHARPE software package. Our study covers independent disk failures and dependent component failures. We propose a new mirroring scheme called Grouped One-to-One scheme that achieves the highest reliability among all schemes considered. The results of this paper indicate that dividing the server into independent groups achieves the best compromise between the server reliability and the cost per stream. We further find that the smaller the group size, the better the trade-off between a high server reliability and a low per stream cost     

一种基于节的视频服务器允许接纳控制算法

提出了一种基于“服务器节”的支持压缩多媒体流的服务器中CPU、磁盘、网络和内存等资源管理的方法和允许接纳控制算法。“服务器节”概念定义了一组客户视频服务特性,如播放、快进、慢进和暂停等,并且确定了视频服务所需资源的分配参。一个“服务器节”包括视频服务器、磁盘设备、网络设备和允许接纳控制。它不但能优化使用单个资源,对于给定系统支持最大数量的客户端,保证其服务质量（QoS),而且其允许接纳控制算法能根据系统所有资源的状况,在不影响原有的视频服务基础上,确定对客户端新提出的视频服务是否接受。     

Techniques for increasing the stream capacity of a high-performancemultimedia server

High-performance servers and high-speed networks will form the backbone of the infrastructure required for distributed multimedia information systems. A server for an interactive distributed multimedia system may require thousands of gigabytes of storage space and a high I/O bandwidth. In order to maximize the system utilization, and thus minimize the cost, it is essential that the load be balanced among each of the server's components, viz. the disks, the interconnection network and the scheduler. Many algorithms for maximizing retrieval capacity from the storage system have been proposed in the literature. This paper presents techniques for improving the server capacity by assigning media requests to the nodes of a server so as to balance the load on the interconnection network and the scheduling nodes. Five policies for request assignment-round-robin (RR), minimum link allocation (MLA), minimum contention allocation (MCA), weighted minimum link allocation (WMLA) and weighted minimum contention allocation (WMCA)-are developed. The performance of these policies on a server model developed by the authors (1995) is presented. We also consider the issue of file replication, and develop two schemes for storing the replicas: the parent group-based round-robin placement (PGBRRP) scheme, and the group-wide round-robin placement (GWRRP) scheme. The performance of the request assignment policies in the presence of file replication is presented     

一种NAS系统的设计模型与性能分析研究

与传统的存储服务器相比，附网存储体系具有诸多的优点．它实现了数据在存储设备到客户机之间的直接传输，提高了系统的整体性能，是目前新型的分布式存储解决方案．文中分析了基于附网存储的数据传输特点，并在此基础上提出了系统实现的设计模型．同时，我们对两种不同的存储方案造成的性能差异进行了研究，结果表明，附网存储系统能够有效地减轻服务器的负载，更充分地利用网络资源，如磁盘中缓存、磁盘处理器性能等．     

高速数据采集系统中的存储瓶颈问题及其解决

高速数据采集系统中的实时数据存储是一个技术瓶颈。本文介绍了一种基于PLX PCI9656的PCI高速数据连续实时采集存储系统，连续不间断采集存储速率达130MB／s以上，并能将连续的数据流实时写入硬盘。本系统使用工控服务器作为计算机平台，采用SCSI硬盘组成RAID0磁盘阵列，实现简单，工作稳定可靠。