Reducing I/O variability using dynamic I/O path characterization in petascale storage systems

In petascale systems with a million CPU cores, scalable and consistent I/O performance is becoming increasingly difficult to sustain mainly because of I/O variability. The I/O variability is caused by concurrently running processes/jobs competing for I/O or a RAID rebuild when a disk drive fails. We present a mechanism that stripes across a selected subset of I/O nodes with the lightest workload at runtime to achieve the highest I/O bandwidth available in the system. In this paper, we propose a probing mechanism to enable application-level dynamic file striping to mitigate I/O variability. We implement the proposed mechanism in the high-level I/O library that enables memory-to-file data layout transformation and allows transparent file partitioning using subfiling. Subfiling is a technique that partitions data into a set of files of smaller size and manages file access to them, making data to be treated as a single, normal file to users. We demonstrate that our bandwidth probing mechanism can successfully identify temporally slower I/O nodes without noticeable runtime overhead. Experimental results on NERSC’s systems also show that our approach isolates I/O variability effectively on shared systems and improves overall collective I/O performance with less variation.     

Improving the I/O performance of intermediate multimedia storage nodes

The data retrieval operations in servers and proxies for Media-on-Demand applications represent a severe bottleneck, because a potentially (very) high number of users concurrently retrieve data with high data rates. In the Intermediate Storage Node Concept (INSTANCE) project, we have developed a new architecture for Media-on-Demand storage nodes that maximizes the number of concurrent clients a single node can support. We avoid the traditional bottlenecks, like copy operations, multiple copies of the same data element in main memory, and checksum calculation in communication protocols, by designing, implementing, and tightly integrating three orthogonal techniques: a zero-copy-one-copy memory architecture, network level framing, and integrated error management. In this paper, we describe the INSTANCE storage node, and present an evaluation of our mechanisms. Our experimental performance results show that the integration of these three techniques in NetBSD at least doubles the number of concurrent clients that a single storage node can serve in our testbed.     

Pre-execution data prefetching with I/O scheduling

Parallel applications suffer from I/O latency. Pre-execution I/O prefetching is effective in hiding I/O latency, in which a pre-execution prefetching thread is created and dedicated to fetch the data for the main thread in advance. However, existing pre-execution prefetching works do not pay attention to the relationship between the main thread and the pre-execution prefetching thread. They just simply pre-execute the I/O accesses using the prefetching thread as soon as possible failing to carefully coordinate them with the operations of the main thread. This drawback induces a series of adverse effects on pre-execution prefetching such as diminishing the degree of the parallelism between computation and I/O, delaying the I/O access of main threads, and aggravating the I/O resource competition in the whole system. In this paper, we propose a new method to overcome this drawback by scheduling the I/O operations among the main threads and the pre-execution prefetching threads. The results of extensive experiments on four popular benchmarks in parallel I/O performance area demonstrate the benefits of the proposed approach.     

云存储上的I/O特征获取机制

为有效地获取云存储系统上的I/O特征,从云存储的体系结构出发,根据不同层面的结构特征提出一种层次化的I/O特征获取机制。该机制从全局云平台、集群和局部物理节点两个方面收集和分析云存储系统上的I/O特征数据,通过在云存储虚拟层嵌入监听模块来采集I/O特征,并通过统计分析模块对收集的数据做在线的分析。实验结果表明,该机制能够有效获取云存储系统的I/O特征,为云存储系统性能优化提供参考。     

海量数据分布式存储与检索优化分析研究

本文主要是针对关系型数据库的信息管理系统在处理海量数据的存储与检索方面的一些可选方案进行探讨与分析,并结合实际应用,分别从分布式数据存储、动态表、索引、查询语句优化等4个方面对海量数据的存储与检索给出可行的优化配置解决方案。     

Stock optimization in transportation/storage systems

Optimal stock reorder policies are determined for transportation/storage systems with queueing of user requests. Exact and approximate solution methods are proposed.Translated from Kibernetika i Sistemnyi Analiz, No. 3, pp. 179–182, May–June, 1992.     

A study of hotspot data prediction model in I/O workloads

There is a notable characteristic of the data access pattern: 80% I/O requests only access 20% data. This feature brings about the concept of hotspot data, which refer to the data in the most frequent requested area. The access to these hotspot data has direct influence upon the performance of the storage system's applications. Therefore, how to predict hotspot data is a critical research focus in the optimization of the storage system. In this paper, we propose a hotspot data prediction model based on a Zipf-like distribution, which can estimate and dynamically adjust parameters according to the present statistics of I/O access. We classify the hotspot data from every trace, and analyse the prediction rate through the classified hotspot data's characteristic. We synthesize the analysis results in different time granularities and hotspot data prediction queue lengths. Finally, we use block I/O traces to discuss the effectiveness of this model. The discussion and analysis results indicate that this model can predict the hotspot data efficiently.     

Exploiting user metadata for energy-aware node allocation in a cloud storage system

Cloud computing has gained popularity in recent years delivering various services as cost-effective platforms. However, the increasing energy consumption needs to be addressed in order to preserve the cost-effectiveness of these systems. In this work, we target the storage infrastructure in a cloud system and introduce several energy efficient storage node allocation methods by exploiting the metadata heterogeneity of cloud users. Our proposed methods preserve load balance on demand and switch inactive nodes into low-energy modes to save energy. We provide a mathematical model to estimate the outcome of proposed methods and conduct theoretical and simulative analyses using real-world workloads.     

树形结构数据输入输出控件的设计与实现

为解决现有树形结构数据输入输出控件都不能很好地输入一个树形数据完整信息的问题,提出了对现有树形结构数据输入输出控件进行改进并与其它控件组合成复合控件的方法。该方法分析了树形结构数据在数据库中的存在形式,分别设计并封装了现有树形结构数据输入输出控件,实现了自定义单表和多表树形结构数据输出控件,然后在自定义输出控件的基础上结合其它控件设计了单表和多表树形结构数据输入控件。该控件是数据敏感的,封装后数据的显示只需填写相关的自定义属性。并通过备件编目实例验证了该方法的有效性。     

Swift云存储环境下基于I/O负载均衡的读取策略

Swift是开源云计算平台OpenStack中云存储服务组件,为平台提供廉价、可靠、安全的云存储服务.针对Swift没有充分利用存储系统的备份数据来实现并行读取操作及负载均衡的这一问题,提出了基于I/O负载均衡的并发读策略.实验结果表明,该策略不仅能够充分利用备份数据来达到存储设备读操作的负载均衡,而且能够实现数据的并行读取,提高平台整体读性能.     

Weight/shape structural optimization exploiting rigid movement

The paper explores the possibility of performing limited shape/size optimization to obtain a minimum weight design using rigid body movement. A two step approach is adopted where changes in the shape parameters are made separately from changes in the size parameters. The sizing step uses a conventional optimization, STARS, which is available commercially. For simplicity the method is demonstrated using planar arches subject to a variety of load and boundary conditions.     

I/O的实施

在设备自动化升级过程中．置换I／O连接通常被看作是一种最基本的设备升级。设备内部I／O系统能够减少制造过程与控制系统之间因置换I／O连接而带来的不便。而且．在某些情况下．I／O系统甚至能够成为控制系统。     

微机I/O模块在PLC I/O扩展中的应用

本文在简要介绍PLC常用的I／O扩展技术基础上，重点介绍在I／O控制点较多的PLC控制系统中，如何使用价格相对较低的微机I／O模块来扩展PLC的I／O点，以降低系统的成本，提高经济效益。并以数字电子种LED显示控制为例，详细介绍了PLC控制系统中，采用微机I／O模块来扩展PLC的I／O点时，其硬件连接和梯形图编程方法。     

On-demand utility-based power control routing for energy-aware optimization in mobile ad hoc networks

In this paper, we propose a novel on-demand energy-aware routing protocol, utility-based power control routing (UBPCR), which reduces the trade-offs that arise in the other energy-aware route selection mechanisms that have recently been proposed for mobile ad hoc networks. Our approach is based on an economic framework that represents the degree of link's satisfaction (utility). With UBPCR, the utility function for any transmitter–receiver pair is defined as a measure of the link's preference regarding the signal-to-interference-plus-noise ratio (SINR), the transmit power, and the transmitter's residual battery capacity. During a route-searching process, each intermediate node between the source and the destination is executed via two consecutive phases: the scheduling phase and the transmit power control phase. The scheduling algorithm finds the proper qualified data slot for the receiving channel so that the transmissions of independent transmitters can be coordinated. The transmit power control determines the optimal power, if one exists, that maximizes the corresponding link's utility. Extensive simulations show that the UBPCR protocol can achieve incompatible goals simultaneously and fairly.     

Dynamic power management in energy-aware computer networks and data intensive computing systems

Energy awareness is an important aspect of modern network and computing system design and management, especially in the case of internet-scale networks and data intensive large scale distributed computing systems. The main challenge is to design and develop novel technologies, architectures and methods that allow us to reduce energy consumption in such infrastructures, which is also the main reason for reducing the total cost of running a network. Energy-aware network components as well as new control and optimization strategies may save the energy utilized by the whole system through adaptation of network capacity and resources to the actual traffic load and demands, while ensuring end-to-end quality of service. In this paper, we have designed and developed a two-level control framework for reducing power consumption in computer networks. The implementation of this framework provides the local control mechanisms that are implemented at the network device level and network-wide control strategies implemented at the central control level. We also developed network-wide optimization algorithms for calculating the power setting of energy consuming network components and energy-aware routing for the recommended network configuration. The utility and efficiency of our framework have been verified by simulation and by laboratory tests. The test cases were carried out on a number of synthetic as well as on real network topologies, giving encouraging results. Thus, we come up with well justified recommendations for energy-aware computer network design, to conclude the paper.     

面向对象数据分析框架ROOT及其I/O系统剖析

为了分析系统中产生的海量数据,研究并分析了一个面向对象数据分析框架ROOT系统。首先分析了ROOT系统的核心模块和主要功能,然后阐述了ROOT系统的文件存储结构和输入输出系统。ROOT系统的文件存储采用压缩的二进制格式,大大减少了数据的存储空间和传输时间;针对数据分析领域中的局部性特点,提供了数据对象自动拆分机制,减少了数据分析时数据的吞吐量,显著提高了数据分析效率。最后通过实验验证了ROOT系统在海量数据分析领域的高效性。     

关于I/O模拟的讨论

详细分析Windows的I/O机制,提出了分别在用户级、系统级、驱动级的FO模拟操作,通过实际案例和源代码分享在I/O模拟上的得失,着重介绍鼠标与键盘的模拟.     

利用FactorySuite 2000 I/O Server Toolkt开发I/O Server

本文介绍了利用美国WONDERWARE公司的Wonder-wareR FactorySuiteTM 2000 I/O Server Toolkit如何开发用户自己的I/O Server.