Performance analysis of distributed storage clusters based on kernel and userspace traces

Distributed storage systems are commonly used in modern computing. They are highly scalable and offer data replication and fault tolerance. The complexity of those systems makes them difficult to debug using traditional tools. The existing tools are able to evaluate the overall performance of such systems but they do not provide enough information to find the root cause of performance issues. In this article, we propose a tracing‐based performance analysis framework for storage clusters. We use a tracing strategy that reduces the tracing overhead in production systems. The traces collected from the different storage nodes are correlated and used to generate a data model that represents the cluster. Userspace tracing is used to gather data from the storage daemons, while Kernel tracing is used to provide detailed information about operating system internals such as disk queues, network queues and process scheduling. Efficient data structures are used to store the model and to generate metrics and graphical views. Our tool is used in different real world scenarios and is able to investigate interesting performance problems including I/O latencies, data replication and storage nodes failures.     

一种监控系统的链路跟踪型日志数据的存储设计

随着软件系统越来越复杂化和分布化,为系统提供具有完善功能的监控服务显得越来越重要.APM(application performance management)系统通过采集软件系统运行时的各项指标数据来分析软件的运行状态,例如CPU、内存使用率、垃圾回收的耗时、QPS等指标.此外,APM系统也会在软件运行中生成各种日志...     

The I/O subsystem-a candidate for improvement

A computer system can be partitioned into hardware and the software executing on that hardware. The hardware consists of processor(s), memory, and “everything else”. The “everything else” we generally combine under the umbrella “I/O, whose job it is to manage the availability of information to and from the processor(s) and memory”. That information comes from storage devices, networks, and nonstorage devices. The I/O subsystem is the collection of all three; its influence on performance is a reflection of how well it manages the availability of information to and from all three. The impression today, from both the hardware side and the software side, is that the I/O subsystem can certainly stand improvement. The author considers improvements to the I/O subsystem     

获取访存依赖:并发程序动态分析基础技术综述

并发错误难触发、难调试、难检测.为应对这一挑战,已有动态程序分析技术通过观测或控制并发程序执行实现其质量保障.由于并发程序不确定性主要来自共享内存,实现其动态分析的基本问题即是获取线程访问共享内存的顺序,即获取访存依赖.提出访存依赖获取技术的综述框架,包含四个评价指标（即时性、准确性、高效性、简化性）、两种方法（在线追踪、离线合成）、两类应用（轨迹分析、并发控制）.通过对已有技术的总结和分析框架中的空白,对未来可能的研究方向予以展望.     

基于trace的网络存储系统评测研究

网络存储系统规模的膨胀和应用范围的扩大,使网络存储系统评测研究越来越重要。网络存储系统的评测包括性能、可用性、可管理性和功耗等内容。论述了网络存储系统主要评测指标的研究内容和研究现状,给出了网络存储系统评测的逻辑框架,分析了在多项指标的实际评测中如何用基于trace的方法分别从trace收集、重放和可用性等几方面对其相关研究现状和常用工具进行讨论。通过分析、总结研究中的问题,展望了相关研究在未来的发展方向和前景。     

Design of servo control systems with quadratic optimal control and observed-state feedback control for mems-based storage device

MicroElectroMechanical Systems (MEMS) are very tiny mechanical devices (on the order of 10–1000 μm) such as sensors, valves, gears, and actuators fabricated on the surface of silicon wafers. These microstructures are created using the same photolithographic processes used in manufacturing other semiconductor devices. Therefore, it is possible to integrate several semiconductor devices (e.g., processors and memory) directly with the nonvolatile storage device. The hierarchy gap between RAM and disks is creating a performance bottleneck in computer systems. MEMS-based storage can improve computer systems performance and fill significant assess time, power dissipation, mass, and cost gaps between RAM and disks. MEMS-based storage is very young technology so there are many possibilities for designing, modeling, and performance. We focus on the system-level performance characteristics. This paper explores control systems with quadratic optimal control and observed-state feedback control for MEMS-based storage device. A closed-loop control system actively damps the oscillations using the actuators and reduces seek time by reducing settling time. Optimizations of the control loop can provide better I/O performance. Accurate model of system components is important for analysis of system performance because these devices do not exist yet. At the beginning of development, our control models may provide reasonable feedback and design trade-offs to both hardware and software designers.     

Hypervisor中内存回收技术的改进

虚拟化是云计算的关键技术. Hypervisor在虚拟机与主机硬件之间提供了一个抽象层,允许用户为运行着的虚拟机分配的内存总值超过主机的可用内存,这种技术称为内存过量分配. 为了能够降低这个技术对虚拟机性能的影响,hypervisor必须提供高效率的内存回收机制. 在本论文中,作者提出了一种解决方案：使用非易失性内存作为hypervisor交换页面数据的缓存设备. 作者从系统内存中划分出空间模拟了非易失性内存设备,修改了KVM模块中的算法,并制定了五种测试环境. 通过实验数据证明,相比现有的Ballooning技术与Hypervisor swapping技术,使用非易失性内存并配合低优先级队列算法时,虚拟机性能可提高30%和50%左右.     

分布式追踪技术综述

随着分布式软件系统在各个行业的广泛应用,如何提升系统运维效率,保障其服务的可靠与稳定,得到了学术界与工业界的关注.分布式软件系统其规模庞大、结构复杂、持续更新且大量服务请求并发执行的特点,给分布式软件系统的运维任务带来了严峻的挑战.传统的以组件/节点/进程/线程为中心的系统监控与追踪方法难以支持分布式软件的故障诊断、性能调优、系统理解等运维任务.分布式追踪技术识别并提取出分布式软件系统因处理单个服务请求所产生的因果相关的事件,以服务请求为中心对分布式软件系统的行为进行精准、细粒度地刻画,对提高分布式软件系统的运维效率有重要意义.对分布式追踪技术的研究与应用进行了综述,从追踪数据获取、请求事件提取、因果关系判断及请求路径表示这四个方面总结了分布式追踪技术的现状;同时以基于请求执行路径的故障诊断和性能分析为例,讨论了学术界对分布式追踪技术的应用研究;最后,对分布式追踪技术的数据读写依赖问题、通用性问题和评价问题进行了探讨并对未来的研究方向进行了展望.     

EPE‐Mobile—A framework for early performance estimation of mobile applications

Considering the constrained resources of mobile devices, a thorough performance evaluation of a mobile application is crucial. However, performance evaluation in the mobile domain is still a manual and time‐consuming task. The diversity of mobile devices only increases the complexity of this task. We propose EPE‐Mobile, a framework to automate early performance estimation in mobile applications. It is composed of a configurable library of basic operations and an engine that automatically creates a synthetic program based on the specification of a new app. The synthetic program that EPE‐Mobile generates provides feedback for mobile developers at the first design stages and before the actual implementation of a new application. The fast evaluation can also guide developers in optimizing their applications or in choosing devices with the best trade‐off between cost and performance to run a given application. Finally, developers can reuse the data collection infrastructure of the framework to collect performance data during all development stages. We validate the proposed framework using 4 applications from the Android Play Store. Based on their specifications, 4 synthetic programs were generated and executed on different devices. We compared the results to those obtained from the execution of the actual applications in the same devices. Experimental results show that it is possible to create synthetic applications with similar behavior to that of real applications and, thus, classify devices based on the actual application needs. The framework uses aspect‐oriented programming to collect the metrics of interest. This approach provides increased modularity and separation of concerns, thus facilitating the improvement of the framework itself, by adding other metrics or basic operations.     

基于非易失性存储器的存储引擎性能优化

非易失性存储器具有接近内存的读写速度,可利用其替换传统的存储设备,从而提升存储引擎的性能。但是,传统的存储引擎通常使用通用块接口读写数据,导致了较长的 I/O 软件栈,增加了软件层的读写延迟,进而限制了非易失性存储器的性能优势。针对这一问题,该文以 Ceph 大数据存储系统为基础,研究设计了基于非易失性存储器的新型存储引擎 NVMStore,通过内存映射的方式访问存储设备,根据非易失性存储器的字节可寻址和数据持久化特性,优化数据读写流程,从而减小数据写放大以及软件栈的开销。实验结果表明,与使用非易失性存储器的传统存储引擎相比,NVMStore能够显著提升 Ceph 的小块数据读写性能。     

Analog memristive memory with applications in audio signal processing

Since the development of the HP memristor, much attention has been paid to studies of memris- tive devices and applications, particularly memristor-based nonvolatile semiconductor memory. Owing to its unique properties, theoretically, one could restart a memristor-based computer immediately without the need for reloading the data. Further, current memories are mainly binary and can store only ones and zeros, whereas memristors have multilevel states, which means a single memristor unit can replace many binary transistors and realize higher-density memory. It is believed that memristors can also implement analog storage besides binary and multilevel information memory. In this paper, an implementation scheme for analog memristive memory is considered. A charge-controlled memristor model is derived and the corresponding SPICE model is constructed. Special write and read operations are demonstrated through numerical analysis and circuit simulations. In addition, an audio analog record/play system using a memristor crossbar array is designed. This system can provide great storage capacity （long recording time） and high audio quality with a simple small circuit structure. A series of computer simulations and analyses verify the effectiveness of the proposed scheme.     

Performance metrics and auditing framework using application kernels for high‐performance computer systems

This paper describes XSEDE Metrics on Demand, a comprehensive auditing framework for use by high‐performance computing centers, which provides metrics regarding resource utilization, resource performance, and impact on scholarship and research. This role‐based framework is designed to meet the following objectives: (1) provide the user community with a tool to manage their allocations and optimize their resource utilization; (2) provide operational staff with the ability to monitor and tune resource performance; (3) provide management with a tool to monitor utilization, user base, and performance of resources; and (4) provide metrics to help measure scientific impact. Although initially focused on the XSEDE program, XSEDE Metrics on Demand can be adapted to any high‐performance computing environment. The framework includes a computationally lightweight application kernel auditing system that utilizes performance kernels to measure overall system performance. This allows continuous resource auditing to measure all aspects of system performance including filesystem performance, processor and memory performance, and network latency and bandwidth. Metrics that focus on scientific impact, such as publications, citations and external funding, will be included to help quantify the important role high‐performance computing centers play in advancing research and scholarship. Copyright © 2012 John Wiley & Sons, Ltd.     

NVMRA: utilizing NVM to improve the random write operations for NAND‐flash‐based mobile devices

NAND flash memory has become the major storage media in mobile devices, such as smartphones. However, the random write operations of NAND flash memory heavily affect the I/O performance, thus seriously degrading the application performance in mobile devices. The main reason for slow random write operations is the out‐of‐place update feature of NAND flash memory. Newly emerged non‐volatile memory, such as phase‐change memory, spin transfer torque, supports in‐place updates and presents much better I/O performance than that of flash memory. All these good features make non‐volatile memory (NVM) as a promising solution to improve the random write performance for NAND flash memory. In this paper, we propose a non‐volatile memory for random access (NVMRA) scheme to utilize NVM to improve the I/O performance in mobile devices. NVMRA exploits the I/O behaviors of applications to improve the random write performance for each application. Based on different I/O behaviors, such as random write‐dominant I/O behavior, NVMRA adopts different storing decisions. The scheme is evaluated on a real Android 4.2 platform. The experimental results show that the proposed scheme can effectively improve the I/O performance and reduce the I/O energy consumption for mobile devices. Copyright © 2015 John Wiley & Sons, Ltd.     

Analyzing the feasibility of building a new mass storage system on distributed resources

The average PC now contains a large and increasing amount of storage with an ever greater amount left unused. We believe there is an opportunity for organizations to harness the vast unused storage capacity on their PCs to create a very large, low‐cost, shared storage system. What is needed is the proper storage system architecture and software to exploit and manage the unused portions of existing PC storage devices across an organization and make it reliably accessible to users and applications. We call our vision of such a storage system Storage@desk (SD). This paper describes our first step towards the realization of SD—a study of machine and storage characteristics and usage in a model organization. We studied 729 PCs in an academic institution for 91 days, monitoring the configuration, load and usage of the major machine subsystems, i.e. disk, memory, CPU and network. To further analyze the availability characteristics of storage in an SD system, we performed a trace‐driven simulation of some basic storage allocation strategies. This paper presents the results of our data collection efforts, our analysis of the data, our simulation results and our conclusion that an SD system is indeed feasible and holds promise as a cost‐effective way to create massive storage systems. Copyright © 2007 John Wiley & Sons, Ltd.     

NAND flash memory-based hybrid file system for high I/O performance

As the performance potentials of SSD (Solid State Device) have been recognized, adopting SSD to IT products as HDD replacements is rapidly increasing. Since SSD is organized into multiple flash memory packages, it deploys peculiar device characteristics that do not occur in HDD, such as block-unit erasure overhead. Also, its high cost per capacity is the main obstacle to building a large-scale storage subsystem with only SSDs. An alternative is to build a hybrid storage subsystem where a small portion of SSDs are integrated with HDDs so as to utilize SSD’s performance advantages in a cost-effective way. This study introduces a new form of file system, called N-hybrid (New-Form of hybrid file system), that enables us to support the hybrid device structure combined with both HDD and SSD. Our primary objectives in developing N-hybrid are to provide better I/O bandwidth by exploiting the characteristics of HDD and SSD and to provide a flexible data layout maximizing the usage of tight SSD storage resources. Several experiments were conducted to verify the effectiveness and suitability of N-hybrid.     

Massive Storage Systems

To accommodate the explosively increasing amount of data in many areas such as scientific computing and e-Business, physical storage devices and control components have been separated from traditional computing systems to become a scalable, intelligent storage subsystem that, when appropriately designed, should provide transparent storage interface, effective data allocation, flexible and efficient storage management, and other impressive features. The design goals and desirable features of such a storage subsystem include high performance, high scalability, high availability, high reliability and high security. Extensive research has been conducted in this field by researchers all over the world, yet many issues still remain open and challenging. This paper studies five different online massive storage systems and one offline storage system that we have developed with the research grant support from China. The storage pool with multiple network-attached RAIDs avoids expensive store-and-forward data copying between the server and storage system, improving data transfer rate by a factor of 2-3 over a traditional disk array. Two types of high performance distributed storage systems for local-area network storage are introduced in the paper. One of them is the Virtual Interface Storage Architecture (VISA) where VI as a communication protocol replaces the TCP/IP protocol in the system. VISA's performance is shown to achieve better than that of IP SAN by designing and implementing the vSCSI (Vl-attached SCSI) protocol to support SCSI commands in the VI network. The other is a fault-tolerant parallel virtual file system that is designed and implemented to provide high I/O performance and high reliability. A global distributed storage system for wide-area network storage is discussed in detail in the paper, where a Storage Service Provider is added to provide storage service and plays the role of user agent for the storage system. Object based Storage Systems not only store data but also adopt the attributes and methods of objects that encapsulate the data. The adaptive policy triggering mechanism (APTM), which borrows proven machine learning techniques to improve the scalability of object storage systems, is the embodiment of the idea about smart storage device and facilitates the self-management of massive storage systems. A typical offline massive storage system is used to backup data or store documents, for which the tape virtualization technology is discussed. Finally, a domain-based storage management framework for different types of storage systems is presented in the paper.     

ReduceM: Interactive and Memory Efficient Ray Tracing of Large Models

We present a novel representation and algorithm, ReduceM, for memory efficient ray tracing of large scenes. ReduceM exploits the connectivity between triangles in a mesh and decomposes the model into triangle strips. We also describe a new stripification algorithm, Strip‐RT, that can generate long strips with high spatial coherence. Our approach uses a two‐level traversal algorithm for ray‐primitive intersection. In practice, ReduceM can significantly reduce the storage overhead and ray trace massive models with hundreds of millions of triangles at interactive rates on desktop PCs with 4‐8GB of main memory.     

基于UML的移动电话计费数据采集系统的设计与实现

本文运用统一建模语言，对移动计费数据采集子系统的逻辑结构和物理实现进行了介绍。首先，利用类图和顺序视图对系统中对象的静态结构和动态行为进行了详细分析，然后运用物理视图对系统中构件的实现结构作了介绍，最后给出了系统实现的框架代码。     

物联网和服务架构的食品安全信息平台模型

针对食品供应链管理和质量安全的跟踪和追溯需求,利用物联网技术,参照EPC标准,提出了一种基于物联网和服务架构的食品安全信息平台模型,描述了RFID技术在Internet基础上实现食品的全程跟踪管理的E-R模型、网络架构、技术框架和功能模块,可通过Web Service和智能手机等多种方式查询追溯详细的电子信息.