Enabling dynamic file I/O path selection at runtime for parallel file system |
| |
Authors: | Xiuqiao Li Limin Xiao Meikang Qiu Bin Dong Li Ruan |
| |
Affiliation: | 1. State Key Laboratory of Software Development Environment, Beijing, 100191, China 2. School of Computer Science and Engineering, Beihang University, Beijing, 100191, China 3. Department of Computer Engineering, San Jose State University, One Washington Square, San Jose, CA, 95192-0180, USA
|
| |
Abstract: | Parallel file systems are experiencing more and more applications from various fields. Various applications have different I/O workload characteristics, which have diverse requirements on accessing storage resources. However, parallel file systems often adopt the “one-size-fits-all” solution, which fails to meet specific application needs and hinders the full exploitation of potential performance. This paper presents a framework to enable dynamic file I/O path selection with fine granularity at runtime. The framework adopts a file handle-rich scheme to allow file systems choose corresponding optimizations to serve I/O requests. Consistency control algorithms are proposed to ensure data consistency while changing optimizations at runtime. One case study on our prototype shows that choosing proper optimizations can improve the I/O performance for small files and large files by up to 40 and 64.4 %, respectively. Another case study shows that the data prefetch performance for real-world application traces can be improved by up to 193 % by selecting correct prefetch patterns. Simulations in large-scale environment also show that our method is scalable and both the memory consumption and the consistency control overhead can be negligible. |
| |
Keywords: | |
本文献已被 SpringerLink 等数据库收录! |
|