Cost-intelligent application-specific data layout optimization for parallel file systems

详细信息    查看全文

• 作者：Huaiming Song (1)
  Yanlong Yin (2)
  Yong Chen (3)
  Xian-He Sun (2)
  
• 关键词：Data layout ; I/O performance modeling ; Parallel file systems ; Parallel I/O ; Data ; intensive computing
• 刊名：Cluster Computing
• 出版年：2013
• 出版时间：June 2013
• 年：2013
• 卷：16
• 期：2
• 页码：285-298
• 全文大小：1175KB
• 参考文献：1. Lustre: A scalable, robust, highly-available cluster file system. White Paper, Cluster File Systems, Inc. (2006) [Online]. Available: http://www.lustre.org/
  2. Schmuck, F., Haskin, R.: GPFS: A?shared-disk file system for large computing clusters. In: FAST-2: Proceedings of the 1st USENIX Conference on File and Storage Technologies, p.?19. USENIX Association, Berkeley (2002)
  3. Welch, B., Unangst, M., Abbasi, Z., Gibson, G., Mueller, B., Small, J., Zelenka, J., Zhou, B.: Scalable performance of the Panasas parallel file system. In: FAST-8: Proceedings of the 6th USENIX Conference on File and Storage Technologies, pp.?1-7. USENIX Association, Berkeley (2008)
  4. Carns, P.H., Ligon, W.B. III, Ross, R.B., Thakur, R.: PVFS: A?parallel file system for Linux clusters. In: Proceedings of the 4th Annual Linux Showcase and Conference, pp. 317-27. USENIX Association, Berkeley (2000)
  5. Thakur, R., Gropp, W., Lusk, E.: Data sieving and collective I/O in ROMIO. In: FRONTIERS-9: Proceedings of the 7th Symposium on the Frontiers of Massively Parallel Computation, p. 182. IEEE Computer Society, Washington (1999) CrossRef
  6. Thakur, R., Choudhary, A.: An extended two-phase method for accessing sections of out-of-core arrays. Sci. Program. 5(4), 301-17 (1996)
  7. Seamons, K.E., Chen, Y., Jones, P., Jozwiak, J., Winslett, M.: Server-directed collective I/O in Panda. In: SC-5: Proceedings of the 1995 ACM/IEEE Conference on Supercomputing (CDROM), p.?57. ACM, New York (1995) CrossRef
  8. Chen, Y., Sun, X.-H., Thakur, R., Song, H., Jin, H.: Improving parallel I/O performance with data layout awareness. In: Cluster-0: Proceedings of the IEEE International Conference on Cluster Computing 2010. IEEE Computer Society, Washington (2010)
  9. Ching, A., Choudhary, A., Liao, W.-K., Ross, R., Gropp, W.: Efficient structured data access in parallel file systems. In: Cluster-3: Proceedings of the IEEE International Conference on Cluster Computing (2003)
  10. Ching, A., Choudhary, A., Coloma, K., Liao, W.-K., Ross, R., Gropp, W.: Noncontiguous I/O accesses through MPI-IO. In: CCGRID-3: Proceedings of the 3rd IEEE International Symposium on Cluster Computing and the Grid, p.?104 (2003)
  11. Nitzberg, B., Lo, V.: Collective buffering: improving parallel I/O performance. In: HPDC-7: Proceedings of the 6th IEEE International Symposium on High Performance Distributed Computing, p.?148. IEEE Computer Society, Washington (1997)
  12. Ma, X., Winslett, M., Lee, J., Yu, S.: Faster collective output through active buffering. In: IPDPS-2: Proceedings of the 16th International Parallel and Distributed Processing Symposium, p.?151. IEEE Computer Society, Washington (2002)
  13. Isaila, F., Malpohl, G., Olaru, V., Szeder, G., Tichy, W.: Integrating collective I/O and cooperative caching into the “ClusterFile-parallel file system. In: ICS-4: Proceedings of the 18th Annual International Conference on Supercomputing, pp. 58-7. ACM, New York (2004) CrossRef
  14. Liao, W.-K., Coloma, K., Choudhary, A., Ward, L., Russell, E., Tideman, S.: Collective caching: Application-aware client-side file caching. In: HPDC-5: Proceedings of the 14th IEEE International Symposium on High Performance Distributed Computing, 2005. HPDC-14, pp. 81-0. IEEE Computer Society, Washington (2005) CrossRef
  15. Fu, J.W.C., Patel, J.H.: Data prefetching in multiprocessor vector cache memories. In: ISCA-1: Proceedings of the 18th Annual International Symposium on Computer Architecture, pp.?54-3. ACM, New York (1991)
  16. Dahlgren, F., Dubois, M., Stenstrom, P.: Fixed and adaptive sequential prefetching in shared memory multiprocessors. In: ICPP-3: Proceedings of the 1993 International Conference on Parallel Processing, pp. 56-3. IEEE Computer Society, Washington (1993) CrossRef
  17. Patterson, R.H., Gibson, G.A., Ginting, E., Stodolsky, D., Zelenka, J.: Informed prefetching and caching. In: Proceedings of the 15th ACM Symposium on Operating Systems Principles, pp. 79-5. ACM Press, New York (1995)
  18. Byna, S., Chen, Y., Sun, X.-H., Thakur, R., Gropp, W.: Parallel I/O prefetching using MPI file caching and I/O signatures. In: SC-8: Proceedings of the 2008 ACM/IEEE Conference on Supercomputing, pp.?1-2. IEEE Press, Piscataway (2008)
  19. Lei, H., Duchamp, D.: An analytical approach to file prefetching. In: Proceedings of the USENIX 1997 Annual Technical Conference, pp. 275-88 (1997)
  20. Tran, N., Reed, D.A., Member, S.: Automatic ARIMA time series modeling for adaptive I/O prefetching. IEEE Trans. Parallel Distrib. Syst. 15, 362-77 (2004) CrossRef
  21. Chen, Y., Byna, S., Sun, X.-H., Thakur, R., Gropp, W.: Hiding I/O latency with pre-execution prefetching for parallel applications. In: SC-8: Proceedings of the 2008 ACM/IEEE Conference on Supercomputing, pp. 1-0. IEEE Press, Piscataway (2008)
  22. Rhodes, P.J., Tang, X., Bergeron, R.D., Sparr, T.M.: Iteration aware prefetching for large multidimensional scientific datasets. In: SSDBM-5: Proc. of the 17th International Conference on Scientific and Statistical Database Management, Berkeley, CA, US, pp. 45-4 (2005)
  23. Rubin, S., Bodík, R., Chilimbi, T.: An efficient profile-analysis framework for data-layout optimizations. SIGPLAN Not. 37(1), 140-53 (2002) CrossRef
  24. Wang, Y., Kaeli, D.: Profile-guided I/O partitioning In: ICS-3: Proceedings of the 17th Annual International Conference on Supercomputing, pp. 252-60. ACM, New York (2003) CrossRef
  25. Hsu, W.W., Smith, A.J., Young, H.C.: The automatic improvement of locality in storage systems. ACM Trans. Comput. Syst. 23(4), 424-73 (2005) CrossRef
  26. Huang, H., Hung, W., Shin, K.G.: FS2: Dynamic data replication in free disk space for improving disk performance and energy consumption. In: SOSP-5: Proceedings of the Twentieth ACM Symposium on Operating Systems Principles, pp. 263-76. ACM, New York (2005) CrossRef
  27. Bhadkamkar, M., Guerra, J., Useche, L., Burnett, S., Liptak, J., Rangaswami, R., Hristidis, V.: BORG: Block-reORGanization for self-optimizing storage systems In: Proceedings of the 7th Conference on File and Storage Technologies, pp.?183-96. USENIX Association, Berkeley (2009). [Online]. Available: http://portal.acm.org/citation.cfm?id=1525908.1525922
  28. Wang, C., Zhang, Z., Ma, X., Vazhkudai, S.S., Mueller, F.: Improving the availability of supercomputer job input data using temporal replication. Comput. Sci. Res. Dev. 23 (2009)
  29. Song, H., Sun, X.-H., Yin, Y., Chen, Y.: A?cost-intelligent application-specific data layout scheme for parallel file systems. In: HPDC-1: Proceedings of the 20th International ACM Symposium on High Performance Distributed Computing, pp. 37-8 (2011)
  30. Seltzer, M., Chen, P., Ousterhout, J.: Disk scheduling revisited. In: Proceedings of the USENIX Winter Technical Conference, USENIX Winter?-0, pp. 313-24 (1990)
  31. Worthington, B.L., Ganger, G.R., Patt, Y.N.: Scheduling algorithms for modern disk drives. In: SIGMETRICS-4: Proceedings of the 1994 ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems, pp. 241-51 (1994) CrossRef
  32. Lumb, C.R., Schindler, J., Ganger, G.R., Nagle, D.F.: Towards higher disk head utilization: extracting free bandwidth from busy disk drives. In: OSDI-0: Proceedings of the 4th Conference on Symposium on Operating System Design & Implementation, pp. 87-02. USENIX Association, Berkeley (2000)
  33. Zhang, X., Jiang, S.: InterferenceRemoval: Removing interference of disk access for MPI programs through data replication. In: ICS-0: Proceedings of the 24th International Conference on Supercomputing, pp. 223-32 (2010) CrossRef
  34. Isaila, F., Tichy, W.F.: Clusterfile: a flexible physical layout parallel file system. In: Cluster-1: Proceedings of the 3rd IEEE International Conference on Cluster Computing, p.?37 (2001)
  35. Wang, F., Xin, Q., Hong, B., Brandt, S.A., Miller, E.L., Long, D.D.E., Mclarty, T.T.: File system workload analysis for large scientific computing applications. In: Proceedings of the 21st IEEE/12th NASA Goddard Conference on Mass Storage Systems and Technologies, pp. 139-52, Apr. 2004
  36. Ligon, W.B., Ross, R.B.: Implementation and performance of a parallel file system for high performance distributed applications. In: HPDC-6: Proceedings of the 5th IEEE International Symposium on High Performance Distributed Computing, p.?471. IEEE Computer Society, Washington (1996) CrossRef
  37. Ruemmler, C., Wilkes, J.: An introduction to disk drive modeling. IEEE Comput. 27, 17-8 (1994) CrossRef
  38. Tian, Y., Klasky, S., Abbasi, H., Lofstead, J., Grout, R., Podhorszki, N., Liu, Q., Wang, Y., Yu, W.: EDO: Improving read performance for scientific applications through elastic data organization. In: Cluster-1: Proceedings of the IEEE International Conference on Cluster Computing. Cluster, vol.?11 (2011)
  39. Vijayakumar, K., Mueller, F., Ma, X., Roth, P.C.: Scalable I/O tracing and analysis. In: PDSW-9: Proceedings of the 4th Annual Workshop on Petascale Data Storage, pp. 26-1. ACM, New York (2009) CrossRef
  40. Yun, H.-C., Lee, S.-K., Lee, J., Maeng, S.: An efficient lock protocol for home-based lazy release consistency. In: CCGRID-1: Proceedings of the 1st International Symposium on Cluster Computing and the Grid, p.?527. IEEE Computer Society, Washington (2001)
  41. Phanishayee, A., Krevat, E., Vasudevan, V., Andersen, D.G., Ganger, G.R., Gibson, G.A., Seshan, S.: Measurement and analysis of TCP throughput collapse in cluster-based storage systems. In: FAST-8: Proceedings of the 6th USENIX Conference on File and Storage Technologies, pp. 1-4. USENIX Association, Berkeley (2008)
  42. Vasudevan, V., Phanishayee, A., Shah, H., Krevat, E., Andersen, D.G., Ganger, G.R., Gibson, G.A., Mueller, B.: Safe and effective fine-grained TCP retransmissions for datacenter communication. In: SIGCOMM-9: Proceedings of the ACM SIGCOMM 2009 Conference on Data Communication, pp. 303-14. ACM, New York (2009). [Online]. Available: http://doi.acm.org/10.1145/1592568.1592604 CrossRef
  43. Vasudevan, V., Shah, H., Phanishayee, A., Krevat, E., Andersen, D., Ganger, G., Gibson, G.: Solving TCP incast in cluster storage systems (poster presentation). In: FAST-9: Proceedings of the 7th USENIX Conference on File and Storage Technologies (2009)
  
• 作者单位：Huaiming Song (1)
  Yanlong Yin (2)
  Yong Chen (3)
  Xian-He Sun (2)
  
  1. R&D Center, Dawning Information Industry Co., Ltd., Beijing, 100084, China
  2. Department of Computer Science, Illinois Institute of Technology, Chicago, IL, 60616, USA
  3. Department of Computer Science, Texas Tech University, Lubbock, TX, 79409, USA
  
• ISSN：1573-7543

文摘

Parallel file systems have been developed in recent years to ease the I/O bottleneck of high-end computing system. These advanced file systems offer several data layout strategies in order to meet the performance goals of specific I/O workloads. However, while a layout policy may perform well on some I/O workload, it may not perform as well for another. Peak I/O performance is rarely achieved due to the complex data access patterns. Data access is application dependent. In this study, a cost-intelligent data access strategy based on the application-specific optimization principle is proposed. This strategy improves the I/O performance of parallel file systems. We first present examples to illustrate the difference of performance under different data layouts. By developing a cost model which estimates the completion time of data accesses in various data layouts, the layout can better match the application. Static layout optimization can be used for applications with dominant data access patterns, and dynamic layout selection with hybrid replications can be used for applications with complex I/O patterns. Theoretical analysis and experimental testing have been conducted to verify the proposed cost-intelligent layout approach. Analytical and experimental results show that the proposed cost model is effective and the application-specific data layout approach can provide up to a?74% performance improvement for data-intensive applications.