地震资料处理PC集群并行效率研究
|
摘要
石油地震资料处理需要海量存储和巨量计算,一直是高性能计算的重要应用领域。基于Linux的集群计算机系统由于在性价比、可靠性和可扩展性方面的明显优势,已成为目前地震资料处理的主流平台。对地震资料处理PC集群的并行效率进行研究,可以充分利用现有的软硬件资源,提高地震资料处理集群的整体效能,提高地质效果和经济效益,具有重要的实际意义。
本文在分析地震资料处理系统的特点的基础上,找出了影响地震资料处理PC集群并行效率的关键因素,主要是I/O瓶颈问题和串行程序在并行环境中的低运行效率。为了解决上述问题,本文主要从并行文件系统和并行编程两方面开展研究工作。
本文首先利用普通PC机和MPI技术构建了一个PC集群,对其进行了LINPACK基准测试,分析了影响集群性能的几个关键因素并提出了相关建议。
为了解决地震资料处理集群中日益严重的I/O瓶颈问题,本文在研究并行文件系统理论的基础上,将Lustre并行文件系统部署到了实验集群中,利用iozone对NFS和Lustre进行了对比测试。结果表明,Lustre能够更好地适应并发I/O的要求,可有效缓解I/O瓶颈问题。为了提高Lustre的安全性,本文设计了一个基于PKI的Lustre安全模型。
在研究相关并行程序设计和开发理论的基础上,对地震资料处理软件中的核心算法快速傅里叶变换和矩阵乘法操作,利用MPI、OpenMP和CUDA等技术进行了并行化,并与相关串行程序在执行耗时方面进行了比较。测试结果表明,通过对计算密集型算法进行并行化,可以明显缩短程序运行周期。
将Lustre并行文件系统和并行编程技术应用到实际地震资料处理系统,通过对实际地震资料的处理测试表明,Lustre和并行编程技术可以显著缩短地震资料处理周期,提高程序运行效率。
此外,本文还研究了编译器、运行环境、数学库等因素对程序效率的影响,得到了一些有意义的结论,并对进一步提高并行效率提出了几点建议。
Petroleum seismic data processing needs mass storage and computing, which has been an important application field of high performance computing. Linux-based cluster system has obvious advantages at the cost performance, reliability and scalability and has become the mainstream platform for seismic data processing. It can make full use of existing hardware and software resources to research the parallel efficiency of PC cluster for seismic data processing. It can not only improve the overall efficiency of seismic data processing cluster, but also improve the geological effects and economic benefits. Therefore, the issue has important practical significance.
This paper analyzes the characteristics of seismic data processing system to identify the key factors that affect the parallel efficiency of PC cluster for seismic data processing,which are I/O bottleneck and serial programs’low operating efficiency in parallel environment. In order to solve the above problems, this paper conducts research in two areas: parallel file system and parallel programming.
Firstly, a PC cluster is built using ordinary PC and MPI. We analyze the key factors that affect cluster performance through LINPACK benchmark and provide relevant suggestions.
In order to solve the increasingly serious I/O bottleneck problem of seismic data processing cluster, we install Lustre parallel file system into the test cluster and make a comparison test between NFS and Lustre using iozone. The results show that Lustre can better meet the concurrent I/O requirements and effectively alleviate the I/O bottleneck. In order to improve the security of Lustre, this paper designs a Lustre security model based on PKI.
The core algorithms of seismic data processing are FFT and matrix multiplication. We parallelize them using MPI, OpenMP and CUDA, and make a comparison in execute time between the parallel programs and related serial programs. Test results show that it can significantly reduce the run cycle by parallelizing the computing-intensive algorithms.
We apply Lustre parallel file system and parallel programming techniques to real seismic data processing system. The real seismic data processing tests show that Lustre and parallel programming techniques can significantly reduce the seismic data processing cycle and improve the efficiency of procedures.
In addition, this paper also investigates the effects of the compilers, runtime environment, math libraries and other factors on program efficiency and obtains some meaningful conclusions. Finally, several suggestions are given on further improve the parallel efficiency.
本文在分析地震资料处理系统的特点的基础上,找出了影响地震资料处理PC集群并行效率的关键因素,主要是I/O瓶颈问题和串行程序在并行环境中的低运行效率。为了解决上述问题,本文主要从并行文件系统和并行编程两方面开展研究工作。
本文首先利用普通PC机和MPI技术构建了一个PC集群,对其进行了LINPACK基准测试,分析了影响集群性能的几个关键因素并提出了相关建议。
为了解决地震资料处理集群中日益严重的I/O瓶颈问题,本文在研究并行文件系统理论的基础上,将Lustre并行文件系统部署到了实验集群中,利用iozone对NFS和Lustre进行了对比测试。结果表明,Lustre能够更好地适应并发I/O的要求,可有效缓解I/O瓶颈问题。为了提高Lustre的安全性,本文设计了一个基于PKI的Lustre安全模型。
在研究相关并行程序设计和开发理论的基础上,对地震资料处理软件中的核心算法快速傅里叶变换和矩阵乘法操作,利用MPI、OpenMP和CUDA等技术进行了并行化,并与相关串行程序在执行耗时方面进行了比较。测试结果表明,通过对计算密集型算法进行并行化,可以明显缩短程序运行周期。
将Lustre并行文件系统和并行编程技术应用到实际地震资料处理系统,通过对实际地震资料的处理测试表明,Lustre和并行编程技术可以显著缩短地震资料处理周期,提高程序运行效率。
此外,本文还研究了编译器、运行环境、数学库等因素对程序效率的影响,得到了一些有意义的结论,并对进一步提高并行效率提出了几点建议。
Petroleum seismic data processing needs mass storage and computing, which has been an important application field of high performance computing. Linux-based cluster system has obvious advantages at the cost performance, reliability and scalability and has become the mainstream platform for seismic data processing. It can make full use of existing hardware and software resources to research the parallel efficiency of PC cluster for seismic data processing. It can not only improve the overall efficiency of seismic data processing cluster, but also improve the geological effects and economic benefits. Therefore, the issue has important practical significance.
This paper analyzes the characteristics of seismic data processing system to identify the key factors that affect the parallel efficiency of PC cluster for seismic data processing,which are I/O bottleneck and serial programs’low operating efficiency in parallel environment. In order to solve the above problems, this paper conducts research in two areas: parallel file system and parallel programming.
Firstly, a PC cluster is built using ordinary PC and MPI. We analyze the key factors that affect cluster performance through LINPACK benchmark and provide relevant suggestions.
In order to solve the increasingly serious I/O bottleneck problem of seismic data processing cluster, we install Lustre parallel file system into the test cluster and make a comparison test between NFS and Lustre using iozone. The results show that Lustre can better meet the concurrent I/O requirements and effectively alleviate the I/O bottleneck. In order to improve the security of Lustre, this paper designs a Lustre security model based on PKI.
The core algorithms of seismic data processing are FFT and matrix multiplication. We parallelize them using MPI, OpenMP and CUDA, and make a comparison in execute time between the parallel programs and related serial programs. Test results show that it can significantly reduce the run cycle by parallelizing the computing-intensive algorithms.
We apply Lustre parallel file system and parallel programming techniques to real seismic data processing system. The real seismic data processing tests show that Lustre and parallel programming techniques can significantly reduce the seismic data processing cycle and improve the efficiency of procedures.
In addition, this paper also investigates the effects of the compilers, runtime environment, math libraries and other factors on program efficiency and obtains some meaningful conclusions. Finally, several suggestions are given on further improve the parallel efficiency.
引文
[1]赵改善.高性能计算在石油物探中的应用现状与前景[J].高性能计算发展与应用,2009(4):19-23
[2]王勇超.高性能计算技术应用研究[D].西安:西安理工大学,2007
[3] TOP500 Supercomputing Sites[EB/OL],http://www.top500.org/,2009-11-17
[4]赵改善,包红林.集群计算技术及其在石油工业中的应用[J].石油物探,2001,40(3):118-126
[5]戴猛.Linux集群在石油行业的应用现状及发展趋势[J].硅谷,2009(7):115-116
[6] Rajkumar Buyya.高性能集群计算:结构与系统(第一卷)[M].郑纬民,石威,汪东升,等译.北京:电子工业出版社,2001:6-24
[7]姚成浪.基于MPI的集群系统设计与实现[D].哈尔滨:哈尔滨工程大学,2007
[8]陈国良,安虹,陈崚,等.并行算法实践[M].北京:高等教育出版社,2004
[9]赵传雪.基于LINUX PC集群数据处理算法研究[D].西安:西安石油大学,2006
[10]熊盛武,王鲁,杨婕.构建高性能集群计算机系统的关键技术[J].微计算机信息,2006,22(1-3):86-88
[11]刘素芹.基于应用网格环境的复杂地表波动方程基准面静校正研究[D].东营:中国石油大学,2008
[12]牛涛.Pc集群环境MPI并行DFT运算的研究[D].广州:华南师范大学,2003
[13]林娜.MPICH作业递交方式的研究与改进[D].上海:复旦大学,2006
[14]都志辉.高性能计算之并行编程技术——MPI并行程序设计[M].北京:清华大学出版社,2001:6-7
[15] George Em Karniadakis and Robert M. Kirby II.Parallel Scientific Computing in C++ and MPI:A seamless approach to parallel algorithms and their implementation[M]. Cambridge:Cambridge University Press,2003
[16]崔永良,刘羽,郭慧玲.MPI-2新特性及MPICH配置方法[J].山东理工大学学报(自然科学版),2008,22(5):44-47
[17] William Gropp,Ewing Lusk,Rajeev Thakur.Using MPI-2:advanced features of the message-passing interface[M].Cambridge:MIT Press,1999
[18]丁慧清.PC Linux机群系统并行计算环境的组建与性能测试[D].北京:北京工业大学,2004
[19]王勇超,张璟,王新卫,等.基于MPICH2的高性能计算集群系统研究[J].计算机技术与发展,2008,18(9):101-104
[20]朱亚超.Beowulf集群系统性能评测技术研究[D].武汉:华中科技大学,2006
[21]伍卫国,陈长虹,张虎,等.并行文件系统管理工具的设计与实现[J].计算机工程,2007,33(18):279-282
[22]霍严梅,杨可新,胡亮,等.并行文件系统研究综述[J].小型微型计算机系统,2008,29(9):1631-1636
[23]董勇,周恩强,陈娟.基于Infiniband技术构建高性能分布式文件系统—Lustre[J].计算机工程与应用,2005(22):103-107,228
[24]杨昕,沈文海.Lustre并行文件系统的发展及在气象领域的应用前景[J].应用气象学报,2008,19(2):243-249
[25]林松涛.基于Lustre文件系统的并行I/O技术研究[D].长沙:国防科学技术大学,2004
[26]张媛,卢泽新,刘亚萍.NFS over Lustre性能评测与分析[J].计算机工程,2007,33(10):274-276
[27] Christopher Smith.Linux NFS-HOWTO[EB/OL],http://nfs.sourceforge.net/nfs-howto/,2006-05-02
[28]曹立强,罗红兵,张晓霞.集群环境中影响NFS文件系统带宽的测试与分析[J].计算机工程,2007,33(19):125-127
[29]张丹丹,姚继锋.基于软RAID的Lustre性能影响要素简析[A].2008年全国高性能计算学术年会论文集[C].无锡:国家并行计算机工程技术研究中心,2008:427-430
[30]王扬,周可,李春花,等.集群文件系统的性能最优化配置[J].计算机研究与发展,2007,44(增刊):168-172
[31] Sun Microsystems , Inc. LustreTM 1.6 Operations Manual[EB/OL] ,http://dlc.sun.com/pdf/820-3681/820-3681.pdf,2009-05
[32]常兴华.Lustre存储系统安全模型设计[J].信息安全与通信保密,2008(4):80-82
[33]黄建华,何希.基于动态口令体制的Kerberos协议改进[J].计算机安全,2009(2):66-69
[34] Frederick Butler,Iliano Cervesato,Aaron D. Jaggard,et al.Formal analysis of Kerberos5[J].Theoretical Computer Science,2006,367(1-2):57-87
[35]范洪生,叶震,侯保花.基于公钥密码体制的Kerberos协议的改进[J].计算机技术与发展,2006,16(4):224-227
[36]何蕴婷,张宗福.基于PKI的文件安全传输方案研究[J].电脑知识与技术,2008,3(5):885-887
[37] Ali Nasrat Haidar,Ali E. Abdallah.Formal modelling of PKI based authentication[J]. Electronic Notes in Theoretical Computer Science,2009,235:55-70
[38] Andrew Nash,William Duane,Celia Joseph,et al.公钥基础设施(PKI):实现和管理电子安全[M].张玉涛,陈建奇,杨波,等译.北京:清华大学出版社,2002
[39]刘士栋,杨林,王建新.分布式网络安全服务Kerberos和PKI比较分析[A].全国第八届通信保密与信息安全现状研讨会论文集[C].成都:中国电子科技集团公司第三十研究所,2002:106-113
[40] Rohit Chandra,Leonardo Dagum,Dave Kohr,et al.Parallel Programming in OpenMP[M].San Fransisco:Morgan Kaufmann Publishers,2001
[41] Barbara Chapman,Gabriele Jost,Ruud van der Pas.Using OpenMP:Portable Shared Memory Parallel Programming[M].Cambridge:MIT Press,2007
[42]蔡佳佳,李名世,郑锋.多核微机基于OpenMP的并行计算[J].计算机技术与发展,2007,17(10):87-91
[43]殷顺昌.OpenMP并行程序性能分析[D].长沙:国防科学技术大学,2006
[44]吴少刚.机群系统OpenMP研究[D].北京:中国科学院计算技术研究所,2004
[45]周伟明.多核计算与程序设计[M].武汉:华中科技大学出版社,2009:75-123
[46] OpenMP Architecture Review Board.OpenMP Application Program Interface,Version 2.5[EB/OL],http://www.openmp.org/mp-documents/spec25.pdf,2005-05
[47]张舒,褚艳利,赵开勇,等.GPU高性能运算之CUDA[M].北京:中国水利水电出版社,2009
[48]陈大伟.基于通用GPU的JPEG2000算法的并行实现[D].南京:南京邮电大学,2008
[49]钱悦.图形处理器CUDA编程模型的应用研究[J].计算机与数字工程,2008,36(12):177-180
[50]张健,陈瑞.图形处理器在通用计算中的应用[J].计算机工程与设计,2009,30(14):3359-3361
[51]马淑芳.基于GPU加速的分形地形生成方法[D].大连:大连理工大学,2008
[52]张军华,雷凌,仝兆岐.PC Cluster技术的国内外现状与发展趋势[J].石油物探,2003,42(4):557-561
[53]沈孝科.利用FFTW实现快速傅里叶变换[J].考试周刊,2008(52):181-182
[54]肖江,胡柯良,邓元勇.基于CUDA的矩阵乘法和FFT性能测试[J].计算机工程,2009,35(10):7-10
[2]王勇超.高性能计算技术应用研究[D].西安:西安理工大学,2007
[3] TOP500 Supercomputing Sites[EB/OL],http://www.top500.org/,2009-11-17
[4]赵改善,包红林.集群计算技术及其在石油工业中的应用[J].石油物探,2001,40(3):118-126
[5]戴猛.Linux集群在石油行业的应用现状及发展趋势[J].硅谷,2009(7):115-116
[6] Rajkumar Buyya.高性能集群计算:结构与系统(第一卷)[M].郑纬民,石威,汪东升,等译.北京:电子工业出版社,2001:6-24
[7]姚成浪.基于MPI的集群系统设计与实现[D].哈尔滨:哈尔滨工程大学,2007
[8]陈国良,安虹,陈崚,等.并行算法实践[M].北京:高等教育出版社,2004
[9]赵传雪.基于LINUX PC集群数据处理算法研究[D].西安:西安石油大学,2006
[10]熊盛武,王鲁,杨婕.构建高性能集群计算机系统的关键技术[J].微计算机信息,2006,22(1-3):86-88
[11]刘素芹.基于应用网格环境的复杂地表波动方程基准面静校正研究[D].东营:中国石油大学,2008
[12]牛涛.Pc集群环境MPI并行DFT运算的研究[D].广州:华南师范大学,2003
[13]林娜.MPICH作业递交方式的研究与改进[D].上海:复旦大学,2006
[14]都志辉.高性能计算之并行编程技术——MPI并行程序设计[M].北京:清华大学出版社,2001:6-7
[15] George Em Karniadakis and Robert M. Kirby II.Parallel Scientific Computing in C++ and MPI:A seamless approach to parallel algorithms and their implementation[M]. Cambridge:Cambridge University Press,2003
[16]崔永良,刘羽,郭慧玲.MPI-2新特性及MPICH配置方法[J].山东理工大学学报(自然科学版),2008,22(5):44-47
[17] William Gropp,Ewing Lusk,Rajeev Thakur.Using MPI-2:advanced features of the message-passing interface[M].Cambridge:MIT Press,1999
[18]丁慧清.PC Linux机群系统并行计算环境的组建与性能测试[D].北京:北京工业大学,2004
[19]王勇超,张璟,王新卫,等.基于MPICH2的高性能计算集群系统研究[J].计算机技术与发展,2008,18(9):101-104
[20]朱亚超.Beowulf集群系统性能评测技术研究[D].武汉:华中科技大学,2006
[21]伍卫国,陈长虹,张虎,等.并行文件系统管理工具的设计与实现[J].计算机工程,2007,33(18):279-282
[22]霍严梅,杨可新,胡亮,等.并行文件系统研究综述[J].小型微型计算机系统,2008,29(9):1631-1636
[23]董勇,周恩强,陈娟.基于Infiniband技术构建高性能分布式文件系统—Lustre[J].计算机工程与应用,2005(22):103-107,228
[24]杨昕,沈文海.Lustre并行文件系统的发展及在气象领域的应用前景[J].应用气象学报,2008,19(2):243-249
[25]林松涛.基于Lustre文件系统的并行I/O技术研究[D].长沙:国防科学技术大学,2004
[26]张媛,卢泽新,刘亚萍.NFS over Lustre性能评测与分析[J].计算机工程,2007,33(10):274-276
[27] Christopher Smith.Linux NFS-HOWTO[EB/OL],http://nfs.sourceforge.net/nfs-howto/,2006-05-02
[28]曹立强,罗红兵,张晓霞.集群环境中影响NFS文件系统带宽的测试与分析[J].计算机工程,2007,33(19):125-127
[29]张丹丹,姚继锋.基于软RAID的Lustre性能影响要素简析[A].2008年全国高性能计算学术年会论文集[C].无锡:国家并行计算机工程技术研究中心,2008:427-430
[30]王扬,周可,李春花,等.集群文件系统的性能最优化配置[J].计算机研究与发展,2007,44(增刊):168-172
[31] Sun Microsystems , Inc. LustreTM 1.6 Operations Manual[EB/OL] ,http://dlc.sun.com/pdf/820-3681/820-3681.pdf,2009-05
[32]常兴华.Lustre存储系统安全模型设计[J].信息安全与通信保密,2008(4):80-82
[33]黄建华,何希.基于动态口令体制的Kerberos协议改进[J].计算机安全,2009(2):66-69
[34] Frederick Butler,Iliano Cervesato,Aaron D. Jaggard,et al.Formal analysis of Kerberos5[J].Theoretical Computer Science,2006,367(1-2):57-87
[35]范洪生,叶震,侯保花.基于公钥密码体制的Kerberos协议的改进[J].计算机技术与发展,2006,16(4):224-227
[36]何蕴婷,张宗福.基于PKI的文件安全传输方案研究[J].电脑知识与技术,2008,3(5):885-887
[37] Ali Nasrat Haidar,Ali E. Abdallah.Formal modelling of PKI based authentication[J]. Electronic Notes in Theoretical Computer Science,2009,235:55-70
[38] Andrew Nash,William Duane,Celia Joseph,et al.公钥基础设施(PKI):实现和管理电子安全[M].张玉涛,陈建奇,杨波,等译.北京:清华大学出版社,2002
[39]刘士栋,杨林,王建新.分布式网络安全服务Kerberos和PKI比较分析[A].全国第八届通信保密与信息安全现状研讨会论文集[C].成都:中国电子科技集团公司第三十研究所,2002:106-113
[40] Rohit Chandra,Leonardo Dagum,Dave Kohr,et al.Parallel Programming in OpenMP[M].San Fransisco:Morgan Kaufmann Publishers,2001
[41] Barbara Chapman,Gabriele Jost,Ruud van der Pas.Using OpenMP:Portable Shared Memory Parallel Programming[M].Cambridge:MIT Press,2007
[42]蔡佳佳,李名世,郑锋.多核微机基于OpenMP的并行计算[J].计算机技术与发展,2007,17(10):87-91
[43]殷顺昌.OpenMP并行程序性能分析[D].长沙:国防科学技术大学,2006
[44]吴少刚.机群系统OpenMP研究[D].北京:中国科学院计算技术研究所,2004
[45]周伟明.多核计算与程序设计[M].武汉:华中科技大学出版社,2009:75-123
[46] OpenMP Architecture Review Board.OpenMP Application Program Interface,Version 2.5[EB/OL],http://www.openmp.org/mp-documents/spec25.pdf,2005-05
[47]张舒,褚艳利,赵开勇,等.GPU高性能运算之CUDA[M].北京:中国水利水电出版社,2009
[48]陈大伟.基于通用GPU的JPEG2000算法的并行实现[D].南京:南京邮电大学,2008
[49]钱悦.图形处理器CUDA编程模型的应用研究[J].计算机与数字工程,2008,36(12):177-180
[50]张健,陈瑞.图形处理器在通用计算中的应用[J].计算机工程与设计,2009,30(14):3359-3361
[51]马淑芳.基于GPU加速的分形地形生成方法[D].大连:大连理工大学,2008
[52]张军华,雷凌,仝兆岐.PC Cluster技术的国内外现状与发展趋势[J].石油物探,2003,42(4):557-561
[53]沈孝科.利用FFTW实现快速傅里叶变换[J].考试周刊,2008(52):181-182
[54]肖江,胡柯良,邓元勇.基于CUDA的矩阵乘法和FFT性能测试[J].计算机工程,2009,35(10):7-10