基于事件的并行程序性能可视化系统研究与实现
|
摘要
随着并行技术的发展和广泛应用,并行程序性能可视化成为开发高效并行程序的一个重要方面。通过形象、直观和富有语义的图形来展示在监测并行程序执行过程中获取的性能数据,帮助用户理解和把握并行程序实际的动态执行过程,从而有效定位并行程序的性能瓶颈点,为并行程序的性能优化提供可视参考。
本文针对并行程序性能可视化领域,总结归纳并行程序性能监测及可视化方法;基于事件的方式设计并行程序的性能监测库,在并行程序的执行过程中自动的获取性能数据;根据并行程序的执行特点形象的抽象出状态、消息和进程图形,通过对性能数据的静态浏览和动画重演,展示出并行程序在执行过程中的各个性能状态。本文的主要研究工作包括以下三个方面:
1)根据事件的含义,对并行程序中重要的函数事件进行归类,建立并行程序的监测事件集,以更改系统连接库的方式来设计并行程序的性能监测库,从而能自动获取并行程序在执行过程中产生的性能数据。
2)根据并行程序执行特性,建立并行程序的图形化抽象,通过形象、直观且易于理解的图形来对性能数据进行静态浏览和动态重演,从而展示出并行程序在实际执行过程中的各个性能状态。
3)设计和开发一个集图形化编程、远程提交编译计算、性能数据自动监测采集和性能数据可视化于一体的图形化并行程序开发系统。
With development and wide application of the parallel technology, parallel performance visualization becomes an important aspect of developing an efficient parallel program. The performance data collected during the execution of parallel program is displayed by means of imaginal, lively and rich semantical graphics. It can help the users to understand and grasp the practical dynamic process of parallel execution, and also can help users to effectively locate the performance bottleneck and can give visual reference for optimizing the parallel performance.
According to the field of parallel performance visualization, parallel performance monitoring and visualization methods are summarized in this paper. Parallel performance monitoring library is designed based on the event model, and performance data can be automatically collected by using it; State graphics, message graphics and process graphics are generated according to the features of execution of parallel program. With the help of visualization by means of static browsing and dynamic repeating, users can easily understand each performance state during the parallel execution. The main research content of this paper including the following three parts:
1) Important function events of parallel program are classified according to the meaning of event. And on the basis, the monitoring events set can be easily established. The parallel program monitoring library is designed by the way of changing the system library, so the performance data can be automatically collected during the parallel execution.
2) Parallel program graphical abstracts are established according to the characteristics of the parallel program execution, the performance data is visualized using the imaginal, lively and rich semantical graphics through static browsing and dynamic repeating, it can help users to understand each performance state of the parallel execution.
3) A graphical parallel program develop system is designed and developed with the function such as: graphical programming, remote submitting, remote compiling, remote computing, automatically collecting performance data and visualization.
本文针对并行程序性能可视化领域,总结归纳并行程序性能监测及可视化方法;基于事件的方式设计并行程序的性能监测库,在并行程序的执行过程中自动的获取性能数据;根据并行程序的执行特点形象的抽象出状态、消息和进程图形,通过对性能数据的静态浏览和动画重演,展示出并行程序在执行过程中的各个性能状态。本文的主要研究工作包括以下三个方面:
1)根据事件的含义,对并行程序中重要的函数事件进行归类,建立并行程序的监测事件集,以更改系统连接库的方式来设计并行程序的性能监测库,从而能自动获取并行程序在执行过程中产生的性能数据。
2)根据并行程序执行特性,建立并行程序的图形化抽象,通过形象、直观且易于理解的图形来对性能数据进行静态浏览和动态重演,从而展示出并行程序在实际执行过程中的各个性能状态。
3)设计和开发一个集图形化编程、远程提交编译计算、性能数据自动监测采集和性能数据可视化于一体的图形化并行程序开发系统。
With development and wide application of the parallel technology, parallel performance visualization becomes an important aspect of developing an efficient parallel program. The performance data collected during the execution of parallel program is displayed by means of imaginal, lively and rich semantical graphics. It can help the users to understand and grasp the practical dynamic process of parallel execution, and also can help users to effectively locate the performance bottleneck and can give visual reference for optimizing the parallel performance.
According to the field of parallel performance visualization, parallel performance monitoring and visualization methods are summarized in this paper. Parallel performance monitoring library is designed based on the event model, and performance data can be automatically collected by using it; State graphics, message graphics and process graphics are generated according to the features of execution of parallel program. With the help of visualization by means of static browsing and dynamic repeating, users can easily understand each performance state during the parallel execution. The main research content of this paper including the following three parts:
1) Important function events of parallel program are classified according to the meaning of event. And on the basis, the monitoring events set can be easily established. The parallel program monitoring library is designed by the way of changing the system library, so the performance data can be automatically collected during the parallel execution.
2) Parallel program graphical abstracts are established according to the characteristics of the parallel program execution, the performance data is visualized using the imaginal, lively and rich semantical graphics through static browsing and dynamic repeating, it can help users to understand each performance state of the parallel execution.
3) A graphical parallel program develop system is designed and developed with the function such as: graphical programming, remote submitting, remote compiling, remote computing, automatically collecting performance data and visualization.
引文
[1]Paradyn Parallel Performance Tools.http://www.paradyn.org/html/overview.html.2008-3-13.
[2]A Performance Visualization Tool for MPI.http://www.csar.uiuc.edu/software /paragraph/.2008-3-13.
[3]PAPI.http://icl.cs.utk.edu/papi/overview/index.html.2008-3-13.
[4]Vampir.http://www.vampir-ng.de/.2008-3-13.
[5]Performance Visualization for Parallel Programs.http://www-unix.mcs.anl.gov/perfvis/download/index.htm.2008-3-13.
[6]张兆庆,刘强.并行程序行为及性能可视化工具—Para Vision[J],软件世界,1996,5:86-87.
[7]mpiP:Lightweight,Scalable MPI Profiling.http://mpip.sourceforge.net/.2008-3-14.
[8]TAU-Tuning and Analysis utilities,http://www.cs.uoregon.edu/research/tau/home.php/.2008-3-14.
[9]Renaissance Computing Institute.http://www.renci.org/resources/software/svpablo/.2008-3-14.
[10]Deep MPI.http://www.crescentbaysoftware.com/deep_mpi_top.html.2008-3-14.
[11]XPVM:A Graphical Console and Monitor for PVM http://www.netlib.org/utk /icl/xpvm/xpvm.html.2008-3-14.
[12]XMPI-A Run/Debug GUI for MPI.http://www.lam-mpi.org/software/xmpi/.2008-3-14.
[13]HPM Tool Kit.http://www.alphaworks.ibm.com/tech/hpmtoolkit/.2008-3-14.
[14]Intel(?)VTune~(TM)Performance Analyzer.http://www.intel.com/cd/software/products/asmo-na/eng/vtune/.2008-3-14.
[15]徐杰锋,舒继武,郑纬民.并行处理可视化监测环境[J].清华大学学报(自然科学版),2003 43(4):532-535.
[16]马捷.并行程序性能可视化分析系统的研究与实现[D].西安:西安交通大学硕士学位论文,1998.7.
[17]刘华,徐炜民,孙强.基于MPI并行程序的性能评测可视化工具[J].计算机工程,2004.5,30(10):82-84.
[18]T.LeBlance,J.Mellor-Crummey and R.Fowler.Analyzing Parallel Program Execution Using Multiple Views. J.Parallel&Distributed Computing, Vol. 9,No.2, June 1990, pp.203-217.
[19] M.T.Heath, A.D.Malony and D.T.Rover. Parallel Performance Visualization:from Practice to Theory. IEEE Parallel and Distributed Technology, Winter 1995, pp44-60.
[20] A.Couch. Categories and Context in Scalable Execution Visualization.J.Parallel & Distributed Computing, Vol.18,No.2,pp.195-204.
[21] M.T.Heath, J.A.Etheridge. Visualizing the Performance of Parallel Programs.IEEE Software, Sep.1991,pp.29-39.
[22] A.D.Malony et al. An Integrated Performance data Collection, Analysis, and Visualization System. Proc.Fourth Conf. Hypercube Concurrent Computers Application,Golden gate Enterprises, Los Altos,Calif.1989,pp,229-236.
[23] D.A.Reed et al.Scalable performance analysis:the Pablo performance analysis environment. Proc.the Scalable Parallel Libraries Conf. Los Alamitos,Calif.1993,pp.104-113.
[24] D.Hart, E.Kraemer and G.C.Roman. Interactive Visual Exploration of Distribted Computations.Proc.11th International Parallel Processing Symposium, April 1997.
[25] J.K.Hollingsworth,R.B.Irvin and B.P.Miller. The Integration of Application and System Based Metrics in A Parallel Performance Tool. 1991 ACM SIGPLAN Symposiun on Principles and Practice of Parallel Programming,April 1991,pp.189-220.
[26] T.Fahringer. Estimating and optimizing performance for parallel programs,IEEE Computer4,1995,Vol.28,No. 11 ,pp,47-56.
[27] D.Kranzlmuler, R.Koppler,S.Grabner, et al.Parallel Program Visualization with MUCH. Parallel Computation,Springer Verlag, 1996,pp.148-160.
[28] J.L.Sharnowski,Betty H.C.Cheng. A Formal Approach to Mcdeling Expected Behavior in Parallel Program Visualizations. Proc. Parallel Architectures and Languages Europe(PARLE'94),Lecture Notes in Computer Science,Vol.817,Springer Verlay, July 1994,pp.202-213.
[29] Ming C.Hao,A.Waheed,A.Karp,et.al.Multiple Views of Parallel Application Execution. Debugging and Performance Tuning for Parallel Computing Systems, IEEE Computer Society Press,1996,pp.199-206.
[30] J.Muthukumarasamy,J.T.Stasko. Visualizing Program Execution on Large Data Sets Using Semantic Zooming. Graphics,Visualization and Usability Center,Georgia Institute of Technology,Atlanta GA, Technical Report GIT-GVU-95-02,January 1995.
[31]王恩柱.基于模板的可视化并行程序设计平台的研究与实现[D].合肥工业大学硕士学位论文.2007.6.
[32]V.S.Sunderam,PVM:a framework for parallel distributed computing.Practice and Experience archive.2(4):315-339,1990.
[33]M.Snir and W.Gropp,MPI:The Complete Reference:The MIT Press,1996.
[34]Verdian,The Portable Batch System Software,Avaliable at:http://www.OpenPBS.org/,2006,5.
[35]M.H.P.C.Center,Maui scheduler,Avaliable at:http://mauischeduler.sourceforge.net/,2006,5.
[36]R.Flanery,A.Geist,B.Luethke,and S.L.Scott,Cluster Command & Control (C3)Tool Suite,In:The Third Austrian-Hungarian Workship on Distributed and Parallel Systems(DAPSYS 2000)in conjunction with EuroPVM/MPI 2000,Balatonfured,Lake Balaton,Hungary,2000.
[37]J.Hsieh,T.Leng,and Y.-C.Fang,OSCAR:A Turnkey Solution for Cluster Computing.Dell Power Solutions.(1),2001.
[38]P.M.Papadopoulos,M.J.Katz,and G.Bruno,NPACI rocks:Tools and techniques for easily deploying manageable linux
[39]EIsholf I J P.A distributed debugger for Amoeba.In:Proceedings of the ACM SIGPLAN/SIGOPS Workshop on Parallel and Distributed Debugging,published in ACM SIGPLAN Notices,1989,24(1):1-10.
[40]Bates P.Debugging heterogeneous distributed systems using event-based models of behavior.In:Proceedings of the ACM SIGPLAN/SIGOPS Workshop on Parallel and Distributed Debugging,published in ACM SIGPLAN Notices,1989,24(1):11-12.
[41]时培值,李三立,孟杰.高性能计算机系统中可视化负载信息的获取及性能分析工具:THPTH[J].小型微型计算机系统,2002.
[42]卫兴武,路强,刘晓平,一种MPI并行程序性能监测库的实现技术,刘晓平等.计算机技术与应用进展2007[C].宁波:2007,556-560.
[43]黄四牛,熊新平.DIS中时间同步问题研究[J].系统仿真学报,2000,12(5):9-10,17.
[44]王延华,洪飞,吴恩华.基于VTK库医学图像处理子系统设计和实现[J],计算机工程与应用,2003.8,39(8):205-207.
[45]What Is VTK?.http://www.vtk.org/what-is-vtk.php 2008.5
[2]A Performance Visualization Tool for MPI.http://www.csar.uiuc.edu/software /paragraph/.2008-3-13.
[3]PAPI.http://icl.cs.utk.edu/papi/overview/index.html.2008-3-13.
[4]Vampir.http://www.vampir-ng.de/.2008-3-13.
[5]Performance Visualization for Parallel Programs.http://www-unix.mcs.anl.gov/perfvis/download/index.htm.2008-3-13.
[6]张兆庆,刘强.并行程序行为及性能可视化工具—Para Vision[J],软件世界,1996,5:86-87.
[7]mpiP:Lightweight,Scalable MPI Profiling.http://mpip.sourceforge.net/.2008-3-14.
[8]TAU-Tuning and Analysis utilities,http://www.cs.uoregon.edu/research/tau/home.php/.2008-3-14.
[9]Renaissance Computing Institute.http://www.renci.org/resources/software/svpablo/.2008-3-14.
[10]Deep MPI.http://www.crescentbaysoftware.com/deep_mpi_top.html.2008-3-14.
[11]XPVM:A Graphical Console and Monitor for PVM http://www.netlib.org/utk /icl/xpvm/xpvm.html.2008-3-14.
[12]XMPI-A Run/Debug GUI for MPI.http://www.lam-mpi.org/software/xmpi/.2008-3-14.
[13]HPM Tool Kit.http://www.alphaworks.ibm.com/tech/hpmtoolkit/.2008-3-14.
[14]Intel(?)VTune~(TM)Performance Analyzer.http://www.intel.com/cd/software/products/asmo-na/eng/vtune/.2008-3-14.
[15]徐杰锋,舒继武,郑纬民.并行处理可视化监测环境[J].清华大学学报(自然科学版),2003 43(4):532-535.
[16]马捷.并行程序性能可视化分析系统的研究与实现[D].西安:西安交通大学硕士学位论文,1998.7.
[17]刘华,徐炜民,孙强.基于MPI并行程序的性能评测可视化工具[J].计算机工程,2004.5,30(10):82-84.
[18]T.LeBlance,J.Mellor-Crummey and R.Fowler.Analyzing Parallel Program Execution Using Multiple Views. J.Parallel&Distributed Computing, Vol. 9,No.2, June 1990, pp.203-217.
[19] M.T.Heath, A.D.Malony and D.T.Rover. Parallel Performance Visualization:from Practice to Theory. IEEE Parallel and Distributed Technology, Winter 1995, pp44-60.
[20] A.Couch. Categories and Context in Scalable Execution Visualization.J.Parallel & Distributed Computing, Vol.18,No.2,pp.195-204.
[21] M.T.Heath, J.A.Etheridge. Visualizing the Performance of Parallel Programs.IEEE Software, Sep.1991,pp.29-39.
[22] A.D.Malony et al. An Integrated Performance data Collection, Analysis, and Visualization System. Proc.Fourth Conf. Hypercube Concurrent Computers Application,Golden gate Enterprises, Los Altos,Calif.1989,pp,229-236.
[23] D.A.Reed et al.Scalable performance analysis:the Pablo performance analysis environment. Proc.the Scalable Parallel Libraries Conf. Los Alamitos,Calif.1993,pp.104-113.
[24] D.Hart, E.Kraemer and G.C.Roman. Interactive Visual Exploration of Distribted Computations.Proc.11th International Parallel Processing Symposium, April 1997.
[25] J.K.Hollingsworth,R.B.Irvin and B.P.Miller. The Integration of Application and System Based Metrics in A Parallel Performance Tool. 1991 ACM SIGPLAN Symposiun on Principles and Practice of Parallel Programming,April 1991,pp.189-220.
[26] T.Fahringer. Estimating and optimizing performance for parallel programs,IEEE Computer4,1995,Vol.28,No. 11 ,pp,47-56.
[27] D.Kranzlmuler, R.Koppler,S.Grabner, et al.Parallel Program Visualization with MUCH. Parallel Computation,Springer Verlag, 1996,pp.148-160.
[28] J.L.Sharnowski,Betty H.C.Cheng. A Formal Approach to Mcdeling Expected Behavior in Parallel Program Visualizations. Proc. Parallel Architectures and Languages Europe(PARLE'94),Lecture Notes in Computer Science,Vol.817,Springer Verlay, July 1994,pp.202-213.
[29] Ming C.Hao,A.Waheed,A.Karp,et.al.Multiple Views of Parallel Application Execution. Debugging and Performance Tuning for Parallel Computing Systems, IEEE Computer Society Press,1996,pp.199-206.
[30] J.Muthukumarasamy,J.T.Stasko. Visualizing Program Execution on Large Data Sets Using Semantic Zooming. Graphics,Visualization and Usability Center,Georgia Institute of Technology,Atlanta GA, Technical Report GIT-GVU-95-02,January 1995.
[31]王恩柱.基于模板的可视化并行程序设计平台的研究与实现[D].合肥工业大学硕士学位论文.2007.6.
[32]V.S.Sunderam,PVM:a framework for parallel distributed computing.Practice and Experience archive.2(4):315-339,1990.
[33]M.Snir and W.Gropp,MPI:The Complete Reference:The MIT Press,1996.
[34]Verdian,The Portable Batch System Software,Avaliable at:http://www.OpenPBS.org/,2006,5.
[35]M.H.P.C.Center,Maui scheduler,Avaliable at:http://mauischeduler.sourceforge.net/,2006,5.
[36]R.Flanery,A.Geist,B.Luethke,and S.L.Scott,Cluster Command & Control (C3)Tool Suite,In:The Third Austrian-Hungarian Workship on Distributed and Parallel Systems(DAPSYS 2000)in conjunction with EuroPVM/MPI 2000,Balatonfured,Lake Balaton,Hungary,2000.
[37]J.Hsieh,T.Leng,and Y.-C.Fang,OSCAR:A Turnkey Solution for Cluster Computing.Dell Power Solutions.(1),2001.
[38]P.M.Papadopoulos,M.J.Katz,and G.Bruno,NPACI rocks:Tools and techniques for easily deploying manageable linux
[39]EIsholf I J P.A distributed debugger for Amoeba.In:Proceedings of the ACM SIGPLAN/SIGOPS Workshop on Parallel and Distributed Debugging,published in ACM SIGPLAN Notices,1989,24(1):1-10.
[40]Bates P.Debugging heterogeneous distributed systems using event-based models of behavior.In:Proceedings of the ACM SIGPLAN/SIGOPS Workshop on Parallel and Distributed Debugging,published in ACM SIGPLAN Notices,1989,24(1):11-12.
[41]时培值,李三立,孟杰.高性能计算机系统中可视化负载信息的获取及性能分析工具:THPTH[J].小型微型计算机系统,2002.
[42]卫兴武,路强,刘晓平,一种MPI并行程序性能监测库的实现技术,刘晓平等.计算机技术与应用进展2007[C].宁波:2007,556-560.
[43]黄四牛,熊新平.DIS中时间同步问题研究[J].系统仿真学报,2000,12(5):9-10,17.
[44]王延华,洪飞,吴恩华.基于VTK库医学图像处理子系统设计和实现[J],计算机工程与应用,2003.8,39(8):205-207.
[45]What Is VTK?.http://www.vtk.org/what-is-vtk.php 2008.5