Accelerating large-scale simulation of seismic wave propagation by multi-GPUs and three-dimensional domain decomposition

详细信息    查看全文

• 作者：Taro Okamoto ; Hiroshi Takenaka ; Takeshi Nakamura ; Takayuki Aoki
• 关键词：Seismic wave propagation ; finite ; difference method ; GPU ; parallel computing ; three ; dimensional domain decomposition
• 刊名：Earth, Planets and Space
• 出版年：2010
• 出版时间：December 2010
• 年：2010
• 卷：62
• 期：12
• 页码：939-942
• 全文大小：278KB
• 参考文献：Abdelkhalek, R., H. Calandra, O. Coulaud, J. Roman, and G. Latu, Fast seismic modeling and reverse time migration on a GPU cluster, International Conference on High Performance Computing & Simulation, 36-3, 2009.
  Aoi, S., N. Nishizawa, and T. Aoki, 3-D wave propagation simulation using GPGPU, Programme and Abstracts, Seismol. Soc. Jpn., 2009 Fall Meeting, abstract A12-09, 2009.
  Aoki, T., Full-GPU CFD applications, IPSJ Mag., 50(2), 107-15, 2009.
  Aoki, T., Multi-GPU Scalabilities for Mesh-based HPC Applications, SIAM Conf. Parallel Processing for Scientific Computing (PP10), Seattle, Washington, February 26, 2010.
  Cerjan, C., D. Kosloff, R. Kosloff, and M. Reshef, A nonreflecting boundary conditions for discrete acoustic and elastic wave equations, Geophysics, 50, 705-08, 1985.CrossRef
  Clayton, R. and B. Engquist, Absorbing boundary conditions for acoustic and elastic wave equations, Bull. Seismol. Soc. Am., 67, 1529-540, 1977.
  Furumura, T., Large-scale simulation of seismic wave propagation in 3D heterogeneous structure using the finite-difference method, J. Seismol. Soc. Jpn. (Zisin), 61, S83–S92, 2009.
  Graves, R. W., Simulating seismic wave propagation in 3D elastic media using staggered-grid finite differences, Bull. Seismol. Soc. Am., 86, 1091-106, 1996.
  Komatitsch, D., D. Michéa, and G. Erlebacher, Porting a high-order finite-element earthquake modeling application to NVIDIA graphics cards using CUDA, J. Parallel Distrib. Comput., 69, 451-60, 2009.CrossRef
  Komatitsch, D., G. Erlebacher, D. G?ddek, and D. Michéa, High-order finite-element seismic wave propagation modeling with MPI on a large GPU cluster, J. Comp. Phys., 229, 7692-714, 2010.CrossRef
  Michea, D. and D. Komatitsch, Accelerating a three-dimensional finite-difference wave propagation code using GPU graphics cards, Geophys. J. Int., doi:10.-111/?j.-365-246X.-010.-4616.?x , 2010.
  Micikevicius, P., 3D finite-difference computation on GPUs using CUDA, in GPGPU-2: Proc. 2nd Workshop on General Purpose Processing on Graphics Processing Units, pp. 79-4, Washington DC, USA, 2009.
  Ogawa, S., T. Aoki, and A. Yamanaka, Multi-GPU scalability of phase-field simulation for phase transition- TFlop/s performance on 40 GPUs, Trans. IPSS Japan, Advanced Computing Systems, 3,67-5, 2010.
  Okamoto, T., H. Takenaka, and T. Nakamura, Computation of Seismic Wave Propagation With GPGPU, Programme and Abstracts, Seismol. Soc. Jpn,, 2009 Fall Meeting, abstract P3-2, 2009.
  Okamoto, T., H. Takenaka, and T. Nakamura, Simulation of seismic wave propagation by GPU, Proceedings ofSymposium on AdvancedComput-ing Systems and Infrastructures, 141-42, 2010.
  Olsen, K. B., S. M. Day, J. B. Minster, Y. Cui, A. Chourasia, D. Okaya, P. Maechling, and T. Jordan, TeraShake2: Spontaneous rupture simulations of Mw 7.7 Earthquakes on the Southern San Andreas Fault, Bull Seismol. Soc. Am., 98, 1162-185, 2008.CrossRef
  Takenaka, H., T. Nakamura, T. Okamoto, and Y. Kaneda, A unified approach implementing land and ocean-bottom topographies in the staggered-grid finite-difference method for seismic wave modeling, Proc. 9th SEGJInt. Symp., CD-ROM Paper No.37, 2009.
  
• 作者单位：Taro Okamoto (1)
  Hiroshi Takenaka (2)
  Takeshi Nakamura (3)
  Takayuki Aoki (4)
  
  1. Department of Earth and Planetary Sciences, Tokyo Institute of Technology, 2-12-1 Ookayama, Meguro, Tokyo, 152-8551, Japan
  2. Department of Earth and Planetary Sciences, Kyushu University, 6-10-1 Hakozaki, Higashi-ku, Fukuoka, 812-8581, Japan
  3. Earthquake and Tsunami Research Project for Disaster Prevention, Japan Agency for Marine-Earth Science and Technology, 3173-25, Showa-machi, Kanazawa-ku, Yokohama, Kanagawa, 236-0001, Japan
  4. Global Scientific Information and Computing Center, Tokyo Institute of Technology, 2-12-1 Ookayama, Meguro, Tokyo, 152-8550, Japan
  
• 刊物类别：Earth Sciences, general; Geology; Geophysics/Geodesy;
• 刊物主题：Earth Sciences, general; Geology; Geophysics/Geodesy;
• 出版者：Springer Berlin Heidelberg
• ISSN：1880-5981

文摘

We adopted the GPU (graphics processing unit) to accelerate the large-scale finite-difference simulation of seismic wave propagation. The simulation can benefit from the high-memory bandwidth of GPU because it is a “memory intensive-problem. In a single-GPU case we achieved a performance of about 56 GFlops, which was about 45-fold faster than that achieved by a single core of the host central processing unit (CPU). We confirmed that the optimized use of fast shared memory and registers were essential for performance. In the multi-GPU case with three-dimensional domain decomposition, the non-contiguous memory alignment in the ghost zones was found to impose quite long time in data transfer between GPU and the host node. This problem was solved by using contiguous memory buffers for ghost zones. We achieved a performance of about 2.2 TFlops by using 120 GPUs and 330 GB of total memory: nearly (or more than) 2200 cores of host CPUs would be required to achieve the same performance. The weak scaling was nearly proportional to the number of GPUs. We therefore conclude that GPU computing for large-scale simulation of seismic wave propagation is a promising approach as a faster simulation is possible with reduced computational resources compared to CPUs. Key words Seismic wave propagation finite-difference method GPU parallel computing three-dimensional domain decomposition