地球外核热对流运动的区域分解多重网格并行数值模拟
|
摘要
旋转球层中热对流运动的数值模拟是地球发电机模型的重要组成部分,对研究地球发电机作用机理具有重要意义.本文设计一个基于国产超级计算平台并行性能良好的地球外核热对流运动并行数值模型.时间积分方案采用与Crank-Nicolson格式和二阶Adams-Bashford公式相结合的近似分解分步法,空间离散基于立方球网格的二阶精度有限体积格式.所得到的两个大规模稀疏线性代数方程组采用带预处理的Krylov子空间迭代法进行求解.为加速迭代求解过程及提高并行性能,迭代过程采用区域分解多重网格的多层限制型加法Schwarz预处理子,减少了求解程序的计算时间,提高了数值模型的并行性能,模型被很好地扩展到上万处理器核数.数值模拟结果与基准模型算例0的参考值吻合得很好.
Numerical simulations of thermal convection in a rotating spherical shell play an important role in dynamo models. In this paper, we present a parallel numerical model with high performance for the Earth's outer core convection based on a homegrown supercomputer. An approximate factorization fractional method combined with Crank-Nicolson scheme and second-order Adams-Bashford formula is employed for temporal integration. Spatial terms are discretized by a second-order finite volume scheme based on cubed-sphere grid. Two resultant large sparse linear algebraic equations are solved by preconditioned Krylov subspace iterative method. To accelerate convergence rate and improve parallel performance, linear solver is preconditioned with multilevel restricted additive Schwarz preconditioner based on domain decomposition and multigrid. The preconditioner reduces compute time and improve parallel performance of the solvers, which scales well to over ten thousand processor cores. Numerical results are in good agreement with reference solutions of the benchmark Case 0.
Numerical simulations of thermal convection in a rotating spherical shell play an important role in dynamo models. In this paper, we present a parallel numerical model with high performance for the Earth's outer core convection based on a homegrown supercomputer. An approximate factorization fractional method combined with Crank-Nicolson scheme and second-order Adams-Bashford formula is employed for temporal integration. Spatial terms are discretized by a second-order finite volume scheme based on cubed-sphere grid. Two resultant large sparse linear algebraic equations are solved by preconditioned Krylov subspace iterative method. To accelerate convergence rate and improve parallel performance, linear solver is preconditioned with multilevel restricted additive Schwarz preconditioner based on domain decomposition and multigrid. The preconditioner reduces compute time and improve parallel performance of the solvers, which scales well to over ten thousand processor cores. Numerical results are in good agreement with reference solutions of the benchmark Case 0.
引文
[1] WICHT J, TILGNER A. Theory and modeling of planetary dynamos[J]. Space Science Reviews, 2010, 152(1/4):501-542.
[2] ROBERTS P H, KING E M. On the genesis of the Earth’s magnetism[J]. Reports on Progress in Physics, 2013, 76(9):096801.
[3] GLATZMAIER G A, ROBERTS P H. A three-dimensional self-consistent computer simulation of a geomagnetic field reversal[J]. Nature, 1995, 377(6546):203-209.
[4] KUANG W J, BLOXHAM J. An Earth-like numerical dynamo model[J]. Nature, 1997, 389(6649):371-374.
[5] CHRISTENSEN U, OLSON P, GLATZMAIER G A. Numerical modelling of the geodynamo: A systematic parameter study[J]. Geophysical Journal International, 1999, 138(2):393-409.
[6] VANTIEGHEM S, SHEYKO A, JACKSON A. Applications of a finite-volume algorithm for incompressible MHD problems[J]. Geophysical Journal International, 2016, 204(2):1376-1395.
[7] KAGEYAMA A, YOSHIDA M. Geodynamo and mantle convection simulations on the Earth Simulator using the Yin-Yang grid[J]. Journal of Physics: Conference Series, 2005, 16:325-338.
[8] HARDER H, HANSEN U. A finite-volume solution method for thermal convection and dynamo problems in spherical shells[J]. Geophysical Journal International, 2005, 161(2):522-532.
[9] MATSUI H, OKUDA H. Thermal convection analysis in a rotating shell by a parallel finite-element method-development of a thermal-hydraulic subsystem of GeoFEM[J]. Concurrency and Computation: Practice and Experience, 2002, 14(6/7):465-481.
[10] CHAN K H, LI L G, LIAO X H. Modelling the core convection using finite element and finite difference methods[J]. Physics of the Earth and Planetary Interiors, 2006, 157(1):124-138.
[11] YIN L, YANG C, MA S Z, et al. Parallel numerical simulation of the thermal convection in the Earth’s outer core on the cubed-sphere[J]. Geophysical Journal International, 2017, 209(3):1934-1954.
[12] GLATZMAIER G A. Geodynamo simulations-how realistic are they?[J]. Annual Review of Earth and Planetary Sciences, 2002, 30(1):237-257.
[13] MATSUI H, HEIEN E, AUBERT J, et al. Performance benchmarks for a next generation numerical dynamo model[J]. Geochemistry, Geophysics, Geosystems, 2016, 17(5):1586-1607.
[14] AURNOU J M, CALKINS M A, CHENG J S, et al. Rotating convective turbulence in Earth and planetary cores[J]. Physics of the Earth and Planetary Interiors, 2015, 246:52-71.
[15] CHRISTENSEN U R, AUBERT J, CARDIN P, et al. A numerical dynamo benchmark[J]. Physics of the Earth and Planetary Interiors, 2001, 128(1):25-34.
[16] HEJDA P, RESHETNYAK M. Control volume method for the thermal convection problem in a rotating spherical shell: Test on the benchmark solution[J]. Studia Geophysica et Geodaetica, 2004, 48(4):741-746.
[17] SADOURNY R. Conservative finite-difference approximations of the primitive equations on quasi-uniform spherical grids[J]. Monthly Weather Review, 1972, 100(2):136-144.
[18] RONCHI C, IACONO R, PAOLUCCI P S. The “cubed sphere”: A new method for the solution of partial differential equations in spherical geometry[J]. Journal of Computational Physics, 1996, 124(1):93-114.
[19] NAIR R D, THOMAS S J, LOFT R D. A discontinuous Galerkin transport scheme on the cubed sphere[J]. Monthly Weather Review, 2005, 133(4):814-828.
[20] ROSSMANITH J A. A wave propagation method for hyperbolic systems on the sphere[J]. Journal of Computational Physics, 2006, 213(2):629-658.
[21] PUTMAN W M, LIN S J. Finite-volume transport on various cubed-sphere grids[J]. Journal of Computational Physics, 2007, 227(1):55-78.
[22] CHEN C G, XIAO F. Shallow water model on cubed-sphere by multi-moment finite volume method[J]. Journal of Computational Physics, 2008, 227(10):5019-5044.
[23] ULLRICH P A, JABLONOWSKI C, VAN LEER B. High-order finite-volume methods for the shallow-water equations on the sphere[J]. Journal of Computational Physics, 2010, 229(17):6104-6134.
[24] YANG C, CAO J W, CAI X C. A fully implicit domain decomposition algorithm for shallow water equations on the cubed-sphere[J]. SIAM Journal on Scientific Computing, 2010, 32(1):418-438.
[25] YANG C, CAI X C. A parallel well-balanced finite volume method for shallow water equations with topography on the cubed-sphere[J]. Journal of Computational and Applied Mathematics, 2011, 235(18):5357-5366.
[26] YANG C, CAI X C. Parallel multilevel methods for implicit solution of shallow water equations with nonsmooth topography on the cubed-sphere[J]. Journal of Computational Physics, 2011, 230(7):2523-2539.
[27] CHOBLET G. Modelling thermal convection with large viscosity gradients in one block of the ‘cubed sphere’[J]. Journal of Computational Physics, 2005, 205(1):269-291.
[28] IVAN L, DE STERCK H, NORTHRUP S A, et al. Multi-dimensional finite-volume scheme for hyperbolic conservation laws on three-dimensional solution-adaptive cubed-sphere grids[J]. Journal of Computational Physics, 2013, 255:205-227.
[29] IVAN L, DE STERCK H, SUSANTO A, et al. High-order central ENO finite-volume scheme for hyperbolic conservation laws on three-dimensional cubed-sphere grids[J]. Journal of Computational Physics, 2015, 282:157-182.
[30] LIU J, HU Q F, HAN G X, et al. Numerical parallel computing of unbalanced stiff equations in distributed memory environments[J]. Chinese J Comput Phys, 2002, 19(1):86-88.
[31] XU X W, MO Z Y. Scalability analysis for parallel algebraic multigrid algorithms[J]. Chinese J Comput Phys, 2007, 24(4):387-394.
[32] CAO J W, XU X, WANG Y N. Parallel algorithms for separable elliptic equation based on GPU[J]. Chinese J Comput Phys, 2015, 32(4):475-481.
[33] 谷同祥, 等. 迭代方法和预处理技术[M]. 北京:科学出版社, 2015.
[34] DEMMEL J W. Applied numerical linear algebra[M]. SIAM, 1997.
[35] SAAD Y. Iterative methods for sparse linear systems[M]. 2nd ed. SIAM, 2003.
[36] DUKOWICZ J K, DVINSKY A S. Approximate factorization as a high order splitting for the implicit incompressible flow equations[J]. Journal of Computational Physics, 1992, 102(2):336-347.
[37] TOSELLI A, WIDLUND O B. Domain decomposition methods-algorithms and theory[M]. Springer, 2005.
[38] CAI X C, KEYES D E, VENKATAKRISHNAN V. Newton-Krylov-Schwarz: An implict solver for CFD[M]. Institute for Computer Applications in Science and Engineering (ICASE), 1995.
[39] CAI X C, SARKIS M. A restricted additive Schwarz preconditioner for general sparse linear systems[J]. SIAM Journal on Scientific Computing, 1999, 21(2):792-797.
[40] MANDEL J. Hybrid domain decomposition with unstructured subdomains[C]//Domain Decomposition Methods in Science and Engineering: The Sixth International Conference on Domain Decomposition, Contemporary Mathematics, 1994, 157:103-112.
[41] BALAY S, BROWN J, BUSCHELMAN K, et al. PETSc users manual revision 3.4[R]. Argonne National Lab, 2013.
[42] FU H H, LIAO J F, YANG J Z, et al. The Sunway TaihuLight supercomputer: System and applications[J]. Science China Information Sciences, 2016, 59:1-16.
[2] ROBERTS P H, KING E M. On the genesis of the Earth’s magnetism[J]. Reports on Progress in Physics, 2013, 76(9):096801.
[3] GLATZMAIER G A, ROBERTS P H. A three-dimensional self-consistent computer simulation of a geomagnetic field reversal[J]. Nature, 1995, 377(6546):203-209.
[4] KUANG W J, BLOXHAM J. An Earth-like numerical dynamo model[J]. Nature, 1997, 389(6649):371-374.
[5] CHRISTENSEN U, OLSON P, GLATZMAIER G A. Numerical modelling of the geodynamo: A systematic parameter study[J]. Geophysical Journal International, 1999, 138(2):393-409.
[6] VANTIEGHEM S, SHEYKO A, JACKSON A. Applications of a finite-volume algorithm for incompressible MHD problems[J]. Geophysical Journal International, 2016, 204(2):1376-1395.
[7] KAGEYAMA A, YOSHIDA M. Geodynamo and mantle convection simulations on the Earth Simulator using the Yin-Yang grid[J]. Journal of Physics: Conference Series, 2005, 16:325-338.
[8] HARDER H, HANSEN U. A finite-volume solution method for thermal convection and dynamo problems in spherical shells[J]. Geophysical Journal International, 2005, 161(2):522-532.
[9] MATSUI H, OKUDA H. Thermal convection analysis in a rotating shell by a parallel finite-element method-development of a thermal-hydraulic subsystem of GeoFEM[J]. Concurrency and Computation: Practice and Experience, 2002, 14(6/7):465-481.
[10] CHAN K H, LI L G, LIAO X H. Modelling the core convection using finite element and finite difference methods[J]. Physics of the Earth and Planetary Interiors, 2006, 157(1):124-138.
[11] YIN L, YANG C, MA S Z, et al. Parallel numerical simulation of the thermal convection in the Earth’s outer core on the cubed-sphere[J]. Geophysical Journal International, 2017, 209(3):1934-1954.
[12] GLATZMAIER G A. Geodynamo simulations-how realistic are they?[J]. Annual Review of Earth and Planetary Sciences, 2002, 30(1):237-257.
[13] MATSUI H, HEIEN E, AUBERT J, et al. Performance benchmarks for a next generation numerical dynamo model[J]. Geochemistry, Geophysics, Geosystems, 2016, 17(5):1586-1607.
[14] AURNOU J M, CALKINS M A, CHENG J S, et al. Rotating convective turbulence in Earth and planetary cores[J]. Physics of the Earth and Planetary Interiors, 2015, 246:52-71.
[15] CHRISTENSEN U R, AUBERT J, CARDIN P, et al. A numerical dynamo benchmark[J]. Physics of the Earth and Planetary Interiors, 2001, 128(1):25-34.
[16] HEJDA P, RESHETNYAK M. Control volume method for the thermal convection problem in a rotating spherical shell: Test on the benchmark solution[J]. Studia Geophysica et Geodaetica, 2004, 48(4):741-746.
[17] SADOURNY R. Conservative finite-difference approximations of the primitive equations on quasi-uniform spherical grids[J]. Monthly Weather Review, 1972, 100(2):136-144.
[18] RONCHI C, IACONO R, PAOLUCCI P S. The “cubed sphere”: A new method for the solution of partial differential equations in spherical geometry[J]. Journal of Computational Physics, 1996, 124(1):93-114.
[19] NAIR R D, THOMAS S J, LOFT R D. A discontinuous Galerkin transport scheme on the cubed sphere[J]. Monthly Weather Review, 2005, 133(4):814-828.
[20] ROSSMANITH J A. A wave propagation method for hyperbolic systems on the sphere[J]. Journal of Computational Physics, 2006, 213(2):629-658.
[21] PUTMAN W M, LIN S J. Finite-volume transport on various cubed-sphere grids[J]. Journal of Computational Physics, 2007, 227(1):55-78.
[22] CHEN C G, XIAO F. Shallow water model on cubed-sphere by multi-moment finite volume method[J]. Journal of Computational Physics, 2008, 227(10):5019-5044.
[23] ULLRICH P A, JABLONOWSKI C, VAN LEER B. High-order finite-volume methods for the shallow-water equations on the sphere[J]. Journal of Computational Physics, 2010, 229(17):6104-6134.
[24] YANG C, CAO J W, CAI X C. A fully implicit domain decomposition algorithm for shallow water equations on the cubed-sphere[J]. SIAM Journal on Scientific Computing, 2010, 32(1):418-438.
[25] YANG C, CAI X C. A parallel well-balanced finite volume method for shallow water equations with topography on the cubed-sphere[J]. Journal of Computational and Applied Mathematics, 2011, 235(18):5357-5366.
[26] YANG C, CAI X C. Parallel multilevel methods for implicit solution of shallow water equations with nonsmooth topography on the cubed-sphere[J]. Journal of Computational Physics, 2011, 230(7):2523-2539.
[27] CHOBLET G. Modelling thermal convection with large viscosity gradients in one block of the ‘cubed sphere’[J]. Journal of Computational Physics, 2005, 205(1):269-291.
[28] IVAN L, DE STERCK H, NORTHRUP S A, et al. Multi-dimensional finite-volume scheme for hyperbolic conservation laws on three-dimensional solution-adaptive cubed-sphere grids[J]. Journal of Computational Physics, 2013, 255:205-227.
[29] IVAN L, DE STERCK H, SUSANTO A, et al. High-order central ENO finite-volume scheme for hyperbolic conservation laws on three-dimensional cubed-sphere grids[J]. Journal of Computational Physics, 2015, 282:157-182.
[30] LIU J, HU Q F, HAN G X, et al. Numerical parallel computing of unbalanced stiff equations in distributed memory environments[J]. Chinese J Comput Phys, 2002, 19(1):86-88.
[31] XU X W, MO Z Y. Scalability analysis for parallel algebraic multigrid algorithms[J]. Chinese J Comput Phys, 2007, 24(4):387-394.
[32] CAO J W, XU X, WANG Y N. Parallel algorithms for separable elliptic equation based on GPU[J]. Chinese J Comput Phys, 2015, 32(4):475-481.
[33] 谷同祥, 等. 迭代方法和预处理技术[M]. 北京:科学出版社, 2015.
[34] DEMMEL J W. Applied numerical linear algebra[M]. SIAM, 1997.
[35] SAAD Y. Iterative methods for sparse linear systems[M]. 2nd ed. SIAM, 2003.
[36] DUKOWICZ J K, DVINSKY A S. Approximate factorization as a high order splitting for the implicit incompressible flow equations[J]. Journal of Computational Physics, 1992, 102(2):336-347.
[37] TOSELLI A, WIDLUND O B. Domain decomposition methods-algorithms and theory[M]. Springer, 2005.
[38] CAI X C, KEYES D E, VENKATAKRISHNAN V. Newton-Krylov-Schwarz: An implict solver for CFD[M]. Institute for Computer Applications in Science and Engineering (ICASE), 1995.
[39] CAI X C, SARKIS M. A restricted additive Schwarz preconditioner for general sparse linear systems[J]. SIAM Journal on Scientific Computing, 1999, 21(2):792-797.
[40] MANDEL J. Hybrid domain decomposition with unstructured subdomains[C]//Domain Decomposition Methods in Science and Engineering: The Sixth International Conference on Domain Decomposition, Contemporary Mathematics, 1994, 157:103-112.
[41] BALAY S, BROWN J, BUSCHELMAN K, et al. PETSc users manual revision 3.4[R]. Argonne National Lab, 2013.
[42] FU H H, LIAO J F, YANG J Z, et al. The Sunway TaihuLight supercomputer: System and applications[J]. Science China Information Sciences, 2016, 59:1-16.