基于深度神经网络的二维流体模拟
|
摘要
提出了一种将深度神经网络与流体模拟相结合的新方法。将具有更多湍流细节的高精度流体模拟结果看作图像中的"风格",利用训练好的深度神经网络的中间层提取特征信息。采用图像风格化技术,将高精度流体模拟结果的湍流信息迁移到低精度流体模拟结果中,使得低精度流体模拟结果同样具有丰富的湍流细节,实现了超分辨率的效果。实时完成低精度流体模拟和湍流迁移,实现了实时的高精度流体模拟。利用流体模拟中的速度信息保证流体模拟在时域上的连续性,使得整个模拟的结果更为真实。采用可以适用于任意风格输入的自适应的实例归一化(adaptive instance normalization,AdaIN)风格化技术,实现了流体模拟的艺术风格控制。
We propose a novel method by combining deep neural network with fluid simulation.High-resolution results with more turbulence details are considered as‘style'and pre-trained networks such as VGG are used to extract features.Our method is capable of transferring turbulence details from high-resolution results to low-resolution ones,achieving real-time performance.The consistency of the whole simulation is guaranteed by the velocity field.Our method provides more artistic control by using the adaptive instance normalization(AdaIN)framework,which supports arbitrary style.
We propose a novel method by combining deep neural network with fluid simulation.High-resolution results with more turbulence details are considered as‘style'and pre-trained networks such as VGG are used to extract features.Our method is capable of transferring turbulence details from high-resolution results to low-resolution ones,achieving real-time performance.The consistency of the whole simulation is guaranteed by the velocity field.Our method provides more artistic control by using the adaptive instance normalization(AdaIN)framework,which supports arbitrary style.
引文
[1]GATYS L A,ECKER A S,BETHGE,M.Image style transfer using convolutional neural networks[C]∥Conference on Computer Vision and Pattern Recognition.New York:IEEE,2016:2414-2423.
[2]JOHNSON J,ALAHI A,FEI-FEI LI.Perceptual losses for real-time style transfer and super-resolution[C]∥BASTIIAN LEIBE.European Conference on Computer Vision.Amsterdam,Netherlands:Springer,2016:694-711.
[3]ULYANOV D,LEBEDEV V,VEDALDI A,et al.Texture networks:feed-forward synthesis of textures and stylized images[J].International Conference on Machine Learning,2016,48(30):1349-1357.
[4]LI C,WAND M.Precomputed real-time texture synthesis with markovian generative adversarial networks[C]∥BASTIIAN LEIBE.European Conference on Computer Vision.Amsterdam,Netherlands:Springer,2016:702-716.
[5]DUMOULIN U,SHLENS J,KUDLUR M.A learned representation for artistic style[J/OL].[2017-10-09].https:∥arxiv.org/abs/1610.07629.
[6]CHEN D,YUAN L,LIAO J,et al.Stylebank:an explicit representation for neural image style transfer[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.New York:IEEE,2017:1897-1906.
[7]LI Y,FANG C,YANG J,et al.Diversified texture synthesis with feed-forward networks[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.New York:IEEE,2017:3920-3928.
[8]ZHANG H,DANA K.Multi-style generative network for real-time transfer[EB/OL].[2017-10-09].https:∥arxiv.org/abs/1703.06953.
[9]CHEN T Q,SCHMIDT M.Fast patch-based style transfer of arbitrary style[EB/OL].[2016-12-23].https:∥arxiv.org/abs/1612.04337.
[10]HUANG X,BELONGIE S.Arbitrary style transfer in real-time with adaptive instance normalization[C]∥Proceedings of the IEEE International Conference on Computer Vision.New York:IEEE,2017:1501-1510.
[11]GOODFELLOW L,POUGET-ABADIE J,MIRZA JM,et al.Generative adversarial nets[C]∥Advances in Neural Information Processing Systems.[S.l.:s.n.],2014:2672-2680.
[12]ZHU J Y,PARK T,ISOLA P,et al.Unpaired imageto-image translation using cycle-consistent adversarial networks[C]∥Proceedings of the IEEE International Conference On Computer Vision.New York:IEEE,2017:2223-2232.
[13]FEDKIW R,STAM J,JENSEN H W.Visual simulation of smoke[C]∥Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques.[S.l.]:ACM,2001:15-22.
[14]BRIDSON R,HOURIHAN J,NORDENSTAM M.Curl-noise for procedural fluid flow[J].ACM Transactions on Graphics(TOG),2007,26(3):46.
[15]KIM T,THUEREY N,JAMES D,et al.Wavelet turbulence for fluid simulation[J].ACM Transactions on Graphics(TOG),2008,27(3):50.
[16]ZHU B,LU W,CONG M,et al.A new grid structure for domain extension[J].Transaction on Graphics,2013,32(4):63.
[17]KYPRIANIDIS J E,COLLOMOSSE J,WANG T,et al.State of the art:a taxonomy of artistic stylization techniques for images and video[J].IEEE Transactions on Visualization and Computer Graphics,2012,19(5):866-885.
[18]EFROS A A,FREEMAN W T.Image quilting for texture synthesis and transfer[C]∥Proceedings of the28th Annual Conference on Computer Graphics and Interactive Techniques.[S.l.]:ACM,2001:341-346.
[19]EFROS A A,LEUNG T K.Texture synthesis by nonparametric sampling[C]∥Proceedings of the seventh IEEE International Conference on Computer Vision.IEEE,1999,2:1033-1038.
[20]ELAD M,MILANFAR P.Style-transfer via texturesynthesis[EB/OL].[2016-11-10].https:∥arxiv.org/abs/1609.03057.
[21]CHEN L,YE J,JIANG L,et al.Synthesizing cloth wrinkles by CNN-based geometry image superresolution[J].Computer Animation and Virtual World,2018,29(3/4):e1810.
[22]HEEGER D J,BERGEN J R.Pyramid-based texture analysis/synthesis[C]∥Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques.[S.l.]:ACM,1995:229-238.
[23]FRIGO O,SABATER A,DELON J,et al.Split and match:example-based adaptive patch sampling for unsupervised style transfer[C]∥Proceedings of the IEEEConference on Computer Vision and Pattern Recognition.New York:IEEE,2016:553-561.
[24]GATYS L,ECKER A,BETHGE M,et al.Controlling perceptual factors in neural style transfer[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.New York:IEEE,2017:3985-3993.
[25]LI C,WAND M.Combining markov random fields and convolutional neural networks for image synthesis[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.New York:IEEE,2016:2479-2486.
[26]ULYANOV D,VEDALDI A,LEMPITSKV M.Improved texture networks:Maximizing quality and diversity in feed-forward stylization and texture synthesis[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.New York:IEEE,2017:6924-6932.
[27]LOSASSO F,GIBOU F,FEDKIW R.Simulating water and smoke with an octree data structure[J].ACMTransactions on Graphics(TOG),2004,23(3):457-462.
[28]KLINGNER B,FELDMAN B,CHENTANEZ N,et al.Fluid animation with dynamic meshes[J].ACMTransactions on Graphics(TOG),2006,25(3):820-825.
[29]CHENTANEZ N,MUELLER M.Real-time eulerian water simulation using a restricted tall cell grid[J].ACM Transactions on Graphics(TOG),2011,30(4):82.
[30]WU K,TRUONG N,YUKSEL C,et al.Fast fluid simulations with sparse volumes on the GPU[J].Computer Graphics Forum,2018,37(2):157-167.
[31]McADAMS A,SIFAKIS E,TERAN J.A parallel multigrid Poisson solver for fluids simulation on large grids[C]∥Proceedings of the 2010 ACM SIG-GRAPH/Eurographics Symposium on Computer Animation.[S.l.]:Eurographics Association,2010:65-74.
[32]KIM B,LIU Y,LLAMAS I,et al.Flow-fixer:using BFECC for fluid simulation[C]∥Conference:Proceedings of the Eurographics Workshop on Natural Phenomena.Dublin,Ireland:Eurographics Association,2005:51-56.
[33]SELLE A,FEDKIW R,KIM B,et al.An unconditionally stable MacCormack method[J].Journal of Scientific Computing,2008,35(2/3):350-371.
[34]SELLE A RASMUSSEN M,FEDKIW R.A vortex particle method for smoke,water and explosions[J].ACM Transactions on Graphics(TOG),2005,24(3):910-914.
[35]PFAFF T,THUEREY N,COHEN J,et al.Scalable fluid simulation using anisotropic turbulence particles[J].ACM Transactions on Graphics(TOG),2010,29(6):174.
[36]TOMPSON M,SCHLACHTER K,SPRECHMANNP,et al.Accelerating eulerian fluid simulation with convolutional networks[EB/OL].[2017-04-05].https:∥arxiv.org/abs/1607.03597.
[37]CHU M,THUEREY N.Data-driven synthesis of smoke flows with cnn-based feature descriptors[J].Transaction on Graphics,2017,36(4):69.
[38]XIE Y,FRANZ E,CHU M,et al.TempoGAN:a temporally coherent,volumetric GAN for super-resolution fluid flow[J].Transaction on Graphics,2018,37(4):95.
[39]IOFFE S,SZEGEDY C.Batch normalization:accelerating deep network training by reducing internal covariate shift[J].Arxiv Preprint Arxiv,2015:1502.03167.
[40]IOFFE S.Batch renormalization:towards reducing minibatch dependence in batch-normalized models[EB/OL].[2017-06-29].https:∥arxiv.org/abs/1702.03275.
[41]THUEREY N,PFAFF T.Mantaflow[M/OL].[2017-10-14].http:∥mantaflow.com.
[42]BRIDSON R,MUELLER M.Fluid simulation[C]∥SIGGRAPH 2007Course Notes.San Diego,California.USA:ACM,2007:1-81.
[43]HARLOW F,WELCH J.Numerical calculation of time-dependent viscous incompressible flow of fluid with free surface[J].Physics of Fluids,1965,8(12):2182-2189.
[44]SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[J].Arxiv Preprint Arxiv,2014:1409.1556.
[45]KRIZHEVSKY A,SUTSKEVER I,HINTON G E.ImageNet classification with deep convolutional neural networks[C]∥Advances in Neural Information Processing Systems.[S.l.:s.n.],2012:1097-1105.
[2]JOHNSON J,ALAHI A,FEI-FEI LI.Perceptual losses for real-time style transfer and super-resolution[C]∥BASTIIAN LEIBE.European Conference on Computer Vision.Amsterdam,Netherlands:Springer,2016:694-711.
[3]ULYANOV D,LEBEDEV V,VEDALDI A,et al.Texture networks:feed-forward synthesis of textures and stylized images[J].International Conference on Machine Learning,2016,48(30):1349-1357.
[4]LI C,WAND M.Precomputed real-time texture synthesis with markovian generative adversarial networks[C]∥BASTIIAN LEIBE.European Conference on Computer Vision.Amsterdam,Netherlands:Springer,2016:702-716.
[5]DUMOULIN U,SHLENS J,KUDLUR M.A learned representation for artistic style[J/OL].[2017-10-09].https:∥arxiv.org/abs/1610.07629.
[6]CHEN D,YUAN L,LIAO J,et al.Stylebank:an explicit representation for neural image style transfer[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.New York:IEEE,2017:1897-1906.
[7]LI Y,FANG C,YANG J,et al.Diversified texture synthesis with feed-forward networks[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.New York:IEEE,2017:3920-3928.
[8]ZHANG H,DANA K.Multi-style generative network for real-time transfer[EB/OL].[2017-10-09].https:∥arxiv.org/abs/1703.06953.
[9]CHEN T Q,SCHMIDT M.Fast patch-based style transfer of arbitrary style[EB/OL].[2016-12-23].https:∥arxiv.org/abs/1612.04337.
[10]HUANG X,BELONGIE S.Arbitrary style transfer in real-time with adaptive instance normalization[C]∥Proceedings of the IEEE International Conference on Computer Vision.New York:IEEE,2017:1501-1510.
[11]GOODFELLOW L,POUGET-ABADIE J,MIRZA JM,et al.Generative adversarial nets[C]∥Advances in Neural Information Processing Systems.[S.l.:s.n.],2014:2672-2680.
[12]ZHU J Y,PARK T,ISOLA P,et al.Unpaired imageto-image translation using cycle-consistent adversarial networks[C]∥Proceedings of the IEEE International Conference On Computer Vision.New York:IEEE,2017:2223-2232.
[13]FEDKIW R,STAM J,JENSEN H W.Visual simulation of smoke[C]∥Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques.[S.l.]:ACM,2001:15-22.
[14]BRIDSON R,HOURIHAN J,NORDENSTAM M.Curl-noise for procedural fluid flow[J].ACM Transactions on Graphics(TOG),2007,26(3):46.
[15]KIM T,THUEREY N,JAMES D,et al.Wavelet turbulence for fluid simulation[J].ACM Transactions on Graphics(TOG),2008,27(3):50.
[16]ZHU B,LU W,CONG M,et al.A new grid structure for domain extension[J].Transaction on Graphics,2013,32(4):63.
[17]KYPRIANIDIS J E,COLLOMOSSE J,WANG T,et al.State of the art:a taxonomy of artistic stylization techniques for images and video[J].IEEE Transactions on Visualization and Computer Graphics,2012,19(5):866-885.
[18]EFROS A A,FREEMAN W T.Image quilting for texture synthesis and transfer[C]∥Proceedings of the28th Annual Conference on Computer Graphics and Interactive Techniques.[S.l.]:ACM,2001:341-346.
[19]EFROS A A,LEUNG T K.Texture synthesis by nonparametric sampling[C]∥Proceedings of the seventh IEEE International Conference on Computer Vision.IEEE,1999,2:1033-1038.
[20]ELAD M,MILANFAR P.Style-transfer via texturesynthesis[EB/OL].[2016-11-10].https:∥arxiv.org/abs/1609.03057.
[21]CHEN L,YE J,JIANG L,et al.Synthesizing cloth wrinkles by CNN-based geometry image superresolution[J].Computer Animation and Virtual World,2018,29(3/4):e1810.
[22]HEEGER D J,BERGEN J R.Pyramid-based texture analysis/synthesis[C]∥Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques.[S.l.]:ACM,1995:229-238.
[23]FRIGO O,SABATER A,DELON J,et al.Split and match:example-based adaptive patch sampling for unsupervised style transfer[C]∥Proceedings of the IEEEConference on Computer Vision and Pattern Recognition.New York:IEEE,2016:553-561.
[24]GATYS L,ECKER A,BETHGE M,et al.Controlling perceptual factors in neural style transfer[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.New York:IEEE,2017:3985-3993.
[25]LI C,WAND M.Combining markov random fields and convolutional neural networks for image synthesis[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.New York:IEEE,2016:2479-2486.
[26]ULYANOV D,VEDALDI A,LEMPITSKV M.Improved texture networks:Maximizing quality and diversity in feed-forward stylization and texture synthesis[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.New York:IEEE,2017:6924-6932.
[27]LOSASSO F,GIBOU F,FEDKIW R.Simulating water and smoke with an octree data structure[J].ACMTransactions on Graphics(TOG),2004,23(3):457-462.
[28]KLINGNER B,FELDMAN B,CHENTANEZ N,et al.Fluid animation with dynamic meshes[J].ACMTransactions on Graphics(TOG),2006,25(3):820-825.
[29]CHENTANEZ N,MUELLER M.Real-time eulerian water simulation using a restricted tall cell grid[J].ACM Transactions on Graphics(TOG),2011,30(4):82.
[30]WU K,TRUONG N,YUKSEL C,et al.Fast fluid simulations with sparse volumes on the GPU[J].Computer Graphics Forum,2018,37(2):157-167.
[31]McADAMS A,SIFAKIS E,TERAN J.A parallel multigrid Poisson solver for fluids simulation on large grids[C]∥Proceedings of the 2010 ACM SIG-GRAPH/Eurographics Symposium on Computer Animation.[S.l.]:Eurographics Association,2010:65-74.
[32]KIM B,LIU Y,LLAMAS I,et al.Flow-fixer:using BFECC for fluid simulation[C]∥Conference:Proceedings of the Eurographics Workshop on Natural Phenomena.Dublin,Ireland:Eurographics Association,2005:51-56.
[33]SELLE A,FEDKIW R,KIM B,et al.An unconditionally stable MacCormack method[J].Journal of Scientific Computing,2008,35(2/3):350-371.
[34]SELLE A RASMUSSEN M,FEDKIW R.A vortex particle method for smoke,water and explosions[J].ACM Transactions on Graphics(TOG),2005,24(3):910-914.
[35]PFAFF T,THUEREY N,COHEN J,et al.Scalable fluid simulation using anisotropic turbulence particles[J].ACM Transactions on Graphics(TOG),2010,29(6):174.
[36]TOMPSON M,SCHLACHTER K,SPRECHMANNP,et al.Accelerating eulerian fluid simulation with convolutional networks[EB/OL].[2017-04-05].https:∥arxiv.org/abs/1607.03597.
[37]CHU M,THUEREY N.Data-driven synthesis of smoke flows with cnn-based feature descriptors[J].Transaction on Graphics,2017,36(4):69.
[38]XIE Y,FRANZ E,CHU M,et al.TempoGAN:a temporally coherent,volumetric GAN for super-resolution fluid flow[J].Transaction on Graphics,2018,37(4):95.
[39]IOFFE S,SZEGEDY C.Batch normalization:accelerating deep network training by reducing internal covariate shift[J].Arxiv Preprint Arxiv,2015:1502.03167.
[40]IOFFE S.Batch renormalization:towards reducing minibatch dependence in batch-normalized models[EB/OL].[2017-06-29].https:∥arxiv.org/abs/1702.03275.
[41]THUEREY N,PFAFF T.Mantaflow[M/OL].[2017-10-14].http:∥mantaflow.com.
[42]BRIDSON R,MUELLER M.Fluid simulation[C]∥SIGGRAPH 2007Course Notes.San Diego,California.USA:ACM,2007:1-81.
[43]HARLOW F,WELCH J.Numerical calculation of time-dependent viscous incompressible flow of fluid with free surface[J].Physics of Fluids,1965,8(12):2182-2189.
[44]SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[J].Arxiv Preprint Arxiv,2014:1409.1556.
[45]KRIZHEVSKY A,SUTSKEVER I,HINTON G E.ImageNet classification with deep convolutional neural networks[C]∥Advances in Neural Information Processing Systems.[S.l.:s.n.],2012:1097-1105.
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |