Instantaneous foveated preview for progressive Monte Carlo rendering
|
摘要
Progressive rendering,for example Monte Carlo rendering of 360° content for virtual reality headsets,is a time-consuming task.If the 3D artist notices an error while previewing the rendering,they must return to editing mode,make the required changes,and restart rendering.We propose the use of eye-tracking-based optimization to significantly speed up previewing of the artist’s points of interest.The speed of the preview is further improved by sampling with a distribution that closely follows the experimentally measured visual acuity of the human eye,unlike the piecewise linear models used in previous work.In a comprehensive user study,the perceived convergence of our proposed method was 10 times faster than that of a conventional preview,and often appeared to be instantaneous.In addition,the participants rated the method to have only marginally more artifacts in areas where it had to start rendering from scratch,compared to conventional rendering methods that had already generated image content in those areas.
Progressive rendering,for example Monte Carlo rendering of 360° content for virtual reality headsets,is a time-consuming task.If the 3D artist notices an error while previewing the rendering,they must return to editing mode,make the required changes,and restart rendering.We propose the use of eye-tracking-based optimization to significantly speed up previewing of the artist’s points of interest.The speed of the preview is further improved by sampling with a distribution that closely follows the experimentally measured visual acuity of the human eye,unlike the piecewise linear models used in previous work.In a comprehensive user study,the perceived convergence of our proposed method was 10 times faster than that of a conventional preview,and often appeared to be instantaneous.In addition,the participants rated the method to have only marginally more artifacts in areas where it had to start rendering from scratch,compared to conventional rendering methods that had already generated image content in those areas.
Progressive rendering,for example Monte Carlo rendering of 360° content for virtual reality headsets,is a time-consuming task.If the 3D artist notices an error while previewing the rendering,they must return to editing mode,make the required changes,and restart rendering.We propose the use of eye-tracking-based optimization to significantly speed up previewing of the artist’s points of interest.The speed of the preview is further improved by sampling with a distribution that closely follows the experimentally measured visual acuity of the human eye,unlike the piecewise linear models used in previous work.In a comprehensive user study,the perceived convergence of our proposed method was 10 times faster than that of a conventional preview,and often appeared to be instantaneous.In addition,the participants rated the method to have only marginally more artifacts in areas where it had to start rendering from scratch,compared to conventional rendering methods that had already generated image content in those areas.
引文
[1]Pharr,M.;Jakob,W.;Humphreys,G.Physical ly Based Rendering:From Theory to Implementation,2nd edn.Morgan Kaufmann,2010.
[2]Strasburger,H.;Rentschler,I.;Ju¨ttner,M.Peripheral vision and pattern recognition:A review.Journal of Vision Vol.11,No.5,13,2011.
[3]Koskela,M.;Viitanen,T.;J¨a¨askel¨ainen,P.;Takala,J.Foveated path tracing.In:Advances in Visual Computing.Lecture Notes in Computer Science,Vol.10072.Bebis,G.;Boyle,R.;Parvin,B.et al.Eds.Springer Cham,723-732,2016.
[4]Koskela,M.;Immonen,K.;Viitanen,T.;Ja¨¨askel¨ainen,P.;Multanen,J.;Takala,J.Foveated instant preview for progressive rendering.In:Proceedings of the SIGGRAPH Asia 2017 Technical Briefs,Article No.10,2017.
[5]Weier,M.;Stengel,M.;Roth,T.;Didyk,P.;Eisemann,E.;Eisemann,M.;Grogorick,S.;Hinkenjann,A.;Kruijff,E.;Magnor,M.;Myszkowski,K.;Slusallek,P.Perception-driven accelerated rendering.Computer Graphics Forum Vol.36,No.2,611-643,2017.
[6]Shibata,T.Head mounted display.Displays Vol.23,Nos.1-2,57-64,2002.
[7]Lee,E.C.;Park,K.R.A robust eye gaze tracking method based on a virtual eyeball model.Machine Vision and Applications Vol.20,No.5,319-337,2009.
[8]Guenter,B.;Finch,M.;Drucker,S.;Tan,D.;Snyder,J.Foveated 3D graphics.ACM Transactions on Graphics Vol.31,No.6,Article No.164,2012.
[9]Vaidyanathan,K.;Salvi,M.;Toth,R.;Foley,T.;Akenine-M¨oller,T.;Nilsson,J.;Munkberg,J.;Hasselgren,J.;Sugihara,M.;Clarberg,P.;Janczak,T.;Lefohn,A.Coarse pixel shading.In:Proceedings of High Performance Graphics,9-18,2014.
[10]Stengel,M.;Grogorick,S.;Eisemann,M.;Magnor,M.Adaptive image-space sampling for gaze-contingent realtime rendering.Computer Graphics Forum Vol.35,No.4,129-139,2016.
[11]Weier,M.;Roth,T.;Kruijff,E.;Hinkenjann,A.;P′erardGayot,A.;Slusallek,P.;Li,Y.Foveated real-time ray tracing for head-mounted displays.Computer Graphics Forum Vol.35,No.7,289-298,2016.
[12]Murphy,H.A.;Duchowski,A.T.;Tyrrell,R.A.Hybrid image/model-based gaze-contingent rendering.ACMTransactions on Applied Perception Vol.5,No.4,Article No.22,2009.
[13]Reddy,M.Perceptually optimized 3D graphics.IEEEComputer Graphics and Applications Vol.21,No.5,68-75,2001.
[14]Pohl,D.;Zhang,X.;Bulling,A.Combining eye tracking with optimizations for lens astigmatism in modern wideangle HMDs.In:Proceedings of the IEEE Virtual Reality,269-270,2016.
[15]Roth,T.;Weier,M.;Maiero,J.;Hinkenjann,A.;Li,Y.Guided high-quality rendering.In:Advances in Visual Computing.Lecture Notes in Computer Science,Vol.9475.Bebis,G.;Boyle,R.;Parvin,B.et al.Eds.Springer Cham,115-125,2015.
[16]Pixar.Renderman 20 documentation:Rendering efficiently.2017.Available at https://renderman.pixar.com/resources/Render Man_20/tutorial Rendering Efficiently.html.
[17]The community of Lux Render.Lux Render documentation:Refine brush.2013.Available at http://www.luxrender.net/wiki/Refine_Brush.
[18]Duchowski,A.T.;Bate,D.;Stringfellow,P.;Thakur,K.;Melloy,B.J.;Gramopadhye,A.K.On spatiochromatic visual sensitivity and peripheral color LOD management.ACM Transactions on Applied Perception Vol.6,No.2,Article No.9,2009.
[19]Viitanen,T.;Koskela,M.;Immonen,K.;M¨akitalo,M.;J¨a¨askel¨ainen,P.;Takala,J.Sparse sampling for real-time ray tracing.In:Proceedings of the13th International Joint Conference on Computer Vision,Imaging and Computer Graphics Theory and Applications,Vol.1,295-302,2018.
[20]Devroye,L.Non-Uniform Random Variate Generation.Springer-Verlag,1986.
[21]Weisstein,E.Lambert W-function.Available at http://mathworld.wolfram.com/Lambert W-Function.html.
[22]Mac Quarrie,A.;Steed,A.Cinematic virtual reality:Evaluating the effect of display type on the viewing experience for panoramic video.In:Proceedings of the IEEE Virtual Reality,45-54,2017.
[23]Stark,R.;Israel,J.H.;W¨ohler,T.Towards hybrid modelling environments-Merging desktop-CAD and virtual reality-technologies.CIRP Annals Vol.59,No.1,179-182,2010.
[24]AMD.Radeon Rays SDK.Available at https://github.com/GPUOpen-Libraries And SDKs/Radeon Rays_SDK.
[25]Rodrigo.What is the latency of FOVE eye tracking?Available at https://support.getfove.com/hc/enus/articles/115000733714-What-is-the-Latency-ofFOVE-Eye-Tracking-.
[26]FOVE Inc.Tech specs.Available at https://www.getfov.com/.
[2]Strasburger,H.;Rentschler,I.;Ju¨ttner,M.Peripheral vision and pattern recognition:A review.Journal of Vision Vol.11,No.5,13,2011.
[3]Koskela,M.;Viitanen,T.;J¨a¨askel¨ainen,P.;Takala,J.Foveated path tracing.In:Advances in Visual Computing.Lecture Notes in Computer Science,Vol.10072.Bebis,G.;Boyle,R.;Parvin,B.et al.Eds.Springer Cham,723-732,2016.
[4]Koskela,M.;Immonen,K.;Viitanen,T.;Ja¨¨askel¨ainen,P.;Multanen,J.;Takala,J.Foveated instant preview for progressive rendering.In:Proceedings of the SIGGRAPH Asia 2017 Technical Briefs,Article No.10,2017.
[5]Weier,M.;Stengel,M.;Roth,T.;Didyk,P.;Eisemann,E.;Eisemann,M.;Grogorick,S.;Hinkenjann,A.;Kruijff,E.;Magnor,M.;Myszkowski,K.;Slusallek,P.Perception-driven accelerated rendering.Computer Graphics Forum Vol.36,No.2,611-643,2017.
[6]Shibata,T.Head mounted display.Displays Vol.23,Nos.1-2,57-64,2002.
[7]Lee,E.C.;Park,K.R.A robust eye gaze tracking method based on a virtual eyeball model.Machine Vision and Applications Vol.20,No.5,319-337,2009.
[8]Guenter,B.;Finch,M.;Drucker,S.;Tan,D.;Snyder,J.Foveated 3D graphics.ACM Transactions on Graphics Vol.31,No.6,Article No.164,2012.
[9]Vaidyanathan,K.;Salvi,M.;Toth,R.;Foley,T.;Akenine-M¨oller,T.;Nilsson,J.;Munkberg,J.;Hasselgren,J.;Sugihara,M.;Clarberg,P.;Janczak,T.;Lefohn,A.Coarse pixel shading.In:Proceedings of High Performance Graphics,9-18,2014.
[10]Stengel,M.;Grogorick,S.;Eisemann,M.;Magnor,M.Adaptive image-space sampling for gaze-contingent realtime rendering.Computer Graphics Forum Vol.35,No.4,129-139,2016.
[11]Weier,M.;Roth,T.;Kruijff,E.;Hinkenjann,A.;P′erardGayot,A.;Slusallek,P.;Li,Y.Foveated real-time ray tracing for head-mounted displays.Computer Graphics Forum Vol.35,No.7,289-298,2016.
[12]Murphy,H.A.;Duchowski,A.T.;Tyrrell,R.A.Hybrid image/model-based gaze-contingent rendering.ACMTransactions on Applied Perception Vol.5,No.4,Article No.22,2009.
[13]Reddy,M.Perceptually optimized 3D graphics.IEEEComputer Graphics and Applications Vol.21,No.5,68-75,2001.
[14]Pohl,D.;Zhang,X.;Bulling,A.Combining eye tracking with optimizations for lens astigmatism in modern wideangle HMDs.In:Proceedings of the IEEE Virtual Reality,269-270,2016.
[15]Roth,T.;Weier,M.;Maiero,J.;Hinkenjann,A.;Li,Y.Guided high-quality rendering.In:Advances in Visual Computing.Lecture Notes in Computer Science,Vol.9475.Bebis,G.;Boyle,R.;Parvin,B.et al.Eds.Springer Cham,115-125,2015.
[16]Pixar.Renderman 20 documentation:Rendering efficiently.2017.Available at https://renderman.pixar.com/resources/Render Man_20/tutorial Rendering Efficiently.html.
[17]The community of Lux Render.Lux Render documentation:Refine brush.2013.Available at http://www.luxrender.net/wiki/Refine_Brush.
[18]Duchowski,A.T.;Bate,D.;Stringfellow,P.;Thakur,K.;Melloy,B.J.;Gramopadhye,A.K.On spatiochromatic visual sensitivity and peripheral color LOD management.ACM Transactions on Applied Perception Vol.6,No.2,Article No.9,2009.
[19]Viitanen,T.;Koskela,M.;Immonen,K.;M¨akitalo,M.;J¨a¨askel¨ainen,P.;Takala,J.Sparse sampling for real-time ray tracing.In:Proceedings of the13th International Joint Conference on Computer Vision,Imaging and Computer Graphics Theory and Applications,Vol.1,295-302,2018.
[20]Devroye,L.Non-Uniform Random Variate Generation.Springer-Verlag,1986.
[21]Weisstein,E.Lambert W-function.Available at http://mathworld.wolfram.com/Lambert W-Function.html.
[22]Mac Quarrie,A.;Steed,A.Cinematic virtual reality:Evaluating the effect of display type on the viewing experience for panoramic video.In:Proceedings of the IEEE Virtual Reality,45-54,2017.
[23]Stark,R.;Israel,J.H.;W¨ohler,T.Towards hybrid modelling environments-Merging desktop-CAD and virtual reality-technologies.CIRP Annals Vol.59,No.1,179-182,2010.
[24]AMD.Radeon Rays SDK.Available at https://github.com/GPUOpen-Libraries And SDKs/Radeon Rays_SDK.
[25]Rodrigo.What is the latency of FOVE eye tracking?Available at https://support.getfove.com/hc/enus/articles/115000733714-What-is-the-Latency-ofFOVE-Eye-Tracking-.
[26]FOVE Inc.Tech specs.Available at https://www.getfov.com/.