单帧图像下的环境光遮蔽估计
|
摘要
环境光遮蔽(ambient occlusion)被广泛用于近似计算低频全局光照、消除间接光照和阴影等计算机图形学和视觉应用中.已有算法直接通过场景的3维几何,或不同光照下的多幅图像计算每个点的环境光遮蔽,存在着对光照和输入图像数量要求高等问题.针对以上不足,提出了一种基于单张图像的环境光遮蔽估计算法.算法利用一个在大量仿真图像数据集上训练的卷积神经网络,直接从自然光照条件下场景的单张图像中恢复每个点的环境光遮蔽.提出并比较了3种不同的神经网络结构设计,实验分析验证了端到端的设计方案可以获得最佳的结果.和已有的环境光遮蔽算法方法比较,所提出的方法不仅计算速度快,而且在数值和视觉上具有更好的结果.
Ambient occlusion has been widely used in many graphics and vision tasks for approxi-mating low frequency global illumination, removing inter-reflection, and inferring scene geometry. Existing methods need either scene geometry or scene appearances under different lightings to compute the ambient occlusion of the scene, which limits the application of these methods in many real applications. In this paper, we present a new method for computing the ambient occlusion from a single image taken under unknown natural lighting. Our method needn't any scene geometry or illumination information. To this end, we utilize a convolutional neural network(CNN), trained on a large amount of synthetic data, to directly recovery the pre-pixel ambient occlusion. We propose three network structures for ambient occlusion estimation, a cascade one and a parallel one that are based on previous CNN solution for intrinsic images, as well as an end-to-end neural network. We analyze their performance and demonstrate that comparing with parallel and cascade designs, the end-to-end design could achieve the best performance. We also valid the efficiency and effectiveness of our method on both synthetic and real images. It is not only faster, but also more accurate than previous methods.
Ambient occlusion has been widely used in many graphics and vision tasks for approxi-mating low frequency global illumination, removing inter-reflection, and inferring scene geometry. Existing methods need either scene geometry or scene appearances under different lightings to compute the ambient occlusion of the scene, which limits the application of these methods in many real applications. In this paper, we present a new method for computing the ambient occlusion from a single image taken under unknown natural lighting. Our method needn't any scene geometry or illumination information. To this end, we utilize a convolutional neural network(CNN), trained on a large amount of synthetic data, to directly recovery the pre-pixel ambient occlusion. We propose three network structures for ambient occlusion estimation, a cascade one and a parallel one that are based on previous CNN solution for intrinsic images, as well as an end-to-end neural network. We analyze their performance and demonstrate that comparing with parallel and cascade designs, the end-to-end design could achieve the best performance. We also valid the efficiency and effectiveness of our method on both synthetic and real images. It is not only faster, but also more accurate than previous methods.
引文
[1]Bavoil L,Sainz M,Dimitrov R.Image-space horizon-based ambient occlusion[C]Proc of the 35th ACM SIGGRAPH.New York:ACM,2008[2017-11-28].https:dl.acm.org/citation.cfm/id=1401061
[2]Bavoil L,Sainz M.Multi-layer dual-resolution screen-space ambient occlusion[C]Proc of the 36th ACM SIGGRAPH.New York:ACM,2009[2017-11-28].https:dl.acm.org/citation.cfm/id=1598035
[3]McGuire M,Osman B,Bukowski M,et al.The alchemy screen-space ambient obscurance algorithm[C]Proc of the38th ACM SIGGRAPH Symp on High Performance Graphics.New York:ACM,2011:25-32
[4]Li Wenyao,Ren Zhong.Self-adaptive multilayer screen space ambient occlusion[J].Journal of Computer-Aided Design&Computer Graphics,2011,23(8):1294-1303(in Chinese)(李文耀,任重.自适应的多层屏幕空间环境光遮蔽[J].计算机辅助设计与图形学学报,2011,23(8):1294-1303)
[5]Langer M S,Zucker S W.Shape-from-shading on a cloudy day[J].Journal of the Optical Society of America:A,1994,11(2):467-478
[6]Prados E,Jindal N,Soatto S.A non-local approach to shape from ambient shading[C]Proc of Scale Space and Variational Methods in Computer Vision.Berlin:Springer,2011:696-708
[7]Wu Chenglei,Wilburn B,Matsushita Y,et al.High-quality shape from multi-view stereo and shading under general illumination[C]Proc of the 29th IEEE Conf on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE,2011:969-976
[8]Aldrian O,Smith W A P.Inverse rendering of faces on a cloudy day[C]Proc of the 12th European Conf on Computer Vision.Berlin:Springer,2012:201-214
[9]Beeler T,Bradley D,Zimmer H,et al.Improved reconstruction of deforming surfaces by cancelling ambient occlusion[C]Proc of the 12th European Conf on Computer Vision.Berlin:Springer,2012:30-43
[10]Laffont P Y,Bousseau A,Paris S,et al.Coherent intrinsic images from photo collections[J].ACM Transactions on Graphics,2012,31(6):202-213
[11]Hauagge D,Wehrwein S,Bala K,et al.Photometric ambient occlusion[C]Proc of the 31st IEEE Conf on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE,2013:2515-2522
[12]Yang Wei,Ji Yu,Lin Haiting,et al.Ambient occlusion via compressive visibility estimation[C]Proc of the 33rd IEEEConf on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE,2015:3882-3889
[13]Ma Yuwei,Shang Yafei,Wan Liang,et al.Photometric ambient occlusion from sparsely sampled illuminations[C]Proc of the 17th IEEE Int Conf on Multimedia&Expo Workshops(ICMEW).Piscataway,NJ:IEEE,2016:1-6
[14]Barrow H G,Tenenbaum J M.Recovering intrinsic scene characteristics from images[J].Computer Vision Systems,1978,2:3-26
[15]Land E H,McCann J J.Lightness and Retinex theory[J].Journal of the Optical Society of America,1971,61(1):1-11
[16]Tappen M F,Freeman W T,Adelson E H.Recovering intrinsic images from a single image[J].IEEE Transactions on Pattern Analysis&Machine Intelligence,2005,27(9):1459-1472
[17]Shen Jianbing,Yang Xiaoshan,Jia Yunde,et al.Intrinsic images using optimization[C]Proc of the 29th IEEE Conf on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE,2011:3481-3487
[18]Barron J T,Malik J.Shape,illumination,and reflectance from shading[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(8):1670-1687
[19]Ren Shaoqing,He Kaiming,Girshick R,et al.Faster R-CNN:Towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis&Machine Intelligence,2017,39(6):1137-1149
[20]Jonathon L,Shelhamer E,Darrell T.Fully convolutional networks for semantic segmentation[C]Proc of the 33rd IEEE Conf on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE,2015:3431-3440
[21]He Kaiming,Zhang Xiangyu,Ren Shaoqing,et al.Deep residual learning for image recognition[C]Proc of the 34th IEEE Conf on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE,2016:770-778
[22]Yu Kai,Jia Lei,Chen Yuqiang,et al.Deep learning:Yesterday,today,and tomorrow[J].Journal of Computer Research and Development,2013,50(9):1799-1804(in Chinese)(余凯,贾磊,陈雨强,等.深度学习的昨天、今天和明天[J].计算机研究与发展,2013,50(9):1799-1804)
[23]Narihira T,Maire M,Yu S X.Learning lightness from human judgement on relative reflectance[C]Proc of the33rd IEEE Conf on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE,2015:2965-2973
[24]Zhou Tinghui,Krahenbuhl P,Efros A A.Learning datadriven reflectance priors for intrinsic image decomposition[C]Proc of the 33rd IEEE Int Conf on Computer Vision.Piscataway,NJ:IEEE.2015:3469-3477
[25]Narihira T,Maire M,Yu S X.Direct intrinsics:Learning albedo-shading decomposition by convolutional regression[C]Proc of the 33rd IEEE Int Conf on Computer Vision.Piscataway,NJ:IEEE,2015:2992-3001
[26]Shi Jian,Dong Yue,Su Hao,et al.Learning nonLambertian object intrinsics across ShapeNet categories[C]Proc of the 35th IEEE Conf on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE,2017:5844-5853
[27]Chang A X,Funkhouser T,Guibas L,et al.ShapeNet:An information-rich 3D model repository[J/OL].2015[2017-01-20].https:arxiv.org/pdf/1512.03012.pdf
[28]Grosse R,Johnson M K,Adelson E H,et al.Ground truth dataset and baseline evaluations for intrinsic image algorithms[C]Proc of the 27th IEEE Int Conf on Computer Vision.Piscataway,NJ:IEEE,2009:2335-2342
[2]Bavoil L,Sainz M.Multi-layer dual-resolution screen-space ambient occlusion[C]Proc of the 36th ACM SIGGRAPH.New York:ACM,2009[2017-11-28].https:dl.acm.org/citation.cfm/id=1598035
[3]McGuire M,Osman B,Bukowski M,et al.The alchemy screen-space ambient obscurance algorithm[C]Proc of the38th ACM SIGGRAPH Symp on High Performance Graphics.New York:ACM,2011:25-32
[4]Li Wenyao,Ren Zhong.Self-adaptive multilayer screen space ambient occlusion[J].Journal of Computer-Aided Design&Computer Graphics,2011,23(8):1294-1303(in Chinese)(李文耀,任重.自适应的多层屏幕空间环境光遮蔽[J].计算机辅助设计与图形学学报,2011,23(8):1294-1303)
[5]Langer M S,Zucker S W.Shape-from-shading on a cloudy day[J].Journal of the Optical Society of America:A,1994,11(2):467-478
[6]Prados E,Jindal N,Soatto S.A non-local approach to shape from ambient shading[C]Proc of Scale Space and Variational Methods in Computer Vision.Berlin:Springer,2011:696-708
[7]Wu Chenglei,Wilburn B,Matsushita Y,et al.High-quality shape from multi-view stereo and shading under general illumination[C]Proc of the 29th IEEE Conf on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE,2011:969-976
[8]Aldrian O,Smith W A P.Inverse rendering of faces on a cloudy day[C]Proc of the 12th European Conf on Computer Vision.Berlin:Springer,2012:201-214
[9]Beeler T,Bradley D,Zimmer H,et al.Improved reconstruction of deforming surfaces by cancelling ambient occlusion[C]Proc of the 12th European Conf on Computer Vision.Berlin:Springer,2012:30-43
[10]Laffont P Y,Bousseau A,Paris S,et al.Coherent intrinsic images from photo collections[J].ACM Transactions on Graphics,2012,31(6):202-213
[11]Hauagge D,Wehrwein S,Bala K,et al.Photometric ambient occlusion[C]Proc of the 31st IEEE Conf on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE,2013:2515-2522
[12]Yang Wei,Ji Yu,Lin Haiting,et al.Ambient occlusion via compressive visibility estimation[C]Proc of the 33rd IEEEConf on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE,2015:3882-3889
[13]Ma Yuwei,Shang Yafei,Wan Liang,et al.Photometric ambient occlusion from sparsely sampled illuminations[C]Proc of the 17th IEEE Int Conf on Multimedia&Expo Workshops(ICMEW).Piscataway,NJ:IEEE,2016:1-6
[14]Barrow H G,Tenenbaum J M.Recovering intrinsic scene characteristics from images[J].Computer Vision Systems,1978,2:3-26
[15]Land E H,McCann J J.Lightness and Retinex theory[J].Journal of the Optical Society of America,1971,61(1):1-11
[16]Tappen M F,Freeman W T,Adelson E H.Recovering intrinsic images from a single image[J].IEEE Transactions on Pattern Analysis&Machine Intelligence,2005,27(9):1459-1472
[17]Shen Jianbing,Yang Xiaoshan,Jia Yunde,et al.Intrinsic images using optimization[C]Proc of the 29th IEEE Conf on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE,2011:3481-3487
[18]Barron J T,Malik J.Shape,illumination,and reflectance from shading[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(8):1670-1687
[19]Ren Shaoqing,He Kaiming,Girshick R,et al.Faster R-CNN:Towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis&Machine Intelligence,2017,39(6):1137-1149
[20]Jonathon L,Shelhamer E,Darrell T.Fully convolutional networks for semantic segmentation[C]Proc of the 33rd IEEE Conf on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE,2015:3431-3440
[21]He Kaiming,Zhang Xiangyu,Ren Shaoqing,et al.Deep residual learning for image recognition[C]Proc of the 34th IEEE Conf on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE,2016:770-778
[22]Yu Kai,Jia Lei,Chen Yuqiang,et al.Deep learning:Yesterday,today,and tomorrow[J].Journal of Computer Research and Development,2013,50(9):1799-1804(in Chinese)(余凯,贾磊,陈雨强,等.深度学习的昨天、今天和明天[J].计算机研究与发展,2013,50(9):1799-1804)
[23]Narihira T,Maire M,Yu S X.Learning lightness from human judgement on relative reflectance[C]Proc of the33rd IEEE Conf on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE,2015:2965-2973
[24]Zhou Tinghui,Krahenbuhl P,Efros A A.Learning datadriven reflectance priors for intrinsic image decomposition[C]Proc of the 33rd IEEE Int Conf on Computer Vision.Piscataway,NJ:IEEE.2015:3469-3477
[25]Narihira T,Maire M,Yu S X.Direct intrinsics:Learning albedo-shading decomposition by convolutional regression[C]Proc of the 33rd IEEE Int Conf on Computer Vision.Piscataway,NJ:IEEE,2015:2992-3001
[26]Shi Jian,Dong Yue,Su Hao,et al.Learning nonLambertian object intrinsics across ShapeNet categories[C]Proc of the 35th IEEE Conf on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE,2017:5844-5853
[27]Chang A X,Funkhouser T,Guibas L,et al.ShapeNet:An information-rich 3D model repository[J/OL].2015[2017-01-20].https:arxiv.org/pdf/1512.03012.pdf
[28]Grosse R,Johnson M K,Adelson E H,et al.Ground truth dataset and baseline evaluations for intrinsic image algorithms[C]Proc of the 27th IEEE Int Conf on Computer Vision.Piscataway,NJ:IEEE,2009:2335-2342