基于视角方向的全景虚拟视图合成
|
摘要
针对3DOF+视频搭建了一套基于视角方向的全景视频虚拟视图合成系统,利用极少视点位置的视频信息即可准确渲染任意视点位置的视频内容,从而为用户提供真实的观看体验。将所合成的新视点位置视图与相机采集的真实视图进行比较。实验结果表明,所合成的虚拟视图平均客观指标PSNR为23dB左右,主观质量良好,较为真实且能够提供良好的交互性与视觉体验。
In order to provide users with a true viewing experience,apanoramic video virtual view synthesis system based on view angle for 3 DOF+ video was built,which could accurately render arbitrary viewpoint position by using video information with few viewpoint positions.Comparing the synthesized new viewpoint position view with the real view collected by the camera,the experimental results show that the synthesized objective view has an average objective index PSNR of about 23 dB,good subjective quality,and is realistic and can provide good interactivity and visual experience.
In order to provide users with a true viewing experience,apanoramic video virtual view synthesis system based on view angle for 3 DOF+ video was built,which could accurately render arbitrary viewpoint position by using video information with few viewpoint positions.Comparing the synthesized new viewpoint position view with the real view collected by the camera,the experimental results show that the synthesized objective view has an average objective index PSNR of about 23 dB,good subjective quality,and is realistic and can provide good interactivity and visual experience.
引文
[1]QIAN F,HAN L J,GOPALAKRISHNAN V.Optimizing 360video delivery over cellularnetworks[C]//Proceedings of the 5th Workshop on All Things Cellular:Operations,Applications and Chanllenges.NewYork:ACM,2016:1-6.
[2]CUMMINGS J J,BAILENSON J N.How immersive is enough?A meta-analysis of the effect of immersive technology on user presence[J].Media Psychology,2016,19(2):272-309.
[3]CHAMPELM L,KOENEN R,LAFRUIT G,et al.Working Draft 0.2of TR:Technical report on architectures for immersive media[R].ISO/IEC JTC1/SC29/WG11,Doc.MPEG-2017N16918,Hobart,2017.
[4]FEHNC.Depth-image-based rendering(DIBR),compression,and transmission for a new approach on 3D-TV[C]//Stereoscopic Displays and Virtual Reality Systems XI.International Society for Optics and Photonics,2004,5291:93-105.
[5]SZELISKI R,SHUM H Y.Creating full view panoramic image mosaics and environment maps[C]//Proceedings of the 24th annual conference on Computer graphics and interactive techniques.NewYork:ACM Press/AddisonWesley Publishing Co.,1997:251-258.
[6]NG K T,CHAN S C,SHUM H Y.Data compression and transmission aspects of panoramic videos[J].IEEETransactions on Circuits and Systems for Video Technology,2005,15(1):82-95.
[7]HUANG K C,CHIEN P Y,CHIEN C A,et al.A 360-degree panoramic video system design[C]//Proceedings of Technical Papers of2014 In ternational Symposium on VLSI Design,Automation and Test.Taiwan,China:IEEE,2014:1-4.
[8]FU C W,WAN L,WONG T T,et al.The rhombic dodecahedron map:an efficient scheme for encoding panoramic video[J].IEEE Transactions on Multimedia,2009,11(4):634-644.
[9]WEGNER K,STANKIEWICZ O,DZIEMBOWSKI A,et al.Omnidirectional 6-DoF/3-DoF+rendering[R].MPEG-I visual of ITU-T SG16WP3and ISO/IEC JTC1/SC29/WG11.MPEG Doc.m40806,2017:1-6.
[10]ZHANG F,LIU F.Casual stereoscopic panorama stitching[C]//Proceedings of 2015IEEE Conference on Computer Vision and Pattern Recognition.Boston,MA,USA:IEEE,2015:2002-2010.
[11]Wikipedia.Equirectangular projection[EB/OL].(2018-02-21)[2018-3-30].https://en.wikipedia.org/wiki/Equirectangular_projection.
[12]贾瑞才.重力/地磁辅助的欧拉角无迹卡尔曼滤波姿态估计[J].光学精密工程,2014,22(12):3280-3286.JIA Rui-cai.Attitude estimation base on gravity/magnetic assisted Euler angle UKF.[J].Optics and Precision Engineering,2014,22(12):3280-3286.
[13]王永,姚太克,周烽,等.望远镜副镜的三自由度并联支撑构型研究与运动分析[J].光学精密工程,2013,21(11):2860-28696.WANG Y,YAO T K,ZHOU F,et al.Type synthesis of 3-DOF parallel support system for telescope secondary mirror[J].Optics and Precision Engineering,2013,21(11):2860-2869.
[14]STANKIEWICZ O,WEGNER K,TANIMOTO M,et al.Enhanced view synthesis reference software(VSRS)for free-viewpoint television[R].ISO/IEC JTC,2013,1:533-541.
[15]LEE C,HO Y S.Common-hole filling for boundary noise removal in VSRS[R].MPEG2010/M18514,Guangzhou,China 2010.
[16]汪泽伟,于有光,杨付正.基于HTC Vive的全景视频显示质量评估[J].液晶与显示,2018,33(1):85-91WANG Z W,YU Y G,YANG F Z.Panoramic video display quality assessment based on HTC Vive[J].Chinese Journal of Liquid Crystals and Displays,2018,33(1):85-91.
[2]CUMMINGS J J,BAILENSON J N.How immersive is enough?A meta-analysis of the effect of immersive technology on user presence[J].Media Psychology,2016,19(2):272-309.
[3]CHAMPELM L,KOENEN R,LAFRUIT G,et al.Working Draft 0.2of TR:Technical report on architectures for immersive media[R].ISO/IEC JTC1/SC29/WG11,Doc.MPEG-2017N16918,Hobart,2017.
[4]FEHNC.Depth-image-based rendering(DIBR),compression,and transmission for a new approach on 3D-TV[C]//Stereoscopic Displays and Virtual Reality Systems XI.International Society for Optics and Photonics,2004,5291:93-105.
[5]SZELISKI R,SHUM H Y.Creating full view panoramic image mosaics and environment maps[C]//Proceedings of the 24th annual conference on Computer graphics and interactive techniques.NewYork:ACM Press/AddisonWesley Publishing Co.,1997:251-258.
[6]NG K T,CHAN S C,SHUM H Y.Data compression and transmission aspects of panoramic videos[J].IEEETransactions on Circuits and Systems for Video Technology,2005,15(1):82-95.
[7]HUANG K C,CHIEN P Y,CHIEN C A,et al.A 360-degree panoramic video system design[C]//Proceedings of Technical Papers of2014 In ternational Symposium on VLSI Design,Automation and Test.Taiwan,China:IEEE,2014:1-4.
[8]FU C W,WAN L,WONG T T,et al.The rhombic dodecahedron map:an efficient scheme for encoding panoramic video[J].IEEE Transactions on Multimedia,2009,11(4):634-644.
[9]WEGNER K,STANKIEWICZ O,DZIEMBOWSKI A,et al.Omnidirectional 6-DoF/3-DoF+rendering[R].MPEG-I visual of ITU-T SG16WP3and ISO/IEC JTC1/SC29/WG11.MPEG Doc.m40806,2017:1-6.
[10]ZHANG F,LIU F.Casual stereoscopic panorama stitching[C]//Proceedings of 2015IEEE Conference on Computer Vision and Pattern Recognition.Boston,MA,USA:IEEE,2015:2002-2010.
[11]Wikipedia.Equirectangular projection[EB/OL].(2018-02-21)[2018-3-30].https://en.wikipedia.org/wiki/Equirectangular_projection.
[12]贾瑞才.重力/地磁辅助的欧拉角无迹卡尔曼滤波姿态估计[J].光学精密工程,2014,22(12):3280-3286.JIA Rui-cai.Attitude estimation base on gravity/magnetic assisted Euler angle UKF.[J].Optics and Precision Engineering,2014,22(12):3280-3286.
[13]王永,姚太克,周烽,等.望远镜副镜的三自由度并联支撑构型研究与运动分析[J].光学精密工程,2013,21(11):2860-28696.WANG Y,YAO T K,ZHOU F,et al.Type synthesis of 3-DOF parallel support system for telescope secondary mirror[J].Optics and Precision Engineering,2013,21(11):2860-2869.
[14]STANKIEWICZ O,WEGNER K,TANIMOTO M,et al.Enhanced view synthesis reference software(VSRS)for free-viewpoint television[R].ISO/IEC JTC,2013,1:533-541.
[15]LEE C,HO Y S.Common-hole filling for boundary noise removal in VSRS[R].MPEG2010/M18514,Guangzhou,China 2010.
[16]汪泽伟,于有光,杨付正.基于HTC Vive的全景视频显示质量评估[J].液晶与显示,2018,33(1):85-91WANG Z W,YU Y G,YANG F Z.Panoramic video display quality assessment based on HTC Vive[J].Chinese Journal of Liquid Crystals and Displays,2018,33(1):85-91.