面向3D呈现的有损和无损混合深度视频编码
|
摘要
针对多个深度视频流提出实时的压缩方法和评估方案,采用一种有损无损混合的编码方法,可以在图像质量和压缩率之间提供一种有效的平衡控制。行程编码(RLE)用于无损压缩,用来保存深度图像素的高位;像素低10bits保存在YUV图像的Y通道,直接使用×264编码。实验表明,所提方法可以在平均时间小于12ms的前提下同时编解码多个深度图。在实时传输中,通过动态调节质量控制级别,可以获取4∶1到20∶1的压缩率。在压缩率10∶1的情况下,主观解码3D重构效果与原始图几乎完全一致。
Color and depth(RGBD)sensors,such as the Microsoft Kinect,are used in many research areas.In the immersive group-to-group telepresence system based on a cluster of RGBD sensors,the color and depth image streams are sent to a remote site over a network connection where they are used for the real-time reconstruction of the captured participants.Depth map coding plays a crucial role in the system that allows distributed groups of users to meet in a 3Dshared virtual world.We develop and evaluate different schemes for the real-time compression of multiple depth image streams.Our analysis suggests that the hybrid lossless-lossy compression approach provides a good tradeoff between quality and compression ratio.The lossless compression based on run length encoding is used to preserve the information of the highest bits of the depth image pixels.The lowest 10-bits of a depth pixel value are directly encoded in the Y channel of a YUV image and encoded by a×264codec.Our experiments show that the proposed method can encode and decode multiple depth image streams in less than 12 ms on average.Depending on the compression level,which can be adjusted during application,we are able to achieve a compression ratio ranging from about 4∶1to 20∶1.Initial results indicate that the quality for 3Dreconstructions is almost indistinguishable from the original for a compression ratio up to 10∶1.
Color and depth(RGBD)sensors,such as the Microsoft Kinect,are used in many research areas.In the immersive group-to-group telepresence system based on a cluster of RGBD sensors,the color and depth image streams are sent to a remote site over a network connection where they are used for the real-time reconstruction of the captured participants.Depth map coding plays a crucial role in the system that allows distributed groups of users to meet in a 3Dshared virtual world.We develop and evaluate different schemes for the real-time compression of multiple depth image streams.Our analysis suggests that the hybrid lossless-lossy compression approach provides a good tradeoff between quality and compression ratio.The lossless compression based on run length encoding is used to preserve the information of the highest bits of the depth image pixels.The lowest 10-bits of a depth pixel value are directly encoded in the Y channel of a YUV image and encoded by a×264codec.Our experiments show that the proposed method can encode and decode multiple depth image streams in less than 12 ms on average.Depending on the compression level,which can be adjusted during application,we are able to achieve a compression ratio ranging from about 4∶1to 20∶1.Initial results indicate that the quality for 3Dreconstructions is almost indistinguishable from the original for a compression ratio up to 10∶1.
引文
[1]Beck S,Kunert A,Kulik A,et al.Froehlich.Immersive group-to-group telepresence[J].IEEE Transactions on Visualization and Computer Graphics,2013,19(4):616-625.
[2]Doumanoglou A,Alexiadis D S,Zarpalas D,et al.Towards real-time and efficient compression of human time-varying-meshes[J].IEEE Transactions on Circuits and Systems for Video Technology,2014,24(12):1.
[3]Ahn J K,Koh Y J,Kim C S.Efficient fine-granular scalable coding of 3D mesh sequences[J].IEEE Transactions on Multimedia,2013,15(3):485-497.
[4]Mekuria R,Sanna M,Izquierdo E,et al.Enabling geometry-based 3-D tele-immersion with fast mesh compression and linear rateless coding[J].IEEE Transactions on Multimedia,2014,16(7):1809-1820.
[5]Raghuraman S,Venkatraman K,Wang Z,et al.A3D teleimmersion streaming approach using skeleton-based prediction[A].Proc.of the 21st ACM International Conference on Multimedia[C].2013,3:721-724.
[6]Cho T,Lee Y,Shin J.A homogenizing filter for depth map compressive sensing using edge-awarded method[A].Proc.of International Conference on ICT Convergence[C].2013,10:591-595.
[7]Tabus I,Schiopu I,Astola J.Context coding of depth map images under the piecewise-constant image model representation[J].IEEE Transactions on Image Processing,2013,22(11):4195-210.
[8]Zhao L,Wang A,Zeng B,et al.Scalable coding of depth images with synthesis-guided edge detection[J].Ksii Transactions on Internet&Information Systems,2015,9(10):4108-4125.
[9]Daribo I,Florencio D,Cheung G.Arbitrarily shaped motion prediction for depth video compression using arithmetic edge coding[J].IEEE Transactions on Image Processing,2014,23(11):4696-708.
[10]Ozkalayc B O,Alatan A A.3D planar representation of stereo depth images for 3DTV applications[J].IEEE Transactions on Image Processing,2014,23(12):5222-5232.
[11]Bal C,Nguyen T.Multiview video plus depth coding with depth-based prediction mode[J].IEEE Transactions on Circuits and Systems for Video Technology,2014,24(6):995-1005.
[12]Schwarz H,Bartnik C,Bosse S,et al.Extension of high efficiency video coding(hevc)for multiview video and depth data[A].Proc.of IEEE International Conference on Image Processing[C].2012,9:205-208.
[13]Oh B T.An adaptive quantization algorithm without side information for depth map coding[J].Image Communication,2014,29(9):962-970.
[14]ZHANG Yu-hua,WANG Yong,ZHU Ce,et al.Optimization of depth modeling modes in 3D-HEVC depth intra coding[J].Journal of Real-Time Image Processing,2016,12(46):1-16.
[15]Merkle P,Muller K,Marpe D,et al.Depth intra coding for3D video based on geometric primitives[J].IEEE Transactions on Circuits&Systems for Video Technology,2015,25(3):570-582.
[16]LEI Jian-jun,LI Zhen-zhen,ZHU Tao,et al.Region-based bit allocation and rate control for depth video in HEVC[J].Multimedia Tools and Applications,2016,75(258):1-17.
[17]Shen L,An P,Zhang Z,et al.A3D-HEVC fast mode decision algorithm for real-time applications[J].Acm Transactions on Multimedia Computing Communications&Applications,2015,11(3):1-23.
[18]ZHAO Zhen-jun,SHEN Li-quan,HUA Qian-qian,et al.Joint bit allocation algorithm for multi-view texture and depth based on 3D-HEVC[J].Journal of Optoelectronics·Laser,2015,26(1):149-155.赵振军,沈礼权,胡乾乾,等.基于3D-HEVC的多视点纹理加深度的联合比特分配算法[J].光电子·激光,2015,26(1):149-155.
[19]LI Zu-tuan,SHAO Feng,JIANG Gang-yi,et al.Low complexity depth video coding based on maximum tolerable depth distortion mode[J].Journal of Optoelectronics·Laser,2015,26(6):1162-1169.李祖团,邵枫,蒋刚毅,等.基于最大可容忍深度失真模型的低复杂度深度视频编码[J].光电子·激光,2015,26(6):1162-1169.
[20]Fu J,Miao D,Yu W,et al.Kinect-like depth data compression[J].IEEE Transactions on Multimedia,2013,15(6):1340-1352.
[21]Yuan H,Kwong S,Liu J,et al.A novel distortion model and lagrangian multiplier for depth maps coding[J].IEEE Transactions on Circuits and Systems for Video Technology,2014,24(3):443-451.
[22]Pece F,Kautz J,Weyrich T.Adapting standard video codecs for depth streaming[A].Proc.of the 17th Eurographics Conference on Virtual Environments[C].2011,4:59-66.
[2]Doumanoglou A,Alexiadis D S,Zarpalas D,et al.Towards real-time and efficient compression of human time-varying-meshes[J].IEEE Transactions on Circuits and Systems for Video Technology,2014,24(12):1.
[3]Ahn J K,Koh Y J,Kim C S.Efficient fine-granular scalable coding of 3D mesh sequences[J].IEEE Transactions on Multimedia,2013,15(3):485-497.
[4]Mekuria R,Sanna M,Izquierdo E,et al.Enabling geometry-based 3-D tele-immersion with fast mesh compression and linear rateless coding[J].IEEE Transactions on Multimedia,2014,16(7):1809-1820.
[5]Raghuraman S,Venkatraman K,Wang Z,et al.A3D teleimmersion streaming approach using skeleton-based prediction[A].Proc.of the 21st ACM International Conference on Multimedia[C].2013,3:721-724.
[6]Cho T,Lee Y,Shin J.A homogenizing filter for depth map compressive sensing using edge-awarded method[A].Proc.of International Conference on ICT Convergence[C].2013,10:591-595.
[7]Tabus I,Schiopu I,Astola J.Context coding of depth map images under the piecewise-constant image model representation[J].IEEE Transactions on Image Processing,2013,22(11):4195-210.
[8]Zhao L,Wang A,Zeng B,et al.Scalable coding of depth images with synthesis-guided edge detection[J].Ksii Transactions on Internet&Information Systems,2015,9(10):4108-4125.
[9]Daribo I,Florencio D,Cheung G.Arbitrarily shaped motion prediction for depth video compression using arithmetic edge coding[J].IEEE Transactions on Image Processing,2014,23(11):4696-708.
[10]Ozkalayc B O,Alatan A A.3D planar representation of stereo depth images for 3DTV applications[J].IEEE Transactions on Image Processing,2014,23(12):5222-5232.
[11]Bal C,Nguyen T.Multiview video plus depth coding with depth-based prediction mode[J].IEEE Transactions on Circuits and Systems for Video Technology,2014,24(6):995-1005.
[12]Schwarz H,Bartnik C,Bosse S,et al.Extension of high efficiency video coding(hevc)for multiview video and depth data[A].Proc.of IEEE International Conference on Image Processing[C].2012,9:205-208.
[13]Oh B T.An adaptive quantization algorithm without side information for depth map coding[J].Image Communication,2014,29(9):962-970.
[14]ZHANG Yu-hua,WANG Yong,ZHU Ce,et al.Optimization of depth modeling modes in 3D-HEVC depth intra coding[J].Journal of Real-Time Image Processing,2016,12(46):1-16.
[15]Merkle P,Muller K,Marpe D,et al.Depth intra coding for3D video based on geometric primitives[J].IEEE Transactions on Circuits&Systems for Video Technology,2015,25(3):570-582.
[16]LEI Jian-jun,LI Zhen-zhen,ZHU Tao,et al.Region-based bit allocation and rate control for depth video in HEVC[J].Multimedia Tools and Applications,2016,75(258):1-17.
[17]Shen L,An P,Zhang Z,et al.A3D-HEVC fast mode decision algorithm for real-time applications[J].Acm Transactions on Multimedia Computing Communications&Applications,2015,11(3):1-23.
[18]ZHAO Zhen-jun,SHEN Li-quan,HUA Qian-qian,et al.Joint bit allocation algorithm for multi-view texture and depth based on 3D-HEVC[J].Journal of Optoelectronics·Laser,2015,26(1):149-155.赵振军,沈礼权,胡乾乾,等.基于3D-HEVC的多视点纹理加深度的联合比特分配算法[J].光电子·激光,2015,26(1):149-155.
[19]LI Zu-tuan,SHAO Feng,JIANG Gang-yi,et al.Low complexity depth video coding based on maximum tolerable depth distortion mode[J].Journal of Optoelectronics·Laser,2015,26(6):1162-1169.李祖团,邵枫,蒋刚毅,等.基于最大可容忍深度失真模型的低复杂度深度视频编码[J].光电子·激光,2015,26(6):1162-1169.
[20]Fu J,Miao D,Yu W,et al.Kinect-like depth data compression[J].IEEE Transactions on Multimedia,2013,15(6):1340-1352.
[21]Yuan H,Kwong S,Liu J,et al.A novel distortion model and lagrangian multiplier for depth maps coding[J].IEEE Transactions on Circuits and Systems for Video Technology,2014,24(3):443-451.
[22]Pece F,Kautz J,Weyrich T.Adapting standard video codecs for depth streaming[A].Proc.of the 17th Eurographics Conference on Virtual Environments[C].2011,4:59-66.