H.264可分级扩展编码算法复杂度的优化
摘要
对于不同接收条件的应用场合,传统的视频编解码技术需要为各种应用环境都构建一套压缩与传输系统。随着视频业务的增多,人们迫切需要一种能够同时满足不同应用条件的可分级编解码技术。H.264可分级扩展标准就是ISO/IEC组织和国际电信联盟为了满足可分级需求而最新制定的高效视频编解码标准。基于最新的编解码技术,H.264可分级扩展标准突破了MPEG-2/4时域、空域和质量分层编码中的漂移(drifting)问题以及多次分级导致的质量急剧下降问题,和MPEG-4 FGS为避免漂移而导致的编码效率低下的问题,该标准以单一的码流高效实现了时域、空域和质量域的可分级。H.264可分级扩展标准的编码器每个编码层除了要完成传统H.264标准所要完成的编码操作以外,还要根据层间的相关性进行适当的层间预测和编码。显然H.264可分级扩展比传统H.264标准的编码器又进一步复杂化,这是不利于扩展标准的应用的。因此,为了更好地应用这一新出现的可分级扩展标准,人们迫切需要一种能够优化编码过程的编码算法。本文即围绕H.264可分级扩展编码算法的优化展开。
本文首先详细分析了扩展标准实现各种可分级的机制,以大量的具体数据,对扩展标准的诸多分级性能作了详细说明,并对层间编码和精细质量可分级包含的关键技术进行了重点分析。
本文然后分析典型配置下参考编码器的编码复杂度,揭示了优化
Different compression and transmission systems have to be constructed for users in different conditions if they are built on the top of traditional non-scalable video encoding and decoding techniques. With the development of ubiquitous multimedia, scalable video coding techniques are desired for efficiently repurposing and transporting the video contents over heterogeneous networks/devices. The scalable extension of H.264 is the newest work of developing joint standards for ISO/IEC and ITU. Based on the newest encoding and decoding techniques, temporal, spatial and quality scalability is realized with an unified framework in H.264 scalable extension standard at the cost of significantly increasing computational complexity. It motivated us to study the optimization of the encoding algorithm for the H.264 scalability extension so as to reduce the complexity while keeping the coding efficiency of H.264 scalable extension.
This thesis first analyzes the scalable mechanisms of temporal,
本文首先详细分析了扩展标准实现各种可分级的机制,以大量的具体数据,对扩展标准的诸多分级性能作了详细说明,并对层间编码和精细质量可分级包含的关键技术进行了重点分析。
本文然后分析典型配置下参考编码器的编码复杂度,揭示了优化
Different compression and transmission systems have to be constructed for users in different conditions if they are built on the top of traditional non-scalable video encoding and decoding techniques. With the development of ubiquitous multimedia, scalable video coding techniques are desired for efficiently repurposing and transporting the video contents over heterogeneous networks/devices. The scalable extension of H.264 is the newest work of developing joint standards for ISO/IEC and ITU. Based on the newest encoding and decoding techniques, temporal, spatial and quality scalability is realized with an unified framework in H.264 scalable extension standard at the cost of significantly increasing computational complexity. It motivated us to study the optimization of the encoding algorithm for the H.264 scalability extension so as to reduce the complexity while keeping the coding efficiency of H.264 scalable extension.
This thesis first analyzes the scalable mechanisms of temporal,
引文
[1] 余松煜,周源华,吴时光.数字图像处理.电子工业出版社,1987.
[2] ITU-T, “Video codec for audiovisual services at 64 kbits”. International Telecommunications Union, Geneva, Switzerland, ITU-T Recommendation H.261, 1993.
[3] ITU-T, “Video coding for low bit-rate communication”. International Telecommunications Union, Geneva, Switzerland, ITU-T Recommendation H.263, 1998.
[4] ISO/IEC JTC1/SC29/WG11, MPEG ISO CD 11172-1991, “Coding of Moving Pictures an Associated Audio for Digital Storage Media at up to about 1.5Mbit/s”. Nov. 1991.
[5] ISO/IEC –DIS 13818-2, “Information Tech. – Generic Coding of Moving Pictures and Associated Audio Information - part 2: Video Rep”. 1994.
[6] ISO/IEC JTC 1/SC 29/WG11, 14496-2, “Information technology – Generic coding of audio-visual objects –Part 2: Visual”. MPEG99/N 2688, Seoul, Mar. 1999.
[7] Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG, “Draft ITU-T Recommendation and Final Draft International Standard of Joint Video Specification (ITU-T Rec. H.264 | ISO/IEC 14496-10 AVC),” Doc. G050r1, Mar. 2003.
[8] 数字音视频编解码技术标准工作组.信息技术 先进音视频编码 第二部分:视频(报批稿) .2005.03.
[9] N. Kamaci and Y. Altunbasak, “Performance comparison of the emerging H.264 video coding standard with the existing standards”, IEEE Int. Conf. Multimedia and Expo, Baltimore, MD, July 2003.
[10] “Overview of H.264”, H.264/MPEG4 Part10 White Paper, http://www.vcodex.com.
[11] “Intra Prediction”, H.264/MPEG4 Part10 White Paper, http://www.vcodex.com.
[12] 杜博.视频编码标准 H.264 编码算法的研究及优化(硕士论文).上海交通大学,2005 年 12 月.
[13] “Inter Prediction”, H.264/MPEG4 Part10 White Paper, http://www.vcodex.com.
[14] ITU-T and ISO/IEC JTC1, “Scalable video coding draft 6”, JVT-S201, Apr. 2006.
[15] 骆政屹,余松煜,宋利,杨小康.H.264 可分级扩展技术的介绍和分析.中国图像图形学报,2006 年 11 月.
[16] ITU-T and ISO/IEC JTC1, JSVM 5.7 software, cvs://garconient. rwthiaachen. de:/cvs. /jvt, June 2006.
[17] S. Heiko, T. Hinz, D. Marple, et al, “Constrained inter-layer prediction for single-loop decoding in scalability”, IEEE International Conference on Image Processing (ICIP). Genoa, Italy, 2005.
[18] W.-J. Han, “Modified intraBL design using smoothed reference”, JVT-R091,Jan. 2006.
[19] X. Wang, Marta Karczewicz, “CE7 Report, multiple FGS layer coding for low-delay applications”, JVT-R077, Jan. 2006.
[20] Amonou Isabelle, Cammas Nathalie, Kervadec Sylvain, et al, “Complexity reduction of FGS passes”, JVT-R069, Jan. 2006.
[21] 吴伟陵.信息处理与编码.人民邮电出版社,2003.
[22] A. Ortega, Kannan Ramchandran, “Rate-distortion methods for image and video compression”, IEEE Signal Precessing Magazine, vol.15, pp.23-50, Nov.1998.
[23] JVT Reference Software, http://bs.hhi.de/~suehring/tml/download.
[24] ITU-T and ISO/IEC JTC1, JSVM 5.7 software, cvs://garconient. rwthiaachen. de:/cvs. /jvt, Aug. 2006.
[25] H. Li, Z. G. Li, C. Wen, “Fast mode decision algorithm for inter-frame coding in fully scalable video coding”, IEEE Transactions on Circuits and Systems for Video Technology, vol.16, pp.889-895, July 2006.
[26] L. Yang, Y. Chen, J. Zhai, et al, “Low complexity intra prediction for enhancement layer”, JVT-Q084, Oct. 2005.
[27] E. S. Larsen and D. K. McAllister, “Fast matrix multiplies using graphics hardware”, IEEE Supercomputing Proceeding, Nov. 2001.
[28] M. Rumpf and R. Strzodka, “Level set segmentation in graphics hardware”, IEEE Conference on Image Processing (ICIP), 2001.
[29] C. J. Thompson, S. Hahn and M. Oskin, “Using modern graphics architectures for general-purpose computing: a framework and analysis”, ACM/IEEE MICRO-35, Nov. 2002.
[30] P. colantoni, N. Boukala and J. D. Rugna, “Fast and accurate color image processing using 3-D graphics cards”, The 8th Int. Fall Workshop: Vision Modeling and Visualization, Munich, Germany, Nov. 2003.
[31] K. Moreland and E. Angel, “The FFT on a GPU”, SIGGRAPH/Eurographics Workshop Graphics Hardware, July 2003.
[32] M. Hopf, T. Ertl, “Hardware accelerated wavelet transformation”, EG/IEEE TCVG Symp, Visualization, 2000.
[33] 房波.基于通用可编程 GPU 的视频编解码器――架构、算法与实现(硕士论文).浙江大学,2005 年 1 月.
[34] G. Shen, G. Gao, S. Li, et al, “Accelerate video decoding with generic GPU”, IEEE Transactions on Circuits and Systems for Video Technology, vol.15, pp.685-693, May 2005.
[35] R. Fernando, M. J. Kilgard 著,洪伟,刘亚妮,李骑等译.Cg 教程.人民邮电出版社,2004.
[36] 王鹏.H.264 视频编码算法优化及其在 DSP 上的实现(硕士论文).上海交通大学,2006 年 1 月.
[37] 陈维安.H.264 软件编解码器的优化(硕士论文).上海交通大学,2005年 12 月.
[38] 周泽健.H.264 自适应快速模式选择的研究(硕士论文).上海交通大学,2006 年 1 月.
[2] ITU-T, “Video codec for audiovisual services at 64 kbits”. International Telecommunications Union, Geneva, Switzerland, ITU-T Recommendation H.261, 1993.
[3] ITU-T, “Video coding for low bit-rate communication”. International Telecommunications Union, Geneva, Switzerland, ITU-T Recommendation H.263, 1998.
[4] ISO/IEC JTC1/SC29/WG11, MPEG ISO CD 11172-1991, “Coding of Moving Pictures an Associated Audio for Digital Storage Media at up to about 1.5Mbit/s”. Nov. 1991.
[5] ISO/IEC –DIS 13818-2, “Information Tech. – Generic Coding of Moving Pictures and Associated Audio Information - part 2: Video Rep”. 1994.
[6] ISO/IEC JTC 1/SC 29/WG11, 14496-2, “Information technology – Generic coding of audio-visual objects –Part 2: Visual”. MPEG99/N 2688, Seoul, Mar. 1999.
[7] Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG, “Draft ITU-T Recommendation and Final Draft International Standard of Joint Video Specification (ITU-T Rec. H.264 | ISO/IEC 14496-10 AVC),” Doc. G050r1, Mar. 2003.
[8] 数字音视频编解码技术标准工作组.信息技术 先进音视频编码 第二部分:视频(报批稿) .2005.03.
[9] N. Kamaci and Y. Altunbasak, “Performance comparison of the emerging H.264 video coding standard with the existing standards”, IEEE Int. Conf. Multimedia and Expo, Baltimore, MD, July 2003.
[10] “Overview of H.264”, H.264/MPEG4 Part10 White Paper, http://www.vcodex.com.
[11] “Intra Prediction”, H.264/MPEG4 Part10 White Paper, http://www.vcodex.com.
[12] 杜博.视频编码标准 H.264 编码算法的研究及优化(硕士论文).上海交通大学,2005 年 12 月.
[13] “Inter Prediction”, H.264/MPEG4 Part10 White Paper, http://www.vcodex.com.
[14] ITU-T and ISO/IEC JTC1, “Scalable video coding draft 6”, JVT-S201, Apr. 2006.
[15] 骆政屹,余松煜,宋利,杨小康.H.264 可分级扩展技术的介绍和分析.中国图像图形学报,2006 年 11 月.
[16] ITU-T and ISO/IEC JTC1, JSVM 5.7 software, cvs://garconient. rwthiaachen. de:/cvs. /jvt, June 2006.
[17] S. Heiko, T. Hinz, D. Marple, et al, “Constrained inter-layer prediction for single-loop decoding in scalability”, IEEE International Conference on Image Processing (ICIP). Genoa, Italy, 2005.
[18] W.-J. Han, “Modified intraBL design using smoothed reference”, JVT-R091,Jan. 2006.
[19] X. Wang, Marta Karczewicz, “CE7 Report, multiple FGS layer coding for low-delay applications”, JVT-R077, Jan. 2006.
[20] Amonou Isabelle, Cammas Nathalie, Kervadec Sylvain, et al, “Complexity reduction of FGS passes”, JVT-R069, Jan. 2006.
[21] 吴伟陵.信息处理与编码.人民邮电出版社,2003.
[22] A. Ortega, Kannan Ramchandran, “Rate-distortion methods for image and video compression”, IEEE Signal Precessing Magazine, vol.15, pp.23-50, Nov.1998.
[23] JVT Reference Software, http://bs.hhi.de/~suehring/tml/download.
[24] ITU-T and ISO/IEC JTC1, JSVM 5.7 software, cvs://garconient. rwthiaachen. de:/cvs. /jvt, Aug. 2006.
[25] H. Li, Z. G. Li, C. Wen, “Fast mode decision algorithm for inter-frame coding in fully scalable video coding”, IEEE Transactions on Circuits and Systems for Video Technology, vol.16, pp.889-895, July 2006.
[26] L. Yang, Y. Chen, J. Zhai, et al, “Low complexity intra prediction for enhancement layer”, JVT-Q084, Oct. 2005.
[27] E. S. Larsen and D. K. McAllister, “Fast matrix multiplies using graphics hardware”, IEEE Supercomputing Proceeding, Nov. 2001.
[28] M. Rumpf and R. Strzodka, “Level set segmentation in graphics hardware”, IEEE Conference on Image Processing (ICIP), 2001.
[29] C. J. Thompson, S. Hahn and M. Oskin, “Using modern graphics architectures for general-purpose computing: a framework and analysis”, ACM/IEEE MICRO-35, Nov. 2002.
[30] P. colantoni, N. Boukala and J. D. Rugna, “Fast and accurate color image processing using 3-D graphics cards”, The 8th Int. Fall Workshop: Vision Modeling and Visualization, Munich, Germany, Nov. 2003.
[31] K. Moreland and E. Angel, “The FFT on a GPU”, SIGGRAPH/Eurographics Workshop Graphics Hardware, July 2003.
[32] M. Hopf, T. Ertl, “Hardware accelerated wavelet transformation”, EG/IEEE TCVG Symp, Visualization, 2000.
[33] 房波.基于通用可编程 GPU 的视频编解码器――架构、算法与实现(硕士论文).浙江大学,2005 年 1 月.
[34] G. Shen, G. Gao, S. Li, et al, “Accelerate video decoding with generic GPU”, IEEE Transactions on Circuits and Systems for Video Technology, vol.15, pp.685-693, May 2005.
[35] R. Fernando, M. J. Kilgard 著,洪伟,刘亚妮,李骑等译.Cg 教程.人民邮电出版社,2004.
[36] 王鹏.H.264 视频编码算法优化及其在 DSP 上的实现(硕士论文).上海交通大学,2006 年 1 月.
[37] 陈维安.H.264 软件编解码器的优化(硕士论文).上海交通大学,2005年 12 月.
[38] 周泽健.H.264 自适应快速模式选择的研究(硕士论文).上海交通大学,2006 年 1 月.