基于比特率控制过程的快速帧内更新算法
|
摘要
使用预测编码产生的视频流对包交换网络中的传输错误异常敏感。由于预测编码的使用，导致错误在时间和空间上不断累积，造成视频质量严重下降。因此，如何提高视频信号的抗误码能力就成为视频编码领域中的一个热点问题。
本文主要致力于研究如何通过帧内更新来彻底消除错误。文章首先简要介绍了视频编码标准的原理和主要技术，分析了差错扩散过程和常用的差错控制技术。接着分析了现有几种帧内更新技术的优缺点，指出基于率失真框架的帧内更新技术是最理想的解决方案。然而，传统的以拉格朗日方程为基础的帧内更新算法由于需要对更新宏块分别进行帧内模式和帧间模式的实际编码和解码操作，以得到编码产生的数据量和失真度，因而带来了极大的计算复杂度，不适合某些实时视频业务的需要。在详细分析了TMN8中的率失真模型和比特率控制过程之后，本文提出了一种基于比特率控制的快速帧内更新算法。首先通过对端到端的失真度分析，提出将失真度分为量化失真，扩散失真和掩盖失真三部分分开计算，然后在TMN8的率失真模型之上提出了一个估算数据量受限情况下的最小总体量化失真度的公式。在此基础上，通过比较更新一个宏块减小的扩散失真度和增加的整帧量化失真度，编码器可以快速判断是否需要对此宏块进行帧内更新，而不需要进行实际的编码解码操作，因此极大地降低了更新算法的计算复杂度。大量模拟实验证明，本算法在极大提高计算速度的同时，取得了与传统算法近似甚至更好的效果。
Predictive video coding for transmission over packet-switched networks is very sensitive to channel errors, since the prediction loop propagates errors in both the spatial and the temporal directions, and causes substantial degradation in video quality. Consequently, research on enhancing the error resilience of video transmission has been widely addressed.
This paper is mainly concerned with Intra-update. First, the principle of video coding and main compression technologies are introduced, and the error propagation problem and corresponding error control methods are analyzed. In succession, the advantages and disadvantages of common Intra-update arithmetic are pointed, and the rate-distortion framework based Intra-update method is referred as a perfect solution. However, based on Lagrange equation, classical Intra-update methods need to do actual encoding and decoding operations to calculate the rate and distortion, so the computational complexity is very huge, so that it is not suitable to some real-time applications. By analyzing the rate-distortion model and rate control method in TMN8, this paper proposes a rate control based fast Intra-update method. Through the end-to-end distortion analysis, the total distortion is suggested to be divided into quantization distortion, propagated distortion and concealment distortion to be calculated individually. A dis
tortion modal based on rate-distortion modal in TMN8 is proposed to approximately estimate the total quantization distortion when the available rate is prescribed. Through comparing the increased quantization distortion and reduced propagation distortion, the encoder can quickly decide whether need to Intra-update a macroblock without actual encoding and decoding operations. Thus, it reduces the computational complexity significantly. Simulation results demonstrate that the proposed method achieves the same or even better gains over traditional method.
本文主要致力于研究如何通过帧内更新来彻底消除错误。文章首先简要介绍了视频编码标准的原理和主要技术，分析了差错扩散过程和常用的差错控制技术。接着分析了现有几种帧内更新技术的优缺点，指出基于率失真框架的帧内更新技术是最理想的解决方案。然而，传统的以拉格朗日方程为基础的帧内更新算法由于需要对更新宏块分别进行帧内模式和帧间模式的实际编码和解码操作，以得到编码产生的数据量和失真度，因而带来了极大的计算复杂度，不适合某些实时视频业务的需要。在详细分析了TMN8中的率失真模型和比特率控制过程之后，本文提出了一种基于比特率控制的快速帧内更新算法。首先通过对端到端的失真度分析，提出将失真度分为量化失真，扩散失真和掩盖失真三部分分开计算，然后在TMN8的率失真模型之上提出了一个估算数据量受限情况下的最小总体量化失真度的公式。在此基础上，通过比较更新一个宏块减小的扩散失真度和增加的整帧量化失真度，编码器可以快速判断是否需要对此宏块进行帧内更新，而不需要进行实际的编码解码操作，因此极大地降低了更新算法的计算复杂度。大量模拟实验证明，本算法在极大提高计算速度的同时，取得了与传统算法近似甚至更好的效果。
Predictive video coding for transmission over packet-switched networks is very sensitive to channel errors, since the prediction loop propagates errors in both the spatial and the temporal directions, and causes substantial degradation in video quality. Consequently, research on enhancing the error resilience of video transmission has been widely addressed.
This paper is mainly concerned with Intra-update. First, the principle of video coding and main compression technologies are introduced, and the error propagation problem and corresponding error control methods are analyzed. In succession, the advantages and disadvantages of common Intra-update arithmetic are pointed, and the rate-distortion framework based Intra-update method is referred as a perfect solution. However, based on Lagrange equation, classical Intra-update methods need to do actual encoding and decoding operations to calculate the rate and distortion, so the computational complexity is very huge, so that it is not suitable to some real-time applications. By analyzing the rate-distortion model and rate control method in TMN8, this paper proposes a rate control based fast Intra-update method. Through the end-to-end distortion analysis, the total distortion is suggested to be divided into quantization distortion, propagated distortion and concealment distortion to be calculated individually. A dis
tortion modal based on rate-distortion modal in TMN8 is proposed to approximately estimate the total quantization distortion when the available rate is prescribed. Through comparing the increased quantization distortion and reduced propagation distortion, the encoder can quickly decide whether need to Intra-update a macroblock without actual encoding and decoding operations. Thus, it reduces the computational complexity significantly. Simulation results demonstrate that the proposed method achieves the same or even better gains over traditional method.
引文
[1] Q.F. Zhu and L. Kerofsky, "Joint source coding, transport processing and error concealment for H.323-based packet video," in Proc. SPIE, VCIP'99, vol. 3653, San Jose, CA, Jan. 1999, pp. 52-62.
[2] ISO/IEC DIS 11172-2(MPEG-1). Information technology - Coding of moving pictures and associated audio for digital storage media at up to about 1.5 mbit/s - part 2: Video. 1993.
[3] ISO/IEC. Document 13818-2 (MPEG-2). Information technology - Generic coding of moving pictures and associated audio, part 2: video, recommendation H.262. 1994.
[4] Recommendation H.263. Video coding for low bit-rate communication. 1996.
[5] ITU-T Standardization Sector, SG 16, VCEG. H.26L Test Model Long-Term Number 8.
[6] 彭强，诸昌钤．视频通信差错恢复技术．计算机应用研究，2001，(7)：9-12．
[7] Villasenor J D, et al. Robust video coding algorithms and systems. Proceeding of The IEEE, 1999, 87(10): 1724-1733.
[8] Wang Yao, et al. Error resilient video coding techniques. IEEE Signal Processing Magazine, 2000, 17(4):61-82.
[9] Wang Yao, et al. Error control and concealment for video communication: a review. Proceedings of the IEEE 1998, 86(5):974-997
[10] Aravind R, et al. Packet loss resilience of MPEG-2 scalable video coding algorithms. IEEE Trans Circuits Syst. Video Technol., 1996, 6:426-435.
[11] Zhu Q F, et al. Coding and cell loss recovery for DCT-based packet video, IEEE Trans Circuits Syst. Video Technol., 1993, 3:248-258.
[12] Girod B., Farber N. Feedback-Based Error Control for Mobile Video Transmission. Proceeding of the IEEE, 1999; 87(10): 1707-1723.
[13] 李伟．基于H．263+视频传输的误差恢复技术．计算机工程与应用，2003，10：165-165．
[14] 高绍帅，涂国防．基于H．263视频解码的错误检测和掩盖。通信学报，2001，12：82-87．
[15] 梁凡，魏晓晖，肖自美等．视频压缩编码的差错恢复技术。中国图像图形学报，2000；05A(5)：374-379．
[16]卢官明．DCT编码压缩图像的传输差错控制．南京邮电学院学报，2002；2(3)：90-94．
[17]罗忠，梁柱，王兆华．IP视频通信中的自适应错误掩盖算法．天津大学学报(自然科学与工程技术版)，2003；36(4)：424-429。
[18]Vembu S, et al. The source-channel separation theorem revisited, IEEE Trans. Inform Theory, 1995, 41:44-54.
[19]Li Hongzhi, et al. Joint source and channel optimized block TCQ with layered transmission and RCPC. Proceeding of International Conference on Image Processing, 1998, 1:644-648.
[20]Marpe D, et al. A two-layered wavelet-based algorithm for efficient lossless and lossy image compression. IEEE Transactions on Circuits and Systems for Video Technology, 2000, 10(7): 1094-1102.
[21]Chung D, et al. Multiple description image coding using signal decomposition and reconstruction based on lapped orthogonal transforms, IEEE Trans. Circuits and Systems for Video Technology, 1999,9(6):895-908.
[22]Kim Chang-Su, et al. An error detection and recovery algorithm for compressed video signal using source level redundancy. IEEE Transactions on Image Processing, 2000, 9(2):209-219.
[23]Leonardo Chiarigline. The MPEG-4 Standard.通信学报 1998,19(9):54-67.
[24]Moccagatta I, et al. Error-resilient coding in JPEG-2000and MPEG-4. IEEE Journal on Selected Areas in Communications, 2000,18(6):899-914.
[25]Brailean J. Wireless multimedia utilizing MPEG-4 error resilient tools. Wireless Communications and Networking Conference, 1999:104-108.
[26]Qiao Daji, et al. A two-step adaptive error recovery scheme for video transmission over wireless networks. Proceeding of Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies. 2000:1698-1704.
[27]沈兰荪，卓力，田栋等．视频编码与低速率传输．电子工业出版社2001．
[28]ITU-T. Draft ITU-T Recommendation H.261. Video codec for audiovisual services at p × 64kbit/s. Jan 1993.
[29]Michele Zorzi. Performance of FEC and ARQ Error Control in Bursty Channels under Delay Constraints. Vtc'98, Ottawa, Canada, May 1998.
[30]T. Turletti, C. Huitema. Video conferencing on the Intemet. IEEE/ACM Trans. Networking, vol. 4, no. 3, pp. 340-351, June 1996.
[31] G. Cote, F. Kossentini. Optimal intra coding of blocks for robust video communication over the Internet. Image Commun., Sept. 1999, pp. 25-34.
[32] P. Haskell, et al. Resynchronization of motion compensated video affected by ATM cell loss. Prec. IEEE Int. Conf. Acoust., Speech, Signal Processing, ICASSP'92, vol. 3, San Francisco, CA, Mar. 1992, pp. 545-548.
[33] J.Y. Liao, et al. Adaptive intra update for video coding over noisy channels. Prec. Int. Conf. Image Processing, ICIP'96, Switzerland, Sept. 1996, pp. 763-766.
[34] E. Steinbach, N. Farber, B. Girod. Standard compatible extension of H.263 for robust video transmission in mobile environments. IEEE Trans. Circuits Syst. Video Technol., vol. 7, pp. 872-881, Dec. 1997.
[35] R. O. Hinds, T. N. Pappas, J. S. Lim. Joint block-based video source/channel coding for packet-switched networks. Prec. SPIE, VCIP'98, vol. 3309, San Jose, CA, Jan. 1998, pp. 124-133.
[36] S. Wenger, G. Cote. Using RFC2429 and H.263+ at low to medium bit-rates for low-latency applications, presented at the Packet Video Workshop'99, New York, NY, Apr. 1999.
[37] C. Hsu, A. Ortega, and M. Khansari. Rate control for robust video transmission over wireless channels. Prec. SPIE, VCIP'97, vol. 3024, San Jose, CA, Feb. 1997, pp. 1200-1211.
[38] A. Ortega, K. Ramchandran. Rate-distortion methods for image and video compression. IEEE Signal Processing Mag., vol. 15, pp. 23-50, Nov. 1998.
[39] G. J. Sullivan, T. Wiegand. Rate-distortion optimization for video compression. IEEE Signal Processing Mag., vol. 15, pp. 74-90, Nov. 1998.
[40] T. Wiegand, et, al. Rate-distortion optimized mode selection for very low bit rate video coding and the emerging H.263 standard. IEEE Trans. Circuits Syst. Video Technol., vol. 6, pp. 182-190, Apr. 1996.
[41] R. Zhang, S. L. Regunathan, K. Rose, Video Coding with Optimal Intra/Inter Mode Switching for Packet Loss Resilience. IEEE Journal of Selected Areas in Communications, pp. 966-976, vol. 18, no. 6, June 2000.
[42] R. Zhang, S. L. Regunathan, and K. Rose. Prescient Mode Selection for Robust Video Coding. International Conference on Image Processing, Oct. 2001, Greek.
[43] J. Choi and D. Park. A stable feedback control of the buffer state using the controlled Lagrange multiplier method. IEEE Trans. Image Processing, vol. 3, pp. 546-588, Sept. 1994.
[44] Wei Ding, et al. Rate Control of MPEG video coding and recording by rate-quantization modeling. IEEE Trans. Circuits Syst., Video Technol, 1996,6:12-19.
[45] Jordi Ribas-Corbera, Shawmin Lei. Rate Control in DCT Video Coding for Low-Delay Communications. IEEE Transactions on circuits and systems for video technology, vol. 9, pp. 172-185, Feb. 1999.
[46] Tihao Chiang, et al. A new rate control scheme using quadratic rate distortion model. IEEE Trans. On Circuits and Systems for Video Tech., 1997, 7(1):246-250
[47] 田栋，沈壮荪，段占云．一种用于视频通信的比特率控制方法．第三届全球智能控制与智能自动化大会，中国合肥，2000：2512-2516．
[48] 田栋，段占云，沈壮荪．支持ROI优先编码策略的码率控制方案．通信学报，2001，22(6)：21-26．
[49] 低速率视频传输关键技术研究：[博士学位论文]．北京：北京工业大学，2002．
[50] Thomas R G. Video codec test model, near-term, version 8(TMN8). ITU Study Croup 16, June 1997.
[51] RTP payload format for the 1998 version of ITU-T REc. H.263 Video (H.263+). Internet Draft, RFC2429.
[52] Zhihai He, Jianfei Cai,etc. Joint Source Channel Rate-Distortion Analysis for Adaptive Mode Selection and Rate Control in Wireless Video Coding. IEEE Trans. on Circuits and Syst. For Video Technol., 2002; 12(6):511-523
[2] ISO/IEC DIS 11172-2(MPEG-1). Information technology - Coding of moving pictures and associated audio for digital storage media at up to about 1.5 mbit/s - part 2: Video. 1993.
[3] ISO/IEC. Document 13818-2 (MPEG-2). Information technology - Generic coding of moving pictures and associated audio, part 2: video, recommendation H.262. 1994.
[4] Recommendation H.263. Video coding for low bit-rate communication. 1996.
[5] ITU-T Standardization Sector, SG 16, VCEG. H.26L Test Model Long-Term Number 8.
[6] 彭强，诸昌钤．视频通信差错恢复技术．计算机应用研究，2001，(7)：9-12．
[7] Villasenor J D, et al. Robust video coding algorithms and systems. Proceeding of The IEEE, 1999, 87(10): 1724-1733.
[8] Wang Yao, et al. Error resilient video coding techniques. IEEE Signal Processing Magazine, 2000, 17(4):61-82.
[9] Wang Yao, et al. Error control and concealment for video communication: a review. Proceedings of the IEEE 1998, 86(5):974-997
[10] Aravind R, et al. Packet loss resilience of MPEG-2 scalable video coding algorithms. IEEE Trans Circuits Syst. Video Technol., 1996, 6:426-435.
[11] Zhu Q F, et al. Coding and cell loss recovery for DCT-based packet video, IEEE Trans Circuits Syst. Video Technol., 1993, 3:248-258.
[12] Girod B., Farber N. Feedback-Based Error Control for Mobile Video Transmission. Proceeding of the IEEE, 1999; 87(10): 1707-1723.
[13] 李伟．基于H．263+视频传输的误差恢复技术．计算机工程与应用，2003，10：165-165．
[14] 高绍帅，涂国防．基于H．263视频解码的错误检测和掩盖。通信学报，2001，12：82-87．
[15] 梁凡，魏晓晖，肖自美等．视频压缩编码的差错恢复技术。中国图像图形学报，2000；05A(5)：374-379．
[16]卢官明．DCT编码压缩图像的传输差错控制．南京邮电学院学报，2002；2(3)：90-94．
[17]罗忠，梁柱，王兆华．IP视频通信中的自适应错误掩盖算法．天津大学学报(自然科学与工程技术版)，2003；36(4)：424-429。
[18]Vembu S, et al. The source-channel separation theorem revisited, IEEE Trans. Inform Theory, 1995, 41:44-54.
[19]Li Hongzhi, et al. Joint source and channel optimized block TCQ with layered transmission and RCPC. Proceeding of International Conference on Image Processing, 1998, 1:644-648.
[20]Marpe D, et al. A two-layered wavelet-based algorithm for efficient lossless and lossy image compression. IEEE Transactions on Circuits and Systems for Video Technology, 2000, 10(7): 1094-1102.
[21]Chung D, et al. Multiple description image coding using signal decomposition and reconstruction based on lapped orthogonal transforms, IEEE Trans. Circuits and Systems for Video Technology, 1999,9(6):895-908.
[22]Kim Chang-Su, et al. An error detection and recovery algorithm for compressed video signal using source level redundancy. IEEE Transactions on Image Processing, 2000, 9(2):209-219.
[23]Leonardo Chiarigline. The MPEG-4 Standard.通信学报 1998,19(9):54-67.
[24]Moccagatta I, et al. Error-resilient coding in JPEG-2000and MPEG-4. IEEE Journal on Selected Areas in Communications, 2000,18(6):899-914.
[25]Brailean J. Wireless multimedia utilizing MPEG-4 error resilient tools. Wireless Communications and Networking Conference, 1999:104-108.
[26]Qiao Daji, et al. A two-step adaptive error recovery scheme for video transmission over wireless networks. Proceeding of Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies. 2000:1698-1704.
[27]沈兰荪，卓力，田栋等．视频编码与低速率传输．电子工业出版社2001．
[28]ITU-T. Draft ITU-T Recommendation H.261. Video codec for audiovisual services at p × 64kbit/s. Jan 1993.
[29]Michele Zorzi. Performance of FEC and ARQ Error Control in Bursty Channels under Delay Constraints. Vtc'98, Ottawa, Canada, May 1998.
[30]T. Turletti, C. Huitema. Video conferencing on the Intemet. IEEE/ACM Trans. Networking, vol. 4, no. 3, pp. 340-351, June 1996.
[31] G. Cote, F. Kossentini. Optimal intra coding of blocks for robust video communication over the Internet. Image Commun., Sept. 1999, pp. 25-34.
[32] P. Haskell, et al. Resynchronization of motion compensated video affected by ATM cell loss. Prec. IEEE Int. Conf. Acoust., Speech, Signal Processing, ICASSP'92, vol. 3, San Francisco, CA, Mar. 1992, pp. 545-548.
[33] J.Y. Liao, et al. Adaptive intra update for video coding over noisy channels. Prec. Int. Conf. Image Processing, ICIP'96, Switzerland, Sept. 1996, pp. 763-766.
[34] E. Steinbach, N. Farber, B. Girod. Standard compatible extension of H.263 for robust video transmission in mobile environments. IEEE Trans. Circuits Syst. Video Technol., vol. 7, pp. 872-881, Dec. 1997.
[35] R. O. Hinds, T. N. Pappas, J. S. Lim. Joint block-based video source/channel coding for packet-switched networks. Prec. SPIE, VCIP'98, vol. 3309, San Jose, CA, Jan. 1998, pp. 124-133.
[36] S. Wenger, G. Cote. Using RFC2429 and H.263+ at low to medium bit-rates for low-latency applications, presented at the Packet Video Workshop'99, New York, NY, Apr. 1999.
[37] C. Hsu, A. Ortega, and M. Khansari. Rate control for robust video transmission over wireless channels. Prec. SPIE, VCIP'97, vol. 3024, San Jose, CA, Feb. 1997, pp. 1200-1211.
[38] A. Ortega, K. Ramchandran. Rate-distortion methods for image and video compression. IEEE Signal Processing Mag., vol. 15, pp. 23-50, Nov. 1998.
[39] G. J. Sullivan, T. Wiegand. Rate-distortion optimization for video compression. IEEE Signal Processing Mag., vol. 15, pp. 74-90, Nov. 1998.
[40] T. Wiegand, et, al. Rate-distortion optimized mode selection for very low bit rate video coding and the emerging H.263 standard. IEEE Trans. Circuits Syst. Video Technol., vol. 6, pp. 182-190, Apr. 1996.
[41] R. Zhang, S. L. Regunathan, K. Rose, Video Coding with Optimal Intra/Inter Mode Switching for Packet Loss Resilience. IEEE Journal of Selected Areas in Communications, pp. 966-976, vol. 18, no. 6, June 2000.
[42] R. Zhang, S. L. Regunathan, and K. Rose. Prescient Mode Selection for Robust Video Coding. International Conference on Image Processing, Oct. 2001, Greek.
[43] J. Choi and D. Park. A stable feedback control of the buffer state using the controlled Lagrange multiplier method. IEEE Trans. Image Processing, vol. 3, pp. 546-588, Sept. 1994.
[44] Wei Ding, et al. Rate Control of MPEG video coding and recording by rate-quantization modeling. IEEE Trans. Circuits Syst., Video Technol, 1996,6:12-19.
[45] Jordi Ribas-Corbera, Shawmin Lei. Rate Control in DCT Video Coding for Low-Delay Communications. IEEE Transactions on circuits and systems for video technology, vol. 9, pp. 172-185, Feb. 1999.
[46] Tihao Chiang, et al. A new rate control scheme using quadratic rate distortion model. IEEE Trans. On Circuits and Systems for Video Tech., 1997, 7(1):246-250
[47] 田栋，沈壮荪，段占云．一种用于视频通信的比特率控制方法．第三届全球智能控制与智能自动化大会，中国合肥，2000：2512-2516．
[48] 田栋，段占云，沈壮荪．支持ROI优先编码策略的码率控制方案．通信学报，2001，22(6)：21-26．
[49] 低速率视频传输关键技术研究：[博士学位论文]．北京：北京工业大学，2002．
[50] Thomas R G. Video codec test model, near-term, version 8(TMN8). ITU Study Croup 16, June 1997.
[51] RTP payload format for the 1998 version of ITU-T REc. H.263 Video (H.263+). Internet Draft, RFC2429.
[52] Zhihai He, Jianfei Cai,etc. Joint Source Channel Rate-Distortion Analysis for Adaptive Mode Selection and Rate Control in Wireless Video Coding. IEEE Trans. on Circuits and Syst. For Video Technol., 2002; 12(6):511-523