视频去块滤波算法的研究及其实现
|
摘要
目前大多数的视频标准例如H.264、MPEG-4、AVS都是采用基于分块的离散余弦变换(BDCT)的压缩编码方式。这是一种将视频图像分成块并独立处理的方法,其最大的缺点就是在比特率不太高的情况下会出现方块效应,因此,在视频系统中去块滤波技术就显得十分重要。
去块滤波技术主要分为环路滤波和后处理滤波。H.264和AVS都定义了环路去块滤波算法,可以显著地提高运动估计的质量,但是其并没有太多地考虑到主观视觉效果,因此,即使在编解码系统中包含环路去块滤波器,还是需要在接收端添加后处理去块滤波器以提高最终的主观视觉效果。另一方面,和环路滤波相比,后处理去块滤波更具有广泛性和通用性。
本课题的主要研究方向是视频去块滤波算法的研究与实现。首先介绍了方块效应产生的原因、去块滤波器的分类和人类视觉系统。接着介绍了当前主流的环路去块滤波算法和后处理去块滤波算法。然后,根据人类视觉系统的特点,对现有后处理去块滤波算法中的模式、参数以及区域判别方法作了系统的研究及大量的实验,以改善主观视觉效果及降低实现复杂度为目的,对现有算法进行了一定的改进。利用Matlab对滤波结果进行考察,结果表明,本文所提出的改进算法同样具有较好的滤波效果,并且提供了降低设计方案实现成本的理论基础。
之后,本文对改进后的算法进行了RTL级硬件实现并使用0.18um TSMC的工艺库进行综合。结果显示,该算法和硬件结构能够在使用较少存储单元的情况下,仍然达到视频实时处理的速度要求。最后,针对硬件加速器在灵活性上的不足,运用MOVE处理器,利用其配置性强和扩展性高的特点,在确保基本达到性能指标的前提下实现了算法的灵活移植。
Most of the current video encoding and decoding compression standards such as H.264, MPEG-4 and AVS are based on BDCT (Blocked Discrete Cosine Transformation). However, this is a method that dividing the image into blocks and independently processing each block. Its biggest disadvantage is that it will cause blocking effect in low bit rate. Thus it is necessary to include deblocking filter in the whole video system.
Deblocking filter can be divided into two types: in-loop filter and post filter. H.264 and AVS have both defined the in-loop deblocking algorithm to increase the quality of motion estimate. However, it doesn’t concern much about subjective visual effect. Thus post deblocking filter should be added to video systems to increase subjective visual effect even if in-loop filter is included. On the other hand, compared with in-loop filter, post filter is more general and has been widely used. This thesis mainly focuses on the research and implementation of deblocking algorithm in video processing system. The paper first introduces the generation of blocking effect, the classification of deblocking filter and human visual system. Then in-loop and post filter is introduced in detail.
Based on the feature of human visual system, the paper does research and many experiences on the mode, parameter and mode decision method of current post deblocking algorithm to make some improvements. All these are aimed at improving subjective visual effect and decreasing complexity in realization. Using Matlab to observe the result and it shows that the improved algorithm also has good effect while providing the academic basis of decreasing the cost of implementation.
Based on the research, the proposed algorithm is implemented in the paper by hardware accelerator in RTL level and synthesizes it using 0.18um TSMC technical library. It is shown in the results that the proposed algorithm and hardware architecture can meet the real-time processing requirement using less memory. Since the hardware accelerator is not flexible, the paper using MOVE processor to realize the algorithm and making advantage of its high compatibility and strong scalability. By doing this, its flexibility is improved while ensuring the speed and performance.
去块滤波技术主要分为环路滤波和后处理滤波。H.264和AVS都定义了环路去块滤波算法,可以显著地提高运动估计的质量,但是其并没有太多地考虑到主观视觉效果,因此,即使在编解码系统中包含环路去块滤波器,还是需要在接收端添加后处理去块滤波器以提高最终的主观视觉效果。另一方面,和环路滤波相比,后处理去块滤波更具有广泛性和通用性。
本课题的主要研究方向是视频去块滤波算法的研究与实现。首先介绍了方块效应产生的原因、去块滤波器的分类和人类视觉系统。接着介绍了当前主流的环路去块滤波算法和后处理去块滤波算法。然后,根据人类视觉系统的特点,对现有后处理去块滤波算法中的模式、参数以及区域判别方法作了系统的研究及大量的实验,以改善主观视觉效果及降低实现复杂度为目的,对现有算法进行了一定的改进。利用Matlab对滤波结果进行考察,结果表明,本文所提出的改进算法同样具有较好的滤波效果,并且提供了降低设计方案实现成本的理论基础。
之后,本文对改进后的算法进行了RTL级硬件实现并使用0.18um TSMC的工艺库进行综合。结果显示,该算法和硬件结构能够在使用较少存储单元的情况下,仍然达到视频实时处理的速度要求。最后,针对硬件加速器在灵活性上的不足,运用MOVE处理器,利用其配置性强和扩展性高的特点,在确保基本达到性能指标的前提下实现了算法的灵活移植。
Most of the current video encoding and decoding compression standards such as H.264, MPEG-4 and AVS are based on BDCT (Blocked Discrete Cosine Transformation). However, this is a method that dividing the image into blocks and independently processing each block. Its biggest disadvantage is that it will cause blocking effect in low bit rate. Thus it is necessary to include deblocking filter in the whole video system.
Deblocking filter can be divided into two types: in-loop filter and post filter. H.264 and AVS have both defined the in-loop deblocking algorithm to increase the quality of motion estimate. However, it doesn’t concern much about subjective visual effect. Thus post deblocking filter should be added to video systems to increase subjective visual effect even if in-loop filter is included. On the other hand, compared with in-loop filter, post filter is more general and has been widely used. This thesis mainly focuses on the research and implementation of deblocking algorithm in video processing system. The paper first introduces the generation of blocking effect, the classification of deblocking filter and human visual system. Then in-loop and post filter is introduced in detail.
Based on the feature of human visual system, the paper does research and many experiences on the mode, parameter and mode decision method of current post deblocking algorithm to make some improvements. All these are aimed at improving subjective visual effect and decreasing complexity in realization. Using Matlab to observe the result and it shows that the improved algorithm also has good effect while providing the academic basis of decreasing the cost of implementation.
Based on the research, the proposed algorithm is implemented in the paper by hardware accelerator in RTL level and synthesizes it using 0.18um TSMC technical library. It is shown in the results that the proposed algorithm and hardware architecture can meet the real-time processing requirement using less memory. Since the hardware accelerator is not flexible, the paper using MOVE processor to realize the algorithm and making advantage of its high compatibility and strong scalability. By doing this, its flexibility is improved while ensuring the speed and performance.
引文
[1] MPEG-4 Video Group,ISO/IEC 14496-2,“Generic Coding of Audio----Visual Objects”,Part 2:Video,1998
[2] H.261,CCITT Rec,“Video Codec for Audio Visual Services at p x 64kbitsls”, 1993
[3] H.263,ITU-T Rec,“Video Coding for Low Bit Rate Communication”,1996
[4] H.264,ITU-T Rec. ISO/IEC 11496-10,“Advanced Video Coding for generic audiovisual services”,March 2005
[5] AVS 文档,AVS 视频专题组,信息技术----先进音视频编码,第 2 部分:视频,AVS-N1063,2003 年 12 月,第 7 部分:移动视频,AVS-N1208 2005 年 9 月
[6] Yao Wang,Jorn Ostermann,Ya-Qin Zhang 著,侯正信,杨喜,王文全等译,《视频处理与通信》,北京,电子工业出版社,2003 年 6 月,pp169-171
[7] 易峰,视频编码中去块滤波技术的原理与算法研究,[学位论文],浙江大学,2006 年 2 月
[8] 毕厚杰,《新一代视频压缩编码标准——H.264/AVC》,北京,人民邮电出版社,2005 年 5 月,pp138-144
[9] 魏崇奎,基于 HVS 图像质量模型的方法研究,[学位论文],国防科学技术大学,2003 年 11 月
[10] 黎洪松,《数字视频处理》,北京,北京邮电大学出版社,2006 年 8 月,pp4-7,pp40-41
[11] 陈薇薇,基于人类视觉系统生理特性的模型及应用研究,[学位论文],国防科学技术大学,2004 年 11 月
[12] 王楠楠,李桂苓,符合人眼视觉特性的视频质量评价模型,中国图象图形学报,2001 年 6 月,第 6 卷(A 版),第 6 期,pp523-527
[13] 佟雨兵,胡薇薇,杨东凯,张其善,视频质量评价方法综述,计算机辅助设计与图形学学报,2006 年 5 月,第 18 卷,第 5 期,pp735-741
[14] 李永强,沈庆国,朱江,汪莉,数字视频质量评价方法综述,电视技术 2006年第 6 期,pp74-82
[15] 张婉,去块效应滤波器的结构设计,设计应用,2005 年 8 月,pp70-73
[16] Peter List,Anthony Joch,Jani Lainema,Gisle Bjontegaard,and Marta Karczewicz,“Adaptive Deblocking Filter”,IEEE Transactions on circuits and systemsfor video technology,July 2003,Vol.13,No.7,pp614-619
[17] K. Ramkishor,Pravin Karandikar,“A Simple and Efficient Deblocking Algorithm for Low Bit-Rate Video Coding”,http://citeseer.ist.psu.edu/
[18] S.-C. Tai,Y.-R. Chen,C.-Y. Chen and Y.-H. Chen,“Low complexity deblocking method for DCT coded video signals”,IEE Proc.-Vis. Image Signal Process,February 2006,Vol.153,No.1,pp46-56
[19] Sung Deuk Kim,Jaeyoun Yi,Hyun Mun Kim and Jong Beom Ra,“A Deblocking Filter With Two Separate Modes In Block-Based Video Coding”,IEEE Transactions on Circuits and Systems for Video Technology,Feb 1999,Vol.9,No.1, pp.156-160
[20] Kee-Koo Kwon,Sung-Ho Im,Dong-Sun Lim,“Deblocking Algorithm in MPEG-4 Video Coding Using Block Boundary Characteristics and Adaptive Filtering”, Image Processing,IEEE International Conference,Sept. 2005,Volume 3,pp11-14
[21] Thomas P. O’Rourke and Robert L. Stevenson , “ Improved Image Decompression for Reduced Transform Coding Artifacts”,IEEE Trans. on Circuits and Systems for Video Technology,Dec. 1995,Vol.5,pp. 490-499
[22] Shengli Xie,Zhiliang Xu,“An Adaptive De-blocking Algorithm Based on MRF”,In Proceedings of the 17th IEEE International Conference on Tools with Artificial Intelligence (ICTAI’05),14-16 Nov,2005,pp2-3
[23] Ju Jia Zou,Hong Yan,“A Deblocking Method for BDCT Compressed Images Based on Adaptive Projections”,IEEE Transactions on circuits and systems for video technology,2005,15 (3),pp430-435
[24] Yu-Wen Huang,To-Wei Chen,Bing-Yu Hsieh,Tu-Chih Wang,Te-Hao Chang, and Liang-Gee Chen,“Architecture Design for De-blocking Filter in H.264/JVT/AVC” Proc. IEEE Intl. Conf. On Multimedia and Expo,July 2003,Vol.1,pp. 693-696
[25] C. C. Cheng and T. S. Chang,“An Hardware Efficient Deblocking Filter for H.264/AVC”,IEEE Consumer Electronics,2005. ICCE. 2005 Digest of Technical Papers. International Conference on 8-12 Jan. 2005,pp.235- 236
[26] Bin Sheng,Wen Gao and Di Wu,“An Implemented Architecture of Deblocking Filter for H.264/AVC”,International Conference on Image Processing (ICIP),2004,pp. 665-668
[27] 黄琼珍,郑世宝,杨宇红,H.264/AVC 中去块效应环路滤波的 VLSI 实现,微电子学与计算机,2006,23(4),pp133-136
[28] Henk Corporaal and Hans (J.M.) Mulder. “MOVE:A framework for high-performance processor design”,In Supercomputing-91,Albuquerque,November 1991,pp 692-701
[29] Henk Corporaal,IMEC,Leuven,Belgium/Eindhoven Univ. Tech.,Eindhoven,the Netherlands,“Transport triggering:Principles and consequences”,Workshop on Transport Triggered Architectures,November 25-26 2002 at Nokia Research Center,Visiokatu 1,Tampere,Finland
[30] H.Corporaal,Microprocessor Architectures:From VLIW to TTA. Chichester,UK:John Wiley & Sons,1997
[31] Henk Corporaal and Paul van der Arend,“MOVE32INT,a Sea of Gates realization of a high performance Transport Triggered Architecture”,Microprocessing and Microprogramming,September 1993,Vol.38,pp53-60
[32] J. Heikkinen,J. Takala and J. Sertamo,“Code compression on transport triggered architectures”, accepted to Int. Workshop on System-on-Chip for Real-Time Applications,Banff,Canada,July 6–7 2002
[33] J.Janssen,“Compilation strategies for transport triggered architectures”, [Dissertation],Delft University of Technology,Delft,The Netherlands,Sep.2001
[34] Jaakko Sertamo,“Processor Generator for Transport Triggered Architectures”,[Dissertation],Tampere University of Technology,Aug,2003
[35] M.J.Q. van der Lem,“Implementation of realtime video compression,conform the MPEG standard,using the TTA architecture”,[Dissertation],Delft University of Technology,Delft,The Netherlands,October 1996
[2] H.261,CCITT Rec,“Video Codec for Audio Visual Services at p x 64kbitsls”, 1993
[3] H.263,ITU-T Rec,“Video Coding for Low Bit Rate Communication”,1996
[4] H.264,ITU-T Rec. ISO/IEC 11496-10,“Advanced Video Coding for generic audiovisual services”,March 2005
[5] AVS 文档,AVS 视频专题组,信息技术----先进音视频编码,第 2 部分:视频,AVS-N1063,2003 年 12 月,第 7 部分:移动视频,AVS-N1208 2005 年 9 月
[6] Yao Wang,Jorn Ostermann,Ya-Qin Zhang 著,侯正信,杨喜,王文全等译,《视频处理与通信》,北京,电子工业出版社,2003 年 6 月,pp169-171
[7] 易峰,视频编码中去块滤波技术的原理与算法研究,[学位论文],浙江大学,2006 年 2 月
[8] 毕厚杰,《新一代视频压缩编码标准——H.264/AVC》,北京,人民邮电出版社,2005 年 5 月,pp138-144
[9] 魏崇奎,基于 HVS 图像质量模型的方法研究,[学位论文],国防科学技术大学,2003 年 11 月
[10] 黎洪松,《数字视频处理》,北京,北京邮电大学出版社,2006 年 8 月,pp4-7,pp40-41
[11] 陈薇薇,基于人类视觉系统生理特性的模型及应用研究,[学位论文],国防科学技术大学,2004 年 11 月
[12] 王楠楠,李桂苓,符合人眼视觉特性的视频质量评价模型,中国图象图形学报,2001 年 6 月,第 6 卷(A 版),第 6 期,pp523-527
[13] 佟雨兵,胡薇薇,杨东凯,张其善,视频质量评价方法综述,计算机辅助设计与图形学学报,2006 年 5 月,第 18 卷,第 5 期,pp735-741
[14] 李永强,沈庆国,朱江,汪莉,数字视频质量评价方法综述,电视技术 2006年第 6 期,pp74-82
[15] 张婉,去块效应滤波器的结构设计,设计应用,2005 年 8 月,pp70-73
[16] Peter List,Anthony Joch,Jani Lainema,Gisle Bjontegaard,and Marta Karczewicz,“Adaptive Deblocking Filter”,IEEE Transactions on circuits and systemsfor video technology,July 2003,Vol.13,No.7,pp614-619
[17] K. Ramkishor,Pravin Karandikar,“A Simple and Efficient Deblocking Algorithm for Low Bit-Rate Video Coding”,http://citeseer.ist.psu.edu/
[18] S.-C. Tai,Y.-R. Chen,C.-Y. Chen and Y.-H. Chen,“Low complexity deblocking method for DCT coded video signals”,IEE Proc.-Vis. Image Signal Process,February 2006,Vol.153,No.1,pp46-56
[19] Sung Deuk Kim,Jaeyoun Yi,Hyun Mun Kim and Jong Beom Ra,“A Deblocking Filter With Two Separate Modes In Block-Based Video Coding”,IEEE Transactions on Circuits and Systems for Video Technology,Feb 1999,Vol.9,No.1, pp.156-160
[20] Kee-Koo Kwon,Sung-Ho Im,Dong-Sun Lim,“Deblocking Algorithm in MPEG-4 Video Coding Using Block Boundary Characteristics and Adaptive Filtering”, Image Processing,IEEE International Conference,Sept. 2005,Volume 3,pp11-14
[21] Thomas P. O’Rourke and Robert L. Stevenson , “ Improved Image Decompression for Reduced Transform Coding Artifacts”,IEEE Trans. on Circuits and Systems for Video Technology,Dec. 1995,Vol.5,pp. 490-499
[22] Shengli Xie,Zhiliang Xu,“An Adaptive De-blocking Algorithm Based on MRF”,In Proceedings of the 17th IEEE International Conference on Tools with Artificial Intelligence (ICTAI’05),14-16 Nov,2005,pp2-3
[23] Ju Jia Zou,Hong Yan,“A Deblocking Method for BDCT Compressed Images Based on Adaptive Projections”,IEEE Transactions on circuits and systems for video technology,2005,15 (3),pp430-435
[24] Yu-Wen Huang,To-Wei Chen,Bing-Yu Hsieh,Tu-Chih Wang,Te-Hao Chang, and Liang-Gee Chen,“Architecture Design for De-blocking Filter in H.264/JVT/AVC” Proc. IEEE Intl. Conf. On Multimedia and Expo,July 2003,Vol.1,pp. 693-696
[25] C. C. Cheng and T. S. Chang,“An Hardware Efficient Deblocking Filter for H.264/AVC”,IEEE Consumer Electronics,2005. ICCE. 2005 Digest of Technical Papers. International Conference on 8-12 Jan. 2005,pp.235- 236
[26] Bin Sheng,Wen Gao and Di Wu,“An Implemented Architecture of Deblocking Filter for H.264/AVC”,International Conference on Image Processing (ICIP),2004,pp. 665-668
[27] 黄琼珍,郑世宝,杨宇红,H.264/AVC 中去块效应环路滤波的 VLSI 实现,微电子学与计算机,2006,23(4),pp133-136
[28] Henk Corporaal and Hans (J.M.) Mulder. “MOVE:A framework for high-performance processor design”,In Supercomputing-91,Albuquerque,November 1991,pp 692-701
[29] Henk Corporaal,IMEC,Leuven,Belgium/Eindhoven Univ. Tech.,Eindhoven,the Netherlands,“Transport triggering:Principles and consequences”,Workshop on Transport Triggered Architectures,November 25-26 2002 at Nokia Research Center,Visiokatu 1,Tampere,Finland
[30] H.Corporaal,Microprocessor Architectures:From VLIW to TTA. Chichester,UK:John Wiley & Sons,1997
[31] Henk Corporaal and Paul van der Arend,“MOVE32INT,a Sea of Gates realization of a high performance Transport Triggered Architecture”,Microprocessing and Microprogramming,September 1993,Vol.38,pp53-60
[32] J. Heikkinen,J. Takala and J. Sertamo,“Code compression on transport triggered architectures”, accepted to Int. Workshop on System-on-Chip for Real-Time Applications,Banff,Canada,July 6–7 2002
[33] J.Janssen,“Compilation strategies for transport triggered architectures”, [Dissertation],Delft University of Technology,Delft,The Netherlands,Sep.2001
[34] Jaakko Sertamo,“Processor Generator for Transport Triggered Architectures”,[Dissertation],Tampere University of Technology,Aug,2003
[35] M.J.Q. van der Lem,“Implementation of realtime video compression,conform the MPEG standard,using the TTA architecture”,[Dissertation],Delft University of Technology,Delft,The Netherlands,October 1996