视频扫描格式转换算法设计及其芯片实现
|
摘要
随着新型显示设备、新的电视广播格式和各类多媒体设备的发展，观众需要高质量的视频格式转换技术。视频后处理芯片可以完成各种格式转换，包括去隔行、空间缩放、帧率变换等等，进一步提高观赏的效果。目前我国数字视频后处理芯片的市场基本被国外公司所垄断(Trident、PixelWorks)，研制具有自主知识产权的芯片具有非常重大的意义。
论文主要针对后处理芯片中的关键技术—去隔行，进行了深入的研究：提出了自有的运动补偿去隔行算法，该算法已经申报4项国家发明专利；自有的基于边沿的运动自适应去隔行算法，该算法的专利目前正在申报中。
论文首先介绍了隔行电视存在的若干个缺陷：对于显示类似于电影质量的图像而言，隔行电视的基本垂直分辨率是不够的；隔行电视还会产生许多视觉虚像，例如边沿闪烁、爬行和锯齿等，接着简单回顾了学术界已经提出的经典的去隔行技术。
论文提出的算法最终是面向芯片实现的，因此论文接下来主要介绍了业界具有领先水平的两款视频后处理芯片中的去隔行算法：飞利浦的基于运动补偿的方法和Faroudja的基于边沿检测的方法。论文从主观和客观两个角度对比分析了自有的运动补偿算法和飞利浦一代芯片SAA4991、二代芯片SAA4992中的运动补偿算法的性能：自有算法对于图像的水平边有效抑制了SAA4991存在的闪烁现象，减小了SAA4992存在的拖尾问题；自有的边沿算法消除了传统算法在物体边沿处存在的锯齿，可以得到清晰平滑的边沿，基本达到了DCDi算法的水平。
论文最后对今后的工作做了展望：基于边沿的图像空间缩放技术还有待进一步的评测，还有很多算法例如电影模式的检测和相应处理、画质增强算法要去设计和改善。
The combination of new displays, new TV broadcasting formats and the advent of multi-media equipment has rapidly increased the need for high quality consumer video format conversion. Digital video post-processing IC is designed for consumer video format conversion to attain high quality image, which mainly includes de-interlacing spatial conversions and picture rate conversion. At the present time, the domestic market is almost monopolized by foreign companies such as Trident PixelWorks. It is very important to develop our own Digital video post-processing IC.
This dissertation mainly focuses on the key technology in the Digital video post-processing IC-de-interlacing. In the dissertation, the author proposes our own motion compensated de-interlacing algorithm, which has been applied for four patents and our own edge-based motion adaptive de-interlacing algorithm, which is being applied for patents.
The author first introduces several problems that interlaced video suffers from: The basic vertical resolution is inadequate for displaying filmlike quality, and interlaced video generates visual artifacts, including edge flicker, shimmering, and diagonal jaggedness. Then the author reviews some representative de-interlacing algorithm that has been proposed.
As the algorithm proposed in this dissertation is designed for IC, the following part of the dissertation introduce two leading algorithm that has been implemented in IC: Philips's motion compensated de-interlacing algorithm and Faroudja's edge-based de-interlacing algorithm. In the dissertation ,the author uses objective and subjective measurement respectively to compare among Philips's 1st (C's algorithm, Philips's 2nd IC's algorithm and our own motion compensated algorithm. Simulation shows our algorithm solves the problem of horizontal edge flicker in SAA4991, alleviates the problem of "tear" in SAA4992.The author also uses subjective measurement to compare among our edge-based motion adaptive de-interlacing algorithm, traditional intra-field interpolation algorithm and Faroudja's DCDi algorithm. Simulation shows our own edge-based algorithm can remove the sawtooth on the object's edge, and has basically reach the level of DCDi.
At last, this dissertation point out some work to do in the future: the edge-based scaling technology is to be further analyzed ; film mode detection and the following processing algorithm, picture enhancement algorithm are to be improved.
论文主要针对后处理芯片中的关键技术—去隔行，进行了深入的研究：提出了自有的运动补偿去隔行算法，该算法已经申报4项国家发明专利；自有的基于边沿的运动自适应去隔行算法，该算法的专利目前正在申报中。
论文首先介绍了隔行电视存在的若干个缺陷：对于显示类似于电影质量的图像而言，隔行电视的基本垂直分辨率是不够的；隔行电视还会产生许多视觉虚像，例如边沿闪烁、爬行和锯齿等，接着简单回顾了学术界已经提出的经典的去隔行技术。
论文提出的算法最终是面向芯片实现的，因此论文接下来主要介绍了业界具有领先水平的两款视频后处理芯片中的去隔行算法：飞利浦的基于运动补偿的方法和Faroudja的基于边沿检测的方法。论文从主观和客观两个角度对比分析了自有的运动补偿算法和飞利浦一代芯片SAA4991、二代芯片SAA4992中的运动补偿算法的性能：自有算法对于图像的水平边有效抑制了SAA4991存在的闪烁现象，减小了SAA4992存在的拖尾问题；自有的边沿算法消除了传统算法在物体边沿处存在的锯齿，可以得到清晰平滑的边沿，基本达到了DCDi算法的水平。
论文最后对今后的工作做了展望：基于边沿的图像空间缩放技术还有待进一步的评测，还有很多算法例如电影模式的检测和相应处理、画质增强算法要去设计和改善。
The combination of new displays, new TV broadcasting formats and the advent of multi-media equipment has rapidly increased the need for high quality consumer video format conversion. Digital video post-processing IC is designed for consumer video format conversion to attain high quality image, which mainly includes de-interlacing spatial conversions and picture rate conversion. At the present time, the domestic market is almost monopolized by foreign companies such as Trident PixelWorks. It is very important to develop our own Digital video post-processing IC.
This dissertation mainly focuses on the key technology in the Digital video post-processing IC-de-interlacing. In the dissertation, the author proposes our own motion compensated de-interlacing algorithm, which has been applied for four patents and our own edge-based motion adaptive de-interlacing algorithm, which is being applied for patents.
The author first introduces several problems that interlaced video suffers from: The basic vertical resolution is inadequate for displaying filmlike quality, and interlaced video generates visual artifacts, including edge flicker, shimmering, and diagonal jaggedness. Then the author reviews some representative de-interlacing algorithm that has been proposed.
As the algorithm proposed in this dissertation is designed for IC, the following part of the dissertation introduce two leading algorithm that has been implemented in IC: Philips's motion compensated de-interlacing algorithm and Faroudja's edge-based de-interlacing algorithm. In the dissertation ,the author uses objective and subjective measurement respectively to compare among Philips's 1st (C's algorithm, Philips's 2nd IC's algorithm and our own motion compensated algorithm. Simulation shows our algorithm solves the problem of horizontal edge flicker in SAA4991, alleviates the problem of "tear" in SAA4992.The author also uses subjective measurement to compare among our edge-based motion adaptive de-interlacing algorithm, traditional intra-field interpolation algorithm and Faroudja's DCDi algorithm. Simulation shows our own edge-based algorithm can remove the sawtooth on the object's edge, and has basically reach the level of DCDi.
At last, this dissertation point out some work to do in the future: the edge-based scaling technology is to be further analyzed ; film mode detection and the following processing algorithm, picture enhancement algorithm are to be improved.
引文
[
[1] DURAND C X, FAGUY D. Rational zoom of bit maps using b-spline interpolation in computerized 2-D animation[J].Computer Graphics Forum,1990,9:27-37.
[2] 孙庆杰，张晓鹏，吴恩华。一种基于Bezier插值曲面的图像放大方法[J]．软件学报，1999，10(6)：570-574．
[3] UNSER M, ALDROUBI A, EDEN M. Enlargement or reduction of digital images with minimum loss of information[J]. IEEE Trans Image Process,1995,4(3):247-258.
[4] RAMPONI G.Warped distance for space-variant linear image interpolation[J].IEEE Trans Imag Proc, 1999,8(5):629-639.
[5] LEHMANN T M, GONNER C, SPITZER K. Survey: interpolation methods in medical image processing[J]. IEEE Trans Med Imag, 1999, 18(11): 1049-1075.
[6] UNSER M, ALDROUBI A, EDEN M. Fast B-spline transforms for continuous image representation and interpolation[J]. IEEE Trans Pattern Anal Machine Intell. 1991,13:277-285.
[7] S.Pigeon and P.Guillotel, Advantages and Drawbacks of Interlaced and Progressive Scanning Formats[J], HAMLET report,RACE 2110,1996.
[8] G.de Haan, J.Kettenis, and B.De Loore, IC for motion-compensated 100Hz TV with natural-motion movie-mode[J], IEEE Tr. on Cons. Electr.,May 1996,pp. 165-174.
[9] Microsoft, Broadcast-Enabled Computer Hard-ware Requirements, WinHEC'97, Broadcast Techologies White Paper, 1997,pp.11-12.
[10] J.W.Woods and Soo-chul Han, Hierarchical Motion Compensated De-interlacing[J], Proc. of the SPIE,Vol.1605,1991,pp.805-810.
[11] F.M Wang, D.Anastassiou, and A.N.Netravali, Time-Recursive Deinterlacing for IDTV and Pyramid Coding[J], Signal processing: Image Communications 2, Elsevier, 1990, pp.365-374.
[12] J.L.Yen, On Nonuniform Sampling of Bandwidth-Limited Signals[J], IRE Tr. On Circ. Th., vol.CT-3,Dec.1956,pp.251-257.
[13] P.Delogne,L.Cuvelier, B.Maison,B.Van Caillie,and L..Vandendorpe, improved Interpolation,Motion Estimation and Compensation for Interlaced Pictures[J], IEEE Tr. on Im. Proc,Vol.3,no.5,Sep.1994,pp.482-491.
[14] L.Vanderdorpe, L.Cuvelier, B.Masion, P.Quelez, and P.Delogne, Motion-compensated conversion from interlaced to progressive formats[J],Signal Processing:Image Communication 6, Elsevier, 1994,pp.193-211.
[15] E.B.Bellers and G.de Haan, Advanced deinterlacing techniques[J], Proc.of the Int. Workshop on HDTV, Oct.1996, Los Angeles,Session.A2.
[16] T.Koga et al. Motion compensated interframe coding for video conferencing[J]. Proc. Nat.Telecommunications Conf.,"1981,pp.G5.3.1-G5.3.5.
[17] J.R.Jain, A.K.Jain.Displacement measurement and its application in interframe image coding[J]. IEEE Trans.Commun., Dec.1981, vol.COMM-29, pp.1799-1808.
[18] A.Puri,H.M.Hang,,D.L.Schiling. An efficient block-matching algorithm for motion compensated coding[J].Proc.ICASSP, 1987,pp.1063-1066.
[19] M.Chanbari.The cross-search algorithm for motion estimation[J]. IEEE Trans.Commun., July 1990,vol.38, pp.950-953.
[20] K.H.K.Chow, M.L.Liou. Genetic motion search algorithm for video compression[J]. IEEE Trans.Circuits Syst.Video Technol,Dec.1993, vol.3,pp.440-445.
[21] M.Bierling. Displacement estimation by hierarchical block-matching[J].Proc.SPIE Conf. Visual Communications and Image Processing, 1988,vol.1001, pp.942-951.
[22] Y-Q.Zhang, S.Zafar. Motion-compensated wavelet transform coding for color video compression[J]. IEEE Trans.Circuits Syst.Video Technol,September 1992,vol.2,pp.285-296.
[23] K.M.Nam,J.-S.Kim,R.-H.Park et al. A fast hierarchical motion vector estimation algorithm using mean pyramid[J]. IEEE Trans. Circuits Syst.Video Technol, August 1995,vol.5,no.4, pp.344-351,
[24] G.de Haan, and P.W.A.C. Biezen, Sub-pixel motion estimation with 3-D recursive search block-matching[J], Signal Processing: Image Communication 6,1994, pp.229-239.
[25] G.De. Haan, P.W.A.C. Biezen,H.Huiijen and O.A.Ojo,Ture motion estimation with 3-D recursive serrch block-matching[J],IEEE Trans.Circuits & Systems for video Technology,Vol.3,No.5 October 1993, pp.368-379.
[26] 王维东、陈涛、张明、谢磊，视频运动估计方法[P]，03115133．7，杭州国芯科技有限公司，2003．1．
[27] E.B.Beliers and G.de Haan, Majority selection de--interlacing; An
advanced motion-compensated spatio-temporal interpolation technique for interlaced video[J], Electronic Imaging 2000, Proc. IVCP, San Hose, Vol.3974, Jan. 2000, pp.386-395.
[28]王维东、陈涛、张明、谢磊，采用错误保护运动补偿解交织的方法和设备[P]，3115134．5，杭州国芯科技有限公司，2003．1．
[29]王维东、陈涛、张明、谢磊，采用误差修正运动补偿解交织的方法和设备[P]，03115132．9，杭州国芯科技有限公司，2003．1．
[30]D.Gillies, M.Plantholt,and D.Westerkamp, Motion adaptive field rate up-conversion algorithms for 900line/100 Hz/2:1 display[J], IEEE Trans.Consumer Electronics,Vol.36,No.2,pp.149-160,May 1990.
[31]N.Suzuki, M.Kageyama, H.Yoshigi,and T.Fukinuki,Improved synthetic motion signal for perfect motion-adaptive pro-scan conversion in IDTV Recirvers[J],IEEE Trans.Consumer Electronics,Vol.35,No.3,pp.266-270,Aug 1989.
[32]M.Suyigama, et.al.,High quality digital TV with frame store processing[J],IEEE Trans. Consumer Electronics, Vol.CE-33, pp.98-107, Aug.1987.
[33]K.Kawai S, Yasuki, M.Yamada, T.Hirata, S.Makino, IDTV Receiver[J],IEEE Trans. Consumer Electronics,Vol.CE-33,No.3,pp.181-190,Aug.1987.
[34]T.Doyle and M.Looymans, Progressive Scan Conversion using Edge Information[J],Signal Processing of HDTV,Ⅱ, L.Chiariglione,Ed.,Elsevier Science Publishiers, 1990,pp.711-721.
[35]G.de Haan and E.B.Beliers, Deinterlacing--An overview[J], Proceedings of the IEEE, Vol.86, No.9, Sep.1998, pp.1839-1857.
[36]J.J.Campbell, Adaptive diagonal interpolation for image resolution enhancement[P], US-patent no.6133957,Oct.2001, Faroudja Labs.Inc.
[37]G.de Haan and R.Lodder, De-interlacing of video data using motion vectors and edge information[J],Digest of the ICCE'02,Jun.2002,pp.70-71.
[1] DURAND C X, FAGUY D. Rational zoom of bit maps using b-spline interpolation in computerized 2-D animation[J].Computer Graphics Forum,1990,9:27-37.
[2] 孙庆杰，张晓鹏，吴恩华。一种基于Bezier插值曲面的图像放大方法[J]．软件学报，1999，10(6)：570-574．
[3] UNSER M, ALDROUBI A, EDEN M. Enlargement or reduction of digital images with minimum loss of information[J]. IEEE Trans Image Process,1995,4(3):247-258.
[4] RAMPONI G.Warped distance for space-variant linear image interpolation[J].IEEE Trans Imag Proc, 1999,8(5):629-639.
[5] LEHMANN T M, GONNER C, SPITZER K. Survey: interpolation methods in medical image processing[J]. IEEE Trans Med Imag, 1999, 18(11): 1049-1075.
[6] UNSER M, ALDROUBI A, EDEN M. Fast B-spline transforms for continuous image representation and interpolation[J]. IEEE Trans Pattern Anal Machine Intell. 1991,13:277-285.
[7] S.Pigeon and P.Guillotel, Advantages and Drawbacks of Interlaced and Progressive Scanning Formats[J], HAMLET report,RACE 2110,1996.
[8] G.de Haan, J.Kettenis, and B.De Loore, IC for motion-compensated 100Hz TV with natural-motion movie-mode[J], IEEE Tr. on Cons. Electr.,May 1996,pp. 165-174.
[9] Microsoft, Broadcast-Enabled Computer Hard-ware Requirements, WinHEC'97, Broadcast Techologies White Paper, 1997,pp.11-12.
[10] J.W.Woods and Soo-chul Han, Hierarchical Motion Compensated De-interlacing[J], Proc. of the SPIE,Vol.1605,1991,pp.805-810.
[11] F.M Wang, D.Anastassiou, and A.N.Netravali, Time-Recursive Deinterlacing for IDTV and Pyramid Coding[J], Signal processing: Image Communications 2, Elsevier, 1990, pp.365-374.
[12] J.L.Yen, On Nonuniform Sampling of Bandwidth-Limited Signals[J], IRE Tr. On Circ. Th., vol.CT-3,Dec.1956,pp.251-257.
[13] P.Delogne,L.Cuvelier, B.Maison,B.Van Caillie,and L..Vandendorpe, improved Interpolation,Motion Estimation and Compensation for Interlaced Pictures[J], IEEE Tr. on Im. Proc,Vol.3,no.5,Sep.1994,pp.482-491.
[14] L.Vanderdorpe, L.Cuvelier, B.Masion, P.Quelez, and P.Delogne, Motion-compensated conversion from interlaced to progressive formats[J],Signal Processing:Image Communication 6, Elsevier, 1994,pp.193-211.
[15] E.B.Bellers and G.de Haan, Advanced deinterlacing techniques[J], Proc.of the Int. Workshop on HDTV, Oct.1996, Los Angeles,Session.A2.
[16] T.Koga et al. Motion compensated interframe coding for video conferencing[J]. Proc. Nat.Telecommunications Conf.,"1981,pp.G5.3.1-G5.3.5.
[17] J.R.Jain, A.K.Jain.Displacement measurement and its application in interframe image coding[J]. IEEE Trans.Commun., Dec.1981, vol.COMM-29, pp.1799-1808.
[18] A.Puri,H.M.Hang,,D.L.Schiling. An efficient block-matching algorithm for motion compensated coding[J].Proc.ICASSP, 1987,pp.1063-1066.
[19] M.Chanbari.The cross-search algorithm for motion estimation[J]. IEEE Trans.Commun., July 1990,vol.38, pp.950-953.
[20] K.H.K.Chow, M.L.Liou. Genetic motion search algorithm for video compression[J]. IEEE Trans.Circuits Syst.Video Technol,Dec.1993, vol.3,pp.440-445.
[21] M.Bierling. Displacement estimation by hierarchical block-matching[J].Proc.SPIE Conf. Visual Communications and Image Processing, 1988,vol.1001, pp.942-951.
[22] Y-Q.Zhang, S.Zafar. Motion-compensated wavelet transform coding for color video compression[J]. IEEE Trans.Circuits Syst.Video Technol,September 1992,vol.2,pp.285-296.
[23] K.M.Nam,J.-S.Kim,R.-H.Park et al. A fast hierarchical motion vector estimation algorithm using mean pyramid[J]. IEEE Trans. Circuits Syst.Video Technol, August 1995,vol.5,no.4, pp.344-351,
[24] G.de Haan, and P.W.A.C. Biezen, Sub-pixel motion estimation with 3-D recursive search block-matching[J], Signal Processing: Image Communication 6,1994, pp.229-239.
[25] G.De. Haan, P.W.A.C. Biezen,H.Huiijen and O.A.Ojo,Ture motion estimation with 3-D recursive serrch block-matching[J],IEEE Trans.Circuits & Systems for video Technology,Vol.3,No.5 October 1993, pp.368-379.
[26] 王维东、陈涛、张明、谢磊，视频运动估计方法[P]，03115133．7，杭州国芯科技有限公司，2003．1．
[27] E.B.Beliers and G.de Haan, Majority selection de--interlacing; An
advanced motion-compensated spatio-temporal interpolation technique for interlaced video[J], Electronic Imaging 2000, Proc. IVCP, San Hose, Vol.3974, Jan. 2000, pp.386-395.
[28]王维东、陈涛、张明、谢磊，采用错误保护运动补偿解交织的方法和设备[P]，3115134．5，杭州国芯科技有限公司，2003．1．
[29]王维东、陈涛、张明、谢磊，采用误差修正运动补偿解交织的方法和设备[P]，03115132．9，杭州国芯科技有限公司，2003．1．
[30]D.Gillies, M.Plantholt,and D.Westerkamp, Motion adaptive field rate up-conversion algorithms for 900line/100 Hz/2:1 display[J], IEEE Trans.Consumer Electronics,Vol.36,No.2,pp.149-160,May 1990.
[31]N.Suzuki, M.Kageyama, H.Yoshigi,and T.Fukinuki,Improved synthetic motion signal for perfect motion-adaptive pro-scan conversion in IDTV Recirvers[J],IEEE Trans.Consumer Electronics,Vol.35,No.3,pp.266-270,Aug 1989.
[32]M.Suyigama, et.al.,High quality digital TV with frame store processing[J],IEEE Trans. Consumer Electronics, Vol.CE-33, pp.98-107, Aug.1987.
[33]K.Kawai S, Yasuki, M.Yamada, T.Hirata, S.Makino, IDTV Receiver[J],IEEE Trans. Consumer Electronics,Vol.CE-33,No.3,pp.181-190,Aug.1987.
[34]T.Doyle and M.Looymans, Progressive Scan Conversion using Edge Information[J],Signal Processing of HDTV,Ⅱ, L.Chiariglione,Ed.,Elsevier Science Publishiers, 1990,pp.711-721.
[35]G.de Haan and E.B.Beliers, Deinterlacing--An overview[J], Proceedings of the IEEE, Vol.86, No.9, Sep.1998, pp.1839-1857.
[36]J.J.Campbell, Adaptive diagonal interpolation for image resolution enhancement[P], US-patent no.6133957,Oct.2001, Faroudja Labs.Inc.
[37]G.de Haan and R.Lodder, De-interlacing of video data using motion vectors and edge information[J],Digest of the ICCE'02,Jun.2002,pp.70-71.