基于FFmpeg的H.264解码器实现
摘要
FFmpeg是一个应用于音频和视频领域的开源程序,由于它开源、免费和跨平台的特点,备受开发者的青睐。它支持超过90种解码器和协议,例如H.261、H.263、H.264等。FFmpeg由C实现,不仅适用于PC软件,还应用于各种嵌入式设备。它为记录、转换和流化音频、视频提供了一个完整的解决方案,包括了先进的音频、视频编解码器库libavcodec,所有这些特性保证了可移植性和编解码器质量。
H.264／MPEG-4第十部分或视频压缩标准A VC (Advanced Video Coding),是目前最为常用的录制、压缩和分配高清晰度视频的标准之一。它是由国际电信联盟远程通信标准化组织(ITU-T)的视频编码专家组(VCEG)与国际标准化组织(ISO)和国际电工委员会(IEC)的运动图像专家组(MPEG)共同开发的一个面向块的基于运动补偿的编解码标准,是两个组织共同合作的产物。H.264被公认为是最好的蓝光光盘的编解码标准之一,所有蓝光光盘播放机必须能够支持H.264解码。它还被广泛的用于流媒体网络资源,如VIMEO, YouTube和iTunes Store中的视频,网络软件如Adobe Flash Player和Microsoft Silverlight, DVB和SBTVD的广播服务,直接广播卫星电视服务,有线广播服务,电视服务,实时视频会议。作为最新的视频压缩标准,在压缩和网络传输质量上H.264/AVC相比以往的标准向前迈进了一大步。
本论文首先介绍了H.264视频标准的应用,解码的关键技术和不同于以往视频编码标准的新特点,然后介绍FFmpeg的编译使用和其中主要的结构体和函数,在此基础上,重点对FFmpeg解码流程和FFmpeg中H.264解码器解码流程进行分析。
在对以上问题进行系统的分析以后,提出一种从FFmpeg中提取特定功能代码的方法,并利用此方法成功提取出H.264解码器,经过测试,提取出的H.264解码器与FFmpeg解码结果相同,抽取出的解码器代码量相比FFmpeg代码显著降低,总文件数由899个降至60个,代码行数由302327行降至26576行。为工程中H.264解码器的使用提供了便捷之选。
FFmpeg is an open source used in the field of audio and video, because of its open source, free and cross-platform features, it is favored by the developers. It supports over 90 codecs and protocols, such as H.261, H.263, H.264 and so on. It is realized by C, and being incorporated in not only PC software, but also a variety of embedded devices. It provides a complete solution to record, transcode and stream audio/video and includes advanced audio and video codec library libavcodec, all of which guarantee the transplantation and codec quality.
H.264/MPEG-4 Part 10 or AVC (Advanced Video Coding) is a standard for video compression, and is currently one of the most commonly used formats for recording, compression, and distribution of high definition video. H.264/MPEG-4 AVC is a block-oriented motion-compensation-based codec standard developed by the ITU-T Video Coding Experts Group (VCEG) together with the International Organization for Standardization (ISO)/International Electrotechnical Commission (IEC) Moving Picture Experts Group (MPEG). It was the product of a partnership effort known as the Joint Video Team (JVT). H.264 is perhaps best known as being one of the codec standards for Blu-ray Discs; all Blu-ray Disc players must be able to decode H.264. It is also widely used by streaming internet sources, such as videos from Vimeo, YouTube, and the iTunes Store, web software such as the Adobe Flash Player and Microsoft Silverlight, broadcast services for DVB and SBTVD, direct-broadcast satellite television services, cable television services, and real-time videoconferencing.
This paper first describes the H.264 video standard applications, the decoding key technologies and new features which are different from the other standards, and then introduced the compilation and usages of FFmpeg, the major structures and functions of FFmpeg, on this basis, focusing on the FFmpeg decoder process and FFmpeg H.264 decoder in thedecoding process analysis.
After a systematic analysis of these questions, propose a method to extract specific function from the FFmpeg code, and using this method successfully extracted theH.264 decoder. Tests showed that, the H.264 decoder extracted from FFmpeg had the same result as FFmpeg decoding. The amount of code compared to FFmpeg code is significantly reduced.
H.264／MPEG-4第十部分或视频压缩标准A VC (Advanced Video Coding),是目前最为常用的录制、压缩和分配高清晰度视频的标准之一。它是由国际电信联盟远程通信标准化组织(ITU-T)的视频编码专家组(VCEG)与国际标准化组织(ISO)和国际电工委员会(IEC)的运动图像专家组(MPEG)共同开发的一个面向块的基于运动补偿的编解码标准,是两个组织共同合作的产物。H.264被公认为是最好的蓝光光盘的编解码标准之一,所有蓝光光盘播放机必须能够支持H.264解码。它还被广泛的用于流媒体网络资源,如VIMEO, YouTube和iTunes Store中的视频,网络软件如Adobe Flash Player和Microsoft Silverlight, DVB和SBTVD的广播服务,直接广播卫星电视服务,有线广播服务,电视服务,实时视频会议。作为最新的视频压缩标准,在压缩和网络传输质量上H.264/AVC相比以往的标准向前迈进了一大步。
本论文首先介绍了H.264视频标准的应用,解码的关键技术和不同于以往视频编码标准的新特点,然后介绍FFmpeg的编译使用和其中主要的结构体和函数,在此基础上,重点对FFmpeg解码流程和FFmpeg中H.264解码器解码流程进行分析。
在对以上问题进行系统的分析以后,提出一种从FFmpeg中提取特定功能代码的方法,并利用此方法成功提取出H.264解码器,经过测试,提取出的H.264解码器与FFmpeg解码结果相同,抽取出的解码器代码量相比FFmpeg代码显著降低,总文件数由899个降至60个,代码行数由302327行降至26576行。为工程中H.264解码器的使用提供了便捷之选。
FFmpeg is an open source used in the field of audio and video, because of its open source, free and cross-platform features, it is favored by the developers. It supports over 90 codecs and protocols, such as H.261, H.263, H.264 and so on. It is realized by C, and being incorporated in not only PC software, but also a variety of embedded devices. It provides a complete solution to record, transcode and stream audio/video and includes advanced audio and video codec library libavcodec, all of which guarantee the transplantation and codec quality.
H.264/MPEG-4 Part 10 or AVC (Advanced Video Coding) is a standard for video compression, and is currently one of the most commonly used formats for recording, compression, and distribution of high definition video. H.264/MPEG-4 AVC is a block-oriented motion-compensation-based codec standard developed by the ITU-T Video Coding Experts Group (VCEG) together with the International Organization for Standardization (ISO)/International Electrotechnical Commission (IEC) Moving Picture Experts Group (MPEG). It was the product of a partnership effort known as the Joint Video Team (JVT). H.264 is perhaps best known as being one of the codec standards for Blu-ray Discs; all Blu-ray Disc players must be able to decode H.264. It is also widely used by streaming internet sources, such as videos from Vimeo, YouTube, and the iTunes Store, web software such as the Adobe Flash Player and Microsoft Silverlight, broadcast services for DVB and SBTVD, direct-broadcast satellite television services, cable television services, and real-time videoconferencing.
This paper first describes the H.264 video standard applications, the decoding key technologies and new features which are different from the other standards, and then introduced the compilation and usages of FFmpeg, the major structures and functions of FFmpeg, on this basis, focusing on the FFmpeg decoder process and FFmpeg H.264 decoder in thedecoding process analysis.
After a systematic analysis of these questions, propose a method to extract specific function from the FFmpeg code, and using this method successfully extracted theH.264 decoder. Tests showed that, the H.264 decoder extracted from FFmpeg had the same result as FFmpeg decoding. The amount of code compared to FFmpeg code is significantly reduced.
引文
[1]Wikipedia. FFmpeg[EB/OL]. http://en.wikipedia.org/wiki/FFmpeg.
[2]Wikipedia. H.264/MPEG-4 AVC[EB/OL]. http://en.wikipedia. org/wiki/H.264/MPEG-4_AVC.
[3]郭晓强,解伟.H.264/AVC视频压缩编码标准的新发展[EB/OL].http://video.coal.cn.
[4]刘马飞,曾学文,倪宏Windows平台下应用FFMPEG实现H.264视频回放[J].微计算机应用,2008,29(11)：61-65.
[5]姚莹琳.H.264大尺寸解码器的实现与优化[D].北京：北京邮电大学2006.
[6]林炜.基于DSP的H.264运动估计算法的研究与实现[D].成都：电子科技大学,2007.
[7]MAJOR A, YI Y, NOUSIAS I, et al. H.264 Decoder Implementation on a Dynamically Reconfigurable Instruction Cell Based Architecture. IEEE International SOC Conference, Austin,2006:49-52.
[8]皇甫泽毅.H.264解码在嵌入式Linux上的优化实践[D].武汉：华中科技大学,2006.
[9]李世平.H.264开源解码器评测[EB/OL]. http://blog. csdn. negsunshinel314/archive.
[10]JVT (Joint Video Team of ITU-T and ISO/IEC). Recommendation and final draft international standard of joint video specification (ITU-T Rec. H.264/ISO/IEC 14496-10 AVC)[S],2003.
[11]欧阳合.H.264和MPEG-4视频压缩：新一代多媒体的视频编码技术[M].北京：国防科技大学出版社,2004.
[12]席道光.基于H.264/AVC标准的视频编码系统应用研究[D].浙江：浙江工业大学,2007。
[13]SETH K, KAMAKOTI V, SRINIVASAN S. Efficient motion vector recovery algorithm for H.264 using directional interpolation [J]. Image Processing, IET, 2010,4(2):132-141.
[14]毕厚杰.新一代视频压缩编码标准——H.264/AVC[M].北京：人民邮电出版社,2005.
[15]KIM W J, CHO K, CHUNG K S, et al. Stage-based frame-partitioned parallelization of H.264/AVC decoding [J].IEEE Transactions on Consumer Electronics,2010,56(2):1088-1096.
[16]PESCADOR F, SANZ C, GARRIDO M J, et al. A DSP Based H.264 decoder for a multi-format IP set-top box [J]. IEEE Transactions on Consumer Electronics,2008,54(1):145-153.
[17]OSTERMANN J, BORMANS J, LIST P, et al. Video coding with H.264/AVC:tools, performance, and complexity [J].Circuits and Systems Magazine, IEEE,2004,4(1):7-28.
[18]WIEGAND T, SULLIVAN G J, BJONTEGAARD G, et al. Overview of the H.264/AVC video coding standard [J]. IEEE Transactions on Circuits and Systems for Video Technology,1993,13(7):560-576.
[19]PENG Z, DON X, WEN G. Variable-bin-rate CABAC engine for H.264/AVC high definition real-time decoding [J]. IEEE Transactions on Very Large Scale Integration (VLSI) Systems,2009,17(3):417-426.
[20]杜建超.基于H.264/AVC的视频容错技术研究[M].西安：西安电子科技大学,2006.
[21]唐玲娜.H.264视频解码优化及DSP实现[D].成都：电子科技大学,2009.
[22]ZHENG J, CHAU L P. Error-resilient coding of H.264 based on periodic macroblock [J]. IEEE Transactions on Broadcasting,2006,52(2):223-229.
[23]LIU Z, FENG L L, YANG S, et al. Motion feature and hadamard coefficient-based fast multiple reference frame motion estimation for H.264 [J]. IEEE Transactions on Circuits and Systems for Video Technology,2008,18(5):620-632.
[24]刘鹏飞,刘志,安平,张兆杨.H.264/AVC解码器优化的研究[J].图像图形学报,2006(11)：1627-1630.
[25]郭宝龙,倪伟,闰允一.通信中的视频信号处理[M].北京：电子工业出版社2007.
[26]SUDENG H, HAN L W, SAM K W, et al. Frame level rate control for H.264/AVC with novel Rate-Quantization model.2010 IEEE International Conference on,2010:226-231.
[27]KUO T Y, CHAN C H. Fast variable block size motion estimation for H.264 using likelihood and correlation of motion field [J]. IEEE Transactions on Circuits and Systems for Video Technology,2006,16(10):1185-1195.
[28]基于H.264的低码率视频通信软件的设计与实现[D].广州：中山大学,2007.
[29]陈维安,李典,余松煜.H.264软件解码器的优化[J].数据采集与处理.2005(20)：493-498.
[30]李辉武.基于ARM9的H.264解码器的优化与实现[D].广州：广东工业大学,2009.
[31]QIANG S M, WEN K X, XIANG L F. Research on Architecture of Multimedia and Its Design Based on Android[C]. International Conference on Internet Technology and Applications, Wuhan,2010:4-6.
[32]马洪堂.基于FFmpeg的视频转换系统的模块研究[J].电脑知识与技术,2009,5(12)：3545-3546术.
[33]肖辉基.基于TMS320DM642的H.264解码器研究与实现[D].成都：电子科技大学,2009.
[34]王颖.H.264视频编码器在ADSP-BF561上的实现及优化[D].成都：西南交通大学,2007.
[35]FFMPEG功能[EB/OL]. http://www.chinavideo.org/
[36]睿鹏,刘佳玲.基于FFMPEG的通用视频插件[J].电脑知识与技术,2010,6(10)：2475-2476.
[37]MAO S, JIE S, XIANG L F. Design and Implementation of Media Player Based on Android[C].2010 6th International Conference on Wireless Communications, Networking and Mobile Computing, Chengdu,2010:4pp.
[38]单海涛,方向忠.基于FFmpeg的高清数字电影软件编码系统的设计[J].信息技术,2007(1)：96-99.
[39]JIN H, JIA M H. The Research of Plug-in Extension Technology Based on Android Multimedia Player Platfrom.2011 International Conference on Computer Science and Service System, Nanjing,2011:874-7.
[40]TONG L Y,TURNER D, STOVER D, et al. Incorporating Video in Platform-Independent Video Games Using Open-Source Software.2010 3rd IEEE International Conference on Computer Science and Information Technology, Chengdu,2010:466-70.
[41]刘马飞,曾学文,倪宏Windows平台下应用FFMPEG实现H.264视频回放[J].微计算机应用,2008,29(11)：61-65.
[42]杨丽军FFmpeg在媒体资产管理系统中的应用[J].微计算机应用,2011,34(15)：147-149.
[43]FFMPEG sourcecode[EB/OL]. http://ffmpeg.mplayerhq.hu/
[44]FFMPEG Working group.FFMPEGO.5 released[EB/OL]. http://svn.ffmpeg.org/ffmpeg/branches/0.5/RELEASE
[45]FFMPEG SDK sourcecode [EB/OL].http://www.ffmpeg.com.cn,2009.
[46]FFmpeg 在 windows下的编译[EB/OL].http://www.chinavideo.org.
[2]Wikipedia. H.264/MPEG-4 AVC[EB/OL]. http://en.wikipedia. org/wiki/H.264/MPEG-4_AVC.
[3]郭晓强,解伟.H.264/AVC视频压缩编码标准的新发展[EB/OL].http://video.coal.cn.
[4]刘马飞,曾学文,倪宏Windows平台下应用FFMPEG实现H.264视频回放[J].微计算机应用,2008,29(11)：61-65.
[5]姚莹琳.H.264大尺寸解码器的实现与优化[D].北京：北京邮电大学2006.
[6]林炜.基于DSP的H.264运动估计算法的研究与实现[D].成都：电子科技大学,2007.
[7]MAJOR A, YI Y, NOUSIAS I, et al. H.264 Decoder Implementation on a Dynamically Reconfigurable Instruction Cell Based Architecture. IEEE International SOC Conference, Austin,2006:49-52.
[8]皇甫泽毅.H.264解码在嵌入式Linux上的优化实践[D].武汉：华中科技大学,2006.
[9]李世平.H.264开源解码器评测[EB/OL]. http://blog. csdn. negsunshinel314/archive.
[10]JVT (Joint Video Team of ITU-T and ISO/IEC). Recommendation and final draft international standard of joint video specification (ITU-T Rec. H.264/ISO/IEC 14496-10 AVC)[S],2003.
[11]欧阳合.H.264和MPEG-4视频压缩：新一代多媒体的视频编码技术[M].北京：国防科技大学出版社,2004.
[12]席道光.基于H.264/AVC标准的视频编码系统应用研究[D].浙江：浙江工业大学,2007。
[13]SETH K, KAMAKOTI V, SRINIVASAN S. Efficient motion vector recovery algorithm for H.264 using directional interpolation [J]. Image Processing, IET, 2010,4(2):132-141.
[14]毕厚杰.新一代视频压缩编码标准——H.264/AVC[M].北京：人民邮电出版社,2005.
[15]KIM W J, CHO K, CHUNG K S, et al. Stage-based frame-partitioned parallelization of H.264/AVC decoding [J].IEEE Transactions on Consumer Electronics,2010,56(2):1088-1096.
[16]PESCADOR F, SANZ C, GARRIDO M J, et al. A DSP Based H.264 decoder for a multi-format IP set-top box [J]. IEEE Transactions on Consumer Electronics,2008,54(1):145-153.
[17]OSTERMANN J, BORMANS J, LIST P, et al. Video coding with H.264/AVC:tools, performance, and complexity [J].Circuits and Systems Magazine, IEEE,2004,4(1):7-28.
[18]WIEGAND T, SULLIVAN G J, BJONTEGAARD G, et al. Overview of the H.264/AVC video coding standard [J]. IEEE Transactions on Circuits and Systems for Video Technology,1993,13(7):560-576.
[19]PENG Z, DON X, WEN G. Variable-bin-rate CABAC engine for H.264/AVC high definition real-time decoding [J]. IEEE Transactions on Very Large Scale Integration (VLSI) Systems,2009,17(3):417-426.
[20]杜建超.基于H.264/AVC的视频容错技术研究[M].西安：西安电子科技大学,2006.
[21]唐玲娜.H.264视频解码优化及DSP实现[D].成都：电子科技大学,2009.
[22]ZHENG J, CHAU L P. Error-resilient coding of H.264 based on periodic macroblock [J]. IEEE Transactions on Broadcasting,2006,52(2):223-229.
[23]LIU Z, FENG L L, YANG S, et al. Motion feature and hadamard coefficient-based fast multiple reference frame motion estimation for H.264 [J]. IEEE Transactions on Circuits and Systems for Video Technology,2008,18(5):620-632.
[24]刘鹏飞,刘志,安平,张兆杨.H.264/AVC解码器优化的研究[J].图像图形学报,2006(11)：1627-1630.
[25]郭宝龙,倪伟,闰允一.通信中的视频信号处理[M].北京：电子工业出版社2007.
[26]SUDENG H, HAN L W, SAM K W, et al. Frame level rate control for H.264/AVC with novel Rate-Quantization model.2010 IEEE International Conference on,2010:226-231.
[27]KUO T Y, CHAN C H. Fast variable block size motion estimation for H.264 using likelihood and correlation of motion field [J]. IEEE Transactions on Circuits and Systems for Video Technology,2006,16(10):1185-1195.
[28]基于H.264的低码率视频通信软件的设计与实现[D].广州：中山大学,2007.
[29]陈维安,李典,余松煜.H.264软件解码器的优化[J].数据采集与处理.2005(20)：493-498.
[30]李辉武.基于ARM9的H.264解码器的优化与实现[D].广州：广东工业大学,2009.
[31]QIANG S M, WEN K X, XIANG L F. Research on Architecture of Multimedia and Its Design Based on Android[C]. International Conference on Internet Technology and Applications, Wuhan,2010:4-6.
[32]马洪堂.基于FFmpeg的视频转换系统的模块研究[J].电脑知识与技术,2009,5(12)：3545-3546术.
[33]肖辉基.基于TMS320DM642的H.264解码器研究与实现[D].成都：电子科技大学,2009.
[34]王颖.H.264视频编码器在ADSP-BF561上的实现及优化[D].成都：西南交通大学,2007.
[35]FFMPEG功能[EB/OL]. http://www.chinavideo.org/
[36]睿鹏,刘佳玲.基于FFMPEG的通用视频插件[J].电脑知识与技术,2010,6(10)：2475-2476.
[37]MAO S, JIE S, XIANG L F. Design and Implementation of Media Player Based on Android[C].2010 6th International Conference on Wireless Communications, Networking and Mobile Computing, Chengdu,2010:4pp.
[38]单海涛,方向忠.基于FFmpeg的高清数字电影软件编码系统的设计[J].信息技术,2007(1)：96-99.
[39]JIN H, JIA M H. The Research of Plug-in Extension Technology Based on Android Multimedia Player Platfrom.2011 International Conference on Computer Science and Service System, Nanjing,2011:874-7.
[40]TONG L Y,TURNER D, STOVER D, et al. Incorporating Video in Platform-Independent Video Games Using Open-Source Software.2010 3rd IEEE International Conference on Computer Science and Information Technology, Chengdu,2010:466-70.
[41]刘马飞,曾学文,倪宏Windows平台下应用FFMPEG实现H.264视频回放[J].微计算机应用,2008,29(11)：61-65.
[42]杨丽军FFmpeg在媒体资产管理系统中的应用[J].微计算机应用,2011,34(15)：147-149.
[43]FFMPEG sourcecode[EB/OL]. http://ffmpeg.mplayerhq.hu/
[44]FFMPEG Working group.FFMPEGO.5 released[EB/OL]. http://svn.ffmpeg.org/ffmpeg/branches/0.5/RELEASE
[45]FFMPEG SDK sourcecode [EB/OL].http://www.ffmpeg.com.cn,2009.
[46]FFmpeg 在 windows下的编译[EB/OL].http://www.chinavideo.org.