基于SH-Mobile的HE-AAC v2解码设计与实现
|
摘要
随着计算机技术的飞速发展,数字音频在消费电子、网络、广播、移动通信和数字影视等领域都有着广泛应用。互联网与无线通信协议的结合,移动终端的应用日益普及,使对更高效编解码系统的需求不断持续。本文中实现的HE-AAC v2解码器是手持数字电视终端需求的功能之一,旨在实现低比特率,高音质的完全解码,未来必将成为移动多媒体广泛应用的技术之一。
本文首先分析了HE-AAC v2主要技术构成,及其编解码器的工作原理,深入分析了先进音频编码(AAC)、频带复制(SBR)和参数立体声(PS)中各算法模块的原理。按照两类不同的AAC文件格式——音频数据交换格式和音频数据传输比特流,分别分析了HE-AAC v2的各种比特流结构。对比特流中的原始数据块中的八类元素做了详细介绍。本文还着重了介绍了完成解码功能的HE-AAC v2解码中间件在软件设计中的作用,分析了其提供的应用函数接口(API),以及在使用这些API时所需的注意事项,根据两种不同的比特流结构,给出对两类比特流的解码流程。
本文通过对HE-AAC v2技术的理论研究,比特流结构分析,借助瑞萨公司HE-AAC v2中间件,在基于SH-Mobile平台的地面数字多媒体广播(T-DMB)终端上,实现了对HE-AAC v2音频格式的文件的解码功能。所做的主要工作如下:
(1)通过对T-DMB终端软件的结构和各功能的分析,提出了前后端共同驱动中间件解码的方案,即前端通过判断输入缓冲中的剩余数据量来不断从SD卡上读取文件,后端通过中断方式驱动中间件解码的不断进行。
(2)结合T-DMB终端软件已有的模块和任务,从总体上设计了解码实现的流程。其中包括从SD卡读出文件,逐个数掘块的解码,解码后的PCM输出。创建多媒体音频解码控制任务,该任务利用中间件,与其他任务相互协作,控制解码的过程与状态。
(3)根据总体设计,进一步详细设计了解码功能实现的任务构成,以及多媒体音频解码控制任务的三种状态(解码启动,解码过程中,解码停止)的实现流程。包括前后端共同驱动文件读取和解码不断进行的具体流程。
(4)编写代码,调试,并设计测试用例、制作样本进行了测试。
经过测试,本设计可支持采样率为16、22.05、24、32、44.1和48kHz,比特率为16～128kbps的各种HE-AAC v2音频文件。通过与适用于PC的同类解码器Nero AAC对比发现,解码性能更胜于它。
As the rapid development of computer technology, digital audio has a wide range of applications in consumer electronics, networks, broadcasting, mobile communication and digital television etc. Internet combined with wireless communication and mobile terminal application growing popularity will make the need for more efficiency codec system continue. HE-AAC v2 decoder realized in the paper is one of the functions of portable digital TV. It means to realize the completed decode with high audio qualification at low bitrate. It will surely be one of the most popular technology applied in mobile multimedia area.
Firstly, HE-AAC v2's main technology and codec's working principle are analyzed. The algorithm principles of compress tools in AAC, SBR and PS areanalyzed deeply. Then, according to two different types of AAC file------audio datainterchange format and andio data transmition stream, the bitestream structures of HE-AAC v2 are analyzed. Seven types element in raw data block in bitstream are introduced. The function of HE-AAC v2 middleware which completes the decoding is focused on. The API provided by middleware and the attension issues when use those application interface are introduced. At last, two types of bitstream's decode process are given out according to their bitstream structures.
Through the study of HE-AAC v2 theory and analysis of bitstream structure, the completed decoding of HE-AAC v2 compress format file on SH-mobile is realized successfully with HE-AAC v2 middleware provided by Renesas co..The major work done are:
(1) Through the analysis of the structure and each functions of T-DMB terminal software, raise a programme which can co-drive decoding from front and end. Files are read from SD card continuously by judging the rest data size in input buffer in front. Continuous decoding is driven by interrupt in the end.
(2) With the existed modual and task in T-DMB terminal software, design the decoding programme in whole. It includes reading files from SD card, decoding every data block and outputting the decoded PCM. Create a task named multimedia audio decode control task. The task controls the decoding process and status by cooperating with other tasks with HE-AAC v2 middleware.
(3) Accoding to the whole designation, design the constructed tasks in the realization of decoding and the flow of realize three status(start decoding, decoding and stop decoding) in multimedia audio decode control task. It includes the specific flow of how to drive reading file and decoding.
(4) Code in C language, debug, design testcase and make samples to test the function.
By playing different HE-AAC v2 audio samples, the design in this paper can support kinds of HE-AAC v2 audio files whose sample rate is 16、22.05、24、32、44.1 or 48kHz, and bitrate is beteen 16 and 128kbps. And, through the compare of the decoder in this paper and Nero AAC which is a AAC decoder userd on PC, it can be found that the decode qualities are more well.
本文首先分析了HE-AAC v2主要技术构成,及其编解码器的工作原理,深入分析了先进音频编码(AAC)、频带复制(SBR)和参数立体声(PS)中各算法模块的原理。按照两类不同的AAC文件格式——音频数据交换格式和音频数据传输比特流,分别分析了HE-AAC v2的各种比特流结构。对比特流中的原始数据块中的八类元素做了详细介绍。本文还着重了介绍了完成解码功能的HE-AAC v2解码中间件在软件设计中的作用,分析了其提供的应用函数接口(API),以及在使用这些API时所需的注意事项,根据两种不同的比特流结构,给出对两类比特流的解码流程。
本文通过对HE-AAC v2技术的理论研究,比特流结构分析,借助瑞萨公司HE-AAC v2中间件,在基于SH-Mobile平台的地面数字多媒体广播(T-DMB)终端上,实现了对HE-AAC v2音频格式的文件的解码功能。所做的主要工作如下:
(1)通过对T-DMB终端软件的结构和各功能的分析,提出了前后端共同驱动中间件解码的方案,即前端通过判断输入缓冲中的剩余数据量来不断从SD卡上读取文件,后端通过中断方式驱动中间件解码的不断进行。
(2)结合T-DMB终端软件已有的模块和任务,从总体上设计了解码实现的流程。其中包括从SD卡读出文件,逐个数掘块的解码,解码后的PCM输出。创建多媒体音频解码控制任务,该任务利用中间件,与其他任务相互协作,控制解码的过程与状态。
(3)根据总体设计,进一步详细设计了解码功能实现的任务构成,以及多媒体音频解码控制任务的三种状态(解码启动,解码过程中,解码停止)的实现流程。包括前后端共同驱动文件读取和解码不断进行的具体流程。
(4)编写代码,调试,并设计测试用例、制作样本进行了测试。
经过测试,本设计可支持采样率为16、22.05、24、32、44.1和48kHz,比特率为16～128kbps的各种HE-AAC v2音频文件。通过与适用于PC的同类解码器Nero AAC对比发现,解码性能更胜于它。
As the rapid development of computer technology, digital audio has a wide range of applications in consumer electronics, networks, broadcasting, mobile communication and digital television etc. Internet combined with wireless communication and mobile terminal application growing popularity will make the need for more efficiency codec system continue. HE-AAC v2 decoder realized in the paper is one of the functions of portable digital TV. It means to realize the completed decode with high audio qualification at low bitrate. It will surely be one of the most popular technology applied in mobile multimedia area.
Firstly, HE-AAC v2's main technology and codec's working principle are analyzed. The algorithm principles of compress tools in AAC, SBR and PS areanalyzed deeply. Then, according to two different types of AAC file------audio datainterchange format and andio data transmition stream, the bitestream structures of HE-AAC v2 are analyzed. Seven types element in raw data block in bitstream are introduced. The function of HE-AAC v2 middleware which completes the decoding is focused on. The API provided by middleware and the attension issues when use those application interface are introduced. At last, two types of bitstream's decode process are given out according to their bitstream structures.
Through the study of HE-AAC v2 theory and analysis of bitstream structure, the completed decoding of HE-AAC v2 compress format file on SH-mobile is realized successfully with HE-AAC v2 middleware provided by Renesas co..The major work done are:
(1) Through the analysis of the structure and each functions of T-DMB terminal software, raise a programme which can co-drive decoding from front and end. Files are read from SD card continuously by judging the rest data size in input buffer in front. Continuous decoding is driven by interrupt in the end.
(2) With the existed modual and task in T-DMB terminal software, design the decoding programme in whole. It includes reading files from SD card, decoding every data block and outputting the decoded PCM. Create a task named multimedia audio decode control task. The task controls the decoding process and status by cooperating with other tasks with HE-AAC v2 middleware.
(3) Accoding to the whole designation, design the constructed tasks in the realization of decoding and the flow of realize three status(start decoding, decoding and stop decoding) in multimedia audio decode control task. It includes the specific flow of how to drive reading file and decoding.
(4) Code in C language, debug, design testcase and make samples to test the function.
By playing different HE-AAC v2 audio samples, the design in this paper can support kinds of HE-AAC v2 audio files whose sample rate is 16、22.05、24、32、44.1 or 48kHz, and bitrate is beteen 16 and 128kbps. And, through the compare of the decoder in this paper and Nero AAC which is a AAC decoder userd on PC, it can be found that the decode qualities are more well.
引文
[1]山石.aacPlus:为3G而来.通讯世界,2007.11(115期).p40.
[2]任军刚.Enhanced aacPlus编解码器的研究和实现.[西安电子科技大学硕士学位论文].西安:西安电子科技大学.2006年1月.p1-p75.
[3]李静,梁海滨.3G时代的天籁之音——访Coding Technology中国首席代表费托尼.TELECOMMUNICTION TECHNOLOGY.2007.3.p126-p128.
[4]Coding Technologies.MPEG-4 aacPlus Reference Implementations.p1-p2.
[5]Stefan Meltzer,Gerald Moser.MPEG-4 HE-AAC v2-audio coding for today's digital media world.EBU TECHNICAL REVIEW-January 2006.p1-pl2.
[6]Martin Wolters,Kristofer Kjorling,Daniel Homm,et al.A closer look into MPEG-4 High Efficiency AAC.Audio Engineering Society Convention Paper Presented at the 115th Convention 2003 October 10-13 New York,NY,USA.p1-pl6.
[7]Coding Technologies and its Partners.Embedded MPEG-4 aacPlus Libraries.p1-p2.
[8]3GPP TS 26.403:Enhanced aacPlus general audio codec;Encoder Specification AAC part.p4-p22.
[9]Osamu Shimada,Toshiyuki Nomura,Akihiko Sugiyama,Masahiro Serizawa.DSP Implementation of the 3GPP Enhanced aacPlus Decoder.Media and Information Research Laboratories,NEC Corporation,Japan.p1-p2.
[10]谦君.3G移动语音编码标准介绍.无线技术,2006,9.p1-p2.
[11]刘倩如.HEAAC解码器的软硬件协同设计与实现.[上海交通大学硕士学位论文].上海:上海交通大学.2007年4月.p1-p4.
[12]胡多传.MPEG-2 AAC音频编解码的研究及实现.[安徽农业大学硕士学位论文].安徽:安徽农业大学.2005年.p1-p4.
[13]ISO/IEC 14496-3:2005,Information technology -Coding of audio-visual objects-Part 3:Audio.p455-p796.
[14]汪国有,张成兴,廖容,陈振学.MPEG-4 AAC实时音频编码器设计与实现研究.计算机与数字工程,第33卷.p1-p5.
[15]胡剑凌,徐盛,陈健.MPEG-4多媒体通信音频编码.计算机工程.第26卷,第7期,2000年7月.p1-p5.
[16]许信玉.MPEG-4技术及其应用.有限电视技术.2004年,第11期.p41-p42.
[17]骆华.MPEG-4通用音频编码技术及解码器实现.应用技术.2005.1.p27-p29.
[18]杨永祥.MPEG-4 AAC音频解码器的DSP程序优化设计与实现研究.[华中科技大学硕士学位论文].武汉:华中科技大学.2005年.p15-p23.
[19]黄春明,陈常谦.心理声学模型及其在MPEG-2AAC中的应用.电声技术.2004年第11期.p44-p47.
[20]罗龙智.基于aacplus v2的音频直播系统的设计与实现.[电子科技大学硕士学位论文].成都:电子科技大学.2007年.p28-p33.
[21]姜晔,吴镇扬.感知音频编码中预回声的产生机理与抑制方法.电声技术.2000,11.p15-p18
[22]杜伟韬,杨占昕.AAC编码器的滤波器组——原理与实现.北京广播学院学报(自然科学版).2005年3月,第12卷,第1期.p1-p5.
[23]梅优良.基于RISC的MPEG4-AAC编解码研究.[浙江大学硕士学位论文].杭州:浙江大学.2006年.p10-p25.
[24]赵喜林,林涛.AAC音频解码器中TNS模块的设计及其ASIC实现.微电子学与计算机.2006年第23卷第1期.p1-p2.
[25]Erik Schuijers,Jeroen Breebaart,Heiko Purnhagen,Jonas Engdegard.Low complexity parametric stereo coding.In 116th AES Convention,Berlin,May 8-11,2004.p1-p6.
[26]党辰,戴葵,王苏峰,等.高频重建技术SBR的研究与实现.电子学报.2004年12月,第12A期.P2-p4.
[27]Sang-Uk Ryu,Kenneth Rose,Joon-Hyuk Chang.Effective High Frequency Regeneration Based On Sinusolidal Modeling For MPEG-4 HE-AAC.2005 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics.October 16-19,2005,New Paltz,NY.p1-p4.
[28]CHONG Kok Seng,Naoya TANAKA,Toshiyuki NOMURA,et al.Low Power Spectral Band Replication Technology for the MPEG-4 Audio Standard.ICICS-PCM 2003,December 2003.p15-p18.
[29]崔晓东.MPEG-4 aacplus/DTS多声道音频传输编码.电视工程.2006年第3期.p11-p14.
[30]3GPP TS 26.404:Enhanced aacPlus general audio codec;Encoder Specification SBR part.p4-p32.
[31]Per Ekstrand.Bandwidth Extension Of Audio Signals By Spectral Band Replication.Coding Technologies,Proc.1st IEEE Benelux Workshop on Model based Processing and Coding of Audio(MPCA-2002),Leuven,Belgium,November 15,2002.p1-p6.
[32]焦慧颖,安建平,卜祥元.数字广播音频编码中的频带复制技术(SBR).中国有限电视.2005(05).p425-p427.
[33]Clark Novak,Yelos Systems,Cleveland,Ohio.Spectral Band Replication and Aaacpls Coding-an overview.2003.Coding Technology.p1-p2.
[34]Oliver Kunz.SBR explained:White paper.Coding Technology.p1-p3.
[35]郭庆巍,张海波,马鸿飞.频带复制技术的分析与测试.理论与实践,质量工程卷.2007年第12期.p51-p54.
[36]Imre Varga.Audio Codec for Mobile Multimedia Applications.2004 IEEE fith Workshop on Multimedia Signal Processing.p1-p4.
[37]3GPP TS 26.405:Enhanced aacPlus general audio codec;Encoder Specification Parametric Stereo part.p4-p13.
[38]孙佳.MPEG-4 AAC音频解码器设计.[山东大学硕士学位论文].山东:山东大学.2007年.p22-p42.
[39]Renesas Technology.SH7343 Hardware Manual.Renesas 32-Bit RISC Microcomputer SH-Mobile Series R8J7343.p12-p32.
[40]Renesas Technology.SH-3/SH-3E/SH3-DSP Software Manual.Renesas 32-Bit RISC Microcomputer SuperH RISC engine Family.p2-p6.
[41]Renesas Technology.SH-4A Software Manual.The SH-4A is a RISC(Reduced Instruction Set Computer)microcomputer which includes a Renesas Technology-original RISC CPU as its core.P8-p22.
[42]刘巍,黄秋元,陈伟.实时操作系统工业标准ITRON的分析.中国电子科学研究院学报.2006年4月,第2期.p1-p4.
[2]任军刚.Enhanced aacPlus编解码器的研究和实现.[西安电子科技大学硕士学位论文].西安:西安电子科技大学.2006年1月.p1-p75.
[3]李静,梁海滨.3G时代的天籁之音——访Coding Technology中国首席代表费托尼.TELECOMMUNICTION TECHNOLOGY.2007.3.p126-p128.
[4]Coding Technologies.MPEG-4 aacPlus Reference Implementations.p1-p2.
[5]Stefan Meltzer,Gerald Moser.MPEG-4 HE-AAC v2-audio coding for today's digital media world.EBU TECHNICAL REVIEW-January 2006.p1-pl2.
[6]Martin Wolters,Kristofer Kjorling,Daniel Homm,et al.A closer look into MPEG-4 High Efficiency AAC.Audio Engineering Society Convention Paper Presented at the 115th Convention 2003 October 10-13 New York,NY,USA.p1-pl6.
[7]Coding Technologies and its Partners.Embedded MPEG-4 aacPlus Libraries.p1-p2.
[8]3GPP TS 26.403:Enhanced aacPlus general audio codec;Encoder Specification AAC part.p4-p22.
[9]Osamu Shimada,Toshiyuki Nomura,Akihiko Sugiyama,Masahiro Serizawa.DSP Implementation of the 3GPP Enhanced aacPlus Decoder.Media and Information Research Laboratories,NEC Corporation,Japan.p1-p2.
[10]谦君.3G移动语音编码标准介绍.无线技术,2006,9.p1-p2.
[11]刘倩如.HEAAC解码器的软硬件协同设计与实现.[上海交通大学硕士学位论文].上海:上海交通大学.2007年4月.p1-p4.
[12]胡多传.MPEG-2 AAC音频编解码的研究及实现.[安徽农业大学硕士学位论文].安徽:安徽农业大学.2005年.p1-p4.
[13]ISO/IEC 14496-3:2005,Information technology -Coding of audio-visual objects-Part 3:Audio.p455-p796.
[14]汪国有,张成兴,廖容,陈振学.MPEG-4 AAC实时音频编码器设计与实现研究.计算机与数字工程,第33卷.p1-p5.
[15]胡剑凌,徐盛,陈健.MPEG-4多媒体通信音频编码.计算机工程.第26卷,第7期,2000年7月.p1-p5.
[16]许信玉.MPEG-4技术及其应用.有限电视技术.2004年,第11期.p41-p42.
[17]骆华.MPEG-4通用音频编码技术及解码器实现.应用技术.2005.1.p27-p29.
[18]杨永祥.MPEG-4 AAC音频解码器的DSP程序优化设计与实现研究.[华中科技大学硕士学位论文].武汉:华中科技大学.2005年.p15-p23.
[19]黄春明,陈常谦.心理声学模型及其在MPEG-2AAC中的应用.电声技术.2004年第11期.p44-p47.
[20]罗龙智.基于aacplus v2的音频直播系统的设计与实现.[电子科技大学硕士学位论文].成都:电子科技大学.2007年.p28-p33.
[21]姜晔,吴镇扬.感知音频编码中预回声的产生机理与抑制方法.电声技术.2000,11.p15-p18
[22]杜伟韬,杨占昕.AAC编码器的滤波器组——原理与实现.北京广播学院学报(自然科学版).2005年3月,第12卷,第1期.p1-p5.
[23]梅优良.基于RISC的MPEG4-AAC编解码研究.[浙江大学硕士学位论文].杭州:浙江大学.2006年.p10-p25.
[24]赵喜林,林涛.AAC音频解码器中TNS模块的设计及其ASIC实现.微电子学与计算机.2006年第23卷第1期.p1-p2.
[25]Erik Schuijers,Jeroen Breebaart,Heiko Purnhagen,Jonas Engdegard.Low complexity parametric stereo coding.In 116th AES Convention,Berlin,May 8-11,2004.p1-p6.
[26]党辰,戴葵,王苏峰,等.高频重建技术SBR的研究与实现.电子学报.2004年12月,第12A期.P2-p4.
[27]Sang-Uk Ryu,Kenneth Rose,Joon-Hyuk Chang.Effective High Frequency Regeneration Based On Sinusolidal Modeling For MPEG-4 HE-AAC.2005 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics.October 16-19,2005,New Paltz,NY.p1-p4.
[28]CHONG Kok Seng,Naoya TANAKA,Toshiyuki NOMURA,et al.Low Power Spectral Band Replication Technology for the MPEG-4 Audio Standard.ICICS-PCM 2003,December 2003.p15-p18.
[29]崔晓东.MPEG-4 aacplus/DTS多声道音频传输编码.电视工程.2006年第3期.p11-p14.
[30]3GPP TS 26.404:Enhanced aacPlus general audio codec;Encoder Specification SBR part.p4-p32.
[31]Per Ekstrand.Bandwidth Extension Of Audio Signals By Spectral Band Replication.Coding Technologies,Proc.1st IEEE Benelux Workshop on Model based Processing and Coding of Audio(MPCA-2002),Leuven,Belgium,November 15,2002.p1-p6.
[32]焦慧颖,安建平,卜祥元.数字广播音频编码中的频带复制技术(SBR).中国有限电视.2005(05).p425-p427.
[33]Clark Novak,Yelos Systems,Cleveland,Ohio.Spectral Band Replication and Aaacpls Coding-an overview.2003.Coding Technology.p1-p2.
[34]Oliver Kunz.SBR explained:White paper.Coding Technology.p1-p3.
[35]郭庆巍,张海波,马鸿飞.频带复制技术的分析与测试.理论与实践,质量工程卷.2007年第12期.p51-p54.
[36]Imre Varga.Audio Codec for Mobile Multimedia Applications.2004 IEEE fith Workshop on Multimedia Signal Processing.p1-p4.
[37]3GPP TS 26.405:Enhanced aacPlus general audio codec;Encoder Specification Parametric Stereo part.p4-p13.
[38]孙佳.MPEG-4 AAC音频解码器设计.[山东大学硕士学位论文].山东:山东大学.2007年.p22-p42.
[39]Renesas Technology.SH7343 Hardware Manual.Renesas 32-Bit RISC Microcomputer SH-Mobile Series R8J7343.p12-p32.
[40]Renesas Technology.SH-3/SH-3E/SH3-DSP Software Manual.Renesas 32-Bit RISC Microcomputer SuperH RISC engine Family.p2-p6.
[41]Renesas Technology.SH-4A Software Manual.The SH-4A is a RISC(Reduced Instruction Set Computer)microcomputer which includes a Renesas Technology-original RISC CPU as its core.P8-p22.
[42]刘巍,黄秋元,陈伟.实时操作系统工业标准ITRON的分析.中国电子科学研究院学报.2006年4月,第2期.p1-p4.