达芬奇环境下异构双核处理器多媒体信息处理系统设计与实现
|
摘要
随着嵌入式技术的不断发展以及无线快速数据通信需求的逐步增长,音视频等多媒体应用的发展也越来越受到人们的重视。人们迫切需要一种费用低廉且高质量的音视频实时通信产品,因此,音视频多媒体的开发具有广阔的发展空间。嵌入式处理器结构的不断发展和改进也为多媒体应用的开发提供了硬件基础。其中比较典型的有TI公司基于ARM+DSP异构双核架构的DM3730处理器,它是专门针对嵌入式系统设计的开放多媒体应用处理器,足以满足现阶段的多媒体通信的需要。而在操作系统方面,采用微内核架构的QNX实时操作系统在嵌入式系统终端领域逐步发展,以其实时性、安全性和可靠性等特点在通信领域占据一席之地。
VoIP技术由于其经济、灵活、功能丰富等优点越来越广泛地应用,而其与无线技术结合产生的无线VoIP通信技术是其最重要的发展趋势之一。经过对现有的无线VoIP通信领域相关技术的研究,本文针对实际情况开展了基于ARM+DSP异构双核平台下的音视频实时通信系统的研究与设计。在本系统设计中,采用了模块化的设计思想,完成了达芬奇子系统软件模块、SIP信令控制模块和无线网络传输模块等主要部分组成。该系统可以极大地提高消费者对于嵌入式音视频实时通信的体验。
本论文通过深入分析ARM+DSP异构双核处理器架构及其软件开发模式,并结合VoIP音视频多媒体通信技术进行系统设计。系统首先完成了整个系统的方案论证,确定各个部分的技术要点,然后根据ARM+DSP异构双核处理器的硬件特点,选定达芬奇软件框架进行异构双核系统应用程序的开发。依据达芬奇软件框架设计的原则和规范,实现了从底层语音和视频算法库的xDM标准封装到上层音视频实时通信应用程序的开发。将采用微内核架构的QNX嵌入式实时操作系统作为ARM核上的操作系统,通过跨平台的QT界面设计软件完成嵌入式终端操作界面的设计,实现了系统通信的控制和音视频播放,人机交互界面简洁友好。最后,搭建了OpenSIPS服务器并完成了终端上的SIP和RTP协议功能。通过对各个模块的功能和性能进行测试,结果显示整个系统运行良好,达到了当初论文设计的初衷,最终可以通过本系统进行音视频通信。
With the development of embedded technology and the gradual growth of the wireless fast data communication need, the multimedia applications of voice and video attract more and more attention. There is an urgent need for a product that can meet the demand for high-quality voice and video communication and can greatly reduce consumer's cost. Therefore, the development of voice and video multimedia applications has a broad space for development. The continuous development and improvement of the embedded processor architecture lays the hardware foundation for the development of multimedia applications, DM3730processor based on ARM+DSP heterogeneous dual-core architecture is specifically designed for the embedded open multimedia communication. With regard to operating system, the microkernel-based QNX real-time operating system has been gradually developed in the field of embedded terminals and has occupied a place in the field of commnunication because of its characteristics of real-time, security and reliability.
VoIP technology is more and more widely applied because of its economic, flexible and feature-rich advantages, combined with wireless technology, wireless VoIP communication technology becomes one of the most important trends in th field. Based on the research of existing wireless VoIP communication technology, this thesis carries out the reaserch on the voice and video communication system based on the heterogeneous dual-core platform for the actual situation. The system adopts the modular design method, including several main parts of the DaVinci software module, the SIP signaling control module and network transmission module and so on. This system can greatly improve the consumer's experience of embedded voice and video realtime communication.
The thesis profoundly analyzes the heterogeneous dual-core processor architecture and its software development model, combined with the VoIP communication technology of the voice and video multimedia, and finally accomplishes the system design. The system completes the entire system demonstration program and determines the main technical points of various parts, and then selects DaVinci software framework for the system development based on the characteristics of heterogeneous dual-core processor hardware, brings about the design from the underlying voice and video algorithm libraries wrapped by the xDM standard to the upper visible VoIP communication application development based on the the DaVinci software design principles and norms. The system adopts the microkernel-based QNX embedded real-time operating system as the operating system on the ARM core, and carries through operating interface development of the terminal development board throught the cross-platform QT interface-designing software to measure up the control of the communication system and voice and video features which is man-machine interface-friendly. Eventually, builds the OpenSIPS server in PC and completes the implementation of SIP protocol module in the terminal development board. Through the system functions and performance tested, the results show that the entire system can be running well, which achieve the original intention of the thesis design. Ultimately, the system can be carried out voice and video communication.
VoIP技术由于其经济、灵活、功能丰富等优点越来越广泛地应用,而其与无线技术结合产生的无线VoIP通信技术是其最重要的发展趋势之一。经过对现有的无线VoIP通信领域相关技术的研究,本文针对实际情况开展了基于ARM+DSP异构双核平台下的音视频实时通信系统的研究与设计。在本系统设计中,采用了模块化的设计思想,完成了达芬奇子系统软件模块、SIP信令控制模块和无线网络传输模块等主要部分组成。该系统可以极大地提高消费者对于嵌入式音视频实时通信的体验。
本论文通过深入分析ARM+DSP异构双核处理器架构及其软件开发模式,并结合VoIP音视频多媒体通信技术进行系统设计。系统首先完成了整个系统的方案论证,确定各个部分的技术要点,然后根据ARM+DSP异构双核处理器的硬件特点,选定达芬奇软件框架进行异构双核系统应用程序的开发。依据达芬奇软件框架设计的原则和规范,实现了从底层语音和视频算法库的xDM标准封装到上层音视频实时通信应用程序的开发。将采用微内核架构的QNX嵌入式实时操作系统作为ARM核上的操作系统,通过跨平台的QT界面设计软件完成嵌入式终端操作界面的设计,实现了系统通信的控制和音视频播放,人机交互界面简洁友好。最后,搭建了OpenSIPS服务器并完成了终端上的SIP和RTP协议功能。通过对各个模块的功能和性能进行测试,结果显示整个系统运行良好,达到了当初论文设计的初衷,最终可以通过本系统进行音视频通信。
With the development of embedded technology and the gradual growth of the wireless fast data communication need, the multimedia applications of voice and video attract more and more attention. There is an urgent need for a product that can meet the demand for high-quality voice and video communication and can greatly reduce consumer's cost. Therefore, the development of voice and video multimedia applications has a broad space for development. The continuous development and improvement of the embedded processor architecture lays the hardware foundation for the development of multimedia applications, DM3730processor based on ARM+DSP heterogeneous dual-core architecture is specifically designed for the embedded open multimedia communication. With regard to operating system, the microkernel-based QNX real-time operating system has been gradually developed in the field of embedded terminals and has occupied a place in the field of commnunication because of its characteristics of real-time, security and reliability.
VoIP technology is more and more widely applied because of its economic, flexible and feature-rich advantages, combined with wireless technology, wireless VoIP communication technology becomes one of the most important trends in th field. Based on the research of existing wireless VoIP communication technology, this thesis carries out the reaserch on the voice and video communication system based on the heterogeneous dual-core platform for the actual situation. The system adopts the modular design method, including several main parts of the DaVinci software module, the SIP signaling control module and network transmission module and so on. This system can greatly improve the consumer's experience of embedded voice and video realtime communication.
The thesis profoundly analyzes the heterogeneous dual-core processor architecture and its software development model, combined with the VoIP communication technology of the voice and video multimedia, and finally accomplishes the system design. The system completes the entire system demonstration program and determines the main technical points of various parts, and then selects DaVinci software framework for the system development based on the characteristics of heterogeneous dual-core processor hardware, brings about the design from the underlying voice and video algorithm libraries wrapped by the xDM standard to the upper visible VoIP communication application development based on the the DaVinci software design principles and norms. The system adopts the microkernel-based QNX embedded real-time operating system as the operating system on the ARM core, and carries through operating interface development of the terminal development board throught the cross-platform QT interface-designing software to measure up the control of the communication system and voice and video features which is man-machine interface-friendly. Eventually, builds the OpenSIPS server in PC and completes the implementation of SIP protocol module in the terminal development board. Through the system functions and performance tested, the results show that the entire system can be running well, which achieve the original intention of the thesis design. Ultimately, the system can be carried out voice and video communication.
引文
[1]黄国睿,张平.多核处理器的关键技术及其发展趋势[J].计算机工程与设计,2009.
[2]汪延东.基于达芬奇平台的IP可视电话终端的研究[D].山东大学,2010.
[3]沈鑫剡.多媒体传输网络与VoIP系统设计[M].北京：人民邮电出版社,2005.
[4]Lau, Richard, Khare, Ram. Chang, William Y. Service assurance for voice over WiFi and 3G networks. Boston:Artech House, c2005.
[5]黄皎,吴春明Wi-Fi与嵌入式设备的融合技术研究[J].计算机工程,2011,04.
[6]于洁.基于SIP协议的VOIP客户端软件的设计与实现[J].电子科技大学,2006.
[7]贺迅.基于CELL异构多核处理器的H.264高清实时编码[D].上海交通大学,2008,11.
[8]陈国兵.嵌入式异构多核体系的片上通信[D].浙江大学,2007,05.
[9]TMS320DSP Algorithm Standard Rules and Guidelines. [EB/OL]. TI.2007,08.
[10]李洪,胥秦秦.H.264在ADSP-BF561上的实现与优化[J].电子技术应用,2009,02.
[11]张起贵,张胜,张刚.最新DSP技术：“达芬奇”系统、框架和组件.国防工业出版社,2009,08.
[12]彭启宗.达芬奇技术-数字图像/视频信号处理新平台.电子工业出版社,2008,09.
[13]DM3730, DM3725 Applications Processor. [EB/OL]. SPRS685.2010.
[14]陈路.达芬奇平台将大幅降低数字视频产品开发门槛.电子工程,2005,11.
[]5]赵勇DAVINCI技术原理与应用指南.东南大学出版社,2008,01.
[16]张辉.基于v412的嵌入式视频驱动程序开发与实现[D].安徽大学,2010,05.
[17]管必聪,吕红.基于TMS320VC5509的G729A算法实现[J].机电工程,2008,02.
[18]王原丽,方存果.基于G.729A语音编解码系统的优化实现[J].武汉理工大学学报,2010,10.
[19]xDAIS-DM (Digital Media) User Guide. [EB/OL]. SPRUEC8B. TI.2007.
[20]毕厚杰,王健.新一代视频压缩编码标准-H.264/AVC北京：人民邮电出版社,2005.
[21]陈兴.基于Davinci的图像压缩编解码与网络传输[D].北方工业大学,2007,06.
[22]高宇.基于H.264/AVC的压缩域视频场景转换检测技术的研究与应用[D].北京工业大学,2008,05.
[23]张晓燕,谢瑶堂.最新视频编码标准H.264及其核心技术[J].北京理工大学,2009.
[24]刘可钊,王树梅.多媒体通信中的音视频同步问题研究[D].南京理工大学,2007.
[25]林朝龙.基于RTP的网络视频传输的应用研究[J].信息化研究,2009,12.
[26]TMS320C6000 Programmer's Guide. [EB/OL]. SPRU198. TI.2010.
[27]TMS320 DSP/BIOS Useris Guide. [EB/OL]. SPRU423E. TI.2004.
[28]陈莉君Linux内核设计与实现.北京：机械工业出版社,2006,01.
[29]邵贝贝.嵌入式实时操作系统uC/OS-Ⅱ北京：北京航空航天大学出版社,2007.
[30]程敬原VxWorks软件开发项目实例完全解析.北京：中国电力出版社,2005,10.
[31]王斑.基于QNX实时系统的测试系统的开发[D].西北工业大学,2006,03.
[32]Rob Krten. QNX Neutrino RTOS-Getting Started with QNX Neutrino:A Guide for Real-time Programmers.2009.
[33]DVSDK Getting Started Guide. [EB/OL]. SPRUEG8. TI.2006.
[34]田黎育TMS320C6000系列DSP编程工具与指南/美国德州仪器公司著.清华大学出版社,2006,09.
[35]Codec Engine Algorithm Creator User's Guide. [EB/OL]. SPRUED6C. TI.2007,09.
[36]许邦建,孙永节.DSP算法与体系结构实现技术.北京：国防工业出版社,2010.
[37]宋建勋,刘峰.基于TMS320DM365多平台实时视频传输系统的设计与实现[J].电视技术,2011,07.
[38]刘微,李向荣.AVS解码器在DSP平台上的优化[J].电子器件,2010,01.
[39]齐美彬,陈俊峰.基于C64x+内核的H.264编码数据Cache优化[J],合肥工业大学学报,2011,04.
[40]TMS320C64x+DSP Cache User's Guide.SPRU862. [EB/OL]. TI.2011.
[41]刘泽新,何彬.SIP代理服务器的设计与实现[J].计算机系统应用,2005,2.
[42]黄永峰,李建庆.下一代网络核心控制协议：SIP及其应用.北京：人民邮电出版社,2009,01.
[43]徐炽,张闻芳.基于SIP协议的VOIP系统设计与实现[J].科学技术与工程,2008.
[44]Yuan Zhang. SIP-based VoIP network and its interworking with the PSTN[J]. Electronics&Communication Engineering Journal,2002,14(6):10-12.
[45]高玉龙.达芬奇技术开发基础、原理与实例.北京：电子工业出版社,2012,06.
[46]Rob Krten. QNX Neutrino RTOS-Getting Started with QNX Neutrino:A Guide for Real-time Programmers. [EB/OL].2009.
[47]丁林松,黄丽琴.QT图形设计与嵌入式开发[M].北京：人民邮电出版社,2009.
[48]Jasmin Blanchette, Mark Summerfield. C++ GUI Programming with Qt 4.Prentice Hall. 2008,02.
[49]Mark Summerfield.Advanced Qt Programming.Addison-Wesley Professional.2010,08.
[50]Flavio E.Goncalves. Building Telephony Systems with OpenSER [M]. April,2008.
[2]汪延东.基于达芬奇平台的IP可视电话终端的研究[D].山东大学,2010.
[3]沈鑫剡.多媒体传输网络与VoIP系统设计[M].北京：人民邮电出版社,2005.
[4]Lau, Richard, Khare, Ram. Chang, William Y. Service assurance for voice over WiFi and 3G networks. Boston:Artech House, c2005.
[5]黄皎,吴春明Wi-Fi与嵌入式设备的融合技术研究[J].计算机工程,2011,04.
[6]于洁.基于SIP协议的VOIP客户端软件的设计与实现[J].电子科技大学,2006.
[7]贺迅.基于CELL异构多核处理器的H.264高清实时编码[D].上海交通大学,2008,11.
[8]陈国兵.嵌入式异构多核体系的片上通信[D].浙江大学,2007,05.
[9]TMS320DSP Algorithm Standard Rules and Guidelines. [EB/OL]. TI.2007,08.
[10]李洪,胥秦秦.H.264在ADSP-BF561上的实现与优化[J].电子技术应用,2009,02.
[11]张起贵,张胜,张刚.最新DSP技术：“达芬奇”系统、框架和组件.国防工业出版社,2009,08.
[12]彭启宗.达芬奇技术-数字图像/视频信号处理新平台.电子工业出版社,2008,09.
[13]DM3730, DM3725 Applications Processor. [EB/OL]. SPRS685.2010.
[14]陈路.达芬奇平台将大幅降低数字视频产品开发门槛.电子工程,2005,11.
[]5]赵勇DAVINCI技术原理与应用指南.东南大学出版社,2008,01.
[16]张辉.基于v412的嵌入式视频驱动程序开发与实现[D].安徽大学,2010,05.
[17]管必聪,吕红.基于TMS320VC5509的G729A算法实现[J].机电工程,2008,02.
[18]王原丽,方存果.基于G.729A语音编解码系统的优化实现[J].武汉理工大学学报,2010,10.
[19]xDAIS-DM (Digital Media) User Guide. [EB/OL]. SPRUEC8B. TI.2007.
[20]毕厚杰,王健.新一代视频压缩编码标准-H.264/AVC北京：人民邮电出版社,2005.
[21]陈兴.基于Davinci的图像压缩编解码与网络传输[D].北方工业大学,2007,06.
[22]高宇.基于H.264/AVC的压缩域视频场景转换检测技术的研究与应用[D].北京工业大学,2008,05.
[23]张晓燕,谢瑶堂.最新视频编码标准H.264及其核心技术[J].北京理工大学,2009.
[24]刘可钊,王树梅.多媒体通信中的音视频同步问题研究[D].南京理工大学,2007.
[25]林朝龙.基于RTP的网络视频传输的应用研究[J].信息化研究,2009,12.
[26]TMS320C6000 Programmer's Guide. [EB/OL]. SPRU198. TI.2010.
[27]TMS320 DSP/BIOS Useris Guide. [EB/OL]. SPRU423E. TI.2004.
[28]陈莉君Linux内核设计与实现.北京：机械工业出版社,2006,01.
[29]邵贝贝.嵌入式实时操作系统uC/OS-Ⅱ北京：北京航空航天大学出版社,2007.
[30]程敬原VxWorks软件开发项目实例完全解析.北京：中国电力出版社,2005,10.
[31]王斑.基于QNX实时系统的测试系统的开发[D].西北工业大学,2006,03.
[32]Rob Krten. QNX Neutrino RTOS-Getting Started with QNX Neutrino:A Guide for Real-time Programmers.2009.
[33]DVSDK Getting Started Guide. [EB/OL]. SPRUEG8. TI.2006.
[34]田黎育TMS320C6000系列DSP编程工具与指南/美国德州仪器公司著.清华大学出版社,2006,09.
[35]Codec Engine Algorithm Creator User's Guide. [EB/OL]. SPRUED6C. TI.2007,09.
[36]许邦建,孙永节.DSP算法与体系结构实现技术.北京：国防工业出版社,2010.
[37]宋建勋,刘峰.基于TMS320DM365多平台实时视频传输系统的设计与实现[J].电视技术,2011,07.
[38]刘微,李向荣.AVS解码器在DSP平台上的优化[J].电子器件,2010,01.
[39]齐美彬,陈俊峰.基于C64x+内核的H.264编码数据Cache优化[J],合肥工业大学学报,2011,04.
[40]TMS320C64x+DSP Cache User's Guide.SPRU862. [EB/OL]. TI.2011.
[41]刘泽新,何彬.SIP代理服务器的设计与实现[J].计算机系统应用,2005,2.
[42]黄永峰,李建庆.下一代网络核心控制协议：SIP及其应用.北京：人民邮电出版社,2009,01.
[43]徐炽,张闻芳.基于SIP协议的VOIP系统设计与实现[J].科学技术与工程,2008.
[44]Yuan Zhang. SIP-based VoIP network and its interworking with the PSTN[J]. Electronics&Communication Engineering Journal,2002,14(6):10-12.
[45]高玉龙.达芬奇技术开发基础、原理与实例.北京：电子工业出版社,2012,06.
[46]Rob Krten. QNX Neutrino RTOS-Getting Started with QNX Neutrino:A Guide for Real-time Programmers. [EB/OL].2009.
[47]丁林松,黄丽琴.QT图形设计与嵌入式开发[M].北京：人民邮电出版社,2009.
[48]Jasmin Blanchette, Mark Summerfield. C++ GUI Programming with Qt 4.Prentice Hall. 2008,02.
[49]Mark Summerfield.Advanced Qt Programming.Addison-Wesley Professional.2010,08.
[50]Flavio E.Goncalves. Building Telephony Systems with OpenSER [M]. April,2008.