基于MPEG-7标准的视频描述与检索
|
摘要
随着计算机以及通信技术的发展，以视频为代表的多媒体数据量和信息量急剧增长。视频数据的日益增加，应用也越来越广泛。现在，在数字图书馆、军事信息系统、Web信息环境、专业视频库等应用，需要对视频数据和视频信息进行组织和管理。同传统的文字信息相比较，视频数据具有信息量大、难以准确描述的特点，因此人们很难从海量的视频信息中找到自己所需的信息。虽然过去开展了大量的视频数据库、视频分析和信息检索的研究，但是缺乏对视频数据进行完整的、规范性的描述，以及建立在这些规范描述之上的视频信息检索方法。本文在分析研究了现有一些基于内容的视频处理和检索方法的基础上，结合MPEG-7标准的新框架，对视频内容规范描述及其检索方法进行了研究，主要的研究工作如下：
1．视频内容分析和规范化描述：根据MPEG-7标准，首先对视频内容进行分析，然后进行规范化的描述。本文在视频内容分析的基础之上，建立了基于MPEG-7标准的视频内容描述模型。该模型从视频数据的特性出发，既综合考虑了视频的各种特征，包括视觉特征、对象空间关系和时间结构，又充分考虑了视频信息检索的要求，采用层次化的描述结构。
2．视频内容描述工具的设计和实现：根据描述定义语言，建立基于MPEG-7标准的视频内容描述工具。描述工具建立的基础是上面提出的描述模型，它对于视频的结构特征，可以实现特征的自动提取并自动生成描述，而语义信息则可以通过手工输入一些基本信息的基础上自动生成描述。根据本文设计的视频内容描述工具，可以建立适合于视频检索的标准化描述，该描述的最终结果采用W3C的XML语言模式。
3．提出一种基于倒排索引的视频索引机制及其索引建立算法：根据上面的描述结果，设计了一种基于倒排索引的视频索引机制及其索引建立算法。在该算法中，设计了更能够代表媒体数据内容的特征以及高效的索引结构。该结构与文档中倒排序方法相结合，提出了一种基于MPEG-7标准的倒排索引机制及其索引建立算法。
4．建立在索引基础上的快速检索算法及其实现：在比较现有XML文档索引和检索算法的基础之上，本文设计并实现了一种基于XML文档的快速检索算法。该算法充分利用了上面提出的倒排序视频索引机制。本文还通过一系列实验对现有的一些XML文档检索算法以及本文设计的检索算法进行对比，最后的实验结果说明了本文算法的有效性和高效性。
综上所述，本文在探讨标准化描述方法的基础上，建立了一个基于MPEG-7标准的视频数据库以及由此建立一套完整的基于内容的检索机制，它具有及其重要的意义。
With the rapid development of multimedia and communication technology, a great deal of video data and information has become available. At present, we need to organize and manage video data and information in many fields, such as digital library, military information system, Web information system and video database etc. Compared to traditional text information, video data contains too much information and is hard to describe, so it becomes difficult to find the proper information that we need. In the past, people have done much research on video database, video analysis and information retrieval, but there lacks complete and standardized description of video data, and video information retrieval method based on the standardized description. In this paper, based on the analysis of current video manipulation and retrieval methods, and according to the new framework of the MPEG-7 standard, the author raised a new method to describe and retrieve video content. The main work includes:
1. Video content analysis and standardized description: According to the MPEG-7 standard, first, the author analyzed video content, then made standardized description of it. Based on the analysis of video content, this paper gives a model of video content description using the MPEG-7 standard. The model starts from the features of video data, it takes into consideration not only structural features of video including visual and spatio-temporal, but also the need to video information retrieval. The model applies a hierarchical structure.
2. Design and implementation of the video content description tool: According to Description Definition Language (DDL), this paper gives a content description tool based on the MPEG-7 standard. The base of the description tool is the description model mentioned in the paper. The tool can automatically extract structural features of video and create the description. Besides, given some necessary information, the tool can also automatically create the description of semantic information. According to the video content description tool, we can build standardized description adapted to video retrieval. The final format of the description uses XML recommended by W3C.
3. A mechanism of video indexing as well as its setting up algorithm based on inverted index: In this algorithm, the author designs an efficient index structure that can better contain features of media data contents. This mechanism of video index is based on invert index, which is similar to the one used in traditional document retrieval.
4. A rapid retrieval algorithm based on the invert index and its implementation: Based on the comparison to the current XML document index and retrieval algorithms, this paper give the rapid retrieval algorithm based on XML document. In the end, the author lists some results of the experiments that compares some current XML document retrieval algorithms to the algorithms given in this paper. These results prove the availability and efficiency of the algorithm.
In a word, based on the method of standardized description, this paper builds a video database and a complete retrieval mechanism based on video contents.
1．视频内容分析和规范化描述：根据MPEG-7标准，首先对视频内容进行分析，然后进行规范化的描述。本文在视频内容分析的基础之上，建立了基于MPEG-7标准的视频内容描述模型。该模型从视频数据的特性出发，既综合考虑了视频的各种特征，包括视觉特征、对象空间关系和时间结构，又充分考虑了视频信息检索的要求，采用层次化的描述结构。
2．视频内容描述工具的设计和实现：根据描述定义语言，建立基于MPEG-7标准的视频内容描述工具。描述工具建立的基础是上面提出的描述模型，它对于视频的结构特征，可以实现特征的自动提取并自动生成描述，而语义信息则可以通过手工输入一些基本信息的基础上自动生成描述。根据本文设计的视频内容描述工具，可以建立适合于视频检索的标准化描述，该描述的最终结果采用W3C的XML语言模式。
3．提出一种基于倒排索引的视频索引机制及其索引建立算法：根据上面的描述结果，设计了一种基于倒排索引的视频索引机制及其索引建立算法。在该算法中，设计了更能够代表媒体数据内容的特征以及高效的索引结构。该结构与文档中倒排序方法相结合，提出了一种基于MPEG-7标准的倒排索引机制及其索引建立算法。
4．建立在索引基础上的快速检索算法及其实现：在比较现有XML文档索引和检索算法的基础之上，本文设计并实现了一种基于XML文档的快速检索算法。该算法充分利用了上面提出的倒排序视频索引机制。本文还通过一系列实验对现有的一些XML文档检索算法以及本文设计的检索算法进行对比，最后的实验结果说明了本文算法的有效性和高效性。
综上所述，本文在探讨标准化描述方法的基础上，建立了一个基于MPEG-7标准的视频数据库以及由此建立一套完整的基于内容的检索机制，它具有及其重要的意义。
With the rapid development of multimedia and communication technology, a great deal of video data and information has become available. At present, we need to organize and manage video data and information in many fields, such as digital library, military information system, Web information system and video database etc. Compared to traditional text information, video data contains too much information and is hard to describe, so it becomes difficult to find the proper information that we need. In the past, people have done much research on video database, video analysis and information retrieval, but there lacks complete and standardized description of video data, and video information retrieval method based on the standardized description. In this paper, based on the analysis of current video manipulation and retrieval methods, and according to the new framework of the MPEG-7 standard, the author raised a new method to describe and retrieve video content. The main work includes:
1. Video content analysis and standardized description: According to the MPEG-7 standard, first, the author analyzed video content, then made standardized description of it. Based on the analysis of video content, this paper gives a model of video content description using the MPEG-7 standard. The model starts from the features of video data, it takes into consideration not only structural features of video including visual and spatio-temporal, but also the need to video information retrieval. The model applies a hierarchical structure.
2. Design and implementation of the video content description tool: According to Description Definition Language (DDL), this paper gives a content description tool based on the MPEG-7 standard. The base of the description tool is the description model mentioned in the paper. The tool can automatically extract structural features of video and create the description. Besides, given some necessary information, the tool can also automatically create the description of semantic information. According to the video content description tool, we can build standardized description adapted to video retrieval. The final format of the description uses XML recommended by W3C.
3. A mechanism of video indexing as well as its setting up algorithm based on inverted index: In this algorithm, the author designs an efficient index structure that can better contain features of media data contents. This mechanism of video index is based on invert index, which is similar to the one used in traditional document retrieval.
4. A rapid retrieval algorithm based on the invert index and its implementation: Based on the comparison to the current XML document index and retrieval algorithms, this paper give the rapid retrieval algorithm based on XML document. In the end, the author lists some results of the experiments that compares some current XML document retrieval algorithms to the algorithms given in this paper. These results prove the availability and efficiency of the algorithm.
In a word, based on the method of standardized description, this paper builds a video database and a complete retrieval mechanism based on video contents.
引文
[1] 李国辉．基于内容的多媒体信息存取技术，计算机世界专题．2000．5．29．
[2] Overview of the MPEG-7 Standard. ISO/IEC JTC1/SC29/W611 N4509 Pattaya, December 2001.
[3] 李国辉．基于内容的多媒体信息存取与MPEG-7．计算机世界专题．2000．5．29．
[4] MPEG-7: ISO/IEC JTC 1/SC 29/WG 11/N3966March 2001, Singapore
[5] MPEG-7 Applications, Demos and Projects. ISO/IEC JTC1/SC29/WG11, N3546. Beijing, July 2000
[6] D. Zhong and S. -F. Chang, "An Integrated System for Content-Based Video Object Segmentation and Retrieval", IEEE Transactions on Circuits and Systems for Video Technology, Vol. 9, No. 8, pp. 1259-1268, Dec. 1999.
[7] 李国辉．MPEG-7 应用．计算机世界专题．2000．5．29．
[8] 李国辉，曹莉华，多媒体数据库系统的设计考虑，第六届全国多媒体技术学术会议论文集，1997．10。
[9] 李国辉，王辰，薛峰，几种典型的基于内容检索系统，计算机世界报技术专题， 1998．5．18。
[10] Guohui Li, Jun Zhang and Defeng Wu. A Content-based Multimedia Database Engine: MIR. The Second IEEE Pacific-Rim Conference on Multimedia, Beijing, 2001.
[11] 李国辉等，．MPEG-7 的概念和 MPEG-21 的启动．世界网络与多媒体， 2001，9(4)：33-35．
[12] 李国辉、曹莉华等．基于内容的多媒体数据查询和检索．小型微型计算机系统．vol．19，No．4，1998．pp1-8．
[13] A. B. Benitez and J. R. Smith, "New Frontiers for Intelligent Content-Based Retrieval", Proceedings of the IS&T/SPIE 2001 Conference on Storage and Retrieval for Media Databases, Vol. 4315, San Jose, CA, Jan. 24-26, 2001.
[14] 胡晓峰、李国辉．多媒体系统，人民邮电出版社，1997．9。
[15] 李国辉，曹莉华，柳伟，基于内容检索的概念和方法，全国第五届多媒体技术学术会议论文集，1996．10．
[16] 柳伟，曹莉华，基于内容的图像检索技术，计算机世界报技术专题，1998．5．18。
[17] 李国辉等，信息组织与检索，科学出版社。
[18] A. B. Benitez and S. -F. Chang, "Validation Experiments on Structural, Conceptual, Collection, and Access Description Schemes for MPEG-7", Digest of the IEEE 2000 International Conference on Consumer Electronics (ICCE-2000), Los Angeles, CA, June 13-15, 2000.
[19] A. B. Benitez and J. R. Smith, "New Frontiers for Intelligent Content-Based Retrieval", Proceedings of the IS&T/SPIE 2001 Conference on Storage and Retrieval for Media Databases, Vol. 4315, San Jose, CA, Jan. 24-26, 2001.
[20] MPEG, Working Documents for MPEG-7 Standard, http://www.cselt.it/mpeg/working_documents.htm.
[21] Y. C. Chang, M. L. Lo, J. R. Smith, "Issues and solutions for storage, retrieval, and search of MPEG-7 documents", Proceedings of IS&T/SPIE 2000 Conference on Internet Multimedia Management Systems, Vol. 4210, Boston, MA, Nov. 6-8, 2000.
[22] ISO MPEG-7. Text of ISO/IEC CD 15938-1 Information Technology-Multimedia Content Description Interface-Part 1 Systems, ISO/IEC JTC 1/SC 29/WG 11 N3701, October 2000
[23] ISO MPEG-7. Text of ISO/IEC CD 15938-2 Information Technology-Multimedia Content Description Interface-Part 2 Description Definition Language, ISO/IEC JTC 1/SC 29/WG 11 N3702, October 2000
[24] ISO MPEG-7. Text of ISO/IEC CD 15938-4 Information Technology-Multimedia Content Description Interface-Part 4 Audio, ISO/IEC JTC 1/SC 29/WG 11 N3704, October 2000
[25] ISO MPEG-7. Text of ISO/IEC CD 15938-3 Information Technology-Multimedia Content Description Interface-Part 3 Visual, ISO/IEC JTC 1/SC 29/WG 11 N3703, October 2000
[26] ISO MPEG-7. Text of ISO/IEC 15938-5/CD Information Technology-Multimedia Content Description Interface-Part 5 Multimedia Description Schemes, ISO/IEC JTC 1/SC 29/WG 11 N3705, October 2000
[27] MPEG MDS Group, "Text of ISO/IEC 15938-5 FCD Information Technology-Multimedia Content Description Interface-Part 5 Multimedia Description Schemes", ISO/IEC JTCI/SC29/WG11 MPEGO1/M7O09, Singapore, March 2001.
[28] 汤义，李国辉，倪泞，MPEG-7 标准描述多媒体内容的方法，计算机工程与科学，已录用。
[29] 汤义，李国辉，倪泞，基于 MPEG-7 标准的视频描述，微处理机，已录用。
[30] MPEG MDS Group, "MPEG-7 Wultimedia Description Schemes XM (Version 7.0)", SO/IEC JTC 1/SC 29/WG 11/N3964 March 2001, Singapore.
[31] IEEE Transactions on Circuits and Systems for Video Technology, Special Issue on MPEG-7, June 2001.
[32] 倪泞，李国辉，汤义，一种基于 MPEG-7 的图像内容描述工具，第十一届全国多媒体技术学术会议论文集，2002．10。
[33] World Wide Web Consortium. XML Schema, Parts O, 1, and 2. W3C Working Draft, April 7, 2000. See http://www.w3.org/TR/xmlschema-O,-1,and-2.
[34] XML Schema Part O. Primer, W3C Candidate Recommendation, 24 October
2000, http://www.w3． org/TR/xmlschema-0/
[35] XML Schema Part 1． Structures, W3C Candidate Recommendation , 24 October 2000, http://www. w3． org/TR/xralschema-1/
[36] XML Schema Part 2． Datatypes, W3C Candidate Recommendation, 24 October 2000, http://www. w3． org/TR/xmlschema-2/
[37] Extensible Markup Language (XML) 1． 0 (Second Edition), W3C Recommendation, 6 October 2000, http://www. w3． org/TR/REC-xml
[38] (美)Didier Martin等著。XML高级编程。严春莹等译。机械工业出版社．
[39] Dongwook Shin Structured querying, indexing, and retrieval for SGML/XML documents. Proceedings of SGML/XML Japan ' 98, pp. 199-214．
[40] McHugh, J., Widom, J., Abiteboul, S., Luo, 0． and Rajaraman, A. Indexing semistructured data. In Stanford Technical Report, January 1998．
[41] R Wilkinson. Effective Retrieval of Structured Documents. ACM-SIGIR, Dublin, pp 311-317, 1994．
[42] Xiaoling Wang, JiRong Wan, YiSheng Dong and Liu WenYin. Enhancive Index for Structured Document Retrieval, 12th International Workshop on Research Issues on Data Engineering (RIDE2002) in Conjunction with ICDE' 02, Sun Jase, USA, February.
[2] Overview of the MPEG-7 Standard. ISO/IEC JTC1/SC29/W611 N4509 Pattaya, December 2001.
[3] 李国辉．基于内容的多媒体信息存取与MPEG-7．计算机世界专题．2000．5．29．
[4] MPEG-7: ISO/IEC JTC 1/SC 29/WG 11/N3966March 2001, Singapore
[5] MPEG-7 Applications, Demos and Projects. ISO/IEC JTC1/SC29/WG11, N3546. Beijing, July 2000
[6] D. Zhong and S. -F. Chang, "An Integrated System for Content-Based Video Object Segmentation and Retrieval", IEEE Transactions on Circuits and Systems for Video Technology, Vol. 9, No. 8, pp. 1259-1268, Dec. 1999.
[7] 李国辉．MPEG-7 应用．计算机世界专题．2000．5．29．
[8] 李国辉，曹莉华，多媒体数据库系统的设计考虑，第六届全国多媒体技术学术会议论文集，1997．10。
[9] 李国辉，王辰，薛峰，几种典型的基于内容检索系统，计算机世界报技术专题， 1998．5．18。
[10] Guohui Li, Jun Zhang and Defeng Wu. A Content-based Multimedia Database Engine: MIR. The Second IEEE Pacific-Rim Conference on Multimedia, Beijing, 2001.
[11] 李国辉等，．MPEG-7 的概念和 MPEG-21 的启动．世界网络与多媒体， 2001，9(4)：33-35．
[12] 李国辉、曹莉华等．基于内容的多媒体数据查询和检索．小型微型计算机系统．vol．19，No．4，1998．pp1-8．
[13] A. B. Benitez and J. R. Smith, "New Frontiers for Intelligent Content-Based Retrieval", Proceedings of the IS&T/SPIE 2001 Conference on Storage and Retrieval for Media Databases, Vol. 4315, San Jose, CA, Jan. 24-26, 2001.
[14] 胡晓峰、李国辉．多媒体系统，人民邮电出版社，1997．9。
[15] 李国辉，曹莉华，柳伟，基于内容检索的概念和方法，全国第五届多媒体技术学术会议论文集，1996．10．
[16] 柳伟，曹莉华，基于内容的图像检索技术，计算机世界报技术专题，1998．5．18。
[17] 李国辉等，信息组织与检索，科学出版社。
[18] A. B. Benitez and S. -F. Chang, "Validation Experiments on Structural, Conceptual, Collection, and Access Description Schemes for MPEG-7", Digest of the IEEE 2000 International Conference on Consumer Electronics (ICCE-2000), Los Angeles, CA, June 13-15, 2000.
[19] A. B. Benitez and J. R. Smith, "New Frontiers for Intelligent Content-Based Retrieval", Proceedings of the IS&T/SPIE 2001 Conference on Storage and Retrieval for Media Databases, Vol. 4315, San Jose, CA, Jan. 24-26, 2001.
[20] MPEG, Working Documents for MPEG-7 Standard, http://www.cselt.it/mpeg/working_documents.htm.
[21] Y. C. Chang, M. L. Lo, J. R. Smith, "Issues and solutions for storage, retrieval, and search of MPEG-7 documents", Proceedings of IS&T/SPIE 2000 Conference on Internet Multimedia Management Systems, Vol. 4210, Boston, MA, Nov. 6-8, 2000.
[22] ISO MPEG-7. Text of ISO/IEC CD 15938-1 Information Technology-Multimedia Content Description Interface-Part 1 Systems, ISO/IEC JTC 1/SC 29/WG 11 N3701, October 2000
[23] ISO MPEG-7. Text of ISO/IEC CD 15938-2 Information Technology-Multimedia Content Description Interface-Part 2 Description Definition Language, ISO/IEC JTC 1/SC 29/WG 11 N3702, October 2000
[24] ISO MPEG-7. Text of ISO/IEC CD 15938-4 Information Technology-Multimedia Content Description Interface-Part 4 Audio, ISO/IEC JTC 1/SC 29/WG 11 N3704, October 2000
[25] ISO MPEG-7. Text of ISO/IEC CD 15938-3 Information Technology-Multimedia Content Description Interface-Part 3 Visual, ISO/IEC JTC 1/SC 29/WG 11 N3703, October 2000
[26] ISO MPEG-7. Text of ISO/IEC 15938-5/CD Information Technology-Multimedia Content Description Interface-Part 5 Multimedia Description Schemes, ISO/IEC JTC 1/SC 29/WG 11 N3705, October 2000
[27] MPEG MDS Group, "Text of ISO/IEC 15938-5 FCD Information Technology-Multimedia Content Description Interface-Part 5 Multimedia Description Schemes", ISO/IEC JTCI/SC29/WG11 MPEGO1/M7O09, Singapore, March 2001.
[28] 汤义，李国辉，倪泞，MPEG-7 标准描述多媒体内容的方法，计算机工程与科学，已录用。
[29] 汤义，李国辉，倪泞，基于 MPEG-7 标准的视频描述，微处理机，已录用。
[30] MPEG MDS Group, "MPEG-7 Wultimedia Description Schemes XM (Version 7.0)", SO/IEC JTC 1/SC 29/WG 11/N3964 March 2001, Singapore.
[31] IEEE Transactions on Circuits and Systems for Video Technology, Special Issue on MPEG-7, June 2001.
[32] 倪泞，李国辉，汤义，一种基于 MPEG-7 的图像内容描述工具，第十一届全国多媒体技术学术会议论文集，2002．10。
[33] World Wide Web Consortium. XML Schema, Parts O, 1, and 2. W3C Working Draft, April 7, 2000. See http://www.w3.org/TR/xmlschema-O,-1,and-2.
[34] XML Schema Part O. Primer, W3C Candidate Recommendation, 24 October
2000, http://www.w3． org/TR/xmlschema-0/
[35] XML Schema Part 1． Structures, W3C Candidate Recommendation , 24 October 2000, http://www. w3． org/TR/xralschema-1/
[36] XML Schema Part 2． Datatypes, W3C Candidate Recommendation, 24 October 2000, http://www. w3． org/TR/xmlschema-2/
[37] Extensible Markup Language (XML) 1． 0 (Second Edition), W3C Recommendation, 6 October 2000, http://www. w3． org/TR/REC-xml
[38] (美)Didier Martin等著。XML高级编程。严春莹等译。机械工业出版社．
[39] Dongwook Shin Structured querying, indexing, and retrieval for SGML/XML documents. Proceedings of SGML/XML Japan ' 98, pp. 199-214．
[40] McHugh, J., Widom, J., Abiteboul, S., Luo, 0． and Rajaraman, A. Indexing semistructured data. In Stanford Technical Report, January 1998．
[41] R Wilkinson. Effective Retrieval of Structured Documents. ACM-SIGIR, Dublin, pp 311-317, 1994．
[42] Xiaoling Wang, JiRong Wan, YiSheng Dong and Liu WenYin. Enhancive Index for Structured Document Retrieval, 12th International Workshop on Research Issues on Data Engineering (RIDE2002) in Conjunction with ICDE' 02, Sun Jase, USA, February.