基于H.264压缩域的视频对象分割
|
摘要
MPEG和H.264是当今主流的视频编码标准。MPEG-4标准首先提出基于内容的视频编码,有关视频对象分割的研究大多集中在像素域,而基于压缩域的视频对象分割直至近年来才开始引起关注。鉴于实际应用中大多数视频序列已经压缩为某种格式,直接在压缩域内进行视频对象分割,可免除对压缩视频进行完全解码;而且在压缩域内需要处理的数据量也比像素域少很多,因此计算量大大减少,存储数据的空间也大大减少;此外通过从压缩视频熵解码提取出的运动信息可直接用作分割所需的运动特征和纹理特征。因此,从压缩域分割视频对象具有快速的特点,可解决传统的像素域分割难于满足实时分割的要求,更适合于有实时性要求的应用场合。研究基于压缩域的视频对象分割具有较强的理论上和应用上的意义。
H.264是最新的视频编码标准,相比于MPEG编码效率提高了一倍,目前越来越多的应用都在转向采用H.264来取代MPEG。本文基于从H.264压缩域提取出来的运动信息(包括宏块编码模式和运动场),进行视频对象分割。另一方面应指出的是,由于从H.264压缩域提取的运动信息不能完全反映对象的真实运动,所以基于H.264压缩视频的运动分割存在很大困难,体现在:(1)H.264压缩域能用于对象分割的信息比较有限,仅可使用的是宏块编码模式和运动场这两个运动信息,仅靠这些信息提取视频对象比较困难;(2)H.264压缩域的运动信息不够准确,宏块编码模式不能完全反映视频帧背景,运动场不代表对象的真实运动,这些问题使得分割质量难以提高;(3)因为要在压缩域内同时保证分割速度和分割质量,现有的许多像素域的运动分割方法难以应用。
本文通过对上述问题的研究,解决了其中的一些技术难题,包括运动信息的致密化和准确化,利用宏块编码模式辅助检测背景,利用运动场的时空相关性提高分割的效率和质量,结合区域分割和区域分类的方法提高分割的有效性,基于高效的运动估计方法检测运动背景以此简化能量函数的构造。本文的主要工作和创新点包括:
(1)对于原始运动场过于稀疏和噪声大的问题,提出一种有效的运动场预处理技术。先采用空时滤波方法去除属于噪声的伪运动矢量,然后采用后向估计,前向投影的运动场累积方法获得稠密的运动场。
(2)仅利用运动场进行分割难以提高对象分割的效率,提出一种以宏块编码模式辅助运动分割的方法。利用H.264的宏块编码模式所反映的视频帧背景信息进行运动估计,从而缩小了运动估计的范围,提高了对象分割速度。并基于得到的背景信息,采用χ2假设检验的方法提取视频对象。
(3)为更准确的区分对象区域与背景区域,提出一种结合区域生长和区域分类的对象分割方法。以幅度、散度、旋度三个运动特征来描述运动场,然后基于这些运动特征,采用改进的统计区域生长方法将运动场分割为不同的区域,接着采用四阶矩的分类方法进行区域分类,最后采用投影滤波的方法细化分割结果。该方法的特点在于通过先进行区域生长,后进行区域分类的方式,有效地将对象区域与背景区域分开,从而提高了分割的准确性。
(4)针对基于马尔可夫随机场(MRF)的方法难于提取具有背景运动的H.264视频对象的问题,提出一种基于Graph Cuts的视频对象分割方法。首先进行背景检测,采用跨度阈值的方法检测静止背景,结合运动估计的方法检测运动背景,利用检测到的背景信息简化了Graph能量函数的构造,从而可提高Graph Cuts优化算法的运算效率。本方法不仅适用于静止背景的视频对象分割,也适用于运动背景的视频对象分割,且分割质量优于同类方法。
MPEG and H.264 are the leading video coding standard these days. The MPEG standard first proposes the video coding based on the content. The video object segmentation in the compressed domain attracts attention in recent years while most work is focused on the pixel domain before. Since most videos are converted into the compressed format, the video object can be extracted in the compressed domain where the video need not be decompressed completely. At the same time, the computation load decrease and the necessary data storage reduce greatly because the processed data are much less in the pixel domain. In addition, the motion information extracted from the compressed video with entropy decoding are used as the motion characteristic and texture feature. Therefore, the fast object segmentation in the compressed domain can meet with the real time requirements which the traditional pixel domain methods fail to satisfy. The research on video object segmentation in the compressed domain has the academic meaning and applied meaning.
H.264 is the latest video coding standard and its coding efficiency is twice as the MPEG standard. More and more applications turn to the H.264 in place of MPEG. The video object segmentation in this dissertation uses the motion information including the macro-block coding mode and motion field extracted from the H.264 compressed domain. On the other side, some difficulties exist in the object segmentation in the H.264 compressed domain because the extracted motion information can not reflect the true motion. These difficulties are: (1) The required segmentation information in the H.264 compressed domain is limited, the segmentation task in this dissertation is rather hard only based on the macro-block coding mode and motion filed. (2) The motion information in the H.264 compressed domain is not accurate, the macro-block coding mode cannot reflect the video background entirely and the motion field cannot represent the true motion. These problems make it hard to improve the segmentation quality. (3) Many current motion segmentation methods in the pixel domain are not easy to fulfill the requirements of both segmentation speed and segmentation quality in the compressed domain.
Based on the research on the above problems, some trouble issues including motion information densification and correction, the background detection via macro-block coding mode, the improvement for efficiency and quality of segmentation via the spatio-temporal correlation, the improvement for segmentation effectiveness by combining region segmentation and region classification as well as the background detection by fast motion estimation as to simplify the energy function formulation. The major work and innovations in this dissertation are listed as following:
(1) The effective motion preprocessing is proposed to resolve the initial sparse and noisy motion field. The spatio-temporal filtering is exploited to remove the noisy motion vectors. Then the backward estimation and forward projection are used to obtain the dense motion field.
(2) The macro-block coding mode is used in the object segmentation to improve the processing speed because it is hard to increase the segmentation efficiency based on motion field only. The background of video frames indicated by H.264 macro-block coding mode can narrow the range of motion estimation and improve the processing speed. At last, theχ2 testing is used to extract the video object based on the estimated background.
(3) In order to classify object region and background region more accurately, the method combining region growing and region classification is proposed in the dissertation. At first, the magnitude, the divergence and the curl are used to characterize the motion field, the modified statistical region growing is applied to divide the motion field into different regions. Then the 4-order moment is used for region classification and the final result is refined by the projection filtering. The way combining region growing and region classification can classify the object region and background region as well as improve the segmentation results.
(4) To the problem that Markov random filed (MRF) is hard to extract object from the H.264 video with moving background. The segmentation method using Graph Cuts is proposed in the dissertation. At beginning, the static background is detected by the span thresholding while the moving background is detected by the motion estimation. The obtained information about background simplifies the energy function of Graph as well as improves the computation efficiency of Graph Cuts method. The proposed methods can be used to extract object from the video with static background or moving background. Moreover, the segmentation quality is better than the similar method.
H.264是最新的视频编码标准,相比于MPEG编码效率提高了一倍,目前越来越多的应用都在转向采用H.264来取代MPEG。本文基于从H.264压缩域提取出来的运动信息(包括宏块编码模式和运动场),进行视频对象分割。另一方面应指出的是,由于从H.264压缩域提取的运动信息不能完全反映对象的真实运动,所以基于H.264压缩视频的运动分割存在很大困难,体现在:(1)H.264压缩域能用于对象分割的信息比较有限,仅可使用的是宏块编码模式和运动场这两个运动信息,仅靠这些信息提取视频对象比较困难;(2)H.264压缩域的运动信息不够准确,宏块编码模式不能完全反映视频帧背景,运动场不代表对象的真实运动,这些问题使得分割质量难以提高;(3)因为要在压缩域内同时保证分割速度和分割质量,现有的许多像素域的运动分割方法难以应用。
本文通过对上述问题的研究,解决了其中的一些技术难题,包括运动信息的致密化和准确化,利用宏块编码模式辅助检测背景,利用运动场的时空相关性提高分割的效率和质量,结合区域分割和区域分类的方法提高分割的有效性,基于高效的运动估计方法检测运动背景以此简化能量函数的构造。本文的主要工作和创新点包括:
(1)对于原始运动场过于稀疏和噪声大的问题,提出一种有效的运动场预处理技术。先采用空时滤波方法去除属于噪声的伪运动矢量,然后采用后向估计,前向投影的运动场累积方法获得稠密的运动场。
(2)仅利用运动场进行分割难以提高对象分割的效率,提出一种以宏块编码模式辅助运动分割的方法。利用H.264的宏块编码模式所反映的视频帧背景信息进行运动估计,从而缩小了运动估计的范围,提高了对象分割速度。并基于得到的背景信息,采用χ2假设检验的方法提取视频对象。
(3)为更准确的区分对象区域与背景区域,提出一种结合区域生长和区域分类的对象分割方法。以幅度、散度、旋度三个运动特征来描述运动场,然后基于这些运动特征,采用改进的统计区域生长方法将运动场分割为不同的区域,接着采用四阶矩的分类方法进行区域分类,最后采用投影滤波的方法细化分割结果。该方法的特点在于通过先进行区域生长,后进行区域分类的方式,有效地将对象区域与背景区域分开,从而提高了分割的准确性。
(4)针对基于马尔可夫随机场(MRF)的方法难于提取具有背景运动的H.264视频对象的问题,提出一种基于Graph Cuts的视频对象分割方法。首先进行背景检测,采用跨度阈值的方法检测静止背景,结合运动估计的方法检测运动背景,利用检测到的背景信息简化了Graph能量函数的构造,从而可提高Graph Cuts优化算法的运算效率。本方法不仅适用于静止背景的视频对象分割,也适用于运动背景的视频对象分割,且分割质量优于同类方法。
MPEG and H.264 are the leading video coding standard these days. The MPEG standard first proposes the video coding based on the content. The video object segmentation in the compressed domain attracts attention in recent years while most work is focused on the pixel domain before. Since most videos are converted into the compressed format, the video object can be extracted in the compressed domain where the video need not be decompressed completely. At the same time, the computation load decrease and the necessary data storage reduce greatly because the processed data are much less in the pixel domain. In addition, the motion information extracted from the compressed video with entropy decoding are used as the motion characteristic and texture feature. Therefore, the fast object segmentation in the compressed domain can meet with the real time requirements which the traditional pixel domain methods fail to satisfy. The research on video object segmentation in the compressed domain has the academic meaning and applied meaning.
H.264 is the latest video coding standard and its coding efficiency is twice as the MPEG standard. More and more applications turn to the H.264 in place of MPEG. The video object segmentation in this dissertation uses the motion information including the macro-block coding mode and motion field extracted from the H.264 compressed domain. On the other side, some difficulties exist in the object segmentation in the H.264 compressed domain because the extracted motion information can not reflect the true motion. These difficulties are: (1) The required segmentation information in the H.264 compressed domain is limited, the segmentation task in this dissertation is rather hard only based on the macro-block coding mode and motion filed. (2) The motion information in the H.264 compressed domain is not accurate, the macro-block coding mode cannot reflect the video background entirely and the motion field cannot represent the true motion. These problems make it hard to improve the segmentation quality. (3) Many current motion segmentation methods in the pixel domain are not easy to fulfill the requirements of both segmentation speed and segmentation quality in the compressed domain.
Based on the research on the above problems, some trouble issues including motion information densification and correction, the background detection via macro-block coding mode, the improvement for efficiency and quality of segmentation via the spatio-temporal correlation, the improvement for segmentation effectiveness by combining region segmentation and region classification as well as the background detection by fast motion estimation as to simplify the energy function formulation. The major work and innovations in this dissertation are listed as following:
(1) The effective motion preprocessing is proposed to resolve the initial sparse and noisy motion field. The spatio-temporal filtering is exploited to remove the noisy motion vectors. Then the backward estimation and forward projection are used to obtain the dense motion field.
(2) The macro-block coding mode is used in the object segmentation to improve the processing speed because it is hard to increase the segmentation efficiency based on motion field only. The background of video frames indicated by H.264 macro-block coding mode can narrow the range of motion estimation and improve the processing speed. At last, theχ2 testing is used to extract the video object based on the estimated background.
(3) In order to classify object region and background region more accurately, the method combining region growing and region classification is proposed in the dissertation. At first, the magnitude, the divergence and the curl are used to characterize the motion field, the modified statistical region growing is applied to divide the motion field into different regions. Then the 4-order moment is used for region classification and the final result is refined by the projection filtering. The way combining region growing and region classification can classify the object region and background region as well as improve the segmentation results.
(4) To the problem that Markov random filed (MRF) is hard to extract object from the H.264 video with moving background. The segmentation method using Graph Cuts is proposed in the dissertation. At beginning, the static background is detected by the span thresholding while the moving background is detected by the motion estimation. The obtained information about background simplifies the energy function of Graph as well as improves the computation efficiency of Graph Cuts method. The proposed methods can be used to extract object from the video with static background or moving background. Moreover, the segmentation quality is better than the similar method.
引文
[1]张兆杨,杨高波,刘志等,视频对象分割提取的原理与应用,科学出版社,2009
[2] ISO/IEC JTC1/WG11/N4043. Overview of the MPEG-4 standard. March, 2001
[3] R.C. Gonzalez, R.E. Woods, Digital Image Processing, Prentice Hall, 2002
[4] T. Wiegand, G. Sullivan, G. Bjontegaard, et al., Overview of the H.264/AVC video coding standard, IEEE Transactions on Circuits and Systems for Video Technology, vol.13, no.7, pp560-576, 2003
[5] T. Stockhammer, M. M. Hannuksela and T. Wigand, H.264/AVC in wireless environments, IEEE Transactions on Circuits and Systems for Video Technology, vol.13, no.7, pp657-673, 2003
[6] S. Wenger, H.264/AVC over IP, IEEE Transactions on Circuits and Systems for Video Technology, vol.13, no.7, pp645-656, 2003
[7] H. Xu, A.A. Younis and M.R. Kabuka, Automatic moving object extraction of content-based applications, IEEE Transactions on Circuit and Systems for Video Technology, vol.14, no.6, pp.796-812,2004
[8] E. Izquierdo and M. Ghanbari, Key components for an advanced segmentation system, IEEE Transactions on Multimedia, vol.4, pp97-113, 2002
[9] S. Wu, A. Amin, Automatic thresholding of gray-level using multi-stage approach, IEEE Proceedings of Seventh International Conference on Document Analysis and Recognition, pp493-497,2003
[10] L. Vincent, Grayscale area openings and closings, their efficient implementation and application, EURASIP Workshop on Mathematical Morphology and Its Applications to Signal Processing, Spain, pp22-27,1993
[11] H.D. Cheng, X.H. Jiang and Y.Sun et al., Color image segmentation: advantages and prospects, Pattern Recognition, vol.34, pp2259-2281,2001
[12] D.Comaniciu and P. Meer, Mean shift: a robust approach toward feature space analysis, IEEE Transactions on Pattern Analysis Machine Intelligence, vol.24, no.5, pp603-619, 2005
[13] M. Ozden, E. Polat, A color image segmentation approach for content-based image retrieval, Pattern Recognition, vol.40, pp1318-1325,2007
[14] C.H. Chen, L.F. Pau, Texture analysis, the handbook of pattern recognition and computer vision, World Scientific Publishing, 1998
[15] C.I. Chang, Y. Du and J. Wang, et al., Survey and comparative analysis of entropy and relative entropy thresholding techniques, IEE Proceedings on Visual Image Signal Process, vol.153, no.6, pp837-850, 2006
[16] R.J. Radke, S. Andra and O. Al-Kofahi et al., Image change detection algothms: a systematic survey, IEEE transactions on image processing, vol.14, no.3, pp294-307, 2005
[17] Sukmarg O, Rao K R. Fast object detection and segmentation in MPEG compressed domain, Proceedings of IEEE TENCON, Malaysia, vol.3, pp364-368,2000
[18] C.D. Creusere, G. Dahman, Object detection and localization in compressed video, Proceedings of Asilomar Conference on Signals, Systems and Computers, Pacific Grove, CA, vol.1, pp93–97, 2001
[19] S. Ji and H.W. Park, Moving object segmentation in DCT-based compressed video, Electronics Letters, vol.36, no.21, pp1769-1770,2000
[20] H.L. Eng and K.K. Ma, Unsupervised image object segmentation over compressed domain, International Conference on Image Processing, vol.3, pp758-761,2000
[21] Y. Zhong, H.J. Zhang, A.K. Jain, Automatic caption localization in compressed video, IEEE International Conference on Image Processing, vol.2, pp96-100, 1999
[22] S. Ji, H.W. Park, Moving-object segmentation with adaptive sprite for DCT-based video coder, IEEE Conference on Image Processing, vol.3, pp566-569, 2001
[23] H.L. Wang, S.F. Chang, Automatic face region detection in MPEG video sequences, Proceedings of SPIE Electronic Imaging and Multimedia Systems, Vol. 2898, pp 160-168,1996
[24] M.T. Coimbra, M. Davies, Approximating optical flow within the MPEG-2 compressed domain, IEEE Transactions on Circuits System Video Technology, vol.15, no.1,pp 103-107,2005
[25] V. Mezaris, I Kompatsiaris, N V Boulgouris et al., Real-time compressed domain spatiotemporal segmentation and ontologies for video indexing and retrieval, IEEE Transactions on Circuits System Video Technology, vol.14, no.5, pp606-621,2004
[26] R. Venkatesh, K.R. Ramakrishnan, S.H. Srinivasan. Video object segmentation: A compressed domain approach, IEEE Transactions Circuits System Video Technology, vol.14, no.4, pp462-474, 2004
[27] D. Schonfeld, D. Lelescu, VORTEX: video retrieval and tracking from compressed multimedia databases - multiple object tracking from MPEG-2 bitstream, Journal of Visual Communication and Image Representative, no.11, pp154–182, 2000
[28] G. D. Borshukov, G. Bozdagi, Y. Altunbasak, Motion segmentation by multistage affine classification, IEEE Transactions on Image Processing,vol.6, no.11, pp1591-1594, 1997
[29] Z.H. Wang, G.Z. Liu and L. Liu, A Fast and accurate video object detection and segmentation method in the compressed domain, IEEE International Conference of Neural Networks and Signal Processing, pp1209-1212, 2003
[30] M.L. Jamrozik, M.H. Hayes, A compressed domain video object segmentation system, IEEE International Conference on Image Processing, vol.1, pp113-116, 2002
[31] F. Porikli, Real-time video object segmentation for MPEG encoded video sequences, SPIE Conference on Real-Time Imaging VIII, vol.5297, pp195-203, 2004
[32] H.T. Zen, T.H. Wa, Moving object detection from MPEG coded picture, International Conference on Image Processing, pp25-29, 1999
[33] A. Benzougar, P. Bouthemy, MRF-based moving object detection from MPEG coded video, International Conference on Image Processing, vol.3, pp402-405, 2001
[34] X.D. Yu, L.Y. Duan, Q. Tian, Robust moving video object segmentation in the MPEG compressed domain, International Conference on Image Processing, vol.3, pp933-936,2003
[35] M. Coimbra, M. Davies, Segmentation of moving pedestrians within the compressed domain, IEEE International Conferences on Acoustics, Speech, and Signal Processing, vol.3, pp605-608, 2004
[36] QIMERA, www.qimera.org
[37] SCHEMA, http://www.schema-ist.org
[38] V. Mezaris, I. Kompatsiaris and M. G. Strintzis, Object-based MPEG-2 video indexing and retrieval in a collaborative environment, Multimedia Tools and Applications Journal, vol.30, no.3, pp255-272, 2006
[39]刘志,面向基于内容多媒体应用的视频对象分割,上海交通大学博士论文,2005
[40]岑峰,视频监控系统中面向人的目标跟踪技术的研究,上海交通大学博士学位论文,2002
[41]杨高波,基于内容多媒体应用的语义视频对象提取及评价,上海大学博士论文,2004
[42]包红强,基于内容的视频运动对象分割技术研究,上海大学博士论文,2005
[43]徐剑峰,刘志,张兆杨,基于熵模型的压缩域运动对象分割,中国图象图形学报,vol.12, no.10, pp1815-1818,2007
[44] W Zeng, J Du, W Gao et al., Robust moving object segmentation on H.264/AVC compressed video using the block-based MRF model, Real-Time Imaging, vol.11, no.4, pp290-299, 2005
[45] Z Liu, Y Lu, Z.Y. Zhang, Real-time spatiotemporal segmentation of video objects in the H.264 compressed domain,Journal of Visual Communication and Image Representation, vol.18, no.3, pp275-290, 2007
[46]刘方青,石旭利,张兆杨,基于EM聚类的H.264压缩域视频对象实时分割算法,中国图象图形学报,vol.12,vol.10, pp1819-1822,2007
[47] Z. Liu, Z.Y. Zhang, L.Q. Shen, Moving object segmentation in the H.264 compressed domain, Optical Engineering, vol.46, no.1, pp0170031-0170035, 2007
[48] R. Cucchiara, A. Prati, R. Vezzani, Real-time motion segmentation from moving cameras, Real-Time Imaging, vol.10, pp127-143, 2004
[49] A. Martelli (1972). Edge detection using heuristic search methods. Computer Graphics and Image Processing 1, pp169–182, 1972
[50] J. H. Elder, A. Krupnik, L. A. Johnston, Contour grouping with prior models. IEEE Transactions on Pattern Analysis and Machine Intelligence vol.25, no.6, pp661–674, 2003
[51] D. Geiger, A. Gupta, L. A. Costa, J. Vlontzos ,Dynamic programming for detecting, tracking, and matching deformable contours. IEEE Transactions on Pattern Analysis and Machine Intelligence ,vol.17, no.3, pp294–302,1995
[52] J. Illingworth, J. Kittler, Survey of the Hough transform. Computer Vision, Graphics, and Image Processing, vol.44, no.1, pp87–116,1988
[53] A. Bovik, Handbook of Video and Image Processing, 2nd edition, Academic Press, 2005
[54] I. A. Ismaili, D. F. Gillies, Colour image segmentation using regression analysis in RGB space. Machine Graphics & Vision vol.3, no.1, pp373–384,1994
[55] L. Vincent, P. Soille, Watersheds in digital spaces: An efficient algorithm based on immersion simulations. IEEE Transactions on Pattern Analysis and Machine Intelligence vol.13,no.6,pp583–598,1991
[56] B. Cramariuc, M. Gabbouj, and J. Astola. Clustering based region growing algorithm for color image segmentation. In Proc. of the th13 Int’l Conf. on Digital Signal Processing, volume 2, pp857–860, July 1997
[57] V. Rehrmann and L. Priese. Fast and robust segmentation of natural color scenes. In Proc. of Annual Conf. Computer Vision, volume 1, pp598–606, 1998
[58] M. Celenk. Hierarchical color clustering for segmentation of textured images. In Proc. of the 29 Southeastern Symposium on System Theory, pp483–487, March 1997
[59] K. Saarinen. Color image segmentation by a watershed algorithm and region adjacency graph processing. In Proc. of 1994 Int’l Conf. on Image Processing volume III, pp1021–1025, 1994
[60] T. Gevers and A.W.M. Smeulders. Combining region splitting and edge detection through guided Delaunay image subdivision. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp1021–1026, 1997
[61] B. K. P. Horn, and B. G. Schunck, Determining optical flow, Artificial Intelligence, Vol. 17, pp 105–203, 1981
[62] J. W. Brandt, Improved accuracy in gradient-based optical flow estimation, International Journal of Computer Vision, Vol. 25, no.1, pp5–22, 1997
[63] M. J. Black, and P. Anandan, The robust estimation of multiple motions: parametric and piecewise-smooth flow fields, Computer Vision and Image Understanding, Vol. 63, no 1, pp75–104, 1996
[64] Y.Wang, J. Ostermann, Y.Q. Zhang, Video processing and communications, Tsinghua University Press,2003
[65] B Lucas, T. Kanade, An iterative registration technique with an application to stereo vision, In International Joint Conference on Artificial Intelligences, pp674–679, 1981
[66] P.J. Rousseeuw, Least median of squares regression, Journal of American Statistical Association, pp871-880, 1984
[67] P.J. Huber, Robust statistics, John Wiley, 1981
[68] C.V. Stewart, Robust Parameter Estimation in Computer Vision, SIAM review, vol.41, no.3, pp513-537,1999
[69] W.H. Press, S.A. Teukolsky, W.T. Vetterling, et al., Numerical recipes: the art of scientific computing, Cambridge University Press, 2007
[70] M.Fischler, R.Bolles, RANdom Sampling Consensus: a paradigm for model fitting with application to image analysis and automated cartography, Communications of ACM, vol.24, no.6, pp381-395, 1981
[71] Felix Woelk and Reinhard Koch, Robust monocular detection of independent motion by a moving observer, Lecture Notes in Computer Science, ISSU3417, pp209-222,2007
[72] M.Hotter and R.Thoma, Image segmentation based on object-oriented mapping parameter estimation, Signal Processing, vol.15, no.3, pp315-334,1988
[73] D. Zhang, and G. Lu, Segmentation of moving objects in image sequence: a review, Circuits, Systems and Signal Processing, vol.20, no.2, pp143–183, 2001
[74] C. Stauffer, W.E.L. Grimson, Learning patterns of activity using real-time tracking, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.22, no.8, pp747–757, 2000
[75] A. Elgammal, D. Harwood, L. Davis, Background and foreground modeling using nonparametric kernel density estimation for visual surveillance, Proceedings of the IEEE, vol.90, no.7, pp1151–1163, 2002
[76] S.Y. Chien, S.Y. Ma, L.G. Chen, Efficient moving object segmentation algorithm using background registration technique, IEEE Transactions on Circuits System Video Technology, vol.12, no.7, pp577-586, 2002
[77] A.M. Tekalp, Digital video processing, Prentice Hall, 1995
[78] J.Y.A. Wang, E.H. Adelson, Layered representation for motion analysis, IEEE Conference on Computer Vision and Pattern Recognition, pp361–366, New York, 1993
[79] J. Y. A. Wang and E. H. Adelson. Representing moving images with layers. IEEE Transactions on Image Processing, vol.3, no.5, pp625--638, 1994
[80] K.Y. Wong, L. Ye, M.E. Spetsakis, EM clustering of incomplete data applied to motion segmentation. British Machine Vision Conference, pp. 237–246, Kingston, 2004
[81] Z.H. Wang, G.Z. Liu, L. Liu, A fast and accurate video object detection and segmentation method in the compressed domain, IEEE International Neural Networks and Signal Processing , pp1209-1212, Nanjing, 2003
[82] S.Z. Li, Markov Random Field Modeling in Image Analysis, Springer-Verlag, 2001
[83] S. Geman, D. Geman, Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.6, pp721-741, 1984
[84] J. E. Besag, On the statistical analysis of dirty pictures, Journal of the Royal Statistical Society B, vol.48, no.3, pp259-302, 1986
[85] P. Chou, C. Brown, The theory and practice of Bayesian image labeling, International Journal of Computer Vision, vol.4, pp185-210, 1990
[86] D. Greig, B. Porteous, A. Seheult, Exact maximum a posteriori estimation for binary images, Journal of the Royal Statistical Society ,Series B, vol.51, no.2, pp271–279, 1989
[87] E.A. Dinic. Algorithm for solution of a problem of maximum flow in networks withy power estimation. Soviet Math. Dokl., no.11, pp1277-1280, 1970
[88] Andrew V. Goldberg and Robert E. Tarjan. A new approach to the maximum-flow problem. Journal of the Association for computing machinery, vol.35, no.4, pp921-940, 1988
[89] Y. Boykov, O. Veksler, R. Zabih, Fast approximate energy minimization via graph cuts, IEEE Transactions on Pattern Analysis Machine Intelligence, vol.23, no.11, pp1-18,2001
[90] J. Serra, L. Vincent, An overview of morphological filtering, Circuits, Systems and Signal Processing, vol.11, no.1, pp47-108, 1992
[91] P. Salembier and M. Padras, Hierarchical morphological segmentation for image sequence coding, IEEE Trans. on Image Processing vol.3, no.5, pp639–651, 1994
[92] S.F. Chang, W. Chen, H.J. Meng, et al., A fully automated content-based video search engine supporting spatiotemporal queries, IEEE Transacitons on circuits and systems for video technology, vol.8, no.5, pp602-615, 1998
[93] Iain Richardson, H.264 and MPEG-4 video compression: video coding for next-generation multimedia, John Wiley & Sons Inc., 2003
[94] Mario Lefebvre, Applied probability and statistics, Springer, 2006
[95] T. Corpetti, E. Memin and P. Perez, Dense estimation of fluid flows, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.24, no.3, pp365-380, 2002
[96] D. Suter, Motion estimation and vector splines, Proceeding of Conference Computer Vision Pattern Recognition, pp939-942, 1994
[97] S. Gupta, J. Prince, Stochastic models for div-curl optical flow methods, Signal Proceeding Letters, vol.3, no.2, pp32-34, 1996
[98] R. Nock and F. Nielsen, Statistical region merging, IEEE Trans. on Pattern Anal. Mach. Intell., vol.26, no.11, pp1452-1458, 2004
[99] R. Nock,F. Nielsen, Semi-supervised statistical region refinement for color image segmentation. Pattern Recognition,vol.38, no.6, pp835-846, 2005
[100] Y P Su,M T Sun,V. Hsu, Global motion estimation from coarsely sampled motion vector field and the applications. IEEE Trans. Circuits Syst. Video Technol., vol.15, no.2, pp232-241, 2005
[101] Y.P. Su, M.T. Sun and V. Hus, Robust global motion estimation from coarsely sampled motion vector fields, Proceedings of SPIE on Image and Video Communication and Processing, vol.5022, pp98-101, 2003
[102] L.M. Garth, H.V. Poor, Detectionofnon-Gaussiansignal: a paradigm for modern statistical signal processing, Proceeding of IEEE, vol.82, no.7, pp 1061-1095,1994
[103] A. Neri, S. Colonnese, G. Russo, et al. Automatic moving object and background separation, Signal Processing, vol.66, pp219-232, 1998
[104] X. Armangue and J. Salvi, Overall view regarding fundamental matrix estimation, Image Vision Comput., Vol.21,pp205-220,2003
[105] E.P. Ong, M. Spann, Robust optical flow computation based on least-median-of squares regression, International journal of computer vision, vol.31, no.1, pp51-82,1999
[106] Yuri Boykov and Vladimir Kolmogorov. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.26, no.9, pp1124-1137, 2004
[2] ISO/IEC JTC1/WG11/N4043. Overview of the MPEG-4 standard. March, 2001
[3] R.C. Gonzalez, R.E. Woods, Digital Image Processing, Prentice Hall, 2002
[4] T. Wiegand, G. Sullivan, G. Bjontegaard, et al., Overview of the H.264/AVC video coding standard, IEEE Transactions on Circuits and Systems for Video Technology, vol.13, no.7, pp560-576, 2003
[5] T. Stockhammer, M. M. Hannuksela and T. Wigand, H.264/AVC in wireless environments, IEEE Transactions on Circuits and Systems for Video Technology, vol.13, no.7, pp657-673, 2003
[6] S. Wenger, H.264/AVC over IP, IEEE Transactions on Circuits and Systems for Video Technology, vol.13, no.7, pp645-656, 2003
[7] H. Xu, A.A. Younis and M.R. Kabuka, Automatic moving object extraction of content-based applications, IEEE Transactions on Circuit and Systems for Video Technology, vol.14, no.6, pp.796-812,2004
[8] E. Izquierdo and M. Ghanbari, Key components for an advanced segmentation system, IEEE Transactions on Multimedia, vol.4, pp97-113, 2002
[9] S. Wu, A. Amin, Automatic thresholding of gray-level using multi-stage approach, IEEE Proceedings of Seventh International Conference on Document Analysis and Recognition, pp493-497,2003
[10] L. Vincent, Grayscale area openings and closings, their efficient implementation and application, EURASIP Workshop on Mathematical Morphology and Its Applications to Signal Processing, Spain, pp22-27,1993
[11] H.D. Cheng, X.H. Jiang and Y.Sun et al., Color image segmentation: advantages and prospects, Pattern Recognition, vol.34, pp2259-2281,2001
[12] D.Comaniciu and P. Meer, Mean shift: a robust approach toward feature space analysis, IEEE Transactions on Pattern Analysis Machine Intelligence, vol.24, no.5, pp603-619, 2005
[13] M. Ozden, E. Polat, A color image segmentation approach for content-based image retrieval, Pattern Recognition, vol.40, pp1318-1325,2007
[14] C.H. Chen, L.F. Pau, Texture analysis, the handbook of pattern recognition and computer vision, World Scientific Publishing, 1998
[15] C.I. Chang, Y. Du and J. Wang, et al., Survey and comparative analysis of entropy and relative entropy thresholding techniques, IEE Proceedings on Visual Image Signal Process, vol.153, no.6, pp837-850, 2006
[16] R.J. Radke, S. Andra and O. Al-Kofahi et al., Image change detection algothms: a systematic survey, IEEE transactions on image processing, vol.14, no.3, pp294-307, 2005
[17] Sukmarg O, Rao K R. Fast object detection and segmentation in MPEG compressed domain, Proceedings of IEEE TENCON, Malaysia, vol.3, pp364-368,2000
[18] C.D. Creusere, G. Dahman, Object detection and localization in compressed video, Proceedings of Asilomar Conference on Signals, Systems and Computers, Pacific Grove, CA, vol.1, pp93–97, 2001
[19] S. Ji and H.W. Park, Moving object segmentation in DCT-based compressed video, Electronics Letters, vol.36, no.21, pp1769-1770,2000
[20] H.L. Eng and K.K. Ma, Unsupervised image object segmentation over compressed domain, International Conference on Image Processing, vol.3, pp758-761,2000
[21] Y. Zhong, H.J. Zhang, A.K. Jain, Automatic caption localization in compressed video, IEEE International Conference on Image Processing, vol.2, pp96-100, 1999
[22] S. Ji, H.W. Park, Moving-object segmentation with adaptive sprite for DCT-based video coder, IEEE Conference on Image Processing, vol.3, pp566-569, 2001
[23] H.L. Wang, S.F. Chang, Automatic face region detection in MPEG video sequences, Proceedings of SPIE Electronic Imaging and Multimedia Systems, Vol. 2898, pp 160-168,1996
[24] M.T. Coimbra, M. Davies, Approximating optical flow within the MPEG-2 compressed domain, IEEE Transactions on Circuits System Video Technology, vol.15, no.1,pp 103-107,2005
[25] V. Mezaris, I Kompatsiaris, N V Boulgouris et al., Real-time compressed domain spatiotemporal segmentation and ontologies for video indexing and retrieval, IEEE Transactions on Circuits System Video Technology, vol.14, no.5, pp606-621,2004
[26] R. Venkatesh, K.R. Ramakrishnan, S.H. Srinivasan. Video object segmentation: A compressed domain approach, IEEE Transactions Circuits System Video Technology, vol.14, no.4, pp462-474, 2004
[27] D. Schonfeld, D. Lelescu, VORTEX: video retrieval and tracking from compressed multimedia databases - multiple object tracking from MPEG-2 bitstream, Journal of Visual Communication and Image Representative, no.11, pp154–182, 2000
[28] G. D. Borshukov, G. Bozdagi, Y. Altunbasak, Motion segmentation by multistage affine classification, IEEE Transactions on Image Processing,vol.6, no.11, pp1591-1594, 1997
[29] Z.H. Wang, G.Z. Liu and L. Liu, A Fast and accurate video object detection and segmentation method in the compressed domain, IEEE International Conference of Neural Networks and Signal Processing, pp1209-1212, 2003
[30] M.L. Jamrozik, M.H. Hayes, A compressed domain video object segmentation system, IEEE International Conference on Image Processing, vol.1, pp113-116, 2002
[31] F. Porikli, Real-time video object segmentation for MPEG encoded video sequences, SPIE Conference on Real-Time Imaging VIII, vol.5297, pp195-203, 2004
[32] H.T. Zen, T.H. Wa, Moving object detection from MPEG coded picture, International Conference on Image Processing, pp25-29, 1999
[33] A. Benzougar, P. Bouthemy, MRF-based moving object detection from MPEG coded video, International Conference on Image Processing, vol.3, pp402-405, 2001
[34] X.D. Yu, L.Y. Duan, Q. Tian, Robust moving video object segmentation in the MPEG compressed domain, International Conference on Image Processing, vol.3, pp933-936,2003
[35] M. Coimbra, M. Davies, Segmentation of moving pedestrians within the compressed domain, IEEE International Conferences on Acoustics, Speech, and Signal Processing, vol.3, pp605-608, 2004
[36] QIMERA, www.qimera.org
[37] SCHEMA, http://www.schema-ist.org
[38] V. Mezaris, I. Kompatsiaris and M. G. Strintzis, Object-based MPEG-2 video indexing and retrieval in a collaborative environment, Multimedia Tools and Applications Journal, vol.30, no.3, pp255-272, 2006
[39]刘志,面向基于内容多媒体应用的视频对象分割,上海交通大学博士论文,2005
[40]岑峰,视频监控系统中面向人的目标跟踪技术的研究,上海交通大学博士学位论文,2002
[41]杨高波,基于内容多媒体应用的语义视频对象提取及评价,上海大学博士论文,2004
[42]包红强,基于内容的视频运动对象分割技术研究,上海大学博士论文,2005
[43]徐剑峰,刘志,张兆杨,基于熵模型的压缩域运动对象分割,中国图象图形学报,vol.12, no.10, pp1815-1818,2007
[44] W Zeng, J Du, W Gao et al., Robust moving object segmentation on H.264/AVC compressed video using the block-based MRF model, Real-Time Imaging, vol.11, no.4, pp290-299, 2005
[45] Z Liu, Y Lu, Z.Y. Zhang, Real-time spatiotemporal segmentation of video objects in the H.264 compressed domain,Journal of Visual Communication and Image Representation, vol.18, no.3, pp275-290, 2007
[46]刘方青,石旭利,张兆杨,基于EM聚类的H.264压缩域视频对象实时分割算法,中国图象图形学报,vol.12,vol.10, pp1819-1822,2007
[47] Z. Liu, Z.Y. Zhang, L.Q. Shen, Moving object segmentation in the H.264 compressed domain, Optical Engineering, vol.46, no.1, pp0170031-0170035, 2007
[48] R. Cucchiara, A. Prati, R. Vezzani, Real-time motion segmentation from moving cameras, Real-Time Imaging, vol.10, pp127-143, 2004
[49] A. Martelli (1972). Edge detection using heuristic search methods. Computer Graphics and Image Processing 1, pp169–182, 1972
[50] J. H. Elder, A. Krupnik, L. A. Johnston, Contour grouping with prior models. IEEE Transactions on Pattern Analysis and Machine Intelligence vol.25, no.6, pp661–674, 2003
[51] D. Geiger, A. Gupta, L. A. Costa, J. Vlontzos ,Dynamic programming for detecting, tracking, and matching deformable contours. IEEE Transactions on Pattern Analysis and Machine Intelligence ,vol.17, no.3, pp294–302,1995
[52] J. Illingworth, J. Kittler, Survey of the Hough transform. Computer Vision, Graphics, and Image Processing, vol.44, no.1, pp87–116,1988
[53] A. Bovik, Handbook of Video and Image Processing, 2nd edition, Academic Press, 2005
[54] I. A. Ismaili, D. F. Gillies, Colour image segmentation using regression analysis in RGB space. Machine Graphics & Vision vol.3, no.1, pp373–384,1994
[55] L. Vincent, P. Soille, Watersheds in digital spaces: An efficient algorithm based on immersion simulations. IEEE Transactions on Pattern Analysis and Machine Intelligence vol.13,no.6,pp583–598,1991
[56] B. Cramariuc, M. Gabbouj, and J. Astola. Clustering based region growing algorithm for color image segmentation. In Proc. of the th13 Int’l Conf. on Digital Signal Processing, volume 2, pp857–860, July 1997
[57] V. Rehrmann and L. Priese. Fast and robust segmentation of natural color scenes. In Proc. of Annual Conf. Computer Vision, volume 1, pp598–606, 1998
[58] M. Celenk. Hierarchical color clustering for segmentation of textured images. In Proc. of the 29 Southeastern Symposium on System Theory, pp483–487, March 1997
[59] K. Saarinen. Color image segmentation by a watershed algorithm and region adjacency graph processing. In Proc. of 1994 Int’l Conf. on Image Processing volume III, pp1021–1025, 1994
[60] T. Gevers and A.W.M. Smeulders. Combining region splitting and edge detection through guided Delaunay image subdivision. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp1021–1026, 1997
[61] B. K. P. Horn, and B. G. Schunck, Determining optical flow, Artificial Intelligence, Vol. 17, pp 105–203, 1981
[62] J. W. Brandt, Improved accuracy in gradient-based optical flow estimation, International Journal of Computer Vision, Vol. 25, no.1, pp5–22, 1997
[63] M. J. Black, and P. Anandan, The robust estimation of multiple motions: parametric and piecewise-smooth flow fields, Computer Vision and Image Understanding, Vol. 63, no 1, pp75–104, 1996
[64] Y.Wang, J. Ostermann, Y.Q. Zhang, Video processing and communications, Tsinghua University Press,2003
[65] B Lucas, T. Kanade, An iterative registration technique with an application to stereo vision, In International Joint Conference on Artificial Intelligences, pp674–679, 1981
[66] P.J. Rousseeuw, Least median of squares regression, Journal of American Statistical Association, pp871-880, 1984
[67] P.J. Huber, Robust statistics, John Wiley, 1981
[68] C.V. Stewart, Robust Parameter Estimation in Computer Vision, SIAM review, vol.41, no.3, pp513-537,1999
[69] W.H. Press, S.A. Teukolsky, W.T. Vetterling, et al., Numerical recipes: the art of scientific computing, Cambridge University Press, 2007
[70] M.Fischler, R.Bolles, RANdom Sampling Consensus: a paradigm for model fitting with application to image analysis and automated cartography, Communications of ACM, vol.24, no.6, pp381-395, 1981
[71] Felix Woelk and Reinhard Koch, Robust monocular detection of independent motion by a moving observer, Lecture Notes in Computer Science, ISSU3417, pp209-222,2007
[72] M.Hotter and R.Thoma, Image segmentation based on object-oriented mapping parameter estimation, Signal Processing, vol.15, no.3, pp315-334,1988
[73] D. Zhang, and G. Lu, Segmentation of moving objects in image sequence: a review, Circuits, Systems and Signal Processing, vol.20, no.2, pp143–183, 2001
[74] C. Stauffer, W.E.L. Grimson, Learning patterns of activity using real-time tracking, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.22, no.8, pp747–757, 2000
[75] A. Elgammal, D. Harwood, L. Davis, Background and foreground modeling using nonparametric kernel density estimation for visual surveillance, Proceedings of the IEEE, vol.90, no.7, pp1151–1163, 2002
[76] S.Y. Chien, S.Y. Ma, L.G. Chen, Efficient moving object segmentation algorithm using background registration technique, IEEE Transactions on Circuits System Video Technology, vol.12, no.7, pp577-586, 2002
[77] A.M. Tekalp, Digital video processing, Prentice Hall, 1995
[78] J.Y.A. Wang, E.H. Adelson, Layered representation for motion analysis, IEEE Conference on Computer Vision and Pattern Recognition, pp361–366, New York, 1993
[79] J. Y. A. Wang and E. H. Adelson. Representing moving images with layers. IEEE Transactions on Image Processing, vol.3, no.5, pp625--638, 1994
[80] K.Y. Wong, L. Ye, M.E. Spetsakis, EM clustering of incomplete data applied to motion segmentation. British Machine Vision Conference, pp. 237–246, Kingston, 2004
[81] Z.H. Wang, G.Z. Liu, L. Liu, A fast and accurate video object detection and segmentation method in the compressed domain, IEEE International Neural Networks and Signal Processing , pp1209-1212, Nanjing, 2003
[82] S.Z. Li, Markov Random Field Modeling in Image Analysis, Springer-Verlag, 2001
[83] S. Geman, D. Geman, Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.6, pp721-741, 1984
[84] J. E. Besag, On the statistical analysis of dirty pictures, Journal of the Royal Statistical Society B, vol.48, no.3, pp259-302, 1986
[85] P. Chou, C. Brown, The theory and practice of Bayesian image labeling, International Journal of Computer Vision, vol.4, pp185-210, 1990
[86] D. Greig, B. Porteous, A. Seheult, Exact maximum a posteriori estimation for binary images, Journal of the Royal Statistical Society ,Series B, vol.51, no.2, pp271–279, 1989
[87] E.A. Dinic. Algorithm for solution of a problem of maximum flow in networks withy power estimation. Soviet Math. Dokl., no.11, pp1277-1280, 1970
[88] Andrew V. Goldberg and Robert E. Tarjan. A new approach to the maximum-flow problem. Journal of the Association for computing machinery, vol.35, no.4, pp921-940, 1988
[89] Y. Boykov, O. Veksler, R. Zabih, Fast approximate energy minimization via graph cuts, IEEE Transactions on Pattern Analysis Machine Intelligence, vol.23, no.11, pp1-18,2001
[90] J. Serra, L. Vincent, An overview of morphological filtering, Circuits, Systems and Signal Processing, vol.11, no.1, pp47-108, 1992
[91] P. Salembier and M. Padras, Hierarchical morphological segmentation for image sequence coding, IEEE Trans. on Image Processing vol.3, no.5, pp639–651, 1994
[92] S.F. Chang, W. Chen, H.J. Meng, et al., A fully automated content-based video search engine supporting spatiotemporal queries, IEEE Transacitons on circuits and systems for video technology, vol.8, no.5, pp602-615, 1998
[93] Iain Richardson, H.264 and MPEG-4 video compression: video coding for next-generation multimedia, John Wiley & Sons Inc., 2003
[94] Mario Lefebvre, Applied probability and statistics, Springer, 2006
[95] T. Corpetti, E. Memin and P. Perez, Dense estimation of fluid flows, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.24, no.3, pp365-380, 2002
[96] D. Suter, Motion estimation and vector splines, Proceeding of Conference Computer Vision Pattern Recognition, pp939-942, 1994
[97] S. Gupta, J. Prince, Stochastic models for div-curl optical flow methods, Signal Proceeding Letters, vol.3, no.2, pp32-34, 1996
[98] R. Nock and F. Nielsen, Statistical region merging, IEEE Trans. on Pattern Anal. Mach. Intell., vol.26, no.11, pp1452-1458, 2004
[99] R. Nock,F. Nielsen, Semi-supervised statistical region refinement for color image segmentation. Pattern Recognition,vol.38, no.6, pp835-846, 2005
[100] Y P Su,M T Sun,V. Hsu, Global motion estimation from coarsely sampled motion vector field and the applications. IEEE Trans. Circuits Syst. Video Technol., vol.15, no.2, pp232-241, 2005
[101] Y.P. Su, M.T. Sun and V. Hus, Robust global motion estimation from coarsely sampled motion vector fields, Proceedings of SPIE on Image and Video Communication and Processing, vol.5022, pp98-101, 2003
[102] L.M. Garth, H.V. Poor, Detectionofnon-Gaussiansignal: a paradigm for modern statistical signal processing, Proceeding of IEEE, vol.82, no.7, pp 1061-1095,1994
[103] A. Neri, S. Colonnese, G. Russo, et al. Automatic moving object and background separation, Signal Processing, vol.66, pp219-232, 1998
[104] X. Armangue and J. Salvi, Overall view regarding fundamental matrix estimation, Image Vision Comput., Vol.21,pp205-220,2003
[105] E.P. Ong, M. Spann, Robust optical flow computation based on least-median-of squares regression, International journal of computer vision, vol.31, no.1, pp51-82,1999
[106] Yuri Boykov and Vladimir Kolmogorov. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.26, no.9, pp1124-1137, 2004