基于四维矩阵的立体视频压缩算法研究
摘要
随着立体视频的广泛应用,大量的视频数据需要存储和传输,如何对立体视频数据进行高效的压缩编码是当前研究的热点问题。本文以经过校正的立体视频/图像为研究对象,提出以下几种立体视频压缩方法。
论文首先研究了马尔科夫随机场建立视差图模型求立体匹配的问题,针对置信度传播信息时具有时变自适应支持域的特点,提出一种基于时域相关性的置信度传播立体视频压缩算法。
然后针对立体匹配容易在对象边界部分出现较大误差,从而影响恢复图像主观效果的问题,提出一种依据纵向亮度连续性约束准则,将图像每列划分为互不交叠的亮度段,接下来以对象边界的视差值为参考,确定每条亮度段自适应搜索范围的立体视频压缩算法。
最后提出一种将立体视频构建在四维矩阵中,选择代表帧进行运动估计,然后利用四维矩阵DCT变换进一步去除时域、空域以及双目之间冗余的立体视频压缩算法。根据四维矩阵划分方式不同,分为规则型四维矩阵立体视频压缩算法和基于内容的四维矩阵立体视频压缩算法两种。实验结果表明,对于真实的自然场景,视图加深度的立体视频压缩方式由于缺乏遮挡/暴露区的残差补偿,图像的恢复质量将受到明显影响,基于内容的四维矩阵立体视频压缩算法具有更高的压缩效果。
2D video is incomparable with stereo video, which could provide viewers depth sense according to stereo disparity theory. By prevailing of stereo video, enormous of video data need to be stored and transmitted, especially in real-time system, high resolution and multi points-points communications. Recently, 2D encoding theory and technology were already maturity, but not in stereo video and multi points. How to compressing stereo video data became a hotspot in communication and it has also important for application of stereo video. Nowadays, stereo video compression has been paid attention by many researchers.
DISTIMA project was an integrated stereo video communication system, which was based on MPEG-2 coding standard and developed by Germany, France and some other European nations. ATTEST proposed a novel stereo video compression that was based on a more flexible joint transmission of monoscopic color video and associated per-pixel depth information. From this 3D data representation format, one or more“virtual”views of a 3D scene can then be synthesized in real-time at the receiver side by means of so-called depth-image-based rendering (DIBR) techniques. International standard organization MPEG built a stereo video coding special team, which mainly discussed stereo video requirements and techniques in apply, and for purpused of a uniform stereo video standard. There were two stereo video schemes in all, which was depth based image rendering, and another one was block-based estimation with residual compensation.
This paper mainly researched on stereo video coding in 3DTV system, and a pair of rectified images with small baseline were as input. Three stereo video compression algorithms were proposed and organized as follows.
1. More exactly disparity estimation could get smaller compression stream and finer quality of reconstructed image by depth based image rendering. Loop belief propagation could get exactly matching by minimizing global energy, but it would take heavy computational cost.
This paper proposed a temporal correlation based belief propagation (BP) stereo video coding, which was based on BP’s message passing provides a time-varying adaptive support region for stereo matching to deal with textureless regions and depth discontinuities elegantly. In textureless regions, for example, the influence of a message can be passed far away. On the other hand, the influence in discontinuous regions will fall off quickly. Stereo video were organized in group. Disparity of every I stereo pair was calculated by standard loop BP, and disparity of every P stereo pair was computed selectively by difference of temporal frames. Finally, reference video and disparity sequences were encoded.
2. If algorithms of disparity estimation were not referred objects, there will be large residual around object edges. And then this would be effected subjective quality of reconstructed image. So this paper proposed a disparity estimation algorithm based on objects.
Firstly, mainly edges of object were extracted by Canny, then feature points were detected by Harris, and original matching relation was constructed, finally outlier disparities were deleted by fundamental matrix estimation processing. And a sparse disparity map was constructed, and then disparity in object needed to be calculated.
It is a difficult problem for selecting unit for calculating disparity of object. It will be too dispersed to be calculated by pixel-based and it is also difficult for selecting size of block by block-based. Because of disparity discontinuous always occurs with large changes in intensity, this paper proposed every column of image was adaptable divided into non-overlapping segments by continuous constraint of intensity in vertical, and every segment was considered as a unit for matching. And then size of searching windows was referenced by disparity of left and right adjacent object edges. Finally, reference and disparity sequences were encoded.
3. Due to view points, intensity source, and noise effect, some information could not be reconstructed by depth base image rendering, but they could be remedied by residual compensation. This paper proposed a content-based four dimensional (4D) matrix stereo video algorithm. Differ from typical block-based method, this motion estimation was calculated by reference frames, which would reduce almost estimation computation. And then redundant of temporal, spatial and disparity correlation was reduced by 4D matrix DCT. Coefficients after transforming were assembled in low frequency and temporal, and then correlation between coefficients could be enhanced by 4D matrix Z scan, which was fitable for variable length coding.
Several stereo video and image were tested, and experimental results were compared between these methods.
In contrast of standard loop BP algorithm, temporal correlation based BP stereo video coding algorithm could reduce 83.8% computation time under the same reconstructed image quality. So it was an efficient disparity sequence algorithm for stereo video transmitting and free view point rendering. Experiment results showed that this algorithm could preserve details well, but some errors occured in edge of object. This might affect stereo sense because it could not be remedied in further by lacking residual compensation process.
Object and continuous constraint of intensity based stereo video coding could reserve well in edges and details, but not in discontinuous intensity. Object edges could be reconstructed more integrity by this method than temporal correlation based BP stereo video coding.
Stereo video could be reconstructed well by 4D matrix based stereo video coding. With accretion of bit rate, the quality of reconstructed image could get higher. By compared with the other two algorithm proposed in this paper, 4D matrix based stereo video coding may be blockness in textureless region at lower bit rate, but it has higher fidelity, which reserved original image information well and may be helpful for reconstructed stereo sense.
Experiment results showed that it would be finer quality of reconstructed image by depth based image rendering for simple content stereo video or image. But for complexity content stereo video, residual compensation could resolve errors by occlusion and exposure well.
论文首先研究了马尔科夫随机场建立视差图模型求立体匹配的问题,针对置信度传播信息时具有时变自适应支持域的特点,提出一种基于时域相关性的置信度传播立体视频压缩算法。
然后针对立体匹配容易在对象边界部分出现较大误差,从而影响恢复图像主观效果的问题,提出一种依据纵向亮度连续性约束准则,将图像每列划分为互不交叠的亮度段,接下来以对象边界的视差值为参考,确定每条亮度段自适应搜索范围的立体视频压缩算法。
最后提出一种将立体视频构建在四维矩阵中,选择代表帧进行运动估计,然后利用四维矩阵DCT变换进一步去除时域、空域以及双目之间冗余的立体视频压缩算法。根据四维矩阵划分方式不同,分为规则型四维矩阵立体视频压缩算法和基于内容的四维矩阵立体视频压缩算法两种。实验结果表明,对于真实的自然场景,视图加深度的立体视频压缩方式由于缺乏遮挡/暴露区的残差补偿,图像的恢复质量将受到明显影响,基于内容的四维矩阵立体视频压缩算法具有更高的压缩效果。
2D video is incomparable with stereo video, which could provide viewers depth sense according to stereo disparity theory. By prevailing of stereo video, enormous of video data need to be stored and transmitted, especially in real-time system, high resolution and multi points-points communications. Recently, 2D encoding theory and technology were already maturity, but not in stereo video and multi points. How to compressing stereo video data became a hotspot in communication and it has also important for application of stereo video. Nowadays, stereo video compression has been paid attention by many researchers.
DISTIMA project was an integrated stereo video communication system, which was based on MPEG-2 coding standard and developed by Germany, France and some other European nations. ATTEST proposed a novel stereo video compression that was based on a more flexible joint transmission of monoscopic color video and associated per-pixel depth information. From this 3D data representation format, one or more“virtual”views of a 3D scene can then be synthesized in real-time at the receiver side by means of so-called depth-image-based rendering (DIBR) techniques. International standard organization MPEG built a stereo video coding special team, which mainly discussed stereo video requirements and techniques in apply, and for purpused of a uniform stereo video standard. There were two stereo video schemes in all, which was depth based image rendering, and another one was block-based estimation with residual compensation.
This paper mainly researched on stereo video coding in 3DTV system, and a pair of rectified images with small baseline were as input. Three stereo video compression algorithms were proposed and organized as follows.
1. More exactly disparity estimation could get smaller compression stream and finer quality of reconstructed image by depth based image rendering. Loop belief propagation could get exactly matching by minimizing global energy, but it would take heavy computational cost.
This paper proposed a temporal correlation based belief propagation (BP) stereo video coding, which was based on BP’s message passing provides a time-varying adaptive support region for stereo matching to deal with textureless regions and depth discontinuities elegantly. In textureless regions, for example, the influence of a message can be passed far away. On the other hand, the influence in discontinuous regions will fall off quickly. Stereo video were organized in group. Disparity of every I stereo pair was calculated by standard loop BP, and disparity of every P stereo pair was computed selectively by difference of temporal frames. Finally, reference video and disparity sequences were encoded.
2. If algorithms of disparity estimation were not referred objects, there will be large residual around object edges. And then this would be effected subjective quality of reconstructed image. So this paper proposed a disparity estimation algorithm based on objects.
Firstly, mainly edges of object were extracted by Canny, then feature points were detected by Harris, and original matching relation was constructed, finally outlier disparities were deleted by fundamental matrix estimation processing. And a sparse disparity map was constructed, and then disparity in object needed to be calculated.
It is a difficult problem for selecting unit for calculating disparity of object. It will be too dispersed to be calculated by pixel-based and it is also difficult for selecting size of block by block-based. Because of disparity discontinuous always occurs with large changes in intensity, this paper proposed every column of image was adaptable divided into non-overlapping segments by continuous constraint of intensity in vertical, and every segment was considered as a unit for matching. And then size of searching windows was referenced by disparity of left and right adjacent object edges. Finally, reference and disparity sequences were encoded.
3. Due to view points, intensity source, and noise effect, some information could not be reconstructed by depth base image rendering, but they could be remedied by residual compensation. This paper proposed a content-based four dimensional (4D) matrix stereo video algorithm. Differ from typical block-based method, this motion estimation was calculated by reference frames, which would reduce almost estimation computation. And then redundant of temporal, spatial and disparity correlation was reduced by 4D matrix DCT. Coefficients after transforming were assembled in low frequency and temporal, and then correlation between coefficients could be enhanced by 4D matrix Z scan, which was fitable for variable length coding.
Several stereo video and image were tested, and experimental results were compared between these methods.
In contrast of standard loop BP algorithm, temporal correlation based BP stereo video coding algorithm could reduce 83.8% computation time under the same reconstructed image quality. So it was an efficient disparity sequence algorithm for stereo video transmitting and free view point rendering. Experiment results showed that this algorithm could preserve details well, but some errors occured in edge of object. This might affect stereo sense because it could not be remedied in further by lacking residual compensation process.
Object and continuous constraint of intensity based stereo video coding could reserve well in edges and details, but not in discontinuous intensity. Object edges could be reconstructed more integrity by this method than temporal correlation based BP stereo video coding.
Stereo video could be reconstructed well by 4D matrix based stereo video coding. With accretion of bit rate, the quality of reconstructed image could get higher. By compared with the other two algorithm proposed in this paper, 4D matrix based stereo video coding may be blockness in textureless region at lower bit rate, but it has higher fidelity, which reserved original image information well and may be helpful for reconstructed stereo sense.
Experiment results showed that it would be finer quality of reconstructed image by depth based image rendering for simple content stereo video or image. But for complexity content stereo video, residual compensation could resolve errors by occlusion and exposure well.
引文
[1]高文,陈熙霖.计算机视觉——算法与系统[M].北京:清华大学出版社, 1999.
[2] Onural L, Sikora T, Ostermann J, et al. An assessment of 3DTV technologies [C]. Proceedings of NAB Broadcast Engineering Conference, 2006.
[3]朱仲杰.立体视频编码中的关键技术研究[D].浙江大学:信息科学与工程学院, 2003.12.
[4]李奇,冯华君,徐之海,等.计算机立体视觉技术综述[J].光学技术, 1999, (5): 71-73.
[5]梁栋,韦穗,周敏彤.双眼立体感知几何模型的研究[J].中国图象图形学报, 1998, 3(8): 679-683.
[6]陈妤姗.多视点视频编码中关键技术的研究[D].华侨大学:信号与信息处理, 2006.
[7]谭晓军,余志,李军.一种改进的立体摄像机标定方法[J].测绘学报, 2006, 35(2): 138-142.
[8] Hartley R. Multiple view geometry in computer vision [M]. England, Cambridge University Press, 2003
[9]张广军.机器视觉[M].北京:科学出版社, 2005.
[10]侯春萍,阿陆南,俞斯乐.立体成像系统数学模型和视差控制方法[J].天津大学学报, 2005, 38(5): 455-465.
[11]郭锐.基于LDPC码不等错误保护的立体视频通信研究[D].浙江大学:信息科学与工程学院, 2007.
[12] Fehn C. 3D-TV approach using depth-image-based rendering [C]. Proc. of Visualization, Imaging and Image Processing, 2003: 482-487.
[13] Penta S K, Narayanan P. J. Compression of multiple depth maps for IBR [J]. Visual Comput, Sep 2005, 21: 611-618.
[14] Jia Y, Xu Y, Liu W, et al. A miniature stereo vision machine for real-time dense depth mapping [C]. ICVS 2003: 268-277.
[15] Fehn C. Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3D-TV(R/OL) [J/OL]. http://iphome.hhi.de/ fehn/Publications/fehn_EI2004.pdf.
[16] Zitnick C. L, King S. B, Uyttendaele M, et al. High-quality video view interpolation using a layered representation [J]. ACM SIGGRAPH and ACM Trans. On Graphics, 2004: 600-608.
[17] Tam W J, Zhang L. 3D-TV content generation: 2D-to-3D conversion [C]. ICME 2006: 1869-1872.
[18] Eisert P, Steinbach E, Girod B. Automatic reconstruction of stationary 3-D objects from multiple uncalibrated camera views [J/OL]. University of Erlangen-Nuremberg, Germany, http://www.lnt.de/~eisert.
[19] Cha J, Kim S Y, Ho Y S, et al. 3D Video Player System with haptic interaction based on depth image-based representation [J]. IEEE Transactions on Consumer Electronics, 2006, 52(2): 477-484.
[20] Kim S Y, Lee S B, et al. Three-dimensional natural video system based on layered representation of depth maps [J]. IEEE Transactions on Consumer Electronics, 2006, 52(3): 1035-1042.
[21] Martinian E, Behrens A, Xin J, et al. View synthesis for multiview video compression [C]. Proc. Picture Coding Symp, 2006.
[22] Martinian E, Behrens A, Xin J, et al. Extensions of H.264/AVC for multiview video compression [J]. TR2006-048, May 2006.
[23] Iddan G. J, Yahav G. 3D imaging in the studio [C]. Proceedings of SPIE, Vol. 4298, 2001.
[24] Ghaffar R, Jafri N, Khan S A, et al. Depth extraction system using stereo pairs [C]. ICIAR 2004: 512-519.
[25]韩军功.立体图像和视频编码的理论及算法研究[D].西安电子科技大学:通信与信息系统, 2004.
[26]韩军功,卢朝阳.用于立体图像编码的自适应块匹配视差估计算法[J].西安电子科技大学学报(自然科学版), 2004, 31(3): 347-351.
[27] Kang S B, Szeliski R, Chai J. Handling occlusions in dense multi-view stereo [C]. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’2001), Kauai, Hawaii, December 2001.
[28]张勇东,李桂苓.立体视频编码中视差估值算法的研究[J].电子测量与仪器学报, 2002, 16(1): 7-12.
[29] Shukla N K, Rathi V, Chakka V, et al. An efficient adaptive window based disparity map computation algorithm by dense two frame stereo correspondence[C]. ICVGIP 2006: 894–905.
[30]安平,张兆扬.立体图像分层交叠块视差估计与自适应补偿[J].通信学报, 2003, 24(3): 1-7.
[31]顾征,苏显渝.采用色彩相似性约束的立体匹配[J].光电工程, 2007, 34(1): 95-99.
[32] Chen Y S, Cai C H, Liu J L. YUV correction for multi-view video compression [C]. The 18th International Conference on Pattern Recognition, 2006.
[33] Okutomi M, Yoshizaki O, Tomita G. Color stereo matching and its application to 3-D measurement of optic nerve head [C]. The 11th IAPR International Conference on Pattern Recognition, 1992.
[34]朱鲲鹏,王敬东,李鹏.一种基于边缘分割的全局立体匹配算[J].光电子技术, 2008, 28(2): 123-128.
[35] Sand P, Teller S. Video matching [C]. Proc. of the SIGGRAPH 2004: 592-599.
[36]李立春,张小虎,傅丹,等.基于极线局部校正的特征匹配方法[J].光学技术,2008, 34(2): 285-288.
[37]文贡坚,吕金建,王继阳.基于特征的高精度自动图像配准方法[J]. 2008, 19(9): 2293-2301.
[38] Liu Z D, Zhao Y N, Yang J Y. Fast stereo matching method using edge traction [C]. The 2nd International Conference on Intelligent Information Processing, 2004, 10: 93-96.
[39]罗军,江和平,沈振康.基于匹配跟踪置信度的自适应对应像素距离图像匹配跟踪算法[J].信号处理, 2008, 24(1): 72-77.
[40] Pedone M, Heikkila J, Talon J B, et al. Blur and contrast invariant fast stereo matching [C]. ACIVS 2008: 883–890.
[41] Luo Q M, Zhou J L, Yu S S, et al. Stereo matching and occlusion detection with integrity and illusion sensitivity [J]. Pattern Recognition Letters, 2003, (24): 1143-1149.
[42] Ahlvers U, Zoelzer U, Rechmeier S, et al. FFT-based disparity estimation for stereo image coding [C]. DAGM 2003: 257-264.
[43] Hirschmuller H, Scharstein D. Evaluation of cost functions for stereo matching [C]. IEEE CVPR, 2007.
[44] Scharstein D, Szeliski R. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms [J]. International Journal of Computer Vision, 2002, 47(1/2/3): 7–42.
[45]游素亚,徐光佑.立体视觉研究的现状与进展[J].中国图象图形学报, 1997, 2(1): 17-24.
[46]钱曾波,邱振戈,张永强.计算机立体视觉研究的进展[J].测绘学院学报, 2001, 18(4): 267-272.
[47] Birchfield S, Tomasi C. Depth discontinuities by pixel-to-pixel stereo [J]. International Journal of Computer Vision, 1999, 35(3): 269–293.
[48] Ohta Y. Stereo by intra-and inter-scanline search us-ing dynamic programming [J]. IEEE Trans. on Pattern Analysis and Machine Intelligence, 1985, 7(2): 139-154.
[49]林国余,张为公.基于控制点的分层双向动态规划立体匹配算法[J].信息与控制, 2006, 35(3): 411-416.
[50]赵杰,于舒春,蔡鹤皋.金字塔双层动态规划立体匹配算法[J].控制与决策, 2007, 22(1): 69-73.
[51] Barnard S. Disparity ana1ysis of images [J]. IEEE Trans. on PAMI, 1980, 2: 333~340.
[52] Moravec HP. Visual mapping by a robot rover [C]. Proceedings of the 6th IJCAI, 1979: 589~600.
[53] Yedidia J S, Freeman W T, Weiss Y. Understanding of belief propagation and its generalizations [C]. International Joint Conference on Artificial Intelligence (IJCAI 2001), 2001.
[54] Sun J, Shum H Y, Zheng N N. Stereo matching using belief propagation [C]. ECCV 2002.
[55] Tappen M F, Freeman W T. Comparison of graph cuts with belief propagation for stereo using identical MRF parameters [C]. IEEE International Conference on Computer Vision, 2003.
[56]张浩峰,赵春霞.基于颜色与空间距离的置信度传播立体匹配算法[J].系统仿真学报, 2008, 20(13): 3374-3377.
[57]卢阿丽,唐振民,郭龙源,等.置信度传播体视算法加速技术的应用研究[J].系统仿真学报, 2009, 21(2): 348-356.
[58]陈侃.基于马尔可夫随机场图像恢复算法研究[D].南京师范大学:数科院计算机系, 2008.
[59] Siegel M W, Sethuraman S, Mcveigh J. S, et al. Compression and interpolation of 3D-stereoscopic and multiview video [C]. Proc. of SPIE, 1997, 3012: 227-238.
[60] Ding L F, Chien S Y, Huang Y W, et al. Stereo video coding system with hybrid coding based on joint prediction scheme [C]. IEEE International Symposium on Circuits and Systems, 2005: 6082-6085.
[61] Sethuraman S. Stereoscopic image sequence compression using multiresolution and quadtree decomposition based disparity-and motion-adaptive segmentation [D]. Carnegie Mellon University: Dept. of Electrical and computer Engineering, 1996.
[62]骆艳,张兆扬.基于多分辨率四叉树的运动分割及交叠块的立体视频帧估计算法[J].中国图象图形学报, 2002, 7(7A): 716-722.
[63] Ellinas J N, Sangriotis S. Stereo video coding based on interpolated motion and disparity estimation [C]. Proceedings of the 3rd International Symposium on Image and Signal Processing and Analysis, 2003: 301-306.
[64] Choquet B, Dugelay J L, Pele D. A coding scheme for stereoscopic television sequences based on motion estimation-compensation using a 3D approach [J]. Image Processing and Its Applications, July 1995: 188-192.
[65] Richardson I E. G. H.264 and MPEG-4 video compression-video coding for next-generation multimedia[M]. John Wiley&Sons Ltd, England, 2003.
[66]李世平,蒋刚毅,郁梅.基于H.264的立体视频编码方法[J].计算机工程与应用, 2005, 1: 77-79.
[67]胡静俊.立体图像校正与视频编码[D].浙江大学:信息学院, 2006.
[68]卫瑞平.双目立体视频对象压缩编码技术[D].吉林大学:通信工程学院, 2008.
[69]傅蕊.基于H.264的立体序列图像压缩算法研究[D].天津大学:电子信息工程学院, 2007.
[70]王知嘉,刘峰,鲍卫兵.自适应选择联合补偿预测模式的立体视频编码算法[J].中国通信, 2009, 4: 58-62.
[71]陈建乐.多视点视频编码方法的研究[D].浙江大学:信息科学与工程学院, 2006.
[72]李达,黎洪松.一种改进的立体图像编码算法[J].北京师范大学学报(自然科学版),2007, 43 (1): 50-52.
[73] Garbas J U, Fecker U, Troger T, et al. 4D scalable multi-view video coding using disparity compensated view filtering and motion [C]. IEEE 8th Workshop on Multimedia Signal Processing, 2006.
[74]董小琳,侯春萍.立体图像压缩研究[J].电子测量技术, 2006, 29(6): 160-162.
[75] Thanapirom S, Fernando W A C, Edirisinghe E A. A zerotree stereo video encoder [J]. Proceedings of the 2003 International Symposium on Circuits and Systems, 2003: 608-611.
[76] Ellinas J N, Sangriotis M S. Stereo video coding based on quad-tree decomposition of B-P frames by motion and disparity interpolation [J]. IEE Proc.-Vis. Image Signal Process, Oct. 2005, Vol. 152, No. 5: 639-647.
[77]吴勇军,殷勤业,曾雁星,等.基于幅相分离立体匹配的小波立体图像压缩[J].西安交通大学学报, 2002, 36(2): 1245-1248.
[78]许昌满,张兆扬.基于马尔可夫模型和小波变换的立体残差图象编码[J].通信学报, 2002, 6(23): 90-96.
[79]章毓晋.图像分割[M].北京:科学出版社, 2006.
[80]李波,姚春莲,李炜,等.利用相邻帧和背景信息的运动对象检测[J].电子学报, 2008, 11(11): 2154-2157.
[81] Yang W X, Ngan K N, Talon J B. Unsupervised multiple object segmentation of multiview images [C]. Proceedings of Advanced Concepts for Intelligent Vision Systems 2007: 178-189.
[82]朱仲杰,郁梅,蒋刚毅,等.立体视频编码中的目标分割与立体匹配算法[J].软件学报, 2003, 14(11): 1971-1976.
[83]容观澳.计算机图像处理[M].北京:清华大学出版社, 1998.
[84] Nordling V. Efficient compression of stereoscopic video using the MPEG standard [D]. Royal Institute of Technology: Computer Science and Engineering, 2003.
[85] Mueller S K, Merkle P, Rein T, et al. Representation, coding, and rendering of 3D video objects with MPEG-4 and H.264/AVC [M]. John Wiley & Sons Lid, England, 2003.
[86]卢朝阳,周幸妮,高西全,等.三角形网格基活动图像编码——基本框架和I帧编码[J].通信学报, 2002, 23(10): 66-70.
[87]卢朝阳,周幸妮,高西全,等.三角形网格基活动图像编码研究——运动估计、运动补偿和残差图像处理[J].通信学报, 2002, 23(11): 65-73.
[88]韩军功,卢朝阳.基于Delaunay三角形网格的立体图像编码算法[J].计算机辅助设计与图形学学报, 2004, 16(12): 1708-1712.
[89]韩军功,卢朝阳.基于三角形网格的立体视差估计和遮挡点检测混合算法[J].电路与系统学报, 2005, 10(1): 44-49.
[90]吴书凯,都思丹,李华.基于半规则网格的视差估计算法[J].计算机应用, 2008, 28(4): 957-960.
[91] Szeliski R, Zabih R, Scharsein D, et al. A comparative study of energy minimization methods for markov random fields with smoothness-based priors[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(6):1068-1080.
[92] Felzenszwalb P F, Huttenlocher D P. Efficient belief propagation for early vision [J]. International Journal of Computer Vision, 2006, 70(1):41-54.
[93] Weiss Y, Freeman W T. On the optimality of solutions of the max-product belief propagation algorithm in arbitrary graphs [J]. IEEE Transactions on Information Theory, 2001.
[94] Weiss Y, Freeman W T. Interpreting images by propagating bayesian beliefs [J]. Advances in Neural Information Processing Systems, 1997, 9: 908-915.
[95]罗军辉,冯平,哈力旦.A,等. MATLAB7.0在图像处理中的应用[M].北京:机械工业出版社, 2005.
[96] Harris Chris, Stephens M. A combined corner and edge detector [C]. Proceedings of Alvey Vision Conference, 1988: 147-151.
[97] Higgins H C L. A computer algorithm for reconstructing a Scene from two projections [J]. Nature, 1981, 293: 133-135.
[98]陈泽志,吴成柯.一种高精度估计的基础矩阵的线性算法[J].软件学报, 2002, 13(4): 840-845
[99]陈泽志,吴成柯,刘勇.基础矩阵估计的加权归一化线性算法[J].软件学报, 2001, 12(3): 420-426.
[100]孙亦南,刘伟军,马永壮,等.一种加权计算基础矩阵的高精度算法[J].计算机工程, 2005, 15(31): 186-188.
[101]王才良,史泽林,李德强.图像重构中估计基础矩阵的线性迭代算法[J].微计算机信息, 2006, 22(3): 221-223.
[102]申晓凡,左峥嵘,张天序.视图合成中基础矩阵的鲁棒性估计算法[J].计算机与数字工程, 2003, 31(15): 8-12.
[103] Torr P H S, Murray D W. The development and comparison of robust methods for estimating the fundamental matrix[J]. International Journal of Computer Vision, 1997, 24: 271-300.
[104] Huang J F, Shang H L, Cheng C M. Robust fundamental matrix estimation with accurate outlier detection [J]. Journal of Information Science and Engineering, 2007, 23, 1213-1225.
[105] Deriche R, Zhang Z, Luong Q T, et al. Robust recovery of the epipolar geometry for an uncalibrated stereo rig[C]. Computer Vision-ECCV 94, 1994, (800): 567-576.
[106] Hartley R I. In defense of the eight-point algorithm [J]. IEEE Transactions on Pattern and Machine Intelligence, 1997, 19(6): 580-593.
[107]赵岩,陈贺新.彩色视频的四维矩阵离散余弦变换编码[J].中国图象图形学报, 2003, 6(8): 620-624.
[108] Wiegand T, Sullivan G. Draft ITU-T recommendation and final draft international standard of joint video specification[R]. ITU-T Rec. H.264|ISO/IEC 14496-10 AVC, in Joint Video Team(JVT) of ISO/IEC MPEG and ITU-T VCEG, JVT-G050, March 2003: 148-158.
[109] Stelmach L, Tam W J, Meegan D, et al. Stereo image quality: effects of mixed spatio-temporal resolution [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2000, 10(2): 188-193.
[110]吴勇军,殷勤业,方强,等.立体图像压缩研究进展[J].数字电视与数字视频(电视技术), 2002, 241: 20-23.
[111] Pei S C, Lai C L. Very Low bit-rate coding algorithm for stereo video with spatiotemporal HVS model and binary correlation disparity estimator [J]. IEEE Journal on Selected Areas in Communications, 1998, 16(1): 98-107.
[112]杜相文,陈贺新,赵志杰.彩色视频的四维MDCT及矩阵量化编码[J].吉林大学学报(信息科学版), 2003, 8(21): 12-16.
[113] Wiegand T, Sullivan G. Overview of the H.264/AVC video coding standard [J]. IEEE Transactions on Circuits and Systems for Video Technology, Vol.13, No.7, July 2003: 560-576.
[114] Marpe D, Bl?ttermann G, Wiegand T. Adaptive codes for H.26L [S]. ITU-Telecommunications Standardization Sector, Video Coding Experts Group (VCEG), VCEG-L13, January, 2001: 2-7.
[115] Bjontegaard G, Lillevold K. Context-adaptive VLC coding of coefficients [S]. JVT Document JVT-C028, May, 2002: 1-11.
[2] Onural L, Sikora T, Ostermann J, et al. An assessment of 3DTV technologies [C]. Proceedings of NAB Broadcast Engineering Conference, 2006.
[3]朱仲杰.立体视频编码中的关键技术研究[D].浙江大学:信息科学与工程学院, 2003.12.
[4]李奇,冯华君,徐之海,等.计算机立体视觉技术综述[J].光学技术, 1999, (5): 71-73.
[5]梁栋,韦穗,周敏彤.双眼立体感知几何模型的研究[J].中国图象图形学报, 1998, 3(8): 679-683.
[6]陈妤姗.多视点视频编码中关键技术的研究[D].华侨大学:信号与信息处理, 2006.
[7]谭晓军,余志,李军.一种改进的立体摄像机标定方法[J].测绘学报, 2006, 35(2): 138-142.
[8] Hartley R. Multiple view geometry in computer vision [M]. England, Cambridge University Press, 2003
[9]张广军.机器视觉[M].北京:科学出版社, 2005.
[10]侯春萍,阿陆南,俞斯乐.立体成像系统数学模型和视差控制方法[J].天津大学学报, 2005, 38(5): 455-465.
[11]郭锐.基于LDPC码不等错误保护的立体视频通信研究[D].浙江大学:信息科学与工程学院, 2007.
[12] Fehn C. 3D-TV approach using depth-image-based rendering [C]. Proc. of Visualization, Imaging and Image Processing, 2003: 482-487.
[13] Penta S K, Narayanan P. J. Compression of multiple depth maps for IBR [J]. Visual Comput, Sep 2005, 21: 611-618.
[14] Jia Y, Xu Y, Liu W, et al. A miniature stereo vision machine for real-time dense depth mapping [C]. ICVS 2003: 268-277.
[15] Fehn C. Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3D-TV(R/OL) [J/OL]. http://iphome.hhi.de/ fehn/Publications/fehn_EI2004.pdf.
[16] Zitnick C. L, King S. B, Uyttendaele M, et al. High-quality video view interpolation using a layered representation [J]. ACM SIGGRAPH and ACM Trans. On Graphics, 2004: 600-608.
[17] Tam W J, Zhang L. 3D-TV content generation: 2D-to-3D conversion [C]. ICME 2006: 1869-1872.
[18] Eisert P, Steinbach E, Girod B. Automatic reconstruction of stationary 3-D objects from multiple uncalibrated camera views [J/OL]. University of Erlangen-Nuremberg, Germany, http://www.lnt.de/~eisert.
[19] Cha J, Kim S Y, Ho Y S, et al. 3D Video Player System with haptic interaction based on depth image-based representation [J]. IEEE Transactions on Consumer Electronics, 2006, 52(2): 477-484.
[20] Kim S Y, Lee S B, et al. Three-dimensional natural video system based on layered representation of depth maps [J]. IEEE Transactions on Consumer Electronics, 2006, 52(3): 1035-1042.
[21] Martinian E, Behrens A, Xin J, et al. View synthesis for multiview video compression [C]. Proc. Picture Coding Symp, 2006.
[22] Martinian E, Behrens A, Xin J, et al. Extensions of H.264/AVC for multiview video compression [J]. TR2006-048, May 2006.
[23] Iddan G. J, Yahav G. 3D imaging in the studio [C]. Proceedings of SPIE, Vol. 4298, 2001.
[24] Ghaffar R, Jafri N, Khan S A, et al. Depth extraction system using stereo pairs [C]. ICIAR 2004: 512-519.
[25]韩军功.立体图像和视频编码的理论及算法研究[D].西安电子科技大学:通信与信息系统, 2004.
[26]韩军功,卢朝阳.用于立体图像编码的自适应块匹配视差估计算法[J].西安电子科技大学学报(自然科学版), 2004, 31(3): 347-351.
[27] Kang S B, Szeliski R, Chai J. Handling occlusions in dense multi-view stereo [C]. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’2001), Kauai, Hawaii, December 2001.
[28]张勇东,李桂苓.立体视频编码中视差估值算法的研究[J].电子测量与仪器学报, 2002, 16(1): 7-12.
[29] Shukla N K, Rathi V, Chakka V, et al. An efficient adaptive window based disparity map computation algorithm by dense two frame stereo correspondence[C]. ICVGIP 2006: 894–905.
[30]安平,张兆扬.立体图像分层交叠块视差估计与自适应补偿[J].通信学报, 2003, 24(3): 1-7.
[31]顾征,苏显渝.采用色彩相似性约束的立体匹配[J].光电工程, 2007, 34(1): 95-99.
[32] Chen Y S, Cai C H, Liu J L. YUV correction for multi-view video compression [C]. The 18th International Conference on Pattern Recognition, 2006.
[33] Okutomi M, Yoshizaki O, Tomita G. Color stereo matching and its application to 3-D measurement of optic nerve head [C]. The 11th IAPR International Conference on Pattern Recognition, 1992.
[34]朱鲲鹏,王敬东,李鹏.一种基于边缘分割的全局立体匹配算[J].光电子技术, 2008, 28(2): 123-128.
[35] Sand P, Teller S. Video matching [C]. Proc. of the SIGGRAPH 2004: 592-599.
[36]李立春,张小虎,傅丹,等.基于极线局部校正的特征匹配方法[J].光学技术,2008, 34(2): 285-288.
[37]文贡坚,吕金建,王继阳.基于特征的高精度自动图像配准方法[J]. 2008, 19(9): 2293-2301.
[38] Liu Z D, Zhao Y N, Yang J Y. Fast stereo matching method using edge traction [C]. The 2nd International Conference on Intelligent Information Processing, 2004, 10: 93-96.
[39]罗军,江和平,沈振康.基于匹配跟踪置信度的自适应对应像素距离图像匹配跟踪算法[J].信号处理, 2008, 24(1): 72-77.
[40] Pedone M, Heikkila J, Talon J B, et al. Blur and contrast invariant fast stereo matching [C]. ACIVS 2008: 883–890.
[41] Luo Q M, Zhou J L, Yu S S, et al. Stereo matching and occlusion detection with integrity and illusion sensitivity [J]. Pattern Recognition Letters, 2003, (24): 1143-1149.
[42] Ahlvers U, Zoelzer U, Rechmeier S, et al. FFT-based disparity estimation for stereo image coding [C]. DAGM 2003: 257-264.
[43] Hirschmuller H, Scharstein D. Evaluation of cost functions for stereo matching [C]. IEEE CVPR, 2007.
[44] Scharstein D, Szeliski R. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms [J]. International Journal of Computer Vision, 2002, 47(1/2/3): 7–42.
[45]游素亚,徐光佑.立体视觉研究的现状与进展[J].中国图象图形学报, 1997, 2(1): 17-24.
[46]钱曾波,邱振戈,张永强.计算机立体视觉研究的进展[J].测绘学院学报, 2001, 18(4): 267-272.
[47] Birchfield S, Tomasi C. Depth discontinuities by pixel-to-pixel stereo [J]. International Journal of Computer Vision, 1999, 35(3): 269–293.
[48] Ohta Y. Stereo by intra-and inter-scanline search us-ing dynamic programming [J]. IEEE Trans. on Pattern Analysis and Machine Intelligence, 1985, 7(2): 139-154.
[49]林国余,张为公.基于控制点的分层双向动态规划立体匹配算法[J].信息与控制, 2006, 35(3): 411-416.
[50]赵杰,于舒春,蔡鹤皋.金字塔双层动态规划立体匹配算法[J].控制与决策, 2007, 22(1): 69-73.
[51] Barnard S. Disparity ana1ysis of images [J]. IEEE Trans. on PAMI, 1980, 2: 333~340.
[52] Moravec HP. Visual mapping by a robot rover [C]. Proceedings of the 6th IJCAI, 1979: 589~600.
[53] Yedidia J S, Freeman W T, Weiss Y. Understanding of belief propagation and its generalizations [C]. International Joint Conference on Artificial Intelligence (IJCAI 2001), 2001.
[54] Sun J, Shum H Y, Zheng N N. Stereo matching using belief propagation [C]. ECCV 2002.
[55] Tappen M F, Freeman W T. Comparison of graph cuts with belief propagation for stereo using identical MRF parameters [C]. IEEE International Conference on Computer Vision, 2003.
[56]张浩峰,赵春霞.基于颜色与空间距离的置信度传播立体匹配算法[J].系统仿真学报, 2008, 20(13): 3374-3377.
[57]卢阿丽,唐振民,郭龙源,等.置信度传播体视算法加速技术的应用研究[J].系统仿真学报, 2009, 21(2): 348-356.
[58]陈侃.基于马尔可夫随机场图像恢复算法研究[D].南京师范大学:数科院计算机系, 2008.
[59] Siegel M W, Sethuraman S, Mcveigh J. S, et al. Compression and interpolation of 3D-stereoscopic and multiview video [C]. Proc. of SPIE, 1997, 3012: 227-238.
[60] Ding L F, Chien S Y, Huang Y W, et al. Stereo video coding system with hybrid coding based on joint prediction scheme [C]. IEEE International Symposium on Circuits and Systems, 2005: 6082-6085.
[61] Sethuraman S. Stereoscopic image sequence compression using multiresolution and quadtree decomposition based disparity-and motion-adaptive segmentation [D]. Carnegie Mellon University: Dept. of Electrical and computer Engineering, 1996.
[62]骆艳,张兆扬.基于多分辨率四叉树的运动分割及交叠块的立体视频帧估计算法[J].中国图象图形学报, 2002, 7(7A): 716-722.
[63] Ellinas J N, Sangriotis S. Stereo video coding based on interpolated motion and disparity estimation [C]. Proceedings of the 3rd International Symposium on Image and Signal Processing and Analysis, 2003: 301-306.
[64] Choquet B, Dugelay J L, Pele D. A coding scheme for stereoscopic television sequences based on motion estimation-compensation using a 3D approach [J]. Image Processing and Its Applications, July 1995: 188-192.
[65] Richardson I E. G. H.264 and MPEG-4 video compression-video coding for next-generation multimedia[M]. John Wiley&Sons Ltd, England, 2003.
[66]李世平,蒋刚毅,郁梅.基于H.264的立体视频编码方法[J].计算机工程与应用, 2005, 1: 77-79.
[67]胡静俊.立体图像校正与视频编码[D].浙江大学:信息学院, 2006.
[68]卫瑞平.双目立体视频对象压缩编码技术[D].吉林大学:通信工程学院, 2008.
[69]傅蕊.基于H.264的立体序列图像压缩算法研究[D].天津大学:电子信息工程学院, 2007.
[70]王知嘉,刘峰,鲍卫兵.自适应选择联合补偿预测模式的立体视频编码算法[J].中国通信, 2009, 4: 58-62.
[71]陈建乐.多视点视频编码方法的研究[D].浙江大学:信息科学与工程学院, 2006.
[72]李达,黎洪松.一种改进的立体图像编码算法[J].北京师范大学学报(自然科学版),2007, 43 (1): 50-52.
[73] Garbas J U, Fecker U, Troger T, et al. 4D scalable multi-view video coding using disparity compensated view filtering and motion [C]. IEEE 8th Workshop on Multimedia Signal Processing, 2006.
[74]董小琳,侯春萍.立体图像压缩研究[J].电子测量技术, 2006, 29(6): 160-162.
[75] Thanapirom S, Fernando W A C, Edirisinghe E A. A zerotree stereo video encoder [J]. Proceedings of the 2003 International Symposium on Circuits and Systems, 2003: 608-611.
[76] Ellinas J N, Sangriotis M S. Stereo video coding based on quad-tree decomposition of B-P frames by motion and disparity interpolation [J]. IEE Proc.-Vis. Image Signal Process, Oct. 2005, Vol. 152, No. 5: 639-647.
[77]吴勇军,殷勤业,曾雁星,等.基于幅相分离立体匹配的小波立体图像压缩[J].西安交通大学学报, 2002, 36(2): 1245-1248.
[78]许昌满,张兆扬.基于马尔可夫模型和小波变换的立体残差图象编码[J].通信学报, 2002, 6(23): 90-96.
[79]章毓晋.图像分割[M].北京:科学出版社, 2006.
[80]李波,姚春莲,李炜,等.利用相邻帧和背景信息的运动对象检测[J].电子学报, 2008, 11(11): 2154-2157.
[81] Yang W X, Ngan K N, Talon J B. Unsupervised multiple object segmentation of multiview images [C]. Proceedings of Advanced Concepts for Intelligent Vision Systems 2007: 178-189.
[82]朱仲杰,郁梅,蒋刚毅,等.立体视频编码中的目标分割与立体匹配算法[J].软件学报, 2003, 14(11): 1971-1976.
[83]容观澳.计算机图像处理[M].北京:清华大学出版社, 1998.
[84] Nordling V. Efficient compression of stereoscopic video using the MPEG standard [D]. Royal Institute of Technology: Computer Science and Engineering, 2003.
[85] Mueller S K, Merkle P, Rein T, et al. Representation, coding, and rendering of 3D video objects with MPEG-4 and H.264/AVC [M]. John Wiley & Sons Lid, England, 2003.
[86]卢朝阳,周幸妮,高西全,等.三角形网格基活动图像编码——基本框架和I帧编码[J].通信学报, 2002, 23(10): 66-70.
[87]卢朝阳,周幸妮,高西全,等.三角形网格基活动图像编码研究——运动估计、运动补偿和残差图像处理[J].通信学报, 2002, 23(11): 65-73.
[88]韩军功,卢朝阳.基于Delaunay三角形网格的立体图像编码算法[J].计算机辅助设计与图形学学报, 2004, 16(12): 1708-1712.
[89]韩军功,卢朝阳.基于三角形网格的立体视差估计和遮挡点检测混合算法[J].电路与系统学报, 2005, 10(1): 44-49.
[90]吴书凯,都思丹,李华.基于半规则网格的视差估计算法[J].计算机应用, 2008, 28(4): 957-960.
[91] Szeliski R, Zabih R, Scharsein D, et al. A comparative study of energy minimization methods for markov random fields with smoothness-based priors[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(6):1068-1080.
[92] Felzenszwalb P F, Huttenlocher D P. Efficient belief propagation for early vision [J]. International Journal of Computer Vision, 2006, 70(1):41-54.
[93] Weiss Y, Freeman W T. On the optimality of solutions of the max-product belief propagation algorithm in arbitrary graphs [J]. IEEE Transactions on Information Theory, 2001.
[94] Weiss Y, Freeman W T. Interpreting images by propagating bayesian beliefs [J]. Advances in Neural Information Processing Systems, 1997, 9: 908-915.
[95]罗军辉,冯平,哈力旦.A,等. MATLAB7.0在图像处理中的应用[M].北京:机械工业出版社, 2005.
[96] Harris Chris, Stephens M. A combined corner and edge detector [C]. Proceedings of Alvey Vision Conference, 1988: 147-151.
[97] Higgins H C L. A computer algorithm for reconstructing a Scene from two projections [J]. Nature, 1981, 293: 133-135.
[98]陈泽志,吴成柯.一种高精度估计的基础矩阵的线性算法[J].软件学报, 2002, 13(4): 840-845
[99]陈泽志,吴成柯,刘勇.基础矩阵估计的加权归一化线性算法[J].软件学报, 2001, 12(3): 420-426.
[100]孙亦南,刘伟军,马永壮,等.一种加权计算基础矩阵的高精度算法[J].计算机工程, 2005, 15(31): 186-188.
[101]王才良,史泽林,李德强.图像重构中估计基础矩阵的线性迭代算法[J].微计算机信息, 2006, 22(3): 221-223.
[102]申晓凡,左峥嵘,张天序.视图合成中基础矩阵的鲁棒性估计算法[J].计算机与数字工程, 2003, 31(15): 8-12.
[103] Torr P H S, Murray D W. The development and comparison of robust methods for estimating the fundamental matrix[J]. International Journal of Computer Vision, 1997, 24: 271-300.
[104] Huang J F, Shang H L, Cheng C M. Robust fundamental matrix estimation with accurate outlier detection [J]. Journal of Information Science and Engineering, 2007, 23, 1213-1225.
[105] Deriche R, Zhang Z, Luong Q T, et al. Robust recovery of the epipolar geometry for an uncalibrated stereo rig[C]. Computer Vision-ECCV 94, 1994, (800): 567-576.
[106] Hartley R I. In defense of the eight-point algorithm [J]. IEEE Transactions on Pattern and Machine Intelligence, 1997, 19(6): 580-593.
[107]赵岩,陈贺新.彩色视频的四维矩阵离散余弦变换编码[J].中国图象图形学报, 2003, 6(8): 620-624.
[108] Wiegand T, Sullivan G. Draft ITU-T recommendation and final draft international standard of joint video specification[R]. ITU-T Rec. H.264|ISO/IEC 14496-10 AVC, in Joint Video Team(JVT) of ISO/IEC MPEG and ITU-T VCEG, JVT-G050, March 2003: 148-158.
[109] Stelmach L, Tam W J, Meegan D, et al. Stereo image quality: effects of mixed spatio-temporal resolution [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2000, 10(2): 188-193.
[110]吴勇军,殷勤业,方强,等.立体图像压缩研究进展[J].数字电视与数字视频(电视技术), 2002, 241: 20-23.
[111] Pei S C, Lai C L. Very Low bit-rate coding algorithm for stereo video with spatiotemporal HVS model and binary correlation disparity estimator [J]. IEEE Journal on Selected Areas in Communications, 1998, 16(1): 98-107.
[112]杜相文,陈贺新,赵志杰.彩色视频的四维MDCT及矩阵量化编码[J].吉林大学学报(信息科学版), 2003, 8(21): 12-16.
[113] Wiegand T, Sullivan G. Overview of the H.264/AVC video coding standard [J]. IEEE Transactions on Circuits and Systems for Video Technology, Vol.13, No.7, July 2003: 560-576.
[114] Marpe D, Bl?ttermann G, Wiegand T. Adaptive codes for H.26L [S]. ITU-Telecommunications Standardization Sector, Video Coding Experts Group (VCEG), VCEG-L13, January, 2001: 2-7.
[115] Bjontegaard G, Lillevold K. Context-adaptive VLC coding of coefficients [S]. JVT Document JVT-C028, May, 2002: 1-11.