虚拟现实中全景图象拼接
|
摘要
虚拟现实技术在诸多领域有着广泛的应用前景。基于全景图的虚拟现实技术具有设备要求低，能快速实现，效果较好等优点，具有很强的实用价值。但目前基于全景图象的虚拟现实技术在实用化的发展上，还存在图象配准的鲁棒性和精度不够高、图象拼接的速度不够快、虚拟场景的图象降质等问题。制约了该项技术的实用性。
本文深入研究了基于全景图的虚拟现实技术的各个环节，文章的主要工作在于：
1、为了提高图象配准的鲁棒性和精度，提出了一种新颖的图象配准方法。本文根据人类观察事物的行为特点，以局部边缘密度为特征，采用基于Canny准则的边缘检测方法来提取边缘，在此基础上利用模糊聚类算法定位出图象中的特征物体区域。为了克服单模板匹配的缺陷，采用了双模板匹配的方式进行图象配准。
2、提出了基于小波变换的图象多分辨率拼接方法，可加快图象拼接的速度。
3、提出了基于B样条曲线拟合的图象融合方法，以解决常规线性图象融合方法中的图象模糊效应。
4、分析了导致图象拼接累积误差的因素，提出了图象拼接累积误差的消除办法。
5、构建了基于全景图的虚拟场景的观看与展示系统，实现了视点位于柱面内和柱面外的观看模式。
基于这些方法，本文构建了一个基于柱面全景图的虚拟现实系统。实验证明，该系统具有全景图生成速度快、虚拟场景重建质量高的特点，具有较强的实用价值。
The Virtual Reality technology has broad application future in many fileds.The VR technology based on panorama has great value on practicality because of its merits such as low require in equiment, good performance in runtime, high quality of images and so on. On the road towards the utilization of VR based on panorama , there're also some problems existing. They are that the robust and precision of Image Registration is not high enough, Image Stitching is not fast enough, the Reconstructed Scene Image degrade some serious. Those problem refrain the application of this technique.
We have researched all the steps of VR technology based on panorama in depth, and the main work lies in:
1. To Enhance the robust and precision of Image Registration, the paper advanced a novel Image Registration MethodAccording to the human behavior characteristic in observation, we use the local edge density(LED) as feature, and the edges are extracted by the edge detect method based on Canny rules,then the fuzzy clustering algorithm can locate the feature region in image.To overcome the vice of template matching with single template, we registration image by the means of matching with two templates.
2. The paper advanced the Image Multi-Resolution Stitching mehod based on the wavelet transformation, which can increase the speed of Image Stitching.
3. The paper advanced the Image Merging Method based on B spine function curve simulation,which can resolve the effect about images being fuzzied
4. The paper analyzed the causes of the cumulative error produced in the procedure of Image Stitching and gave the solution for it.
5. We established a VR lay out system based on panorama. There're two observation mode: the viewpoint is in the inner of image cylinder , the viewpoint is outside of the image cylinder.
With these methods, we construct a simple VR system based on cylinderical panorama.The experiment result proved that this system can produce panprama in a high speed and reconstruct the virtual scene with high image quality, so it have much value in practice.
本文深入研究了基于全景图的虚拟现实技术的各个环节，文章的主要工作在于：
1、为了提高图象配准的鲁棒性和精度，提出了一种新颖的图象配准方法。本文根据人类观察事物的行为特点，以局部边缘密度为特征，采用基于Canny准则的边缘检测方法来提取边缘，在此基础上利用模糊聚类算法定位出图象中的特征物体区域。为了克服单模板匹配的缺陷，采用了双模板匹配的方式进行图象配准。
2、提出了基于小波变换的图象多分辨率拼接方法，可加快图象拼接的速度。
3、提出了基于B样条曲线拟合的图象融合方法，以解决常规线性图象融合方法中的图象模糊效应。
4、分析了导致图象拼接累积误差的因素，提出了图象拼接累积误差的消除办法。
5、构建了基于全景图的虚拟场景的观看与展示系统，实现了视点位于柱面内和柱面外的观看模式。
基于这些方法，本文构建了一个基于柱面全景图的虚拟现实系统。实验证明，该系统具有全景图生成速度快、虚拟场景重建质量高的特点，具有较强的实用价值。
The Virtual Reality technology has broad application future in many fileds.The VR technology based on panorama has great value on practicality because of its merits such as low require in equiment, good performance in runtime, high quality of images and so on. On the road towards the utilization of VR based on panorama , there're also some problems existing. They are that the robust and precision of Image Registration is not high enough, Image Stitching is not fast enough, the Reconstructed Scene Image degrade some serious. Those problem refrain the application of this technique.
We have researched all the steps of VR technology based on panorama in depth, and the main work lies in:
1. To Enhance the robust and precision of Image Registration, the paper advanced a novel Image Registration MethodAccording to the human behavior characteristic in observation, we use the local edge density(LED) as feature, and the edges are extracted by the edge detect method based on Canny rules,then the fuzzy clustering algorithm can locate the feature region in image.To overcome the vice of template matching with single template, we registration image by the means of matching with two templates.
2. The paper advanced the Image Multi-Resolution Stitching mehod based on the wavelet transformation, which can increase the speed of Image Stitching.
3. The paper advanced the Image Merging Method based on B spine function curve simulation,which can resolve the effect about images being fuzzied
4. The paper analyzed the causes of the cumulative error produced in the procedure of Image Stitching and gave the solution for it.
5. We established a VR lay out system based on panorama. There're two observation mode: the viewpoint is in the inner of image cylinder , the viewpoint is outside of the image cylinder.
With these methods, we construct a simple VR system based on cylinderical panorama.The experiment result proved that this system can produce panprama in a high speed and reconstruct the virtual scene with high image quality, so it have much value in practice.
引文
[1] L.G. Brown. A Survey of Image Registration Techniques. ACM Computer Survey, Vol24, No. 4,1992.
[2] 徐丹，潘志庚，石教英．虚拟现实中基于图像的绘制技术．中国图象图形学报，1998，3(12)：1005～1010．
[3] Chen S E, William L. View Interpolation for Image Synthesis. Computer Graphics(SIGGRAPH'93), 1993:583～590.
[4] Seitz S, Dyer C. Physically-Valid View Synthesis by Image Interpolatioin. IEEE Wrokshop on Representaions of Visual Scenes, Cambridge, ME, 1995:18～25.
[5] Seitz S, Dyer C. View morphing Computer Graphics(SIGGRAPH' 96),1996:21～30.
[6] 江加和，宋子善，沈为群等．虚拟现实中基于图象绘制虚拟场景技术综述．系统工程与电子技术，2000，22(10)：86～89．
[7] Sawhney H S, etal Model-based 2D&3D dominant motion estimation for mosaicing and video representation. Fifth International Conference on Computer Vision(ICCV' 95), MIT, Cambridge, MA, IEEE Computer Society Press. 1995: 583～590.
[8] Adelson E H. Layered representations of Visual Scenes, Cambridge, MA, 1995: 3～9
[9] Adelson E H, Bergen J R. The plenoptic function and the elements of early vision. In M Landy & J. Movshon, editors, Computational Models of visual Processing, chapter 1. MIT Press, Cambridge, MA, 1991.
[10] McMillan L. Bishop G. Plenoptic Modeling: An Image-Based Rendering System. In Computer Graphics(SIGGRAPH' 95), 1995: 39～46.
[11] Levoy M, Hanrahan P. Light field rendering. In Computer Graphics(SIGGRAPH' 96), 1996: 31～42.
[12] Gortler S J, Grzeszczuk R, Szekiski R, Cohen M F. The Lumigraph.
In Computer Graphics(SIGGRAPH'96), 1996: 43～54.
[13] Shum H Y, He L W. Rendering with concentric mosaics. In Proc SIGGRAPH'99, Los Angeles, 1999. 299～306.
[14] S. Chen. Qucik Time VR-an image-based approach to virtual environment navigation. Computer Graphicx(ACMSIGGRAPH'95). Auguest 1995: 29～38.
[15] 胡晓峰，老松杨，张茂军．HVS：一个半现实全景图时空模型．小型微型计算机系统，1994，15(12)：19～23．
[16] 蔡勇，刘学慧，吴恩华．基于图象绘制的虚拟现实系统环境．软件学报，1997，8(10)：721～728．
[17] 刘翔，曾芬芳，高世峰，陈浩．基于IBR技术的全景图生成．华东船舶工业学院学报，2001，15(1)：32～37．
[18] 韦穗．基于图象的虚拟现实技术．安徽大学学报，1998，22(4)：34～39
[19] R. Szeliski, Harry Shum. Create full-view Panoramic mosaics and texture-mapped 3D models. In SIGGRAPH' 97 Conference Proceedings, pp. 251-258, Auguest 1997.
[20] R. Szeliski. Video mosaics for virtual environments. IEEE Computer Graphics and Applications, 22-30, March 1996.
[21] SAmuel Pelg, Joshua Herman. Panoramic Mosaics by Manifold Projection, IEEE CVPR, Proceedings, 1997.
[22] R. Szeliski, J. Coughlan. Hierarchical spline-based image registration. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR' 94),pp. 194-201, June 1994.
[23] 郝爱民，张蓓，赵沁平．计算机虚拟北航的建造与人机交互的实现．北京航空航天大学学报，2000，26(4)：435～438．
[24] 钟力，胡晓峰．全景视频的信息组织和实现方法．小型微型计算机系统，1996，17(12)：1～5．
[25] 孙庆杰，吴恩华．一种新的用于图象拼接的颜色融合方法．工程图学学报，1998，4：65～74．
[26] 钟力，胡晓峰．重叠图象拼接算法．中国图象图形学报，1998，3(5)：367～370．
[27] 齐越，徐玮等．球面虚拟空间的自由漫游．小型微型计算机系统，2001，22(7)：793～795．
[28] 李洪举，吴恩华．基于图象的室内虚拟环境的研究．计算机学报，1999，22(5)：529～534．
[29] 赵唯，李端义．基于小波变换与形态学的一种边界重叠图象的拼接算法．微机发展，2001，6：3～6．
[30] 漆驰，刘强，孙家广．摄像机图像序列的全景图拼接．计算机辅助设计与图形学学报，2001，13(7)：605～609．
[31] 黄铜城，银丽平．视频图象序列的实时镶嵌技术研究．计算机工程，2001，27(4)：124～126．
[32] 胡金辉，胡占义．同心拼图中的深度的计算．计算机学报，2001，24(6)：580～587．
[33] 罗亚波，陈定方．图象与建模相结合的虚拟现实关键技术．交通与计算机，2000，18(6)：58～60．
[34] 许学军，甘咏梅．虚拟场景中全景图象生成技术研究．吉首大学学报，1997，18(4)：51～53．
[35] 张素，陈江，狳乃平．一种多图象的多分辨合成技术．小型微型计算机系统，2000，21(5)：516～519．
[36] S.B Kang. A Survey of Image-Based Rendering Techniques. Technical Report 1997/4, Digital Equipment Corporation, Cambridge Research Lab, August 1997.
[37] 邱茂林，马颂德，李毅．计算机视觉中摄像机定标综述．自动化学报，2000，26(1)：43～55．
[38] F. Du, M. Brady. Self-calibration of the intrinsic parameters of cameras for active vision systems. In CVPR, pages 477-482, New York, NY,June 1993.
[39] Horst A. Beyer. Accurate calibration of CCD-cameras. In IEEE
Proceedings of International Conference on Computer Vision and Pattern Recognition, Urbana Champaign, IL, June 1993.
[40] R. Brunelli, Poggio T. Template matching:Matched spatial filter and beyond. A.I. Memo 1549, M.I.T.,October 1995.
[41] G.P. Stein. Accurate internal Camera Calibration using rataion,with analysis of sources of error. In ICCV95, pp. 230-236, 1995.
[42] Sawhney H., R. Kumar. True Multi-Image Alignment and Its Application to Mosaicing and Lens Distortion Correction. IEEE Transactions on Pattern Analysis and Machine Interlligence 21(3), pp. 245-253, 1999.
[43] 夏良正．数字图象处理．南京：东南大学出版社，1999．
[44] Eric Bourque, Gregory Dudek. Automated Creation of Image-Based Virtual Reality. SPIE Proceedings on Intelligent Systems and Manufacturing, Pittsburgh, PA, Oct. 1997: 292-303.
[45] Castleman K.R. Digitla Image Processing. Beijing: Prentice-Hall&Tsinghua University Press, 1998.
[46] 边肇祺，张学工．模式识别．北京：清华大学出版社，2000．
[47] J.F. Canny. A computational approach to edge detection. IEEE Trans. On Patterb Analysis and Machine Intelligence, 8(6):679-698, Nov 1986.
[48] 刘曙光，刘明远，何钺．基于Canny准则的基数B样条小波边缘检测．信号处理，2001，17(5)：418～423
[49] Mallat S. A theory for multiresolution signal decomposition:The wavelet representation [J]. IEEE Trans Pattern Anal Mach Inte1,1989,11(7): 674-693.
[50] Gregory K. Wallance. The JPEG Still Picture Compression Standard, Communications of the ACM, 1991,34(4):30-44.
[2] 徐丹，潘志庚，石教英．虚拟现实中基于图像的绘制技术．中国图象图形学报，1998，3(12)：1005～1010．
[3] Chen S E, William L. View Interpolation for Image Synthesis. Computer Graphics(SIGGRAPH'93), 1993:583～590.
[4] Seitz S, Dyer C. Physically-Valid View Synthesis by Image Interpolatioin. IEEE Wrokshop on Representaions of Visual Scenes, Cambridge, ME, 1995:18～25.
[5] Seitz S, Dyer C. View morphing Computer Graphics(SIGGRAPH' 96),1996:21～30.
[6] 江加和，宋子善，沈为群等．虚拟现实中基于图象绘制虚拟场景技术综述．系统工程与电子技术，2000，22(10)：86～89．
[7] Sawhney H S, etal Model-based 2D&3D dominant motion estimation for mosaicing and video representation. Fifth International Conference on Computer Vision(ICCV' 95), MIT, Cambridge, MA, IEEE Computer Society Press. 1995: 583～590.
[8] Adelson E H. Layered representations of Visual Scenes, Cambridge, MA, 1995: 3～9
[9] Adelson E H, Bergen J R. The plenoptic function and the elements of early vision. In M Landy & J. Movshon, editors, Computational Models of visual Processing, chapter 1. MIT Press, Cambridge, MA, 1991.
[10] McMillan L. Bishop G. Plenoptic Modeling: An Image-Based Rendering System. In Computer Graphics(SIGGRAPH' 95), 1995: 39～46.
[11] Levoy M, Hanrahan P. Light field rendering. In Computer Graphics(SIGGRAPH' 96), 1996: 31～42.
[12] Gortler S J, Grzeszczuk R, Szekiski R, Cohen M F. The Lumigraph.
In Computer Graphics(SIGGRAPH'96), 1996: 43～54.
[13] Shum H Y, He L W. Rendering with concentric mosaics. In Proc SIGGRAPH'99, Los Angeles, 1999. 299～306.
[14] S. Chen. Qucik Time VR-an image-based approach to virtual environment navigation. Computer Graphicx(ACMSIGGRAPH'95). Auguest 1995: 29～38.
[15] 胡晓峰，老松杨，张茂军．HVS：一个半现实全景图时空模型．小型微型计算机系统，1994，15(12)：19～23．
[16] 蔡勇，刘学慧，吴恩华．基于图象绘制的虚拟现实系统环境．软件学报，1997，8(10)：721～728．
[17] 刘翔，曾芬芳，高世峰，陈浩．基于IBR技术的全景图生成．华东船舶工业学院学报，2001，15(1)：32～37．
[18] 韦穗．基于图象的虚拟现实技术．安徽大学学报，1998，22(4)：34～39
[19] R. Szeliski, Harry Shum. Create full-view Panoramic mosaics and texture-mapped 3D models. In SIGGRAPH' 97 Conference Proceedings, pp. 251-258, Auguest 1997.
[20] R. Szeliski. Video mosaics for virtual environments. IEEE Computer Graphics and Applications, 22-30, March 1996.
[21] SAmuel Pelg, Joshua Herman. Panoramic Mosaics by Manifold Projection, IEEE CVPR, Proceedings, 1997.
[22] R. Szeliski, J. Coughlan. Hierarchical spline-based image registration. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR' 94),pp. 194-201, June 1994.
[23] 郝爱民，张蓓，赵沁平．计算机虚拟北航的建造与人机交互的实现．北京航空航天大学学报，2000，26(4)：435～438．
[24] 钟力，胡晓峰．全景视频的信息组织和实现方法．小型微型计算机系统，1996，17(12)：1～5．
[25] 孙庆杰，吴恩华．一种新的用于图象拼接的颜色融合方法．工程图学学报，1998，4：65～74．
[26] 钟力，胡晓峰．重叠图象拼接算法．中国图象图形学报，1998，3(5)：367～370．
[27] 齐越，徐玮等．球面虚拟空间的自由漫游．小型微型计算机系统，2001，22(7)：793～795．
[28] 李洪举，吴恩华．基于图象的室内虚拟环境的研究．计算机学报，1999，22(5)：529～534．
[29] 赵唯，李端义．基于小波变换与形态学的一种边界重叠图象的拼接算法．微机发展，2001，6：3～6．
[30] 漆驰，刘强，孙家广．摄像机图像序列的全景图拼接．计算机辅助设计与图形学学报，2001，13(7)：605～609．
[31] 黄铜城，银丽平．视频图象序列的实时镶嵌技术研究．计算机工程，2001，27(4)：124～126．
[32] 胡金辉，胡占义．同心拼图中的深度的计算．计算机学报，2001，24(6)：580～587．
[33] 罗亚波，陈定方．图象与建模相结合的虚拟现实关键技术．交通与计算机，2000，18(6)：58～60．
[34] 许学军，甘咏梅．虚拟场景中全景图象生成技术研究．吉首大学学报，1997，18(4)：51～53．
[35] 张素，陈江，狳乃平．一种多图象的多分辨合成技术．小型微型计算机系统，2000，21(5)：516～519．
[36] S.B Kang. A Survey of Image-Based Rendering Techniques. Technical Report 1997/4, Digital Equipment Corporation, Cambridge Research Lab, August 1997.
[37] 邱茂林，马颂德，李毅．计算机视觉中摄像机定标综述．自动化学报，2000，26(1)：43～55．
[38] F. Du, M. Brady. Self-calibration of the intrinsic parameters of cameras for active vision systems. In CVPR, pages 477-482, New York, NY,June 1993.
[39] Horst A. Beyer. Accurate calibration of CCD-cameras. In IEEE
Proceedings of International Conference on Computer Vision and Pattern Recognition, Urbana Champaign, IL, June 1993.
[40] R. Brunelli, Poggio T. Template matching:Matched spatial filter and beyond. A.I. Memo 1549, M.I.T.,October 1995.
[41] G.P. Stein. Accurate internal Camera Calibration using rataion,with analysis of sources of error. In ICCV95, pp. 230-236, 1995.
[42] Sawhney H., R. Kumar. True Multi-Image Alignment and Its Application to Mosaicing and Lens Distortion Correction. IEEE Transactions on Pattern Analysis and Machine Interlligence 21(3), pp. 245-253, 1999.
[43] 夏良正．数字图象处理．南京：东南大学出版社，1999．
[44] Eric Bourque, Gregory Dudek. Automated Creation of Image-Based Virtual Reality. SPIE Proceedings on Intelligent Systems and Manufacturing, Pittsburgh, PA, Oct. 1997: 292-303.
[45] Castleman K.R. Digitla Image Processing. Beijing: Prentice-Hall&Tsinghua University Press, 1998.
[46] 边肇祺，张学工．模式识别．北京：清华大学出版社，2000．
[47] J.F. Canny. A computational approach to edge detection. IEEE Trans. On Patterb Analysis and Machine Intelligence, 8(6):679-698, Nov 1986.
[48] 刘曙光，刘明远，何钺．基于Canny准则的基数B样条小波边缘检测．信号处理，2001，17(5)：418～423
[49] Mallat S. A theory for multiresolution signal decomposition:The wavelet representation [J]. IEEE Trans Pattern Anal Mach Inte1,1989,11(7): 674-693.
[50] Gregory K. Wallance. The JPEG Still Picture Compression Standard, Communications of the ACM, 1991,34(4):30-44.