全方位图像展开算法改进的研究与实现
|
摘要
基于反射镜面的全方位视觉是一项新兴的视觉技术,该视觉系统能一次性的获取水平方向360°的场景图像,与传统视觉系统相比具有视野大的优点,非常适合在实时监控等领域的应用。单张全方位图像包含了水平一周的场景信息,存在严重的扭曲现象,在应用中需要从全方位图像中恢复出所拍摄的景物信息。
本文首先介绍了全方位视觉技术的研究和应用现状。通过对双曲镜面全方位视觉系统中几何光路的分析,详细地阐述了全方位视觉图像的成像过程和使用单张全方位图像展开获得全景图以及透视图的方法。
在展开过程中,使用传统的近邻取样法容易造成展开图像素点不连续,展开质量较差,尤其是对于分辨率不高的全方位图。本文在展开的逆投影坐标变换过程中引入了双线性内插值法,提高了展开质量。并根据逆投影坐标变换公式特点,引入了三角函数查表法,减少了大量三角函数的重复运算,提高了展开性能。
全方位视觉展开运算量巨大,而且操作系统在多媒体处理上会占用较多CPU,两者容易使性能成为全方位视觉展开过程中的一个瓶颈。本文作者开发了用于将全方位图展开成全景图的DirectShow过滤器,使用该过滤器可以充分利用显卡的加速功能,发挥显卡的性能,减少CPU的占用率,达到更流畅的全方位视频展开回放效果。
使用本文开发的过滤器,开发了基于DirectShow的全方位视觉展开软件平台,实现了对全方位图像、全方位视频文件、全方位实时视频流展开成全景图和透视图等功能。本文详细阐述了该软件的设计方法、具体实现,并简要介绍了软件的操作及性能。
Omnidirectional vision is a vision system based on reflector,which captures scene of 360 degrees level direction once.Compared with traditional vision system,omnidirectional vision is of wider view that is suitable for various applications in real-time surveillance fields.One omnidirectional image contains information of one level circle scene, however the image is seriously distorted,which isn't capable for practice application unless the scene information can be correctly extracted from omnidierectional image.
Firstly,current research and applications of omnidierectional vision technology are introduced.Through specific analysis of geometry light routing in hyperboloid omnidirectional vision system,imaging process of omnidirectional vision is explained in detail,and technique for unwrapping one omnidirectional image to get panorama image and rendering is brought up in this paper.
In unwrapping process,traditional methods easily make image pixel discontinuous which causes low image quality,especially for low resolution omnidirectional image.In this paper,quadratic interpolation method is applied for athwart projection transform,which improves image quality.According to athwart projection formula,trigonometric function table-checking method is induced,based on which,massive trigonometric function operation complex is reduced and unwrapping performance is improved.
Because the process of omnidirectional vision unwrapping contains complex operation,on the other side,multimedia process makes OS costs plentiful CPU.Both of above will be a bottleneck for omnidirectional vision unwrapping process.Aiming at this bottleneck,DirectShow filter for unwrapping omnidirectional image is designed and implemented.The filter makes full use of graphics card acceleration promising higher performance of graphics card,cost less CPU,what makes the unwrapped panorama video playback more fluent.
Omnidirectional vision platform,based on DirectShow filter developed in this paper,implements functions of unwrapping omnidirectional image,video,and real-time video stream to panorama or perspective images.The design and specific implementation,including operations of software and performance evaluation,are explained simply.
本文首先介绍了全方位视觉技术的研究和应用现状。通过对双曲镜面全方位视觉系统中几何光路的分析,详细地阐述了全方位视觉图像的成像过程和使用单张全方位图像展开获得全景图以及透视图的方法。
在展开过程中,使用传统的近邻取样法容易造成展开图像素点不连续,展开质量较差,尤其是对于分辨率不高的全方位图。本文在展开的逆投影坐标变换过程中引入了双线性内插值法,提高了展开质量。并根据逆投影坐标变换公式特点,引入了三角函数查表法,减少了大量三角函数的重复运算,提高了展开性能。
全方位视觉展开运算量巨大,而且操作系统在多媒体处理上会占用较多CPU,两者容易使性能成为全方位视觉展开过程中的一个瓶颈。本文作者开发了用于将全方位图展开成全景图的DirectShow过滤器,使用该过滤器可以充分利用显卡的加速功能,发挥显卡的性能,减少CPU的占用率,达到更流畅的全方位视频展开回放效果。
使用本文开发的过滤器,开发了基于DirectShow的全方位视觉展开软件平台,实现了对全方位图像、全方位视频文件、全方位实时视频流展开成全景图和透视图等功能。本文详细阐述了该软件的设计方法、具体实现,并简要介绍了软件的操作及性能。
Omnidirectional vision is a vision system based on reflector,which captures scene of 360 degrees level direction once.Compared with traditional vision system,omnidirectional vision is of wider view that is suitable for various applications in real-time surveillance fields.One omnidirectional image contains information of one level circle scene, however the image is seriously distorted,which isn't capable for practice application unless the scene information can be correctly extracted from omnidierectional image.
Firstly,current research and applications of omnidierectional vision technology are introduced.Through specific analysis of geometry light routing in hyperboloid omnidirectional vision system,imaging process of omnidirectional vision is explained in detail,and technique for unwrapping one omnidirectional image to get panorama image and rendering is brought up in this paper.
In unwrapping process,traditional methods easily make image pixel discontinuous which causes low image quality,especially for low resolution omnidirectional image.In this paper,quadratic interpolation method is applied for athwart projection transform,which improves image quality.According to athwart projection formula,trigonometric function table-checking method is induced,based on which,massive trigonometric function operation complex is reduced and unwrapping performance is improved.
Because the process of omnidirectional vision unwrapping contains complex operation,on the other side,multimedia process makes OS costs plentiful CPU.Both of above will be a bottleneck for omnidirectional vision unwrapping process.Aiming at this bottleneck,DirectShow filter for unwrapping omnidirectional image is designed and implemented.The filter makes full use of graphics card acceleration promising higher performance of graphics card,cost less CPU,what makes the unwrapped panorama video playback more fluent.
Omnidirectional vision platform,based on DirectShow filter developed in this paper,implements functions of unwrapping omnidirectional image,video,and real-time video stream to panorama or perspective images.The design and specific implementation,including operations of software and performance evaluation,are explained simply.
引文
[1]马颂德,张正友.计算机视觉-计算理论与算法基础[M].北京:科学出版社2003.1
[2]Yasushi Yagi,Katsushi Ikeuchi.Research in Japan on Omnidirectional Sensors and their applications[R].International journal of computer vision Mar 11,2004
[3]Yasushi Yagi.Omnidirectional Sensing and Its Applications[J].IEICE Trans.INF&SYST,VOL.E82-D,NO.3 March,1999
[4]Kostas Daniilidis.http://www.cis.upenn.edu/~kostas/omni.html.2005
[5]Rees D W.Panoramic tele-vison viewing system[P].USP:3505465,1970-04.
[6]Yagi,Kawato.Panoramic Scene Analysis with Conic Projection[J].Proc of International Conference on Robots and Systems.1990
[7]Hong,J.Image Based Homing[J].Proc of IEEE international Conference on Robots and Automation.1991
[8]S,K,Nayar.Catadioptric Omnidirectional Camera[J].In International Conference on Computer Vision and Pattern Recongnition 1997
[9]K.Yamazawa,Y.Yagi,and M.Yachida.Omnidirectional Imaging with Hyperboloidal Projection[J].Proc.Int.Conf.on Intelligent Robots and Systems,Vol.2,pp.1029-1034,July 1993
[10]Tomas svoboda,Tomas Pajdla and Vaclav hlavac.Motion estimation using central panoramic cameras[J].In Stefan Hahn,editor,IEEE international Conference on Intelligent Vehicles,pages 335-340,Stuttgart,Germany,Oct 1998.
[11]T.Svoboda.Central Panoramic Cameras Design,Geometry,Egomotion[D].PhD thesis,Czech Technical University,1999.
[12]Tomas Svoboda,Tomas Pajdla.Epipolar Geometry for Central Catadioptric Cameras[J].Czech technical University,2001.3
[13]北京理工大学媒体计算与智能系统实验室.http://www.mcislab.org.cn/ direction/research/pqm/omnidirectional%20camera.htm.2006
[14]XiangHua Ying,Zhanyi hu.Catadioptric Camera Calibration Using Geometric Invariants[J].IEEE Transactions on Pattern Analysis and Machine Intelligence.Oct,2004
[15]Y.Onoe,K.Yamazawa,N.Yokoya,and H.Takemura,Visual surveillance and monitoring system using an omnidirectional video camera[J],Proc.IEEE Intl.Conf.on Pattern Recognition,pp.588-592.1998
[16]Noboru Babaguchi,Yoshihiko fujimoto,Kazumasa Yamazawa,Naokazu Yokoya.A system for visualization and summarization of omnidirectional surveillance video[J].Proc 8th International WorkShop on Multimedia Inforamtion System,pp18-27.Oct.2002
[17]皮文凯.基于全方位视觉的人体运动检测与跟踪[C].北京:北京大学智能科学系2001级硕士论文.2004
[18]Yu Ohara,Ryusuke Sagawa,Tomio Echigo,Yasushi Yagi.Gait Volum:Spatio-temporal Analysis of Walking[J].Omnivis 2004.
[19]Yong Rui,Anoop Gupta JJCadiz.Viewing Meeting Captured by an Omni-Directional Camera[J].Human Factors in Computing Systems CHI 2001,Seattle Washington,pp 450-457.2001
[20]Frank Wallhoff,Martin Zobl,Gerhard Rigoll and Igor Potucek.Face Tracking in Meeting Room Scenarios Using Omnidirectional Views In IEEE Int.Proceedings on International Conference on Pattern Recognition(ICPR),Cambridge,UK,August 2004
[21]Rainer Stiefelhagen,Xilin Chen,Jie Yang.Capturing Interactions in Meetings with Omnidirectional Cameras[J].Womtec-ICCV workshop 2003
[22]Yagi Y,kawato S,Tsuji S.Collision avoidance using omnidirectional images sensor(COPIS)[J].Proc IEEE ICRA,1991
[23]Yagi Y,kawato S,Tsuji S.Real-time omnidirectional image sensor(COPIS)for vision-guided navigation[J].IEEE Trans Robotics and Automation,vol 1,nol,Feb 1994
[24]Yagi Y,Okumara H,Yachida M.Multiple visual sensing system for mobile robot[J]. Proc ICRA 1994
[25] Yagi Y, Yachida M. Evaluating effectivity of map gemeration by tracking vertical edges in omnidirectional image sequence[J].proc of ICRA, 1995
[26] Stefan Harbar, Gaurav S.Sukhatme. Omnidirectional Vision for an Autonous Helicopter[J]. In proceedings of the IEEE International Conference on Robotics and Automation .pp 3602-3609. April 2004
[27] Heung Yeung Shum,Sing Bing Kang. A Review of Image-based Rendering Techniques[J]. IEEE/SPIE Visual Communications and Image Processing (VCIP) 2000 .pp2-13.Perth, June 2000
[28] Daniel GAliaga,Ingrid Carlbom. Plenoptic Stitching: A scalable method for reconstruction 3D interactive walkthroughs[J]. In Proceedings of ACM SIGGRAPH 2001, pp 443-450.August 2001
[29] D. Aliaga, T. Funkhouser, D. Yanovsky, I. Carlbom, Sea of Images[J], Proceedings of IEEE Visualization. pp. 331-338.2001
[30] Hiroshi Ishiguro,Kim C,Ng,Richard Capella,Mohan M.Trivedi. Omnidirectional image-based modeling:three approaches to approximated plenoptic representations[J].Machine Vision and Applications.2003
[31] Jose Gaspar, Etienne Grossmann,Jose Sntos-Victor. Interactive Reconstruction from an Omnidirectional Image[J]. 9th International Symposium on Intelligent Robotic Systems,2001
[32] Petr Doubek, Tomas Svoboda. Reliable 3D reconstruction from a few catadioptric images[J]. IEEE workshop on Omnidirectional vision .pp 71-78.2002
[33] Petri Doubek,Tomas Svoboda. What space can be reconstructed from multiple catadioptric images[J]. In Horst Wildenauer and Walter Kropatsch, editors, Proceedings of the Computer Vision Winter Workshop. PRIP TU Vienna, pp 198-207. February 2002
[34] Roland Bunschoten,Ben krose. Robust Scene Reconstruction from and Omnidirectional Vision System[J].IEEE transactions on robotics and automation.2002
[35]陈焱,杨东勇,沈正伟.全方位图像展开成全景和透视图的实现方法[J].计算机工程.2007.1
[36]Glullo Sandlni,Jose Santos Victor,Tomas Pajdla,Fablo Berton.OMNIVIEWS:Direct Omnidirectional Imaging Based on a Retina-like Sensor[J].IEEE sensors,2002.
[37]曾吉勇,苏显渝.水平场景无畸变的折反射全景成像系统[J].光学学报.2003.5
[38]张健,段滔,刘爱东.全景图像几何畸变校正算法研究及其软件实现[J].光学与光电技术,2003.6
[39]李忠新,刘春庆,王执铨.一种改进的柱面全景视图生成方法[J].小型微型计算机系统.2005.7
[40]凌云峰,朱齐丹,吴自新,张智.全景视觉图像柱面理论展开算法实现及其改进[J].应用科技,2006.9
[41]Sheng-Wen Jeng,Wdn-Hsiang tsai.Precise Image Unwarping of Omnidirectional Cameras with Hyperbolic-Shaped Mirrors[J].CVGIP 2000
[42]尤玉虎,周孝宽.数字图像最佳插值算法研究[J].中国空间科学技术.2005.6
[43]陈良,高成敏.快速离散化双线性插值算法[J].计算机工程与设计.2007.8
[44]Neil A.Dodgson.Quadratic Interpolation for Image Resampling[C].IEEE Transactions on Image Processing.1997.9
[45]Ahmed M.Eltawil,Eugene Grayver,Alireza Tarighat,Jean Francois Frigon and Ali Azam Abbasfar.Implementation of a Digital Timing Recovery Circuit for CDMA Applications[C].IEEE 2005 CUSTOM INTEGRATED CIRCUITS CONFERENCE
[46]Microsoft MSDN.http://msdn.microsoft.com/library/default.asp?url=/library /en-us/gdicpp/GDIPlus/GDIPlus.asp.2006
[47]Microsoft MSDN.http://msdn2.microsoft.com/en-us/library/ms87987 5.aspx
[48]Microsoft Corp.Microsoft DirectX 9.0 SDK Update.December 2004
[49]Mark D.Pesce.Programming Microsoft DirectShow for digital video and television[M],Microsoft Press.2003
[50]陆其明,DirectShow实务精选[M].北京:科学出版社,2004.7
[51]管国辰,一种模块化的流媒体系统开发框架[J].计算机工程.2007.2
[52]Vstone Corp.http://www.vstone.co.jp.2006
[53]Remote Reality.http://www.remotereality.com/vtprod/.2006
[2]Yasushi Yagi,Katsushi Ikeuchi.Research in Japan on Omnidirectional Sensors and their applications[R].International journal of computer vision Mar 11,2004
[3]Yasushi Yagi.Omnidirectional Sensing and Its Applications[J].IEICE Trans.INF&SYST,VOL.E82-D,NO.3 March,1999
[4]Kostas Daniilidis.http://www.cis.upenn.edu/~kostas/omni.html.2005
[5]Rees D W.Panoramic tele-vison viewing system[P].USP:3505465,1970-04.
[6]Yagi,Kawato.Panoramic Scene Analysis with Conic Projection[J].Proc of International Conference on Robots and Systems.1990
[7]Hong,J.Image Based Homing[J].Proc of IEEE international Conference on Robots and Automation.1991
[8]S,K,Nayar.Catadioptric Omnidirectional Camera[J].In International Conference on Computer Vision and Pattern Recongnition 1997
[9]K.Yamazawa,Y.Yagi,and M.Yachida.Omnidirectional Imaging with Hyperboloidal Projection[J].Proc.Int.Conf.on Intelligent Robots and Systems,Vol.2,pp.1029-1034,July 1993
[10]Tomas svoboda,Tomas Pajdla and Vaclav hlavac.Motion estimation using central panoramic cameras[J].In Stefan Hahn,editor,IEEE international Conference on Intelligent Vehicles,pages 335-340,Stuttgart,Germany,Oct 1998.
[11]T.Svoboda.Central Panoramic Cameras Design,Geometry,Egomotion[D].PhD thesis,Czech Technical University,1999.
[12]Tomas Svoboda,Tomas Pajdla.Epipolar Geometry for Central Catadioptric Cameras[J].Czech technical University,2001.3
[13]北京理工大学媒体计算与智能系统实验室.http://www.mcislab.org.cn/ direction/research/pqm/omnidirectional%20camera.htm.2006
[14]XiangHua Ying,Zhanyi hu.Catadioptric Camera Calibration Using Geometric Invariants[J].IEEE Transactions on Pattern Analysis and Machine Intelligence.Oct,2004
[15]Y.Onoe,K.Yamazawa,N.Yokoya,and H.Takemura,Visual surveillance and monitoring system using an omnidirectional video camera[J],Proc.IEEE Intl.Conf.on Pattern Recognition,pp.588-592.1998
[16]Noboru Babaguchi,Yoshihiko fujimoto,Kazumasa Yamazawa,Naokazu Yokoya.A system for visualization and summarization of omnidirectional surveillance video[J].Proc 8th International WorkShop on Multimedia Inforamtion System,pp18-27.Oct.2002
[17]皮文凯.基于全方位视觉的人体运动检测与跟踪[C].北京:北京大学智能科学系2001级硕士论文.2004
[18]Yu Ohara,Ryusuke Sagawa,Tomio Echigo,Yasushi Yagi.Gait Volum:Spatio-temporal Analysis of Walking[J].Omnivis 2004.
[19]Yong Rui,Anoop Gupta JJCadiz.Viewing Meeting Captured by an Omni-Directional Camera[J].Human Factors in Computing Systems CHI 2001,Seattle Washington,pp 450-457.2001
[20]Frank Wallhoff,Martin Zobl,Gerhard Rigoll and Igor Potucek.Face Tracking in Meeting Room Scenarios Using Omnidirectional Views In IEEE Int.Proceedings on International Conference on Pattern Recognition(ICPR),Cambridge,UK,August 2004
[21]Rainer Stiefelhagen,Xilin Chen,Jie Yang.Capturing Interactions in Meetings with Omnidirectional Cameras[J].Womtec-ICCV workshop 2003
[22]Yagi Y,kawato S,Tsuji S.Collision avoidance using omnidirectional images sensor(COPIS)[J].Proc IEEE ICRA,1991
[23]Yagi Y,kawato S,Tsuji S.Real-time omnidirectional image sensor(COPIS)for vision-guided navigation[J].IEEE Trans Robotics and Automation,vol 1,nol,Feb 1994
[24]Yagi Y,Okumara H,Yachida M.Multiple visual sensing system for mobile robot[J]. Proc ICRA 1994
[25] Yagi Y, Yachida M. Evaluating effectivity of map gemeration by tracking vertical edges in omnidirectional image sequence[J].proc of ICRA, 1995
[26] Stefan Harbar, Gaurav S.Sukhatme. Omnidirectional Vision for an Autonous Helicopter[J]. In proceedings of the IEEE International Conference on Robotics and Automation .pp 3602-3609. April 2004
[27] Heung Yeung Shum,Sing Bing Kang. A Review of Image-based Rendering Techniques[J]. IEEE/SPIE Visual Communications and Image Processing (VCIP) 2000 .pp2-13.Perth, June 2000
[28] Daniel GAliaga,Ingrid Carlbom. Plenoptic Stitching: A scalable method for reconstruction 3D interactive walkthroughs[J]. In Proceedings of ACM SIGGRAPH 2001, pp 443-450.August 2001
[29] D. Aliaga, T. Funkhouser, D. Yanovsky, I. Carlbom, Sea of Images[J], Proceedings of IEEE Visualization. pp. 331-338.2001
[30] Hiroshi Ishiguro,Kim C,Ng,Richard Capella,Mohan M.Trivedi. Omnidirectional image-based modeling:three approaches to approximated plenoptic representations[J].Machine Vision and Applications.2003
[31] Jose Gaspar, Etienne Grossmann,Jose Sntos-Victor. Interactive Reconstruction from an Omnidirectional Image[J]. 9th International Symposium on Intelligent Robotic Systems,2001
[32] Petr Doubek, Tomas Svoboda. Reliable 3D reconstruction from a few catadioptric images[J]. IEEE workshop on Omnidirectional vision .pp 71-78.2002
[33] Petri Doubek,Tomas Svoboda. What space can be reconstructed from multiple catadioptric images[J]. In Horst Wildenauer and Walter Kropatsch, editors, Proceedings of the Computer Vision Winter Workshop. PRIP TU Vienna, pp 198-207. February 2002
[34] Roland Bunschoten,Ben krose. Robust Scene Reconstruction from and Omnidirectional Vision System[J].IEEE transactions on robotics and automation.2002
[35]陈焱,杨东勇,沈正伟.全方位图像展开成全景和透视图的实现方法[J].计算机工程.2007.1
[36]Glullo Sandlni,Jose Santos Victor,Tomas Pajdla,Fablo Berton.OMNIVIEWS:Direct Omnidirectional Imaging Based on a Retina-like Sensor[J].IEEE sensors,2002.
[37]曾吉勇,苏显渝.水平场景无畸变的折反射全景成像系统[J].光学学报.2003.5
[38]张健,段滔,刘爱东.全景图像几何畸变校正算法研究及其软件实现[J].光学与光电技术,2003.6
[39]李忠新,刘春庆,王执铨.一种改进的柱面全景视图生成方法[J].小型微型计算机系统.2005.7
[40]凌云峰,朱齐丹,吴自新,张智.全景视觉图像柱面理论展开算法实现及其改进[J].应用科技,2006.9
[41]Sheng-Wen Jeng,Wdn-Hsiang tsai.Precise Image Unwarping of Omnidirectional Cameras with Hyperbolic-Shaped Mirrors[J].CVGIP 2000
[42]尤玉虎,周孝宽.数字图像最佳插值算法研究[J].中国空间科学技术.2005.6
[43]陈良,高成敏.快速离散化双线性插值算法[J].计算机工程与设计.2007.8
[44]Neil A.Dodgson.Quadratic Interpolation for Image Resampling[C].IEEE Transactions on Image Processing.1997.9
[45]Ahmed M.Eltawil,Eugene Grayver,Alireza Tarighat,Jean Francois Frigon and Ali Azam Abbasfar.Implementation of a Digital Timing Recovery Circuit for CDMA Applications[C].IEEE 2005 CUSTOM INTEGRATED CIRCUITS CONFERENCE
[46]Microsoft MSDN.http://msdn.microsoft.com/library/default.asp?url=/library /en-us/gdicpp/GDIPlus/GDIPlus.asp.2006
[47]Microsoft MSDN.http://msdn2.microsoft.com/en-us/library/ms87987 5.aspx
[48]Microsoft Corp.Microsoft DirectX 9.0 SDK Update.December 2004
[49]Mark D.Pesce.Programming Microsoft DirectShow for digital video and television[M],Microsoft Press.2003
[50]陆其明,DirectShow实务精选[M].北京:科学出版社,2004.7
[51]管国辰,一种模块化的流媒体系统开发框架[J].计算机工程.2007.2
[52]Vstone Corp.http://www.vstone.co.jp.2006
[53]Remote Reality.http://www.remotereality.com/vtprod/.2006