摘要
创建一个栩栩如生的虚拟世界是计算机图形学和虚拟现实永恒的追求目标之一。近年来,基于增强现实的交互技术逐渐受到关注,应用一些已有的增强现实技术能够突破现有图形学技术所面临的人机交互效率瓶颈,更好的发挥真实感渲染等图形学技术在人机交互中的作用。增强现实技术利用图形学、计算机视觉等手段,通过投影仪摄像机系统对用户所处的真实环境进行补充。增强现实技术通过将计算机产生的虚拟信息嵌入到真实的场景里,使用户能够切身地体会到一个真实和虚拟并存的世界。在这样的环境中,人能够以最直接的方式和计算机进行沟通,极大地改善人机交互效率。
本文的主要目标是研究基于模型的空间增强现实技术,包括三维物体网格模型的获取、三维建模的加速、模型的后期处理、虚实模型注册、多投影增强校准和重光照渲染。这些基础技术的研究对于推进投影增强现实的应用具有重要意义。本文的主要工作包括以下部分:
首先,本文完整地阐述了空间增强现实技术所涉及的基础问题,提出了空间增强现实流水线概念。通过采用基于模型的投影增强现实技术,本文完整地实施了空间增强现实,并通过设计桌面空间增强现实应用实例,说明了投影增强现实技术在改善人机交互接口上所具有的潜力。
其次,本文对基于结构光的建模技术进行了阐述,并对其进行了加速,通过使用硬件同步的摄像机和投影仪协同工作,提升了建模速度。
第三,本文对结构光建出的模型进行了后期处理,包括:模型简化和多角度的部分模型拼接。这样的后期处理可以保证模型在后期阶段不会带来问题。
第四,本文实现了虚拟和真实场景物体的注册技术,通过标定阶段获取的投影仪内外参可以计算投影参数从而保证虚拟真实场景的正确注册。
第五,本文提出了一种重叠区域的羽化算法来解决多投影环境下空间增强现实的光度不一致性问题。通过羽化衰减重叠区域的投影像素值取得了较好结果。
最后,本文使用了基于球面谐波的重光照技术对现实物体进行增强,并设计实验展示了重光照在空间增强现实中的实际效果。通过以上的这些部分研究,本文充分展示和说明了空间增强现实在改善人机交互领域所具有的巨大潜力。
Creating vivid virtual world has been the eternal destination for computer graphics and virtual reality. Recently, Augment reality based interaction techniques is becoming more and more popular, using such techniques can break the existing bottlenecks within HCI and better develop the potential of rendering techniques in HCI. By utilizing pro-cam systems to augment real scenes, users of such systems can experience a co-existing real and virtual world. In such applications human can directly act to the system as if they are acting to the real world, thus, greatly improve the efficiency of interaction.
The main object of this work is to study model based projection augment reality techniques, including real object mesh modeling, 3d model acquire acceleration, model post processing, virtue and real object registration, multi-projection compensation and relighting. The study of these basic techniques is very important for the application of projection augment reality techniques. The main contribution of this work includes the following parts:
First, all related technique is considered and explained in detail in this work, and the concept of spatial augment reality pipeline is brought. By using the model based projective augment reality techniques, table top spatial augment reality applications are implemented to demonstrate the potential it has in improving the existing human computer interface.
Second, structure light based modeling method is improved by using hardware to synchronize the projector with the camera, modeling efficiency is improved.
Third, post process of the model is also considered, after mesh decimation and part model alignment, model is adapted to work well in the following phases.
Fourth, virtue and real scene registration is implemented by using the parameters retrieved from the calibration. Correct rendering parameters is calculated, thus, making the real and virtue scene correctly aligned.
Fifth, a pixel feathering technique is brought to handle the brightness inconsistency in the overlap area in the multi-projector environment.
Last, spherical harmonic based relighting technique is implemented to show its applicability in spatial augment reality, experiments are designed to demonstrate its actual effect in the real environment. By all these sections, our work is dedicated to show and demonstrate the potential which spatial augment reality has in improving the existing human computer interface.
引文
[1] Raskar, R., Welch, G. Cutts, M., Lake, A., Stesin, L., Fuchs, H.:―The Office of the Future:A Unified Approach to Image Based Modeling and Spatially Immersive Displays‖. In:Proc. of SIGGRAPH’98 (1998)
[2] Underkoffler, J. and Ullmer, B. and Ishii, H. Emancipated pixels: real-world graphics in the luminous room. In: Proceedings of the 26th annual conference on Computer graphics and interactive techniques page385-392 1999
[3] Pinhanez, C. The everywhere displays projector: A device to create ubiquitous graphical interfaces. Springer Ubicomp 2001: Ubiquitous Computing
[4] Raskar, R. and van Baar, J. and Beardsley, P. and Willwacher, T. and Rao, S. and Forlines, C. iLamp: geometrically aware and self-configuring projectors. ACM SIGGRAPH 2005 Courses
[5] Raskar, R. and Welch, G. and Low, K.L. and Bandyopadhyay, D. Shader Lamp: Animating real objects with image-based illumination .Springer Verlag Wien (2001)
[6] Bimber, O. and Raskar, R. Spatial augmented reality: Merging real and virtual worlds .AK Peters Ltd 2005
[7] Yoshida, T. and Horii, C. and Sato, K. A virtual color reconstruction system for real heritage with light projection . Proceedings of VSMM 2003
[8] Cruz-Neira, C. and Sandin, D.J. and DeFanti, T.A. Surround-screen projection-based virtual reality: the design and implementation of the CAVE. Proceedings of the 20th annual conference on Computer graphics and interactive techniques 1993
[9] Ronald T. Azuma―A Survey of Augmented Reality‖, In Presence: Teleoperators and Virtual Environments 6, 4 (August 1997), 355-385.
[10]于晓洋,张加海,董秋玲.结构光三维测量系统的建模与标定.传感器世界,2007年13卷11期:15-18
[11] Zhang, S. and Huang. High-resolution, real-time three-dimensional shape measurement. Optical Engineering 2006
[12]马颂德,张正友.计算机视觉—计算理论与算法基础[M].北京:科学出版社,1998:1
[13]周昆等,一种新的基于边折叠的网格简化算法,第七届多媒体技术学会论文集[C]:84-90
[14] Garland M. et al Surface simplification using quadric error metrics[J] ACM SIGGRAPH, Los Angeles, 1997:209-216
[15] X. Li and I. Guskov, Multi-scale Features for Approximate Alignment of Point-based Surfaces[J], in Proc. Eurographics Symp. on Geom.Proc., 2005.
[16] J. Koenderink and A. van Doorn, Surface Shape and Curvature Scales[J], Img. and Vis. Comp., vol. 10:557–565, 1992.
[17] H.T. Ho and D. Gibbins, Multi-scale Feature Extraction for 3D Surface Registration using Local Shape Variation[J], IVCNZ 2008
[18] A. Mian, M. Bennamoun, and R. Owens, A Novel Representation and Feature Matching Algorithm for Automatic Pairwise Registration of Range Images[J], IJCV, vol. 66:19–40, 2006.
[19] D. Bandy opadhyay, R. Raskar, and H. Fuchs. Dynamic shader Lamp:Painting on real objects. In The Second IEEE and ACM International Symposium on Augmented Reality (ISAR01). Citeseer, 2001.
[20] P. Debevec. Rendering synthetic objects into real scenes: Bridging tra-ditional and image-based graphics with global illumination and high dy-namic range photography. In ACM SIGGRAPH 2008 classes, page 32.ACM, 2008.
[21] P. Debevec, T. Hawkins, C. Tchou, H. Duiker, W. Sarokin, and M. Sagar. Acquiring the re?ectance ?eld of a human face. In Proceedings of the 27th annual conference on Computer graphics and interactive tech-niques, pages 145–156. ACMPress/Addison-Wesley Publishing Co. New York, NY, USA, 2000.
[22] P. Debevec and J. Malik. Recovering high dynamic range radiance maps from photographs. In ACM SIGGRAPH 2008 classes, page 31. ACM, 2008.
[23] Faugeras. Three-Dimensional Computer Vision A Geometric View-point. MIT press, 2006
[24] M. Fiala. Artag revision 1, a ?ducial marker system using digital tech-niques. National Research Council Publication, 47419.
[25] M. Li. Correspondence Analysis Between The Image Formation Pipelines of Graphics and Vision. In Spanish Symposium on Pattern Recognition and Image Analysis, pages 187–192.
[26] P. Lincoln, G.Welch, A. Nashel, A. Ilie, et al. Animatronic Shader Lamp Avatars. In Proceedings of the 2009 8th IEEE International Symposium on Mixed and Augmented Reality, pages 27–33. IEEE Computer Society, 2009.
[27] V. Masselus, P. Peers, P. Dutr′e, and Y. Willems. Relighting with 4D incident light ?elds. In ACM SIGGRAPH 2003 Papers, page 620. ACM, 2003.
[28] R. Raskar, G. Welch, and H. Fuchs. Spatially augmented reality. In First IEEE Workshop on Augmented Reality (IWAR98), pages 11–20. Citeseer,1998.
[29] B. C. Yu Sheng, Theodore C. Yapo. Global Illumination Compensation for Spatially Augmented Reality. In Computer Graphics Forum, Eurographics 2010. Eurographics, 2010.
[30] Z. Zhang et al. A ?exible new technique for camera calibration. IEEE Transactions on pattern analysis and machine intelligence, 22(11):1330–1334, 2000.
[31] Ronald T. Azuma―A Survey of Augmented Reality‖, In Presence: Teleoperators and Virtual Environments 6, 4 (August 1997), 355-385.
[32] Zhang Zhengyou. A Flexible Camera Calibration by Viewing a Plane from Unknown Orientations[A]. ICCV99[C]. 1999.
[33] I. Sato. Appearance Sampling of Real Objects for Variable Illumination. In International Journal of Computer Vision, VOL 75, Number 1, p.29-48, 2007.
[34] R. Green. Spherical Harmonic Lighting: The Gritty Details. Available at http://www.paulsprojects. net/opengl/sh/sh.html, 2003.
[35] Y. Yu, P. Debevec, J. Malik, and T. Hawkins. Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs. In SIGGRAPH 99, p.215-224, 1999.
[36]毕田力,杨旭波,肖双九,基于谐波图片的重光照技术,上海交通大学硕士学位论文2010
[37]王宇超,杨旭波,肖双九,增强现实系统中的嵌入结构光技术研究,上海交通大学硕士学位论文2009
[38]毛松亮,杨旭波,肖双九,结构光建模的模型简化与拼接,上海交通大学学士学位论文2009