增强现实中多视图几何问题的研究
|
摘要
计算机视觉中的多视图几何技术构建了一套完整、成熟的从不同视点观察物体表面的理论。该理论囊括了许多几何知识，尤其是多幅视图之间存在的复杂而又美妙的几何关系已成为计算机理解客观世界的重要途径之一。本论文针对该技术在增强现实中的需求开展研究工作，取得的研究成果包括：
(1)提出了基于数据不确定性的约束求解模型：数据是客观实体信息的载体，由于采样的不精确性，使得它包含了不确定性。在大多数视觉几何约束求解方法中，都忽略或简化了它对精度的影响。本文从数据不确定性出发，分析出图像不确定性的因素，提出了一个基于数据不确定性的约束求解模型(DUCRM)。该模型推导出4种不同层次的二维数据不确定性，并定义了图像不确定性的表达及其几何含义；通过迭代求解时不确定性的非线性扩散结果，DUCRM对观测向量进行了逐步求精，并提高了求解精度。
(2)提出了改进的平面单应约束与对极几何约束的求解算法：针对低分辨率图像的兴趣点检测难、匹配精度低的问题，本文采用基于Harris角点检测与高阶曲线内部约束的混合算子来提高检测与匹配的精度，并结合DUCRM模型提高平面单应约束的求解精度。对极几何约束的线性求解方法对噪声敏感、精度较低；而非线性方法虽然精度高，但计算量大；另外，通用DUCRM模型求解时的奇异矩阵容易导致数值不稳定。针对这些问题，本文提出了一种常数项消除的方法避免了奇异矩阵，该方法根据观测向量的权值中心推导出待求参数矢量的最后一个元素与前8个元素的关系，从而分离出最后一个元素。实验结果表明，结合扩展DUCRM模型，该方法能较大地提高求解精度。
(3)提出了一种视频增强现实中虚实物体遮挡处理的加速算法：虚实物体之间正确、快速、鲁棒的遮挡处理一直是增强现实的难点之一。基于深度图求解方法的关键是如何快速地求出真实物体的深度，本文针对普通PC机提出了一种深度近似的快速方法：先用全局与局部高斯核对前景物体进行提取；接着用提取的物体轮廓线上的点进行立体匹配，立体匹配采用对极几何约束缩小搜索范围；再用三角法将匹配结果反投到空间中，并求出深度均值作为前景物体的深度值；最后根据深度图以及物体的重叠区域进行遮挡处理。该方法具有速度快、精度高的优点。
(4)提出了一种基于反光球的快速真实光照检测与阴影绘制方法：针对双向反射分布函数、基于图像的重光照等方法恢复真实光源计算量大，且必须已知部分场景几何的问题，本文提出一种基于反光球的快速方法：假设真实光源离散地分布在半空间中，根据光源在反光球上的分布情况，可以快速地估计出真实光源的位置和方向。本文将该方法应用于开发的视频增强现实原型系统中，实现了虚拟物体的快速阴影绘制。
本文最后开发了一个增强型海底世界交互系统。它是一个可以容纳多个体验者共同参与，并能与各种虚拟海洋生物进行交互的系统。它综合实现了上面提出的几个算法，并实现了主体人与虚拟化身之间的和谐交互。该原型系统的开发探索了类似系统在科技馆中展出的可能性，具有潜在的商业应用价值。
Multiple View Geometry in Computer Vision has build an all-around and sophisticmapping model between shapes from projective objects and camera's parameters. Thismodel is composed of many geometry knowledge, particularly the complex and grace-ful geometric relationship among several views. It has also become one of the mostimportant fundamental theories of understanding real world for computers. This dis-sertation studies the application of Multiple View Geometry in Augmented Reality, andmain contributions are:
(1) A new vision parameter estimation method using Data Uncertainty-basedConstrain Resolver Model (DUCRM): Data describes the characteristics of real objects.It also carries the uncertainty which is not wanted. In most of current estimation meth-ods, data uncertainty is overlooked and simplified. The accuracy of estimated param-eters thus is affected. This dissertation defines the analytic representation of data un-certainty on image and provides the algebraic equations for uncertainty propagation innonlinear functions, which are all based on data uncertainty theory. It provides a four-layer approach and a revising scheme in minimization iterative process. The accuracyof estimated parameters are thus improved.
(2) Improved planar homography estimation algorithm using mixed operatorand improved epipolar geometry estimation algorithm based on constant elimina-tion: Calculating planar homography between two low resolution images is always adifficult problem because of larger errors in comer detection and matching. We pro-vides a mixed detection algorithm using Harris comer and curve functions to achievebetter results. And with DUCRM, we get a better result compared with other methods.In addition, Epipolar Geometry is also an important vision parameter for perspectiveprojective applications. Traditional linear methods is simple but subject to noise per-turbance. And nonlinear methods give better result by a global searching, which iscomputing expensive. Experiment shows that simply using DUCRM leads to singularproblems which cause numerical calculation unstable. We provide an approach by elim-inating the constant column in measurement function. Results show that the singularmatrix is avoided and computing speed is accelerated.
(3) An accelerated algorithm of occlusion handling between virtual and real ob-jects: Correct, fast and robust occlusion handling between virtual and real objects isalways a critical problem in interactive AR. How to accelerate the computing speed is the key of using depth map based methods. We provides an approximation methodwhich can be implemented on PCs. First, a global and local Gaussian kernel function isused to extract foreground objects. And then, stereo matching is conducted only usingpoints on objects' contour. This process is also accelerated by Epipolar Geometry. Fore-ground object's depth is thus estimated using triangle method and occlusion is detectedby the overlapped region. Compared with other methods, ours is fast and gives betterresults. However, since one object has only one depth value, this method can not beapplied to deformable objects.
(4) A fast shadow rendering algorithm in dynamic scene based on reflective lightball: Although BRDF and Image-based Relighting methods have good models to re-cover light sources, they can not done in real time as for a dynamic scene. We providesa discrete light sample method to approximate the light source in real scene, which us-ing a light reflection ball. This method is applied in a Video-based Augmented Reality(VAR) system simulated by PCs without GPU acceleration.
Based on the estimation model and algorithms described above, we also developedan interactive Video-based Augmented Reality prototype. It is a mixed environmentwhich provides tools to enable observers walk through and interact with virtual seaanimals. In this prototype, algorithms are implemented and tested. And interactivescheme between observers and virtual avatars is also designed and implemented. Thisprototype helps us explore the feasibility of creating and exhibiting such systems inScience Centers.
(1)提出了基于数据不确定性的约束求解模型：数据是客观实体信息的载体，由于采样的不精确性，使得它包含了不确定性。在大多数视觉几何约束求解方法中，都忽略或简化了它对精度的影响。本文从数据不确定性出发，分析出图像不确定性的因素，提出了一个基于数据不确定性的约束求解模型(DUCRM)。该模型推导出4种不同层次的二维数据不确定性，并定义了图像不确定性的表达及其几何含义；通过迭代求解时不确定性的非线性扩散结果，DUCRM对观测向量进行了逐步求精，并提高了求解精度。
(2)提出了改进的平面单应约束与对极几何约束的求解算法：针对低分辨率图像的兴趣点检测难、匹配精度低的问题，本文采用基于Harris角点检测与高阶曲线内部约束的混合算子来提高检测与匹配的精度，并结合DUCRM模型提高平面单应约束的求解精度。对极几何约束的线性求解方法对噪声敏感、精度较低；而非线性方法虽然精度高，但计算量大；另外，通用DUCRM模型求解时的奇异矩阵容易导致数值不稳定。针对这些问题，本文提出了一种常数项消除的方法避免了奇异矩阵，该方法根据观测向量的权值中心推导出待求参数矢量的最后一个元素与前8个元素的关系，从而分离出最后一个元素。实验结果表明，结合扩展DUCRM模型，该方法能较大地提高求解精度。
(3)提出了一种视频增强现实中虚实物体遮挡处理的加速算法：虚实物体之间正确、快速、鲁棒的遮挡处理一直是增强现实的难点之一。基于深度图求解方法的关键是如何快速地求出真实物体的深度，本文针对普通PC机提出了一种深度近似的快速方法：先用全局与局部高斯核对前景物体进行提取；接着用提取的物体轮廓线上的点进行立体匹配，立体匹配采用对极几何约束缩小搜索范围；再用三角法将匹配结果反投到空间中，并求出深度均值作为前景物体的深度值；最后根据深度图以及物体的重叠区域进行遮挡处理。该方法具有速度快、精度高的优点。
(4)提出了一种基于反光球的快速真实光照检测与阴影绘制方法：针对双向反射分布函数、基于图像的重光照等方法恢复真实光源计算量大，且必须已知部分场景几何的问题，本文提出一种基于反光球的快速方法：假设真实光源离散地分布在半空间中，根据光源在反光球上的分布情况，可以快速地估计出真实光源的位置和方向。本文将该方法应用于开发的视频增强现实原型系统中，实现了虚拟物体的快速阴影绘制。
本文最后开发了一个增强型海底世界交互系统。它是一个可以容纳多个体验者共同参与，并能与各种虚拟海洋生物进行交互的系统。它综合实现了上面提出的几个算法，并实现了主体人与虚拟化身之间的和谐交互。该原型系统的开发探索了类似系统在科技馆中展出的可能性，具有潜在的商业应用价值。
Multiple View Geometry in Computer Vision has build an all-around and sophisticmapping model between shapes from projective objects and camera's parameters. Thismodel is composed of many geometry knowledge, particularly the complex and grace-ful geometric relationship among several views. It has also become one of the mostimportant fundamental theories of understanding real world for computers. This dis-sertation studies the application of Multiple View Geometry in Augmented Reality, andmain contributions are:
(1) A new vision parameter estimation method using Data Uncertainty-basedConstrain Resolver Model (DUCRM): Data describes the characteristics of real objects.It also carries the uncertainty which is not wanted. In most of current estimation meth-ods, data uncertainty is overlooked and simplified. The accuracy of estimated param-eters thus is affected. This dissertation defines the analytic representation of data un-certainty on image and provides the algebraic equations for uncertainty propagation innonlinear functions, which are all based on data uncertainty theory. It provides a four-layer approach and a revising scheme in minimization iterative process. The accuracyof estimated parameters are thus improved.
(2) Improved planar homography estimation algorithm using mixed operatorand improved epipolar geometry estimation algorithm based on constant elimina-tion: Calculating planar homography between two low resolution images is always adifficult problem because of larger errors in comer detection and matching. We pro-vides a mixed detection algorithm using Harris comer and curve functions to achievebetter results. And with DUCRM, we get a better result compared with other methods.In addition, Epipolar Geometry is also an important vision parameter for perspectiveprojective applications. Traditional linear methods is simple but subject to noise per-turbance. And nonlinear methods give better result by a global searching, which iscomputing expensive. Experiment shows that simply using DUCRM leads to singularproblems which cause numerical calculation unstable. We provide an approach by elim-inating the constant column in measurement function. Results show that the singularmatrix is avoided and computing speed is accelerated.
(3) An accelerated algorithm of occlusion handling between virtual and real ob-jects: Correct, fast and robust occlusion handling between virtual and real objects isalways a critical problem in interactive AR. How to accelerate the computing speed is the key of using depth map based methods. We provides an approximation methodwhich can be implemented on PCs. First, a global and local Gaussian kernel function isused to extract foreground objects. And then, stereo matching is conducted only usingpoints on objects' contour. This process is also accelerated by Epipolar Geometry. Fore-ground object's depth is thus estimated using triangle method and occlusion is detectedby the overlapped region. Compared with other methods, ours is fast and gives betterresults. However, since one object has only one depth value, this method can not beapplied to deformable objects.
(4) A fast shadow rendering algorithm in dynamic scene based on reflective lightball: Although BRDF and Image-based Relighting methods have good models to re-cover light sources, they can not done in real time as for a dynamic scene. We providesa discrete light sample method to approximate the light source in real scene, which us-ing a light reflection ball. This method is applied in a Video-based Augmented Reality(VAR) system simulated by PCs without GPU acceleration.
Based on the estimation model and algorithms described above, we also developedan interactive Video-based Augmented Reality prototype. It is a mixed environmentwhich provides tools to enable observers walk through and interact with virtual seaanimals. In this prototype, algorithms are implemented and tested. And interactivescheme between observers and virtual avatars is also designed and implemented. Thisprototype helps us explore the feasibility of creating and exhibiting such systems inScience Centers.
引文
[1] M. Bajura, F. Henry, and R. Ohbuchi. Merging virtual reality with the real world: Seenig ultrasound imagery within the patient. Computer Graphics, 26(2):203-210, 1992.
[2] 王涌天，郑伟，刘越等．基于增强现实技术的圆明园现场数字重建．科技导报，24(3)：36-40，2006．
[3] 石教英．虚拟现实基础及实用算法．科学出版社，北京，2002．
[4] G. Burdea and P. Coiffet. Virtual Reality Technology. John Wiley and Sons Press, New Jersey, 1994.
[5] R. Azuma, Y. Baillot, R. Behringer, Steven. Feiner, S. Julier, and B. MacIntyre. Recent advanances in augmented reality. IEEE Computer Graphics and Applications, 21(6):34-47, 2001.
[6] P. Milgram and F. Kishino. A taxonomy of mixed reality visual displays. IEICE Transactions on Information Systems, E77-D(12):1321-1329, 1994.
[7] O. Bimber and R. Raskar. Spatial Augmented Reality: Merging Real and Virtual Worlds. A K Peters Ltd. Press, Wellesley, 2005.
[8] URL www.ai.mit.edu/projects/vision-surgery.
[9] URL www.cs.columbia.edu/graphics.
[10] URL www.mic.atr.co.jp.
[11] URL www.arvika.de.
[12] URL www.mixedrealitylab.org.
[13] H. Kato and M. Billinghurst. Marker tracking and hmd calibration for a videobased augmented reality conferencing system. In Proc of the 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR' 99), pages 85-94, Washington DC,USA, 1999.
[14] URL www.hitl.washington.edu.
[15] M. Billinghurst. Research directions in augmented reality. In Proc. of 16th International Conference on Artificial Reality and Telexistence (ICAT'06), Hangzhou, Zhejiang, China, 2006.
[16] W. Lee and J. Park. Augmented foam: A tangible augmented reality for product design. In Proc. of 4th IEEE/ACM International Symposium on Mixed and Augmented Reality (ISMAR'05), pages 106-109, Vienna,Austria, 2005.
[17] N.T. Thalman and G. Papagiannakis. Virtual worlds and augmented reality in cultural heritage applications. In Proc. of International Workshop on Recording, Modeling and Visualization of Cultural Heritage, Ascona, Switzerland, 2005.
[18] J.E. Swan and J.L. Gabbard. Survey of user-based experimentation in augmented reality. In Proc. of 1st International Conference on Virtual Reality (ICVR'05), Las Vegas,USA, 2005.
[19] M.A. Livingston. Evaluating human factors in augmented reality systems. Computer Graphics and Applications, 25(6):6-9, 2005.
[20] C.T.Q. Tran and H.D.N.A.M. Ta. Magic land: live 3D human capture mixed reality interactive system. In Proc. of IEEE International Workshop on Human-Computer Interaction (HCI'05), pages 1142-1143, Beijing, China, 2005.
[21] P. Supan, I. Stuppacher, and M. Haller. Image-based shadowing in real-time augmented reality. In Proc. of 16th International Conference on Artificial Reality and Telexistence (ICAT'06), Hangzhou, Zhejiang, China, 2006.
[22] P.A. Fortin and P. Hebert. Handling occlusions in real-time augmented reality: Dealing with movable real and virtual objects. In Proc. of 3rd Canadian Conference on Computer and Robot Vision (CVR'06), page 54, Quebec,Canada, 2006.
[23] A. Henryson, M. Billinghurst, and M. Ollila. Face to face collaborative ar on mobile phones. In Proc. of 4th IEEE/ACM Internatinoal Symposium on Mixed and Augmented Reality (ISMAR'05), pages 80-89, Vienna, Austria, 2005.
[24] 哈涌刚，周雅．用于增强现实的头盔显示器的设计．光学技术，26(4)：350—353，2000．
[25] 周雅，闫达远．一种增强现实系统的三维注册方法．中国图象图形学报，05A(5)：430-433，2000．
[26] 陈靖，王涌天．利用光流预测的增强现实系统注册方法．系统仿真学报，14(6)：739—742，2002．
[27] 周雅，晏磊，赵虎．增强现实系统光照模型建立研究．中国图象图形学报：A辑，9(8)：968-971，2004．
[28] 倪剑，闫达远，周雅等．增强现实系统中的深度检测技术研究．计算机应用，26(1)： 132-137，2006．
[29] 朱淼良，姚远，蒋云良．增强现实综述．中国图象图形学报，9(7)：767-775，2004．
[30] 孙敏，陈秀万，张飞舟等．增强现实地理信息系统．北京大学学报：自然科学版，40(6)：906-913，2004．
[31] 常勇，何宗宜．户外增强现实技术及其在地下管网3维可视化中的应用．测绘通报，(11)：54-57，2005．
[32] 任波，李利军，曹伟明等．基于增强现实的交互式城市设计研究．华中科技大学学报：城市科学版，23(2)：32-34，2006．
[33] R. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge, 2003.
[34] 韦穗，杨尚骏，章权兵等．计算机视觉中的多视图几何．安徽大学出版社，合肥，2002．
[35] B. Johansson. Computer Vision Using Rich Features-Geometry and Systems. PhD thesis, LUND University, Swedish, 2002.
[36] R. Azuma. A survey of augmented reality. Presence:Teleoperators and Virtual Environments, 6(4):355-385, 1997.
[37] D. Wagner and D. Schmalstieg. First steps towards handheld augmented reality. In Proc. of 7th Internatinal Conference on Wearable Computers (ISWC'03), pages 127-137, Los Alamitos,Canada, 2003.
[38] M. Moehring, C. Lessig, and O. Bimber. Video see-through ar on consumer cellphones. In Proc. of 3th IEEE/ACM InternatinoaI Symposium on Mixed and Augmented Reality (ISMAR'04), pages 252-253, Los Alamitos,Canada, 2004.
[39] G.D. Stetten, V. Chib, D. Hildebrand, and J. Bursee. Real time tomographic reflection: Phantoms for calibration and biopsy. In Proc. of 1th IEEE/ACM Internatinoal Symposium on Mixed and Augmented Reality (ISMAR'01), pages 11-19, Los Alamitos,Canada, 2001.
[40] K. Hayashi, H.K. Kato, and S. Nishida. Occlusion detection of real objects using contour based stereo matching. In Proc. of ACM International Conference on Augmented Tele-existence, pages 180-186, Christchurch, New Zealand, 2005.
[41] T. Starner, B. Leibe, B. Singletary, K. Lyons, M. Gandy, and J. Pair. Towards augmented reality gaming. In Proc. of IMAGINA'00, Monaco, 2000.
[42] K. Low, G. Welch, A. Lastra, and H. Fuchs. Life-sized projector-based dioramas. In Proc. of ACM Symposium on Virtual Reality Software and Technology (VRST'01), pages 161-168, Banff, Alberta,Canada, 2001.
[43] T. Okuma, K. Kiyokawa, H. Takemura, and N. Yokoya. An augmented reality system using a real-time vision based registration. In Proc. of Fourteenth International Conference on Pattern Recognition (ICPR'98), volume 2, pages 1226-1229, Brisbane, Australia, 1998.
[44] 章毓晋．图像工程下册：图像理解与计算机视觉．清华大学出版社，北京，1999．
[45] P. Gurdjos, P. Sturm, and Y.H. Wu. Euclidean structure from n＞=2 parallel circles: Theory and algorithms. In Proc. of 9th European Conference on Computer Vision (ECCV'06), pages 238-252, Graz, Austria, 2006.
[46] 马颂德，张正友．计算机视觉——计算理论与算法基础．科学出版社，北京，2003．
[47] R. Behringer. Registration for outdoor augmented reality applications using computer vision techniques and hybrid sensors. In Proc. oflEEE VR'99, pages 244-251, Houston, TX, USA, 1999.
[48] Y. Suya, U. Neumann, and R. Azuma. Hybrid inertial and vision tracking for augmented reality registration. In Proc. of IEEE VR'01, pages 260-267, Yokohama, Japan, 2001.
[49] 郑文庭．基于几何和图象混合的虚拟场景实时绘制技术研究．浙江大学博士论文，1999．
[50] J.N. Wang and M.M. Oliverira. Improved scene reconstruction from range images. Computer Graphics Forum, 21(3):521-530, 2002.
[51] I. Sato, Y. Sato, and K. Ikeuchi. Illumination distribution from shadows. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR'99), pages 306-312, Kerkyra, Japan, 1999.
[52] 孙其民，吴恩华．从单幅高动态范围图像恢复环境中物体的材质．软件学报，13(9)：1852—1857，2002．
[53] M. Kanbara and N. Yokoya. Real-time estimation of light source environment for photorealistic augmented reality. In Proc. of 17th Intern. Conference on Pattern Recognition (ICPR'04), pages 911-914, Cambridge, UK, 2004.
[54] S. Diverdi and T. Hollerer. Combing dynamic physical and virtual illumination in augmented reality. Technical report, Department of Computer Science, University of California, 2004.
[55] 史文中．空间数据与空间分析不确定性原理．科学出版社，北京，2005．
[56] M. Bogdan and M. Peter. Estimation of nonlinear errors-in-variables models for computer vision applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(10):1537-1552, 2006.
[57] 王新洲．非线性模型参数估计理论与应用．武汉大学出版社，武汉，2002．
[58] F. Bookstein. Fitting conic sections to scattered data. Computer Graphics and Image Processing, 9:56-71, 1979.
[59] A. Fitzgibbon, M. Pilu, and R. B. Fisher. Direct least square fitting of ellipses. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(5):476-480, 1999.
[60] G.H. Golub and C.F. Van Loan. An analysis of the total least squares. SIAM J Numer Anal, 17:883-893, 1980.
[61] P.D. Sampson. Fitting conic sections to very scattered data: An iterative refinement of the bookstein algorithm. Computer Visioin, Graphics, and Image Processing, 18:97-108, 1982.
[62] K. Kanatani. Renormalization for unbiased estimation. In Proc. of International Conference on Computer Vision (ICCV'93), pages 599-606, Berlin,Germany, 1993.
[63] W. Chojnacki, M.J. Brooks, D.H.A. Van, and D. Gawley. From fns to heiv: A link between two vision parameter estimation methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(2):264-268, 2004.
[64] A. Anubhav, C.V. Jawahar, and P.J. Narayanan. Survey of planar homography estimation techniques. Technical Report IIIT/TR/2005/12, Deemed University, 2005.
[65] H.C. Longuet. A computer algorithm for reconstructing a scene from two projections. Nature, 293:133-135, 1981.
[66] R. Hartley. In defense of the eightpoint algorihtm. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(6):580-593, 1997.
[67] M.J. Brooks, W. Chojnacki, D. Gawley, and A.V.D. Hengel. What value covariance information in estimating vision parameters? In Proc. of International Conference on Computer Vision (ICCV'01), pages 302-308, Vancouver, USA, 2001.
[68] B. Matei. Heteroscedastic Errors-in-Variables Models in Computer Vision. PhD thesis, Rutgers University, USA, 2001.
[69] 陈泽志,吴成柯．一种高精度估计的基础矩阵的线性算法．软件学报,13(4)：840-845,2002．
[70] B.N. Parlett. Acta Numerica. Cambridge University Press, Cambridge, 1995.
[71] E. Anderson, Z. Bai, C. Bischof, and S. Blackford. Lapack users' guide third edition, 1999.
[72] K. Kanatani. Statistical Optimization for Geometric Computation: Theory and Practice. Dover Publications Press, New York, 1996.
[73] F. Devernay and O. Faugeras. Straight lines have to be straight: automatic calibration and removal of distortion from scenes of structured environments. Machine Vision and Applications, 13(1):14-24,2001.
[74] Z. Zhang. A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330-1334,2000.
[75] CB. Lee. Radial undistortion and calibration on an image array. PhD thesis, Massachusetts Institute of Technology, 2000.
[76] H. Farid and A.C. Popescu. Blind removal of lens distortion, the Optical Society of America A, 18(9):2072-2078,2001.
[77] M.P. Kumar, C.V. Jawahar, and P.J. Narayanan. Discrete contours in multiple views: Approximation and recoginition. International Journal of Image and Vision Computing, 22(14):1229-1239,2004.
[78] C. Schmid and A. Zisserman. The geometry and matching of lines and curves over multiple views. International Journal of Computer Vision, 40(3): 199-233, 2000.
[79] R.Y. Tsai and T.S. Huang. The perspective view of three points. IEEE Transactions on Pattern Analysis and Machine Intelligence, (6):13-27,1984.
[80] J. Wang, T.S. Huang, and N. Ahuja. Motion and structure from two perspective views: algorithms, error analysis and error estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(5):451-476,1989.
[81] Z.Y Zhang. Determining the epipolar geometry and its uncertainty: a review. International Journal of Computer Vision, 27(2), 1998.
[82] M. Pollefeys, R. Koch, and V.L. Gool. Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters. International Journal of Computer Vision, 32(1):7-25, 1999.
[83] 马永壮，刘伟军．基于射影空间的视觉基础矩阵鲁棒估计．机器人，27(6)：545-550，2005．
[84] 胡明星，袁保宗，唐晓芳．基于遗传算法不同策略下的基础矩阵估计方法．计算机研究与发展，39(2)：1604-1613，2002．
[85] 张迁，刘政凯，庞彦伟等．基于susan算法的航空影像的自动配准．测绘学报，32(3)：245-250，2003．
[86] K. Kenichi, O. Naoya, and K. Yasushi. Optimal homography computation with a reliability measure. Journal of Machine Vision Applications, E83-D(7):1369-1374, 2000.
[87] G. Gordon, M. Billinghurst, M. Bell, J. Woodfill, B. Kowalik, A. Erendi, and J. Tilander. The use of dense stereo range data in augmented reality. In Proc. of IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'02), Los Alamitos, Canada.
[88] Y. Pang, M.L. Yuan, A.Y.C. Nee, S.K. Ong, and K.Y. Toumi. A markerless registration method for augmented reality based on affine properties. In Proc. of 7th ACM International Conference on Australasian User Interface (AUIC'06), pages 25-32, Hobart, Australia, 2006.
[89] K. Kiyokawa, Y. Kurata, and H. Ohno. An optical see-through display for mutual occlusion of real and virtual environments. In Proc. of IEEE and ACM International Symposium on Augmented Reality(ISAR'01), pages 217-223, New York,USA, 2001.
[90] S. Noda, Y. Ban, K. Sato, and K. Chilhara. An optical see-through mixed reality display with realtime rangefinder and an active pattern light source. Transactions of the Virtual Reality Society of Japan, 4(4):665-670, 1999.
[91] T. Naemura, T. Nitta, A. Mimura, and H. Harashima. Virtual shadows-enhanced interaction in mixed reality environments. In Proc. of IEEE VR'02, pages 293-294, Orlando, USA, 2002.
[92] O. Bimber, B. Frohlich, D. Schmalstieg, and L. M. Encarnagao. The virtual showcase. IEEE Computer Graphics and Application, 21(6):48-55, 2002.
[93] A. Fuhrmann, G. Hesina, F Faure, and M. Gervautz. Occulsion in collaborative augmented environments. Computers and Graphics, 23(6):809-819,1998.
[94] L. Vacchetti, V. Lepetit, and P. Fua. Combining edge and texture information for real-time accurate 3D camera tracking. In Proc, of IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'04), pages 48-56, Arlington, USA, 2004.
[95] Y. Ohta, Y. Sugaya, H. Igarashi, T. Ohtsuki, and K. Taguchi. Share-z: Client/server depth sensing for see-through head mounted displays. In Proc, of 2nd IEEE and ACM International Symposia on Mixed Reality (ISMR'01), pages 64-72, Tokyo, Japan, 2001.
[96] J.E. Swan, M.A. Livingston, H.S. Smallman, D. Brown, Y. Baillot, J.L. Gabbard, and D. Hix. A perceptual matching technique for depth judgments in optical, see-through augmented reality. In Proc, of IEEE VR'06, pages 19-26, Alexandria, Virginia, USA, 2006.
[97] H. Wang, K. Sengupta, P. Kumar, and Sharma.R. Occlusion handling in augmented reality using background-foreground segmentation and projective geometry. Presence: Teleoperators and Virtual Environments, 14(3):264-277,2005.
[98] P. Noriega, B. Bascle, and O. Bernier. Local kernel color histograms for background subtration. In Proc, of International Conference on Computer Vision Theory and Applications (VISAPP'06), pages 67-74, Setubal, Portugal, 2006.
[99] A.K. Jain, M.N. Murty, and P. J. Flynn. Data clustering: A review. ACM Computing Surveys, 31(3):264-323,1999.
[100] G. Rhodes. Fast,robust intersection of 3D line segments. Game Programming Gems 2, pages 191-204, 2001.
[101] URL http: //openvrml. org/index.
[102] M. Billinghurst, H. Kato, and I. Poupyrev. The magicbook: a transitional ar interface. Computers and Graphics, 25(5):745-753,2001.
[103] W. Fuller. Measurement Error Models. John Wiley and Sons Press, New Jersey, 1978.
[104] J.W. Brewer. Kronecker products and matrix calculus in system theory. IEEE Transactions on Circuits and Systems, 25(9):772-781,1978.
[2] 王涌天，郑伟，刘越等．基于增强现实技术的圆明园现场数字重建．科技导报，24(3)：36-40，2006．
[3] 石教英．虚拟现实基础及实用算法．科学出版社，北京，2002．
[4] G. Burdea and P. Coiffet. Virtual Reality Technology. John Wiley and Sons Press, New Jersey, 1994.
[5] R. Azuma, Y. Baillot, R. Behringer, Steven. Feiner, S. Julier, and B. MacIntyre. Recent advanances in augmented reality. IEEE Computer Graphics and Applications, 21(6):34-47, 2001.
[6] P. Milgram and F. Kishino. A taxonomy of mixed reality visual displays. IEICE Transactions on Information Systems, E77-D(12):1321-1329, 1994.
[7] O. Bimber and R. Raskar. Spatial Augmented Reality: Merging Real and Virtual Worlds. A K Peters Ltd. Press, Wellesley, 2005.
[8] URL www.ai.mit.edu/projects/vision-surgery.
[9] URL www.cs.columbia.edu/graphics.
[10] URL www.mic.atr.co.jp.
[11] URL www.arvika.de.
[12] URL www.mixedrealitylab.org.
[13] H. Kato and M. Billinghurst. Marker tracking and hmd calibration for a videobased augmented reality conferencing system. In Proc of the 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR' 99), pages 85-94, Washington DC,USA, 1999.
[14] URL www.hitl.washington.edu.
[15] M. Billinghurst. Research directions in augmented reality. In Proc. of 16th International Conference on Artificial Reality and Telexistence (ICAT'06), Hangzhou, Zhejiang, China, 2006.
[16] W. Lee and J. Park. Augmented foam: A tangible augmented reality for product design. In Proc. of 4th IEEE/ACM International Symposium on Mixed and Augmented Reality (ISMAR'05), pages 106-109, Vienna,Austria, 2005.
[17] N.T. Thalman and G. Papagiannakis. Virtual worlds and augmented reality in cultural heritage applications. In Proc. of International Workshop on Recording, Modeling and Visualization of Cultural Heritage, Ascona, Switzerland, 2005.
[18] J.E. Swan and J.L. Gabbard. Survey of user-based experimentation in augmented reality. In Proc. of 1st International Conference on Virtual Reality (ICVR'05), Las Vegas,USA, 2005.
[19] M.A. Livingston. Evaluating human factors in augmented reality systems. Computer Graphics and Applications, 25(6):6-9, 2005.
[20] C.T.Q. Tran and H.D.N.A.M. Ta. Magic land: live 3D human capture mixed reality interactive system. In Proc. of IEEE International Workshop on Human-Computer Interaction (HCI'05), pages 1142-1143, Beijing, China, 2005.
[21] P. Supan, I. Stuppacher, and M. Haller. Image-based shadowing in real-time augmented reality. In Proc. of 16th International Conference on Artificial Reality and Telexistence (ICAT'06), Hangzhou, Zhejiang, China, 2006.
[22] P.A. Fortin and P. Hebert. Handling occlusions in real-time augmented reality: Dealing with movable real and virtual objects. In Proc. of 3rd Canadian Conference on Computer and Robot Vision (CVR'06), page 54, Quebec,Canada, 2006.
[23] A. Henryson, M. Billinghurst, and M. Ollila. Face to face collaborative ar on mobile phones. In Proc. of 4th IEEE/ACM Internatinoal Symposium on Mixed and Augmented Reality (ISMAR'05), pages 80-89, Vienna, Austria, 2005.
[24] 哈涌刚，周雅．用于增强现实的头盔显示器的设计．光学技术，26(4)：350—353，2000．
[25] 周雅，闫达远．一种增强现实系统的三维注册方法．中国图象图形学报，05A(5)：430-433，2000．
[26] 陈靖，王涌天．利用光流预测的增强现实系统注册方法．系统仿真学报，14(6)：739—742，2002．
[27] 周雅，晏磊，赵虎．增强现实系统光照模型建立研究．中国图象图形学报：A辑，9(8)：968-971，2004．
[28] 倪剑，闫达远，周雅等．增强现实系统中的深度检测技术研究．计算机应用，26(1)： 132-137，2006．
[29] 朱淼良，姚远，蒋云良．增强现实综述．中国图象图形学报，9(7)：767-775，2004．
[30] 孙敏，陈秀万，张飞舟等．增强现实地理信息系统．北京大学学报：自然科学版，40(6)：906-913，2004．
[31] 常勇，何宗宜．户外增强现实技术及其在地下管网3维可视化中的应用．测绘通报，(11)：54-57，2005．
[32] 任波，李利军，曹伟明等．基于增强现实的交互式城市设计研究．华中科技大学学报：城市科学版，23(2)：32-34，2006．
[33] R. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge, 2003.
[34] 韦穗，杨尚骏，章权兵等．计算机视觉中的多视图几何．安徽大学出版社，合肥，2002．
[35] B. Johansson. Computer Vision Using Rich Features-Geometry and Systems. PhD thesis, LUND University, Swedish, 2002.
[36] R. Azuma. A survey of augmented reality. Presence:Teleoperators and Virtual Environments, 6(4):355-385, 1997.
[37] D. Wagner and D. Schmalstieg. First steps towards handheld augmented reality. In Proc. of 7th Internatinal Conference on Wearable Computers (ISWC'03), pages 127-137, Los Alamitos,Canada, 2003.
[38] M. Moehring, C. Lessig, and O. Bimber. Video see-through ar on consumer cellphones. In Proc. of 3th IEEE/ACM InternatinoaI Symposium on Mixed and Augmented Reality (ISMAR'04), pages 252-253, Los Alamitos,Canada, 2004.
[39] G.D. Stetten, V. Chib, D. Hildebrand, and J. Bursee. Real time tomographic reflection: Phantoms for calibration and biopsy. In Proc. of 1th IEEE/ACM Internatinoal Symposium on Mixed and Augmented Reality (ISMAR'01), pages 11-19, Los Alamitos,Canada, 2001.
[40] K. Hayashi, H.K. Kato, and S. Nishida. Occlusion detection of real objects using contour based stereo matching. In Proc. of ACM International Conference on Augmented Tele-existence, pages 180-186, Christchurch, New Zealand, 2005.
[41] T. Starner, B. Leibe, B. Singletary, K. Lyons, M. Gandy, and J. Pair. Towards augmented reality gaming. In Proc. of IMAGINA'00, Monaco, 2000.
[42] K. Low, G. Welch, A. Lastra, and H. Fuchs. Life-sized projector-based dioramas. In Proc. of ACM Symposium on Virtual Reality Software and Technology (VRST'01), pages 161-168, Banff, Alberta,Canada, 2001.
[43] T. Okuma, K. Kiyokawa, H. Takemura, and N. Yokoya. An augmented reality system using a real-time vision based registration. In Proc. of Fourteenth International Conference on Pattern Recognition (ICPR'98), volume 2, pages 1226-1229, Brisbane, Australia, 1998.
[44] 章毓晋．图像工程下册：图像理解与计算机视觉．清华大学出版社，北京，1999．
[45] P. Gurdjos, P. Sturm, and Y.H. Wu. Euclidean structure from n＞=2 parallel circles: Theory and algorithms. In Proc. of 9th European Conference on Computer Vision (ECCV'06), pages 238-252, Graz, Austria, 2006.
[46] 马颂德，张正友．计算机视觉——计算理论与算法基础．科学出版社，北京，2003．
[47] R. Behringer. Registration for outdoor augmented reality applications using computer vision techniques and hybrid sensors. In Proc. oflEEE VR'99, pages 244-251, Houston, TX, USA, 1999.
[48] Y. Suya, U. Neumann, and R. Azuma. Hybrid inertial and vision tracking for augmented reality registration. In Proc. of IEEE VR'01, pages 260-267, Yokohama, Japan, 2001.
[49] 郑文庭．基于几何和图象混合的虚拟场景实时绘制技术研究．浙江大学博士论文，1999．
[50] J.N. Wang and M.M. Oliverira. Improved scene reconstruction from range images. Computer Graphics Forum, 21(3):521-530, 2002.
[51] I. Sato, Y. Sato, and K. Ikeuchi. Illumination distribution from shadows. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR'99), pages 306-312, Kerkyra, Japan, 1999.
[52] 孙其民，吴恩华．从单幅高动态范围图像恢复环境中物体的材质．软件学报，13(9)：1852—1857，2002．
[53] M. Kanbara and N. Yokoya. Real-time estimation of light source environment for photorealistic augmented reality. In Proc. of 17th Intern. Conference on Pattern Recognition (ICPR'04), pages 911-914, Cambridge, UK, 2004.
[54] S. Diverdi and T. Hollerer. Combing dynamic physical and virtual illumination in augmented reality. Technical report, Department of Computer Science, University of California, 2004.
[55] 史文中．空间数据与空间分析不确定性原理．科学出版社，北京，2005．
[56] M. Bogdan and M. Peter. Estimation of nonlinear errors-in-variables models for computer vision applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(10):1537-1552, 2006.
[57] 王新洲．非线性模型参数估计理论与应用．武汉大学出版社，武汉，2002．
[58] F. Bookstein. Fitting conic sections to scattered data. Computer Graphics and Image Processing, 9:56-71, 1979.
[59] A. Fitzgibbon, M. Pilu, and R. B. Fisher. Direct least square fitting of ellipses. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(5):476-480, 1999.
[60] G.H. Golub and C.F. Van Loan. An analysis of the total least squares. SIAM J Numer Anal, 17:883-893, 1980.
[61] P.D. Sampson. Fitting conic sections to very scattered data: An iterative refinement of the bookstein algorithm. Computer Visioin, Graphics, and Image Processing, 18:97-108, 1982.
[62] K. Kanatani. Renormalization for unbiased estimation. In Proc. of International Conference on Computer Vision (ICCV'93), pages 599-606, Berlin,Germany, 1993.
[63] W. Chojnacki, M.J. Brooks, D.H.A. Van, and D. Gawley. From fns to heiv: A link between two vision parameter estimation methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(2):264-268, 2004.
[64] A. Anubhav, C.V. Jawahar, and P.J. Narayanan. Survey of planar homography estimation techniques. Technical Report IIIT/TR/2005/12, Deemed University, 2005.
[65] H.C. Longuet. A computer algorithm for reconstructing a scene from two projections. Nature, 293:133-135, 1981.
[66] R. Hartley. In defense of the eightpoint algorihtm. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(6):580-593, 1997.
[67] M.J. Brooks, W. Chojnacki, D. Gawley, and A.V.D. Hengel. What value covariance information in estimating vision parameters? In Proc. of International Conference on Computer Vision (ICCV'01), pages 302-308, Vancouver, USA, 2001.
[68] B. Matei. Heteroscedastic Errors-in-Variables Models in Computer Vision. PhD thesis, Rutgers University, USA, 2001.
[69] 陈泽志,吴成柯．一种高精度估计的基础矩阵的线性算法．软件学报,13(4)：840-845,2002．
[70] B.N. Parlett. Acta Numerica. Cambridge University Press, Cambridge, 1995.
[71] E. Anderson, Z. Bai, C. Bischof, and S. Blackford. Lapack users' guide third edition, 1999.
[72] K. Kanatani. Statistical Optimization for Geometric Computation: Theory and Practice. Dover Publications Press, New York, 1996.
[73] F. Devernay and O. Faugeras. Straight lines have to be straight: automatic calibration and removal of distortion from scenes of structured environments. Machine Vision and Applications, 13(1):14-24,2001.
[74] Z. Zhang. A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330-1334,2000.
[75] CB. Lee. Radial undistortion and calibration on an image array. PhD thesis, Massachusetts Institute of Technology, 2000.
[76] H. Farid and A.C. Popescu. Blind removal of lens distortion, the Optical Society of America A, 18(9):2072-2078,2001.
[77] M.P. Kumar, C.V. Jawahar, and P.J. Narayanan. Discrete contours in multiple views: Approximation and recoginition. International Journal of Image and Vision Computing, 22(14):1229-1239,2004.
[78] C. Schmid and A. Zisserman. The geometry and matching of lines and curves over multiple views. International Journal of Computer Vision, 40(3): 199-233, 2000.
[79] R.Y. Tsai and T.S. Huang. The perspective view of three points. IEEE Transactions on Pattern Analysis and Machine Intelligence, (6):13-27,1984.
[80] J. Wang, T.S. Huang, and N. Ahuja. Motion and structure from two perspective views: algorithms, error analysis and error estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(5):451-476,1989.
[81] Z.Y Zhang. Determining the epipolar geometry and its uncertainty: a review. International Journal of Computer Vision, 27(2), 1998.
[82] M. Pollefeys, R. Koch, and V.L. Gool. Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters. International Journal of Computer Vision, 32(1):7-25, 1999.
[83] 马永壮，刘伟军．基于射影空间的视觉基础矩阵鲁棒估计．机器人，27(6)：545-550，2005．
[84] 胡明星，袁保宗，唐晓芳．基于遗传算法不同策略下的基础矩阵估计方法．计算机研究与发展，39(2)：1604-1613，2002．
[85] 张迁，刘政凯，庞彦伟等．基于susan算法的航空影像的自动配准．测绘学报，32(3)：245-250，2003．
[86] K. Kenichi, O. Naoya, and K. Yasushi. Optimal homography computation with a reliability measure. Journal of Machine Vision Applications, E83-D(7):1369-1374, 2000.
[87] G. Gordon, M. Billinghurst, M. Bell, J. Woodfill, B. Kowalik, A. Erendi, and J. Tilander. The use of dense stereo range data in augmented reality. In Proc. of IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'02), Los Alamitos, Canada.
[88] Y. Pang, M.L. Yuan, A.Y.C. Nee, S.K. Ong, and K.Y. Toumi. A markerless registration method for augmented reality based on affine properties. In Proc. of 7th ACM International Conference on Australasian User Interface (AUIC'06), pages 25-32, Hobart, Australia, 2006.
[89] K. Kiyokawa, Y. Kurata, and H. Ohno. An optical see-through display for mutual occlusion of real and virtual environments. In Proc. of IEEE and ACM International Symposium on Augmented Reality(ISAR'01), pages 217-223, New York,USA, 2001.
[90] S. Noda, Y. Ban, K. Sato, and K. Chilhara. An optical see-through mixed reality display with realtime rangefinder and an active pattern light source. Transactions of the Virtual Reality Society of Japan, 4(4):665-670, 1999.
[91] T. Naemura, T. Nitta, A. Mimura, and H. Harashima. Virtual shadows-enhanced interaction in mixed reality environments. In Proc. of IEEE VR'02, pages 293-294, Orlando, USA, 2002.
[92] O. Bimber, B. Frohlich, D. Schmalstieg, and L. M. Encarnagao. The virtual showcase. IEEE Computer Graphics and Application, 21(6):48-55, 2002.
[93] A. Fuhrmann, G. Hesina, F Faure, and M. Gervautz. Occulsion in collaborative augmented environments. Computers and Graphics, 23(6):809-819,1998.
[94] L. Vacchetti, V. Lepetit, and P. Fua. Combining edge and texture information for real-time accurate 3D camera tracking. In Proc, of IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'04), pages 48-56, Arlington, USA, 2004.
[95] Y. Ohta, Y. Sugaya, H. Igarashi, T. Ohtsuki, and K. Taguchi. Share-z: Client/server depth sensing for see-through head mounted displays. In Proc, of 2nd IEEE and ACM International Symposia on Mixed Reality (ISMR'01), pages 64-72, Tokyo, Japan, 2001.
[96] J.E. Swan, M.A. Livingston, H.S. Smallman, D. Brown, Y. Baillot, J.L. Gabbard, and D. Hix. A perceptual matching technique for depth judgments in optical, see-through augmented reality. In Proc, of IEEE VR'06, pages 19-26, Alexandria, Virginia, USA, 2006.
[97] H. Wang, K. Sengupta, P. Kumar, and Sharma.R. Occlusion handling in augmented reality using background-foreground segmentation and projective geometry. Presence: Teleoperators and Virtual Environments, 14(3):264-277,2005.
[98] P. Noriega, B. Bascle, and O. Bernier. Local kernel color histograms for background subtration. In Proc, of International Conference on Computer Vision Theory and Applications (VISAPP'06), pages 67-74, Setubal, Portugal, 2006.
[99] A.K. Jain, M.N. Murty, and P. J. Flynn. Data clustering: A review. ACM Computing Surveys, 31(3):264-323,1999.
[100] G. Rhodes. Fast,robust intersection of 3D line segments. Game Programming Gems 2, pages 191-204, 2001.
[101] URL http: //openvrml. org/index.
[102] M. Billinghurst, H. Kato, and I. Poupyrev. The magicbook: a transitional ar interface. Computers and Graphics, 25(5):745-753,2001.
[103] W. Fuller. Measurement Error Models. John Wiley and Sons Press, New Jersey, 1978.
[104] J.W. Brewer. Kronecker products and matrix calculus in system theory. IEEE Transactions on Circuits and Systems, 25(9):772-781,1978.