基于Android终端的移动增强现实技术的研究
|
摘要
随着科技不断地快速进步,移动终端的硬件设施越来越完善,体积小和易于携带的优点,使得移动终端的普及率也越来越高。而且随着物质生活水平的提高,对于感兴趣的未知事物,浏览器搜索方式似乎已经不能满足人们的需求。此外绝大多数智能终端自身已经集成了GPS和传感器等功能完善的硬件模块,为增强现实技术的研究提供了硬件条件。基于以上背景,基于移动智能终端的增强现实的应用研究逐渐得到越来越多研究机构和企业的重视。
本论文旨在基于目前比较流行的Android开源操作系统平台,深入探讨移动增强现实的相关技术,并结合位置服务,提出将位置导航服务与增强现实相结合的解决方案,实现了一个基于Android终端的增强现实系统。
首先,本文介绍了本课题的选题背景和研究意义,讨论了国内外研究现状,引出本课题工作内容以及论文结构。其次,介绍了增强现实的三大关键技术,即智能显示技术、智能跟踪注册技术和智能交互技术,在此基础上,详细阐述了基于计算机视觉和基于传感器的两种增强现实系统。再次,对Android平台的架构及应用开发进行了介绍,介绍了Android增强现实中GPS框架、Sensor框架和OpenGL ES基本知识,然后引出近几年较流行的高通QCAR开发包Vuforia SDK。最后,设计并实现了将Android传感器和文字识别技术相结合的增强现实系统,介绍了系统整体框架和工作流程,详细阐述了包括终端信息获取、数据库系统、文字识别、建筑物识别和增强信息显示在内的各个功能模块,并就最后程序运行的结果进行分析。
本文的创新点是将基于传感器和基于图像识别的增强现实技术相结合,充分利用了这两大研究方向的优势。课题中设计的系统在学校小范围内得以试验,取得了较为理想的效果,对增强现实在移动智能终端上的研究作出了贡献,具有一定的理论与现实意义。
With the rapid development of technology, smart phones are hardware well-facilities, small-sized and easy-to-carry, which makes it more and more popular. Meanwhile, people have higher demands on the way of learning about attractive things rather than by searching with a browser merely. In addition, GPS and Sensor modules are integrated into smart phones, which promotes the development of augmented reality technology. Currently, more and more research institutions and famous enterprises pay attention to mobile augmented reality.
The paper is intended to research key technology of mobile augmented reality and build up an augmented reality system based on android operation system by combining location service with augmented reality.
Firstly, the paper introduces background and significance of this subject, then discusses development situation of augmented reality at home and abroad. My work and structure of the paper are introduced at last. Secondly, the paper introduces key technologies of augmented reality which are displaying, three-diversion registration and interaction. On the base of key technologies, computer vision-based and sensor-based augmented reality systems are elaborated. Thirdly, the paper introduces framework and development of android operation system including GPS, Sensor and OpenGL ES, then presents QCAR ToolKit of Qualcomm based on mobile device. Finally, an augmented reality system combining GPS and Sensors of android device with character recognition is desined. And framework and schedule are introduced, then all the function modules are elaborated in detail sequentialy, and then the paper analyzes the results of test.
The innovation of the paper is combining sensor-based augmented reality technology with character recognition technology. The system in the paper is tested within the range of school and get ideal results, which makes a theoretical and practical significance.
本论文旨在基于目前比较流行的Android开源操作系统平台,深入探讨移动增强现实的相关技术,并结合位置服务,提出将位置导航服务与增强现实相结合的解决方案,实现了一个基于Android终端的增强现实系统。
首先,本文介绍了本课题的选题背景和研究意义,讨论了国内外研究现状,引出本课题工作内容以及论文结构。其次,介绍了增强现实的三大关键技术,即智能显示技术、智能跟踪注册技术和智能交互技术,在此基础上,详细阐述了基于计算机视觉和基于传感器的两种增强现实系统。再次,对Android平台的架构及应用开发进行了介绍,介绍了Android增强现实中GPS框架、Sensor框架和OpenGL ES基本知识,然后引出近几年较流行的高通QCAR开发包Vuforia SDK。最后,设计并实现了将Android传感器和文字识别技术相结合的增强现实系统,介绍了系统整体框架和工作流程,详细阐述了包括终端信息获取、数据库系统、文字识别、建筑物识别和增强信息显示在内的各个功能模块,并就最后程序运行的结果进行分析。
本文的创新点是将基于传感器和基于图像识别的增强现实技术相结合,充分利用了这两大研究方向的优势。课题中设计的系统在学校小范围内得以试验,取得了较为理想的效果,对增强现实在移动智能终端上的研究作出了贡献,具有一定的理论与现实意义。
With the rapid development of technology, smart phones are hardware well-facilities, small-sized and easy-to-carry, which makes it more and more popular. Meanwhile, people have higher demands on the way of learning about attractive things rather than by searching with a browser merely. In addition, GPS and Sensor modules are integrated into smart phones, which promotes the development of augmented reality technology. Currently, more and more research institutions and famous enterprises pay attention to mobile augmented reality.
The paper is intended to research key technology of mobile augmented reality and build up an augmented reality system based on android operation system by combining location service with augmented reality.
Firstly, the paper introduces background and significance of this subject, then discusses development situation of augmented reality at home and abroad. My work and structure of the paper are introduced at last. Secondly, the paper introduces key technologies of augmented reality which are displaying, three-diversion registration and interaction. On the base of key technologies, computer vision-based and sensor-based augmented reality systems are elaborated. Thirdly, the paper introduces framework and development of android operation system including GPS, Sensor and OpenGL ES, then presents QCAR ToolKit of Qualcomm based on mobile device. Finally, an augmented reality system combining GPS and Sensors of android device with character recognition is desined. And framework and schedule are introduced, then all the function modules are elaborated in detail sequentialy, and then the paper analyzes the results of test.
The innovation of the paper is combining sensor-based augmented reality technology with character recognition technology. The system in the paper is tested within the range of school and get ideal results, which makes a theoretical and practical significance.
引文
[1]Ivan Sutherland. A head-mounted three dimensional display. In Proc. FJCC 1968. Washington D.C., USA:Thompson Books,1968,757-764
[2]Thomas P. Caudell, David W. Mizell. Augmented Reality:An Application of Heads-Up Display Technology to Manual Manufacturing Processes. In Proc. Hawaii International Conference on System Sciences 1992. Los Alamitos, CA, USA:IEEE Computer Society Press,1992,659-669
[3]Hirokazu Kato and Mark Billinghurst. Marker tracking and HMD calibration for a video-based augmented reality conferenencing system. In Proc. IWAR 1999, San Francisco,CA, USA,20-21 October 1999,85-94
[4]R. Azuma. Asurvey of augmented reality. Teleoperators and Virtual Environments, 1997,6(4):355-385
[5]B. Thomas, B. Close, J. Donoghue, et al. ARQuake:An outdoor/indoor first person augmented reality application. In Proc. ISWC 2000, Atlanta, GA, USA,18-21 October 2000, 139-146
[6]MILGRAM P,KISHINo F.A Taxonomy of Mixed Reality Visual Display[J].IEICE Trans.InformationSystem.1994,77(12):1321-1329
[7]PINTARIC,T..An adaptive thresholding algorithmm for the augmented reality toolkit.In IEEE Int.Augmented Reality Toolkit Workshop.2003:71
[8]S. Feiner, B. Maclntyre, T. Hollerer, et al. A touring machine:Prototyping 3D mobile augmented reality systems for exploring the urban environment. In Proc. ISWC 1997,Cambridge, MA, USA,13-14 October 1997,74-81
[9]A. Cheok, S. Fong, K. Gohl, et al. Human Pacman:A Mobile Entertainment System with Ubiquitous Computing and Tangible Interaction over a Wide Outdoor Area. In Proc. Mobile HCI 2003, Lecture Notes in Computer Science,2003, 2795:209-223
[10]胡晓明,刘越等.五指七自由度数据手套的研究.光电子技术与信息.2003,8(增刊)：768-771
[11]哈涌刚,周雅.用于增强现实的头盔显示器的设计.光学技术.2000,26(4)：350-353
[12]周雅,闫达远,王涌天,常红,徐彤.一种增强现实系统的三维注册方法.中国图像图形学报.2000,5(5)：430-433
[13]陈靖,王涌天,施琦,闫达远.明德烈,柳健,田金文.非定标的虚实注册方法.红外与激光工程.2002,31(2)：170-174
[14]明德烈,柳健,田金文.增强现实的虚实注册技术研究.中国图像图形学报.2003,8(5)：557-561
[15]陈靖,王聪.增强现实研究进展与核心技术.信息技术与标准化.2012(6)：35-37
[16]Jun Rekimoto and Katashi Nagao.The world through the computer:computer augmented interaction with real world environments.In Proceedings of the 8th annual ACM symposium on user interface and software technology(UIST'95),1995:29-36
[17]Daniel Wagner.Handheld Augmented Reality:[学位论文]Austria:Graz University of Technology,2007
[18]Mark Billinghurst,Hirkazu Kato,Ivan Poupyrev.The magicbook-moving seamlessly between reality and virtuality.IEEE Computer Graphics and Applications,21 (3),2001:6-8
[19]姚远.增强现实应用技术研究：[学位论文]杭州：浙江大学,2006
[20]Feiner S,MacIntyre B,Hollerer T.A touring machine:Prototyping 3D mobile augmented reality systems for exploring the urban environment,1997,19-30
[21]周国众.移动增强现实关键技术及应用.测绘与空间地理信息.2012.9,35(9)：141-144
[22]ARToolKit website:http://www.hitl.washington.edu/artoolkit/
[23]Fiala,M.ARTag,a Fiducial Marker System Using Digital Techniques.In Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition(CVPR'05)-Volume 2,2005:590-596
[24]Daniel Wagner,Dieter Schmalstieg.Artoolkitplus for pose tracking on mobile devices.In Proceedings of 12th Computer Vision Winter Workshop(CVWW'07),2007
[25]Lowe D GDistinctive image features from scale-invariant key points.International Journal of computer Vision,2004,60(2)
[26]Bay H,Tuytelaars T,Gool L V.SURF:Speeded up robust features[C]//ECCV.2006
[27]Rosten E,Drummond T.Machine learning for high-speed comer detection. European Conference on Computer Vision.2006,3951:430-443
[28]Andrei State,Mark A.Livingston,Gentaro Hirota,et al. Techniques for Augmented Reality Systems:Realizing Ultrasound-Guided Needle Biopsies.In Procs.SIGGRAPH 1996, New Orleans, LA, USA,4-9 August 1996,439-446
[29]Andrei State, Gentaro Hirota, David T. Chen, et al. Superior Augmented Reality Registration by Integrating Landmark Tracking and Magnetic Tracking. In Procs. SIGGRAPH 1996, New Orleans, LA, USA,4-9 August 1996,429-438
[30]G. Fritz, C. Seifert, L. Paletta. Urban Object Recognition from Informative Local Features. In Procs. ICRA 2005, Barcelona, Spain,18-22 April 2005,132-138
[31]A.J. Davison. Real-time simultaneous localisation and mapping with a single camera. In Procs.ICCV 2003, Nice, France,14-17 October 2003,1403-1410
[32]G. Klein and D. Murray. Parallel tracking and mapping on a camera phone. In Procs. ISMAR 2009, Orlando, FL, USA,19-22 October 2009,83-86
[33]G. Klein, D. Murray. Parallel tracking and mapping for small AR workspaces. In Procs. ISMAR 2007, Nara, Japan,13-16 November 2007,1-10
[34]G. Fritz. A Mobile Vision System for Urban Detection with Informative Local Descriptors. In Procs. ICVS 2006, New York, NY, USA,5-7 January 2006,30
[35]G. Klein, D. Murray. Compositing for Small Cameras. In Procs. ISMAR 2008, Cambridge, UK,15-18 September 2008,57-60
[36]Z. Y. Zhang. A flexible new technique for camera calibration. IEEE Trans. Pattern Analysis and Machine Intelligence,2000,22(11):1330-1334
[37]Jun Rekimoto.Transvision:A hand-held augmented reality system for collaborative design.In International Conference on Virtual Systems and Multimedia(VSMM' 96),1996:85-90
[38]Christian Geiger,Bernd Kleinnjohann,Christian Reimann,Dirk Stichling.Mobile ar4all.In Proceedings of the IEEE and ACM International Symposium on Augmented Reality(ISAR'01),2001:181-182
[39]D.Mogilev,K.Kiyokawa,M.Billinghurst,J.Pair.Ar pad:an interface for face-to-face ar collaboration. In CHI' 02 extended abstracts on Human factors in computing systems,2002:654-655
[40]周雅,徐彤,阎达远,王涌天,哈涌刚.增强现实中虚拟物体的投影注册算法研究.北京理工大学学报.2000.12,20(6)：738-742
[41]盖索林Google Android开发入门指南.北京：人民邮电出版社,2009
[42]伯内特,张波Android基础教程.北京：人民邮电出版社,2009
[43]https://developer.vuforia.com/
[2]Thomas P. Caudell, David W. Mizell. Augmented Reality:An Application of Heads-Up Display Technology to Manual Manufacturing Processes. In Proc. Hawaii International Conference on System Sciences 1992. Los Alamitos, CA, USA:IEEE Computer Society Press,1992,659-669
[3]Hirokazu Kato and Mark Billinghurst. Marker tracking and HMD calibration for a video-based augmented reality conferenencing system. In Proc. IWAR 1999, San Francisco,CA, USA,20-21 October 1999,85-94
[4]R. Azuma. Asurvey of augmented reality. Teleoperators and Virtual Environments, 1997,6(4):355-385
[5]B. Thomas, B. Close, J. Donoghue, et al. ARQuake:An outdoor/indoor first person augmented reality application. In Proc. ISWC 2000, Atlanta, GA, USA,18-21 October 2000, 139-146
[6]MILGRAM P,KISHINo F.A Taxonomy of Mixed Reality Visual Display[J].IEICE Trans.InformationSystem.1994,77(12):1321-1329
[7]PINTARIC,T..An adaptive thresholding algorithmm for the augmented reality toolkit.In IEEE Int.Augmented Reality Toolkit Workshop.2003:71
[8]S. Feiner, B. Maclntyre, T. Hollerer, et al. A touring machine:Prototyping 3D mobile augmented reality systems for exploring the urban environment. In Proc. ISWC 1997,Cambridge, MA, USA,13-14 October 1997,74-81
[9]A. Cheok, S. Fong, K. Gohl, et al. Human Pacman:A Mobile Entertainment System with Ubiquitous Computing and Tangible Interaction over a Wide Outdoor Area. In Proc. Mobile HCI 2003, Lecture Notes in Computer Science,2003, 2795:209-223
[10]胡晓明,刘越等.五指七自由度数据手套的研究.光电子技术与信息.2003,8(增刊)：768-771
[11]哈涌刚,周雅.用于增强现实的头盔显示器的设计.光学技术.2000,26(4)：350-353
[12]周雅,闫达远,王涌天,常红,徐彤.一种增强现实系统的三维注册方法.中国图像图形学报.2000,5(5)：430-433
[13]陈靖,王涌天,施琦,闫达远.明德烈,柳健,田金文.非定标的虚实注册方法.红外与激光工程.2002,31(2)：170-174
[14]明德烈,柳健,田金文.增强现实的虚实注册技术研究.中国图像图形学报.2003,8(5)：557-561
[15]陈靖,王聪.增强现实研究进展与核心技术.信息技术与标准化.2012(6)：35-37
[16]Jun Rekimoto and Katashi Nagao.The world through the computer:computer augmented interaction with real world environments.In Proceedings of the 8th annual ACM symposium on user interface and software technology(UIST'95),1995:29-36
[17]Daniel Wagner.Handheld Augmented Reality:[学位论文]Austria:Graz University of Technology,2007
[18]Mark Billinghurst,Hirkazu Kato,Ivan Poupyrev.The magicbook-moving seamlessly between reality and virtuality.IEEE Computer Graphics and Applications,21 (3),2001:6-8
[19]姚远.增强现实应用技术研究：[学位论文]杭州：浙江大学,2006
[20]Feiner S,MacIntyre B,Hollerer T.A touring machine:Prototyping 3D mobile augmented reality systems for exploring the urban environment,1997,19-30
[21]周国众.移动增强现实关键技术及应用.测绘与空间地理信息.2012.9,35(9)：141-144
[22]ARToolKit website:http://www.hitl.washington.edu/artoolkit/
[23]Fiala,M.ARTag,a Fiducial Marker System Using Digital Techniques.In Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition(CVPR'05)-Volume 2,2005:590-596
[24]Daniel Wagner,Dieter Schmalstieg.Artoolkitplus for pose tracking on mobile devices.In Proceedings of 12th Computer Vision Winter Workshop(CVWW'07),2007
[25]Lowe D GDistinctive image features from scale-invariant key points.International Journal of computer Vision,2004,60(2)
[26]Bay H,Tuytelaars T,Gool L V.SURF:Speeded up robust features[C]//ECCV.2006
[27]Rosten E,Drummond T.Machine learning for high-speed comer detection. European Conference on Computer Vision.2006,3951:430-443
[28]Andrei State,Mark A.Livingston,Gentaro Hirota,et al. Techniques for Augmented Reality Systems:Realizing Ultrasound-Guided Needle Biopsies.In Procs.SIGGRAPH 1996, New Orleans, LA, USA,4-9 August 1996,439-446
[29]Andrei State, Gentaro Hirota, David T. Chen, et al. Superior Augmented Reality Registration by Integrating Landmark Tracking and Magnetic Tracking. In Procs. SIGGRAPH 1996, New Orleans, LA, USA,4-9 August 1996,429-438
[30]G. Fritz, C. Seifert, L. Paletta. Urban Object Recognition from Informative Local Features. In Procs. ICRA 2005, Barcelona, Spain,18-22 April 2005,132-138
[31]A.J. Davison. Real-time simultaneous localisation and mapping with a single camera. In Procs.ICCV 2003, Nice, France,14-17 October 2003,1403-1410
[32]G. Klein and D. Murray. Parallel tracking and mapping on a camera phone. In Procs. ISMAR 2009, Orlando, FL, USA,19-22 October 2009,83-86
[33]G. Klein, D. Murray. Parallel tracking and mapping for small AR workspaces. In Procs. ISMAR 2007, Nara, Japan,13-16 November 2007,1-10
[34]G. Fritz. A Mobile Vision System for Urban Detection with Informative Local Descriptors. In Procs. ICVS 2006, New York, NY, USA,5-7 January 2006,30
[35]G. Klein, D. Murray. Compositing for Small Cameras. In Procs. ISMAR 2008, Cambridge, UK,15-18 September 2008,57-60
[36]Z. Y. Zhang. A flexible new technique for camera calibration. IEEE Trans. Pattern Analysis and Machine Intelligence,2000,22(11):1330-1334
[37]Jun Rekimoto.Transvision:A hand-held augmented reality system for collaborative design.In International Conference on Virtual Systems and Multimedia(VSMM' 96),1996:85-90
[38]Christian Geiger,Bernd Kleinnjohann,Christian Reimann,Dirk Stichling.Mobile ar4all.In Proceedings of the IEEE and ACM International Symposium on Augmented Reality(ISAR'01),2001:181-182
[39]D.Mogilev,K.Kiyokawa,M.Billinghurst,J.Pair.Ar pad:an interface for face-to-face ar collaboration. In CHI' 02 extended abstracts on Human factors in computing systems,2002:654-655
[40]周雅,徐彤,阎达远,王涌天,哈涌刚.增强现实中虚拟物体的投影注册算法研究.北京理工大学学报.2000.12,20(6)：738-742
[41]盖索林Google Android开发入门指南.北京：人民邮电出版社,2009
[42]伯内特,张波Android基础教程.北京：人民邮电出版社,2009
[43]https://developer.vuforia.com/