Gesture-based interactive augmented reality content authoring system using HMD

详细信息    查看全文

• 作者：Jinwook Shim ; Yoonsik Yang ; Nahyung Kang ; Jonghoon Seo ; Tack-Don Han
• 关键词：Immersive augmented reality ; Augmented reality authoring ; Gesture interaction ; Tangible interaction
• 刊名：Virtual Reality
• 出版年：2016
• 出版时间：March 2016
• 年：2016
• 卷：20
• 期：1
• 页码：57-69
• 全文大小：1,959 KB
• 参考文献：AndAR (2014) AndAR-Android augmented reality. http://​code.​google.​com/​p/​andar/​ . Accessed 30 May 2014
  ARToolKit (2014) ARToolKit. http://​www.​hitl.​washington.​edu/​artoolkit/​ . Accessed 30 May 2014
  Azuma RT (1997) A survey of augmented reality. Presence 6(4):355–385CrossRef
  Azuma R, Bishop G (1994) Improving static and dynamic registration in an optical see-through HMD. In: Proceedings of the 21st ACM annual conference on computer graphics and interactive techniques, pp 197–204
  Azuma R, Baillot Y, Behringer R, Feiner S, Julier S, MacIntyre B (2001) Recent advances in augmented reality. IEEE Comput Graph Appl 21(6):34–47CrossRef
  Bajura M, Neumann U (1995) Dynamic registration correction in video-based augmented reality systems. IEEE Comput Graph Appl 15(5):52–60CrossRef
  Billinghurst M, Kato H, Poupyrev I (2001) The magicbook-moving seamlessly between reality and virtuality. IEEE Comput Graph Appl 21(3):6–8
  Billinghurst M, Hakkarainen M, Woodward C (2008) Augmented assembly using a mobile phone. In: Proceedings of the 7th ACM international conference on mobile and ubiquitous multimedia, pp 84–87
  Buchmann V, Violich S, Billinghurst M, Cockburn A (2004) FingARtips: Gesture based direct manipulation in augmented reality. In: Proceedings of the 2nd ACM international conference on computer graphics and interactive techniques in Australasia and South East Asia, pp 212–221
  Bunnun P, Subramanian S, Mayol-Cuevas WW (2013) In-situ interactive image-based model building for augmented reality from a handheld device. Virtual Real 17(2):137–146CrossRef
  Coquillart S, Göbel M (2004) Authoring of mixed reality applications including multi-marker calibration for mobile devices. In: Eurographics symposium on virtual environments, pp 1–9
  Doliotis P, Athitsos V, Kosmopoulos D, Perantonis S (2012) Hand shape and 3D pose estimation using depth data from a single cluttered frame. In: Bebis G, Boyle R, Parvin B, Koracin D, Fowlkes C (eds) Advances in visual computing. Springer, Berlin, Heidelberg, pp 148–158CrossRef
  Dorfmuller-Ulhaas K, Schmalstieg D (2001) Finger tracking for interaction in augmented environments. In: Proceedings IEEE and ACM international symposium on augmented reality, pp 55–64
  Grimm P, Haller M, Paelke V, Reinhold S, Reimann C, Zauner R (2002) AMIRE-authoring mixed reality. In: The first IEEE international workshop augmented reality toolkit, pp 2–pp
  Hackenberg G, McCall R, Broll W (2011) Lightweight palm and finger tracking for real-time 3d gesture control. In: 2011 IEEE virtual reality conference (VR), pp 19–26
  Harviainen T, Korkalo O, Woodward C (2009) Camera-based interactions for augmented reality. In: Proceedings of the ACM international conference on advances in computer entertainment technology, pp 307–310
  Henrysson A, Billinghurst M (2007) Using a mobile phone for 6 DOF mesh editing. In: Proceedings of the 8th ACM SIGCHI New Zealand chapter’s international conference on computer-human interaction: design centered HCI, pp 9–16
  Hoff WA, Nguyen K, Lyon T (1996) Computer-vision-based registration techniques for augmented reality. In: Photonics East’96, international society for optics and photonics, pp 538–548
  Kato H, Billinghurst M (1999) Marker tracking and HMD calibration for a video-based augmented reality conferencing system. In: Proceedings IWAR’99, pp 85–94
  Kato H, Billinghurst M, Poupyrev I, Imamoto K, Tachibana K (2000) Virtual object manipulation on a table-top AR environment. Proc ISAR 2000:111–119
  Kaufmann H, Schmalstieg D (2003) Mathematics and geometry education with collaborative augmented reality. Comput Graph 27(3):339–345CrossRef
  Langlotz T, Mooslechner S, Zollmann S, Degendorfer C, Reitmayr G, Schmalstieg D (2012) Sketching up the world: in situ authoring for mobile augmented reality. Pers Ubiquit Comput 16(6):623–630CrossRef
  Ledermann F, Schmalstieg D (2005) April: a high-level framework for creating augmented reality presentations. In: Proceedings. VR 2005, pp 187–194
  Lee T, Hollerer T (2007) Handy AR: Markerless inspection of augmented reality objects using fingertip tracking. In: 11th IEEE international symposium on wearable computers, pp 83–90
  Lee U, Tanaka J (2012) Hand controller: image manipulation interface using fingertips and palm tracking with Kinect depth data. In: Proceedings of Asia Pacific conference on computing human interact, pp 705–706
  Lee GA, Nelles C, Billinghurst M, Kim GJ (2004) Immersive authoring of tangible augmented reality applications. In: Proceedings of the 3rd IEEE/ACM international symposium on mixed and augmented reality, pp 172–181
  Li D, Weng D, Li Y, Xie J (2013) Touchablear: a new experience of augmented reality. Proc ICVRV 2013:37–42
  Liang H, Yuan J, Thalmann D (2012) 3D fingertip and palm tracking in depth image sequences. In: Proceedings of the 20th ACM international conference on Multimedia, pp 785–788
  Lund A (2001) Measuring usability with the use questionnaire. Usability Interface 8(2):3–6MathSciNet
  MacIntyre B, Gandy M, Dow S, Bolter JD (2004) DART: A toolkit for rapid design exploration of augmented reality experiences. In: Proceedings of the 17th annual ACM symposium on user interface software and technology, pp 197–206
  Maier P, Tonnis M, Klinker G (2010) Designing and comparing two-handed gestures to confirm links between user controlled objects. Proc ISMAR 2010:251–252
  Ng L, Oon S, Ong S, Nee A (2011) GARDE: a gesture-based augmented reality design evaluation system. Int J Interact Design Manuf (IJIDeM) 5(2):85–94CrossRef
  NyARToolKit (2014) NyARToolKit. http://​arforglass.​org/​?​project=​nyartoolkit/​ . Accessed 30 May 2014
  Oculus VR (2014) Oculus VR developer center. http://​developer.​oculusvr.​com/​ . Accessed 30 May 2014
  Oikonomidis I, Kyriazis N, Argyros AA (2011) Efficient model-based 3D tracking of hand articulations using Kinect. Proc BMVC 2011:1–11
  Oikonomidis I, Kyriazis N, Argyros AA (2012) Tracking the articulated motion of two strongly interacting hands. Proc CVPR 2012:1862–1869
  Okuma T, Kiyokawa K, Takemura H, Yokoya N (1998) An augmented reality system using a real-time vision based registration. In: Proceedings fourteenth international conference on pattern recognition, vol 2, pp 1226–1229
  Ong S, Wang Z (2011) Augmented assembly technologies based on 3D bare-hand interaction. CIRP Ann-Manuf Technol 60(1):1–4CrossRef
  OpenNI (2014) OpenNI. https://​github.​com/​OpenNI/​OpenNI/​ . Accessed 30 May 2014
  Poupyrev I, Tan DS, Billinghurst M, Kato H, Regenbrecht H, Tetsutani N (2002) Developing a generic augmented-reality interface. Computer 35(3):44–50CrossRef
  QCAR (2014) QCAR. http://​developer.​qualcomm.​com/​dev/​augmented-reality/​ . Accessed 30 May 2014
  Radu I, MacIntyre B (2009) Augmented-reality scratch: a tangible programming environment for children. In: Proceedings of conference on interaction design for children, Como, Italy
  Ren Z, Meng J, Yuan J, Zhang Z (2011a) Robust hand gesture recognition with Kinect sensor. In: Proceedings of the 19th ACM international conference on Multimedia, pp 759–760
  Ren Z, Yuan J, Zhang Z (2011b) Robust hand gesture recognition based on finger-earth mover’s distance with a commodity depth camera. In: Proceedings of the 19th ACM international conference on Multimedia, pp 1093–1096
  Schmalstieg D, Fuhrmann A, Hesina G, Szalavári Z, Encarnaçao LM, Gervautz M, Purgathofer W (2002) The studierstube augmented reality project. Presence Teleoper Virtual Environ 11(1):33–54CrossRef
  Seichter H, Looser J, Billinghurst M (2008) Composar: An intuitive tool for authoring AR applications. In: Proceedings of the 7th IEEE/ACM international symposium on mixed and augmented reality, pp 177–178
  Shim J, Kong M, Yang Y, Seo J, Han T-D (2014) Interactive features based augmented reality authoring tool. Proc ICCE 2014:47–50
  SoftKinetic (2014) SoftKinetic: the interface is you. http://​www.​softkinetic.​com/​ . Accessed 30 May 2014
  Thomas BH, Piekarski W (2002) Glove based user interaction techniques for augmented reality in an outdoor environment. Virtual Real 6(3):167–180CrossRef
  Wang Y, Langlotz T, Billinghurst M, Bell T (2009) An authoring tool for mobile phone AR environments. Proc N Z Comput Sci Res Stud Conf 9:1–4
  Whitton MC, Pisano ED, Fuchs H (1996) Technologies for augmented reality systems: Realizing ultrasound-guided needle biopsies. In: Proceedings of the 23rd annual conference on computer graphics and interactive techniques, pp 439–446
  Wilson AD (2010) Using a depth camera as a touch sensor. In: Proceedings ACM international conference on interactive tabletops and surfaces, pp 69–72
  Wozniewski M, Warne P (2011) Towards in situ authoring of augmented reality content. In: Proceedings ISMAR 2011
  Yao Y, Fu Y (2012) Real-time hand pose estimation from RGB-D sensor. Proc ICME 2012:705–710
  
• 作者单位：Jinwook Shim (1)
  Yoonsik Yang (1)
  Nahyung Kang (1)
  Jonghoon Seo (2)
  Tack-Don Han (1)
  
  1. Department of Computer Science, Yonsei University, 50, Yonsei-ro, Seodaemun-gu, Seoul, 03722, Korea
  2. Software Platform R&D Lab, LG Electronics Advanced Research Institute, 19, Yangjae-daero 11gil, Seocho-gu, Seoul, 06772, Korea
  
• 刊物类别：Computer Science
• 刊物主题：Computer Graphics
  Computing Methodologies
  Artificial Intelligence and Robotics
  Image Processing and Computer Vision
  
• 出版者：Springer London
• ISSN：1434-9957

文摘

This paper proposes an augmented reality content authoring system that enables ordinary users who do not have programming capabilities to easily apply interactive features to virtual objects on a marker via gestures. The purpose of this system is to simplify augmented reality (AR) technology usage for ordinary users, especially parents and preschool children who are unfamiliar with AR technology. The system provides an immersive AR environment with a head-mounted display and recognizes users’ gestures via an RGB-D camera. Users can freely create the AR content that they will be using without any special programming ability simply by connecting virtual objects stored in a database to the system. Following recognition of the marker via the system’s RGB-D camera worn by the user, he/she can apply various interactive features to the marker-based AR content using simple gestures. Interactive features applied to AR content can enlarge, shrink, rotate, and transfer virtual objects with hand gestures. In addition to this gesture-interactive feature, the proposed system also allows for tangible interaction using markers. The AR content that the user edits is stored in a database, and is retrieved whenever the markers are recognized. The results of comparative experiments conducted indicate that the proposed system is easier to use and has a higher interaction satisfaction level than AR environments such as fixed-monitor and touch-based interaction on mobile screens. Keywords Immersive augmented reality Augmented reality authoring Gesture interaction Tangible interaction