Face-Off: A Face Reconstruction Technique for Virtual Reality (VR) Scenarios

详细信息    查看全文

• 关键词：Virtual reality ; VR headset ; Face reconstruction ; PCA ; Wearable setup ; Oculus
• 刊名：Lecture Notes in Computer Science
• 出版年：2016
• 出版时间：2016
• 年：2016
• 卷：9913
• 期：1
• 页码：490-503
• 全文大小：1,763 KB
• 参考文献：1.Fove eyetracker, January 2016. http://​www.​getfove.​com/​
  2.Bentsianov, B., Blitzer, A.: Facial anatomy. Clin. Dermatol. 22(1), 3–13 (2004)CrossRef
  3.Burgos-Artizzu, X.P., Fleureau, J., Dumas, O., Tapie, T., LeClerc, F., Mollet, N.: Real-time expression-sensitive hmd face reconstruction. In: SIGGRAPH Asia 2015 Technical Briefs, p. 9. ACM (2015)
  4.Chakrabarti, A., Rajagopalan, A., Chellappa, R.: Super-resolution of face images using kernel pca-based prior. IEEE Trans. Multimedia 9(4), 888–892 (2007)CrossRef
  5.Cristinacce, D., Cootes, T.F.: Feature detection and tracking with constrained local models. In: BMVC, vol. 2, p. 6 (2006)
  6.Ekenel, H.K., Stiefelhagen, R.: Why is facial occlusion a challenging problem? In: Tistarelli, M., Nixon, M.S. (eds.) ICB 2009. LNCS, vol. 5558, pp. 299–308. Springer, Heidelberg (2009). doi:10.​1007/​978-3-642-01793-3_​31 CrossRef
  7.VR Headset, January 2016. http://​www.​wareable.​com/​headgear/​the-best-ar-and-vr-headsets
  8.Hoberman, P., Krum, D.M., Suma, E.A., Bolas, M.: Immersive training games for smartphone-based head mounted displays. In: 2012 IEEE Virtual Reality Short Papers and Posters (VRW), pp. 151–152. IEEE (2012)
  9.Hosoi, T., Nagashima, S., Kobayashi, K., Ito, K., Aoki, T.: Restoring occluded regions using fw-pca for face recognition. In: 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 23–30. IEEE (2012)
  10.Hupont, I., Baldassarri, S., Cerezo, E.: Facial emotional classification: from a discrete perspective to a continuous emotional space. Pattern Anal. Appl. 16(1), 41–54 (2013)MathSciNet CrossRef
  11.Hwang, B.W., Lee, S.W.: Reconstruction of partially damaged face images based on a morphable face model. IEEE Trans. Pattern Anal. Mach. Intell. 25(3), 365–372 (2003)CrossRef
  12.Jäckel, D.: Head-mounted displays. In: Proceedings of RTMI, pp. 1–8 (2013)
  13.Javidi, B., Okano, F.: Three-Dimensional Television, Video, and Display Technologies. Springer, Heidelberg (2002)
  14.Khan, M., Li, H., Rehman, S.: Telepresence mechatronic robot (tebot): toward the design and control of socially interactive bio-inspired system. J. Intell. Fuzzy Syst., Special Issue: Multimedia in Technology Enhanced Learning (2016)
  15.Kimura, S., Fukuomoto, M., Horikoshi, T.: Eyeglass-based hands-free videophone. In: Proceedings of the 2013 International Symposium on Wearable Computers, pp. 117–124. ACM (2013)
  16.Kuo, C.J., Lin, T.G., Huang, R.S., Odeh, S.F.: Facial model estimation from stereo/mono image sequence. IEEE Trans. Multimedia 5(1), 8–23 (2003)CrossRef
  17.Leonardis, A., Bischof, H.: Robust recognition using eigenimages. Comput. Vis. Image Underst. 78(1), 99–118 (2000)CrossRef MATH
  18.Li, H., Trutoiu, L., Olszewski, K., Wei, L., Trutna, T., Hsieh, P.L., Nicholls, A., Ma, C.: Facial performance sensing head-mounted display. ACM Trans. Graph. (Proceedings SIGGRAPH 2015) 34(4), 47 (2015)
  19.Lin, D., Tang, X.: Quality-driven face occlusion detection and recovery. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2007, pp. 1–7. IEEE (2007)
  20.Liu, L., et al.: Vibrotactile rendering of human emotions on the manifold of facial expressions. J. Multimedia 3(3), 18–25 (2008)
  21.McCloy, R., Stone, R.: Virtual reality in surgery. BMJ 323(7318), 912–915 (2001)CrossRef
  22.Mo, Z., Lewis, J.P., Neumann, U.: Face inpainting with local linear representations. In: BMVC, pp. 1–10 (2004)
  23.Reinhard Friedl, M.: Virtual reality and 3d visualizations in heart surgery education. In: The Heart Surgery Forum, vol. 2001, p. 03054 (2002)
  24.Schild, J., LaViola, J., Masuch, M.: Understanding user experience in stereoscopic 3d games. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 89–98. ACM (2012)
  25.Sderstrm, U.: Very low bitrate video communication: a principal component analysis approach. Ph.D. thesis, Ume Unviersity (2008)
  26.Söderström, U., Li, H.: Asymmetrical principal component analysis: theory and its applications to facial video coding. Effective Video Coding for Multimedia Applicationss, Dr. Sudhakar Radhakrishnan (ed.) (2011). InTech, ISBN: 978-953-307-177-0
  27.Stengel, M., Grogorick, S., Eisemann, M., Eisemann, E., Magnor, M.A.: An affordable solution for binocular eye tracking and calibration in head-mounted displays. In: Proceedings of the 23rd Annual ACM Conference on Multimedia Conference, pp. 15–24. ACM (2015)
  28.Storer, M., Roth, P.M., Urschler, M., Bischof, H.: Fast-robust PCA. In: Salberg, A.-B., Hardeberg, J.Y., Jenssen, R. (eds.) SCIA 2009. LNCS, vol. 5575, pp. 430–439. Springer, Heidelberg (2009)CrossRef
  29.Tu, C.-T., Lien, J.-J.J.: Facial occlusion reconstruction: recovering both the global structure and the local detailed texture components. In: Mery, D., Rueda, L. (eds.) PSIVT 2007. LNCS, vol. 4872, pp. 141–151. Springer, Heidelberg (2007)CrossRef
  30.Soderstrom, U., Li, H.: Asymmetrical principal component analysis for video coding. Electron. Lett. 44, 276–277 (2008)CrossRef
  31.V.R. Users, January 2016. http://​www.​statista.​com/​statistics/​426469/​active-virtual-reality-users-worldwide/​
  32.Vincent, O., Folorunso, O.: A descriptive algorithm for sobel image edge detection. In: Proceedings of Informing Science & IT Education Conference (InSITE), vol. 40, pp. 97–107 (2009)
  33.Wall, M.E., Rechtsteiner, A., Rocha, L.M.: Singular value decomposition and principal component analysis. In: Berrar, D.P., Dubitzky, W., Granzow, M. (eds.) A Practical Approach to Microarray Data Analysis, pp. 91–109. Springer, US (2003)CrossRef
  34.Waters, K.: A muscle model for animation three-dimensional facial expression. ACM SIGGRAPH Comput. Graph. 21, 17–24 (1987). ACMCrossRef
  35.Whissell, C.: The dictionary of affect in language. Emot. Theor. Res. Exp. 4(113–131), 94 (1989)
  
• 作者单位：M. S. L. Khan (15)
  Shafiq Ur Réhman (15) (17)
  Ulrik Söderström (15)
  Alaa Halawani (15)
  Haibo Li (16)
  
  15. Department of Applied Physics and Electronics, Umeå University, 90187, Umeå, Sweden
  17. University of East London, London, England
  16. KTH Royal Institute of Technology, 100 44, Stockholm, Sweden
  
• 丛书名：Computer Vision – ECCV 2016 Workshops
• ISBN：978-3-319-46604-0
• 刊物类别：Computer Science
• 刊物主题：Artificial Intelligence and Robotics
  Computer Communication Networks
  Software Engineering
  Data Encryption
  Database Management
  Computation by Abstract Devices
  Algorithm Analysis and Problem Complexity
  
• 出版者：Springer Berlin / Heidelberg
• ISSN：1611-3349
• 卷排序：9913

文摘

Virtual Reality (VR) headsets occlude a significant portion of human face. The real human face is required in many VR applications, for example, video teleconferencing. This paper proposes a wearable camera setup-based solution to reconstruct the real face of a person wearing VR headset. Our solution lies in the core of asymmetrical principal component analysis (aPCA). A user-specific training model is built using aPCA with full face, lips and eye region information. During testing phase, lower face region and partial eye information is used to reconstruct the wearer face. Online testing session consists of two phases, (i) calibration phase and (ii) reconstruction phase. In former, a small calibration step is performed to align test information with training data, while the later uses half face information to reconstruct the full face using aPCA-based trained-data. The proposed approach is validated with qualitative and quantitative analysis.