Real-time smart lighting control using human motion tracking from depth camera

详细信息    查看全文

• 作者：SungYong Chun ; Chan-Su Lee ; Ja-Soon Jang
• 关键词：Human motion detection ; Depth camera ; Lighting control ; Human motion tracking ; Multiple camera ; Multiple target tracking ; Smart lighting control
• 刊名：Journal of Real-Time Image Processing
• 出版年：2015
• 出版时间：December 2015
• 年：2015
• 卷：10
• 期：4
• 页码：805-820
• 全文大小：3,618 KB
• 参考文献：1.Aggarwal, J.K., Ryoo, M.S.: Human activity analysis: a review. ACM Comput. Surv., 43(3), 16:1鈥?6:43 (2012)
  2.Aldrich, M., Zhao, N., Paradiso, J.: Energy efficient control of polychromatic solid state lighting using a sensor network. In Proc. of SPIE, 7784, 778408鈥?78415 (2010)CrossRef
  3.Armato, A., Lanat谩, A.: Comparative study on photometric normalization algorithms for an innovative, robust and real-time eye gaze tracker. J. Real-Time Image Proc. 8, 21鈥?3 (2013)CrossRef
  4.Barandiaran, I., Paloc, C., Gra帽a, M.: Real-time optical markerless tracking for augmented reality applications. J. Real-Time Image Proc., pp. 129鈥?38 (2010)
  5.Barry, R.: Using the FreeRTOS Real Time Kernel-Standard Edition. freeRTOS (2010)
  6.Bertozzi, M., Broggi, A., Fascioli, A.: Stereo inverse perspective mapping: theory and applications. Image Vis. Comput. 16, 585鈥?90 (1998)CrossRef
  7.Bleser, G., Becker, M., Stricker, D.: Real-time vision-based tracking and reconstruction. J. Real Time Image Proc. 2, 161鈥?75 (2007)CrossRef
  8.Caicedo, D., Pandharipande, A., Leus, G.: Occupancy-based illumination control of led lighting systems. Light. Res. Technol. 43, 217鈥?34 (2011)CrossRef
  9.Caragliu, A., Del Bo, C., Nijkamp, P.: Smart cities in Europe. Technical Report 0048, VU University Amsterdam, Faculty of Economics, Business Administration and Econometrics (2009)
  10.Chandaria, J., Thomas, G.A., Stricker, D.: The MATRIS project: real-time markerless camera tracking for augmented reality and broadcast applications. J. Real Time Image Proc., 2, 69鈥?9 (2007)CrossRef
  11.Chao, Y.-W., Choi, W., Pantofaru, C., Savarese, S.: Layout estimation of highly cluttered indoor scenes using geometric and semantic cues. Proc. Image Anal. Process. LNCS 8157, 489鈥?99 (2013)
  12.Chen, C., Liu, K., Kehtarnavaz, N.: Real-time human action recognition based on depth motion maps. J. Real Time Image Proc. 10, 1鈥? (2013)
  13.Michael Ziegler et. al.: Lighting the cities-accelerating the deployment of innovative lighting in european cities. European Commission Report 0048, European Union (2013)
  14.Dodier, R.H., Henze, G.P., Tiller, D.K.: Xin Guo. Building occupancy detection through sensor belief networks. Energy Build. 38(9), 1033鈥?043 (2006)CrossRef
  15.Fairchild, MD.: Color appearance models. Wiley (2005)
  16.Gembris, D., Neeb, M., Gipp, M., Kugel, A., Manner, R.: Correlation analysis on gpu systems using nvidia鈥檚 cuda. J. Real Time Image Proc., pp. 275鈥?80 (2011)
  17.G贸mez-Romero, J., Serrano, M.A., Patricio, M.A., Garc铆a, J., Molina, J.M.: Context-based scene recognition from visual data in smart homes: an information fusion approach. Pers. Ubiquitous Comput. 16(7), 835鈥?57 (2012)CrossRef
  18.Guo, X., Tiller, D.K., Henze, G.P., Waters, C.E.: The performance of occupancy-based lighting control systems: a review. Light. Res. Technol. 42, 415鈥?31 (2010)CrossRef
  19.Happe, M., Lubbers, E., Platzner, M.: A self-adaptive heterogeneous multi-core architecture for embedded real-time video object tracking. J. Real Time Image Proc. 8, 95鈥?10 (2013)CrossRef
  20.Hedau, V., Hoiem, D., Forsyth, D.: Recovering the spatial layout of cluttered rooms. In: Proceedings of the IEEE International Conference on Computer Vision (2009)
  21.Hern谩ndez, J., Montemayor, A.S., Pantrigo, J.J., S谩nchez, 脕.: Human action recognition based on tracking features. In IWINAC, Part I, LNCS 6686, pp. 471鈥?80 (2011)
  22.Javed, O., Rasheed, Z., Shafique, K., Shah, M.: Tracking across multiple cameras with disjoint views. Proc. IEEE Int. Conf. Comput. Vis. 2, 952鈥?57 (2003)CrossRef
  23.Javed,O., Shafique, K., Shah, M.: Appearance modeling for tracking in multiple non-overlapping cameras. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR鈥?5) -Vol 2-Vol 02, pp. 26鈥?3 (2005)
  24.Ji, X., Liu, H.: Advances in view-invariant human motion analysis: a review. IEEE Trans. Syst. Man Cybern. Part C, 40, 13鈥?4 (2010)
  25.Karpinsky, N., Zhang, S.: High-resolution, real-time 3d imaging with fringe analysis. J. Real Time Image Proc. 7, 55鈥?6 (2012)
  26.Ke, S.-R., Le Uyen Thuc, H., Lee, Y.-J., Huang, J.-N., Yoo, J.-H., Choi, K.-H.: A review on video-based human activity recognition. Computers, 2:88鈥?31 (2013)CrossRef
  27.Kulkarni,P., Ganesan, D., Shenoy, P., Lu, Q.: Senseye: a multi-tier camera sensor network. In: Proceedings of the 13th annual ACM International Conference on Multimedia, pp. 229鈥?38 (2005)
  28.Lee, C.-S., Chun, S., Shin, J.: The effect of led lighting color temperature and illumination control on attention and relaxation level. In: Proceedings of CJK Lighting Conference, pp. 21鈥?2 (2012)
  29.Lee, H., Wu, C., Aghajan, H.: Vision-based user-centric light control for smart environments. Pervasive Mob. Comput. 7, 223鈥?40 (2011)CrossRef
  30.Lee,J., Choi, K., Suk, H.-J.: Led lighting for the educational environment: focusing on math and multimedia based tasks for 4th grade elementary school students in South Korea. Proc. AIC, 4, 1569鈥?572 (2013)
  31.Jin Lee, S., Shah, G., Bhattacharya, A., Motai, Y.: Human tracking with an infrared camera using a curve matching framework. EURASIP J. Adv. Signal Process. 2012, 1鈥?5 (2012)
  32.Madden, C., Dahai Cheng, E., Piccardi, M.: Tracking people across disjoint camera views by an illumination-tolerant appearance representation. Mach. Vis. Appl. 18, 233鈥?47 (2007)MATH CrossRef
  33.Mahotra, S., Patlolla, C., Kehtarnavaz, N.: Real-time computation of disparity for hand-pair gesture recognition using a stereo webcam. J. Real Time Image Proc. 7, 257鈥?66 (2012)
  34.Mallot, H.A., B眉lthoff, H.H., Little, J.I., Bohrer, S.: Inverse perspective mapping simplifies optical flow computation and obstacle detection. Biol. Cybern. 64, 177鈥?85 (1991)MATH CrossRef
  35.Marsi, S., Saponara, S.: Integrated video motion estimator with retinex-like pre-processing for robust motion analysis in automotive scenarios: algorithmic and real-time architecture design. J. Real-Time Image Proc. 5, 275鈥?89 (2010)
  36.Meyn, S.P., Surana, A., Lin, Y., Maris Oggianu, S., Narayanan, S., Frewen, T.A.: A sensor-utility-network method for estimation of occupancy in buildings. In: Proceedings of IEEE Conference on Decision and Control (CDC), pp. 1494鈥?500 (2009)
  37.Mogelmose, A., Bahnsen, C., Moeslund, T.B.: Tri-modal person re-identification with rgb, depth and thermal features. In: Proceedings of IEEE Workshop on Perception Beyond Visible, Spectrum, pp. 301鈥?07 (2013)
  38.Newsham, G.R., Arsenault, C.: A camera as a sensor for lighting and shading control. Light. Res. Technol. 41, 143鈥?63 (2009)CrossRef
  39.Oi, N., Takahashi, H.: Preferred combinations between illuminance and color temperature in several settings for daily living activities, pp. 214鈥?15. In Proc. of International Symposium on Design of Artificial, Environments (2007)
  40.Orejfej, O., Liu, Z.: Hon4d: Histogram of oriented 4d normals for activity recognition from depth sequences. In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition (2013)
  41.Paradiso, J.A., Aldich, M., ZHao, N.: Energy-efficient control of solid-state lighting. SPIE Newsroom (2011)
  42.Park, H., Burke, J., Srivastava, M.B.: Design and implementation of a wireless sensor network for intelligent light control. In: Proceedings of the international conference on Information processing in sensor networks, IPSN 鈥?7, pp. 370鈥?79 (2007)
  43.Petrosino, A., Miralto, M., Ferone, A.: A real-time streaming server in the RTlinux environment using VideoLanClient. J. Real-Time Image Proc. 6, 247鈥?56 (2011)
  44.Rambabu, C., Kim, K., Woo, W.: Fast and accurate extraction of moving object silhouette for personalized virtual studio home. J. Real-Time Image Proc. 4, 317鈥?28 (2009)
  45.Rauter, M.: Reliable human detection and tracking in top-view depth images. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition, Workshop, pp. 529鈥?34 (2013)
  46.S谩nchez-Oro, J., Fern谩ndez-L贸pez, D., Cabido, R.: AS. Montemayor, J. Jos茅 Pantrigo. Radar-based road-traffic monitoring in urban environments. Digit. Signal Process. 23(1), 364鈥?74 (2013)MathSciNet CrossRef
  47.Carlos Santos, P., Stork, A., Buaes, A., Eduardo Pereira, C., Jorge, J.: A real-time low-cost marker-based multiple camera tracking solution for virtual reality applications. J. Real-Time Image Proc. 5, 121鈥?28 (2010)
  48.Fred Schubert, E.: Lighting-emitting diodes. Cambridge University Press (2006)
  49.Singhvi, V., Krause, A., Guestrin, C., Garrett, Jr., J.H., Scott Matthews, H.: Intelligent light control using sensor networks. In: Proceedings of the international conference on Embedded networked sensor systems, pp. 218鈥?29. ACM (2005)
  50.Alan, N., Smith.: Lighting. International labor, Organization (2011)
  51.Song, B., Roy-Chowdhury, A.K.: Robust tracking in a camera network: a multi-objective optimization framework. IEEE J. Selected Top.Signal Process. 2(4), 582鈥?96 (2008)CrossRef
  52.Tan, S., Dale, J., Anderson, A., Johnston, A.: Inverse perspective projection and optical flow: a calibration method and a quantitative analysis. Image Vis. Comput. 24, 153鈥?65 (2006)CrossRef
  53.Vacchetti, L., Lepetit, V., Fua, P.: Fusion online and offline information for stable 3d tracking in real-time. Proc. IEEE Conf. Comput. Vis. Pattern Recognit. 2, 241鈥?48 (2003)
  54.Vaisenberg, R., Mehrotra, S., Ramanan, D.: Semartcam scheduler: semantics driven real-time data collection from indoor camera networks to maximize event detection. J. Real-Time Image Proc. 5, 215鈥?30 (2010)
  55.Valberg, A.: Light vision color. Wiley, West Sussex (2005)
  56.Wachs, J.P., K枚lsch, M., Goshorn, D.: Human posture recognition for intelligent vehicles. J. Real-Time Image Proc. 5, 231鈥?44 (2010)
  57.Wang, J., Liu, Z., Chorowski, J., Chen, Z., Wu, Y.: Robust 3d action recognition with random occupancy patterns. Proc. ECCV Part II LNCS 7573, 872鈥?85 (2012)
  58.Wang, J., Liu, Z., Wu, Y., Yuan, J.: Mining actionlet ensemble for action recognition with depth cameras. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition, 1290鈥?297 (2012)
  59.Wen, Y.-J., Bonnell, J., Agogino, A.M.: Energy conservation utilizing wireless dimmable lighting control in a shared-space office. In Proc. of the Annual Conference of the Illuminating Engineering Society (2008)
  60.Wojnicki, I., Ernst, S., Kotulski, L., Sedziwy, A.: Advanced street lighting control. Expert Syst. Appl. 41(4), 999鈥?005 (2014)CrossRef
  61.Xia, L., Aggarwal, J.K.: Spatio-temporal depth cuboid similarity feature for activity recognition using depth camera. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2013)
  62.Yoon, S., Oh, H., Lee, D., Oh, S.: PASU: a personal area situation understanding system using depth camera sensor networks. Pers. Ubiquitous Comput. 17(4), 713鈥?27 (2013)CrossRef
  63.Zhang, J., Kan, C., Schwing, A.G., Urtasun, R.: Estimating the 3d layout of indoor scenes and its clutter from depth sensors. In: Proceedings of IEEE International Conference on Computer Vision (2013)
  64.Zhu, Y., Guo, G.: A study on visible to infrared action recognition. IEEE Signal Process. Lett. 20(9), 897鈥?00 (2013)
  
• 作者单位：SungYong Chun (1)
  Chan-Su Lee (1)
  Ja-Soon Jang (1)
  
  1. Yeungnam University, 214-1 Dae-dong, 712-749, Gyeongsan-si, Gyeongsangbuk-do, Rep. of Korea
  
• 刊物类别：Computer Science
• 刊物主题：Image Processing and Computer Vision
  Multimedia Information Systems
  Computer Graphics
  Pattern Recognition
  Signal,Image and Speech Processing
  
• 出版者：Springer Berlin / Heidelberg
• ISSN：1861-8219

文摘

A smart lighting system provides automatic control of lighting illumination and color temperature for high quality of life as well as energy savings in smart cities. A real-time activity understanding system with accurate human location estimation under varying illumination is required for smart lighting control, since comfortable lighting conditions vary according to human activities. This paper presents a real-time smart lighting control system using human location estimation based on inverse-perspective mapping of depth map images and activity estimation from location, heading direction, and height estimation of the moving person from multiple depth cameras. Lighting control based on estimated proximity to the specific activity area, distance to the target lighting area, and heading direction of the person provides an automatic activity-dependent lighting environment as well as energy savings. We implemented several activity modes such as study mode, dialog mode, and watching TV mode, and applied the proposed lighting control system to a living room lighting control with known furniture, electronics, and lighting locations using multiple Kinect depth cameras. The proposed model is based on localized proximity-based lighting control and can be extended to a more general lighting control by combining with global lighting control schemes. Keywords Human motion detection Depth camera Lighting control Human motion tracking Multiple camera Multiple target tracking Smart lighting control