基于人眼信息特征的人体疲劳检测
|
摘要
人眼状态是反映疲劳程度的重要指标,头部姿势变化、光线等因素对人眼定位造成很大影响,从而影响人眼状态识别以及疲劳检测的准确性,为此提出了一种利用级联卷积神经网络通过检测人眼6个特征点来识别人眼状态进而识别人体疲劳的方法。首先,一级网络采用灰度积分投影结合区域—卷积神经网实现人眼的检测与定位;然后,二级网络将人眼图片进行分割后采用并联子卷积系统进行人眼特征点回归;最后,利用人眼特征点计算人眼开闭度识别当前人眼状态,并根据单位时间闭眼百分比(PERCLOS)准则判断人体疲劳状态。实验结果表明,利用所提方法实现了在归一化误差为0.05时,人眼6特征点的平均检测准确率为95.8%,并根据模拟视频帧的PERCLOS值识别疲劳状态验证了该方法的有效性。
The eye state is an important indicator reflecting the degree of fatigue. Changes in head posture and light have a great influence on human eye positioning, which affects the accuracy of eye state recognition and fatigue detection. A cascade Convolutional Neural Network(CNN) was proposed, by which the human eye state could be identified by detecting six feature points of human eye to identify human body fatigue. Firstly, grayscale integral projection and regional-convolution neural network were used as the first-level network to realize the positioning and detection of human eyes. Then, the secondary network was adopted to divide the human eye image and parallel sub-convolution system was used to perform human eye feature point regression. Finally, human eye feature points were used to calculate the human eye opening and closing degree to identify the current eye state, and the human body fatigue state was judged according to the PERcentage of eyelid CLOSure over the pupil time(PERCLOS) criterion. The experimental results show that the average detection accuracy of six eye feature points reaches 95.8% when the normalization error is 0.05, thus the effectiveness of the proposed method is verified by identifying the fatigue state based on the PERCLOS value of analog video.
The eye state is an important indicator reflecting the degree of fatigue. Changes in head posture and light have a great influence on human eye positioning, which affects the accuracy of eye state recognition and fatigue detection. A cascade Convolutional Neural Network(CNN) was proposed, by which the human eye state could be identified by detecting six feature points of human eye to identify human body fatigue. Firstly, grayscale integral projection and regional-convolution neural network were used as the first-level network to realize the positioning and detection of human eyes. Then, the secondary network was adopted to divide the human eye image and parallel sub-convolution system was used to perform human eye feature point regression. Finally, human eye feature points were used to calculate the human eye opening and closing degree to identify the current eye state, and the human body fatigue state was judged according to the PERcentage of eyelid CLOSure over the pupil time(PERCLOS) criterion. The experimental results show that the average detection accuracy of six eye feature points reaches 95.8% when the normalization error is 0.05, thus the effectiveness of the proposed method is verified by identifying the fatigue state based on the PERCLOS value of analog video.
引文
[1] YOU Z,GAO Y,ZHANG J,et al.A study on driver fatigue recognition based on SVM method [C]// Proceedings of the 2017 4th International Conference on Transportation Information and Safety.Piscataway,NJ:IEEE,2017:693-697.
[2] CHAI R,NAIK G,NGUYEN T N,et al.Driver fatigue classification with independent component by entropy rate bound minimization analysis in an EEG-based system [J].IEEE Journal of Biomedical and Health Informatics,2017,21(3):715-724.
[3] XU J,MIN J,HU J.Real-time eye tracking for the assessment of driver fatigue [J].Healthcare Technology Letters,2018,5(2):54-58.
[4] 唐广发,张会林.人眼疲劳预测技术的研究[J].计算机工程与应用,2016,52(9):213-218.(TANG G F,ZHANG H L.Research on human eye fatigue prediction technology[J].Computer Engineering and Applications,2016,52(9):213-218.)
[5] ZENG S,LI J,JIANG L,et al.A driving assistant safety method based on human eye fatigue detection [C]// Proceedings of the 2017 Control and Decision Conference.Piscataway,NJ:IEEE,2017:6370-6377.
[6] DENG Z,JING R,JIAO L,et al.Fatigue detection based on isophote curve [C]// Proceedings of the 2015 International Conference on Computer and Computational Sciences.Piscataway,NJ:IEEE,2015:146-150.
[7] 李响,谭南林,李国正,等.基于Zernike矩的人眼定位与状态识别[J].电子测量与仪器学报,2015(3):390-398.(LI X,TAN N L,LI G Z,et al.Human eye localization and state recognition based on Zernike moment[J].Journal of Electronic Measurement and Instrumentation,2015(3):390-398)
[8] ARAUJO G M,FML R,JUNIOR W S,et al.Weak classifier for density estimation in eye localization and tracking[J].IEEE Transactions on Image Processing,2017,26(7):3410-3424.
[9] SONG M,TAO D,SUN Z,et al.Visual-context boosting for eye detection[J].IEEE Transactions on Systems,Man and Cybernetics,Part B (Cybernetics),2010,40(6):1460-1467.
[10] LI J,WONG H C,LO S L,et al.Multiple object detection by a deformable part-based model and an R-CNN[J].IEEE Signal Processing Letters,2018,25(2):288-292.
[11] LI J,LIANG X,SHEN S M,et al.Scale-aware fast R-CNN for pedestrian detection [J].IEEE Transactions on Multimedia,2018,20(4):985-996.
[12] ABDULNABI A H,WANG G,LU J,et al.Multi-task CNN model for attribute prediction [J].IEEE Transactions on Multimedia,2015,17(11):1949-1959.
[13] LUO Y,GUAN Y P.Adaptive skin detection using face location and facial structure estimation [J].IET Computer Vision,2017,11(7):550-559.
[14] YANG W,ZHANG Z,ZHANG Y,et al.Real-time digital image stabilization based on regional field image gray projection[J].Journal of Systems Engineering and Electronics,2016,27(1):224-231.
[15] K?NIG D,ADAM M,JARVERS C,et al.Fully convolutional region proposal networks for multispectral person detection[C]// Proceedings of the 2017 Computer Vision and Pattern Recognition Workshops.Washington,DC:IEEE Computer Society,2017:243-250.
[16] FU L,ZHANG J,HUANG K.Mirrored non-maximum suppression for accurate object part localization [C]// Proceedings of the 2015 3rd IAPR Asian Conference on Pattern Recognition.Piscataway,NJ:IEEE,2015:51-55.
[17] MANDAL B,LI L,WANG G S,et al.Towards detection of bus driver fatigue based on robust visual analysis of eye state [J].IEEE Transactions on Intelligent Transportation Systems,2017,18(3):545-557.
[2] CHAI R,NAIK G,NGUYEN T N,et al.Driver fatigue classification with independent component by entropy rate bound minimization analysis in an EEG-based system [J].IEEE Journal of Biomedical and Health Informatics,2017,21(3):715-724.
[3] XU J,MIN J,HU J.Real-time eye tracking for the assessment of driver fatigue [J].Healthcare Technology Letters,2018,5(2):54-58.
[4] 唐广发,张会林.人眼疲劳预测技术的研究[J].计算机工程与应用,2016,52(9):213-218.(TANG G F,ZHANG H L.Research on human eye fatigue prediction technology[J].Computer Engineering and Applications,2016,52(9):213-218.)
[5] ZENG S,LI J,JIANG L,et al.A driving assistant safety method based on human eye fatigue detection [C]// Proceedings of the 2017 Control and Decision Conference.Piscataway,NJ:IEEE,2017:6370-6377.
[6] DENG Z,JING R,JIAO L,et al.Fatigue detection based on isophote curve [C]// Proceedings of the 2015 International Conference on Computer and Computational Sciences.Piscataway,NJ:IEEE,2015:146-150.
[7] 李响,谭南林,李国正,等.基于Zernike矩的人眼定位与状态识别[J].电子测量与仪器学报,2015(3):390-398.(LI X,TAN N L,LI G Z,et al.Human eye localization and state recognition based on Zernike moment[J].Journal of Electronic Measurement and Instrumentation,2015(3):390-398)
[8] ARAUJO G M,FML R,JUNIOR W S,et al.Weak classifier for density estimation in eye localization and tracking[J].IEEE Transactions on Image Processing,2017,26(7):3410-3424.
[9] SONG M,TAO D,SUN Z,et al.Visual-context boosting for eye detection[J].IEEE Transactions on Systems,Man and Cybernetics,Part B (Cybernetics),2010,40(6):1460-1467.
[10] LI J,WONG H C,LO S L,et al.Multiple object detection by a deformable part-based model and an R-CNN[J].IEEE Signal Processing Letters,2018,25(2):288-292.
[11] LI J,LIANG X,SHEN S M,et al.Scale-aware fast R-CNN for pedestrian detection [J].IEEE Transactions on Multimedia,2018,20(4):985-996.
[12] ABDULNABI A H,WANG G,LU J,et al.Multi-task CNN model for attribute prediction [J].IEEE Transactions on Multimedia,2015,17(11):1949-1959.
[13] LUO Y,GUAN Y P.Adaptive skin detection using face location and facial structure estimation [J].IET Computer Vision,2017,11(7):550-559.
[14] YANG W,ZHANG Z,ZHANG Y,et al.Real-time digital image stabilization based on regional field image gray projection[J].Journal of Systems Engineering and Electronics,2016,27(1):224-231.
[15] K?NIG D,ADAM M,JARVERS C,et al.Fully convolutional region proposal networks for multispectral person detection[C]// Proceedings of the 2017 Computer Vision and Pattern Recognition Workshops.Washington,DC:IEEE Computer Society,2017:243-250.
[16] FU L,ZHANG J,HUANG K.Mirrored non-maximum suppression for accurate object part localization [C]// Proceedings of the 2015 3rd IAPR Asian Conference on Pattern Recognition.Piscataway,NJ:IEEE,2015:51-55.
[17] MANDAL B,LI L,WANG G S,et al.Towards detection of bus driver fatigue based on robust visual analysis of eye state [J].IEEE Transactions on Intelligent Transportation Systems,2017,18(3):545-557.