眨眼与哈欠提取新方法及其适用性研究
|
摘要
眨眼与哈欠在基于面部视频的成像式光电容积描记(IPPG)技术中通常被视作运动伪迹干扰。区别于传统的去运动伪迹解决思路,将二者视为特定的面部运动特征并设法加以提取,以期用于人机交互技术研究。方法中,依托二阶盲辨识(SOBI)算法,基于实测数据的统计对比,分析面部R/G/B观测信号的运动敏感性,将运动敏感性最佳的R通道信号作为眨眼与哈欠提取的信号载体。实验统计显示,基于R通道信号可以有效地对眨眼与哈欠进行直接提取,其识别率可达93%与95%,且无需借助于高运算复杂性的盲源分离(BSS)/独立分量分析(ICA)算法,在人机交互技术领域具备较好的实用价值。
Blinking and yawn are commonly seen as motion artifact interference in facial video-based Imaging Photoplethysmography(IPPG). Different from traditional ideas for motion artifact attenuation, they are deemed as specific facial motion features and managed to be extracted, for being engaged in Human-Computer Interaction(HCI)technology. On the basis of experimental statistics, this paper analyzes the sensitivity of the R/G/B observed signals to motion artifact relying on the Second-Order Blind Identification(SOBI)algorithm, and selects the R-channel with optimal sensitivity as carrier for extraction of the features. The experimental statistics show that, based on R-channel signals, it can extract blinking and yawn effectively without help of Blind Source Separation(BSS)/Independent Component Analysis(ICA)algorithm which have high computational complexities, and the recognition rates can reach 93% and 95%. So the R-channel-based method has a good practical value in the field of HCI technology.
Blinking and yawn are commonly seen as motion artifact interference in facial video-based Imaging Photoplethysmography(IPPG). Different from traditional ideas for motion artifact attenuation, they are deemed as specific facial motion features and managed to be extracted, for being engaged in Human-Computer Interaction(HCI)technology. On the basis of experimental statistics, this paper analyzes the sensitivity of the R/G/B observed signals to motion artifact relying on the Second-Order Blind Identification(SOBI)algorithm, and selects the R-channel with optimal sensitivity as carrier for extraction of the features. The experimental statistics show that, based on R-channel signals, it can extract blinking and yawn effectively without help of Blind Source Separation(BSS)/Independent Component Analysis(ICA)algorithm which have high computational complexities, and the recognition rates can reach 93% and 95%. So the R-channel-based method has a good practical value in the field of HCI technology.
引文
[1]Poh M Z.Advancements in noncontact,multiparameter physiological measurements using a Webcam[J].IEEE Transactions on Biomedical Engineering,2011,58(1):7-11.
[2]Poh M Z.Non-contact,automated cardiac pulse measurements using video imaging and blind source separation[J].Opt Exp,2010,18(10):10762-10774.
[3]McDuff D,Gontarek S,Picard R.Remote detection of photoplethysmographic systolic and diastolic peaks using a digital camera[J].IEEE Transactions on Biomedical Engineering,2014,61(12):2948-2954.
[4]Wang W.Exploiting spatial redundancy of image sensor for motion robust r PPG[J].IEEE Transactions on Biomedical Engineering,2015,62(2):415-425.
[5]Pauly L,Sankar D.A novel method for eye tracking and blink detection in video frames[C]//Proceeding of IEEE International Conference on Computer Graphics,Vision and Information Security,2015:252-257.
[6]Tidoni E,Gergondet P,Fusco G,et al.The role of audiovisual feedback in a thought-based control of a humanoid robot:A BCI study in healthy and spinal cord injured people[J].IEEE Trans on Neural Syst Rehabil Eng,2016(1).
[7]Udhaya Kumar S,Vinod V M.EOG based wheelchair control for quadriplegics[C]//Proceeding of International Conference on Innovations in Information,Embedded and Communication Systems,2015:1-4.
[8]武照敏.基于眨眼动作的低成本人机交互系统研究[D].武汉:武汉理工大学,2013.
[9]Hershman R,Henik A,Cohen N.A novel blink detection method based on pupillometry noise.[J].Behavior Research Methods,2018,50(1):107-114.
[10]Singh H,Singh J.Real-time eye blink and wink detection for object selection in HCI systems[J].Journal on Multimodal User Interfaces,2018,12(1):55-65.
[11]Han C H,Im C H.Development of an electrooculogrambased human-computer interface using involuntary eye movement by spatially rotating sound for communication of locked-in patients[J].Scientific Reports,2018,8(1):9505.
[12]Memon Q A,Hassan M Y.Detecting computer vision syndrome using eye blink-An experimental evaluation[J].Journal of Physics(Conference Series),2018,1098:012029.
[13]Hu S,Lin Z.Fatigue driving detection based on machine learning and image processing technology[J].Journal of Physics(Conference Series),2018,1069:012179.
[14]Zhang W,Su J.Driver yawning detection based on long short term memory networks[C]//2017 IEEE Symposium Series on Computational Intelligence,2017:1-5.
[15]何盛鸿.基于EEG和EOG的异步人机接口及其应用[D].广州:华南理工大学,2017.
[16]何璇,吴小培,张超,等.基于面部视频的血流信号提取方法的比较及适用性研究[J].生物医学工程学杂志,2017(2):278-289.
[17]路雪,卫兵,吴小培.基于连续眨眼的人_机交互新模式[J].计算机工程与应用,2018,54(20):263-270.
[18]Wei B,He X,Zhang C,et al.Non-contact,synchronous dynamic measurement of respiratory rate and heart rate based on dual sensitive regions[J].Biomedical Engineering Online,2017,16(1):17.
[19]Wei B,Zhang C,Wu X.Comprehensive comparison study on different ICA/BSS methods in IPPG techniques for obtaining high-quality BVP signal[C]//Proceedings of International Conference on Intelligent Information Processing,2016:48.
[20]Bell A,Sejnowski T.An information-maximization approach to blind separation and blind deconvolution[J].Neural Comput,1995,7(6):1129-1159.
[21]Belouchrani A,Abedmeraim K,Cardoso J F,et al.Ablind source separation technique using second-order statistics[J].IEEE Transactions on Signal Processing,1997,45(2):434-444.
[2]Poh M Z.Non-contact,automated cardiac pulse measurements using video imaging and blind source separation[J].Opt Exp,2010,18(10):10762-10774.
[3]McDuff D,Gontarek S,Picard R.Remote detection of photoplethysmographic systolic and diastolic peaks using a digital camera[J].IEEE Transactions on Biomedical Engineering,2014,61(12):2948-2954.
[4]Wang W.Exploiting spatial redundancy of image sensor for motion robust r PPG[J].IEEE Transactions on Biomedical Engineering,2015,62(2):415-425.
[5]Pauly L,Sankar D.A novel method for eye tracking and blink detection in video frames[C]//Proceeding of IEEE International Conference on Computer Graphics,Vision and Information Security,2015:252-257.
[6]Tidoni E,Gergondet P,Fusco G,et al.The role of audiovisual feedback in a thought-based control of a humanoid robot:A BCI study in healthy and spinal cord injured people[J].IEEE Trans on Neural Syst Rehabil Eng,2016(1).
[7]Udhaya Kumar S,Vinod V M.EOG based wheelchair control for quadriplegics[C]//Proceeding of International Conference on Innovations in Information,Embedded and Communication Systems,2015:1-4.
[8]武照敏.基于眨眼动作的低成本人机交互系统研究[D].武汉:武汉理工大学,2013.
[9]Hershman R,Henik A,Cohen N.A novel blink detection method based on pupillometry noise.[J].Behavior Research Methods,2018,50(1):107-114.
[10]Singh H,Singh J.Real-time eye blink and wink detection for object selection in HCI systems[J].Journal on Multimodal User Interfaces,2018,12(1):55-65.
[11]Han C H,Im C H.Development of an electrooculogrambased human-computer interface using involuntary eye movement by spatially rotating sound for communication of locked-in patients[J].Scientific Reports,2018,8(1):9505.
[12]Memon Q A,Hassan M Y.Detecting computer vision syndrome using eye blink-An experimental evaluation[J].Journal of Physics(Conference Series),2018,1098:012029.
[13]Hu S,Lin Z.Fatigue driving detection based on machine learning and image processing technology[J].Journal of Physics(Conference Series),2018,1069:012179.
[14]Zhang W,Su J.Driver yawning detection based on long short term memory networks[C]//2017 IEEE Symposium Series on Computational Intelligence,2017:1-5.
[15]何盛鸿.基于EEG和EOG的异步人机接口及其应用[D].广州:华南理工大学,2017.
[16]何璇,吴小培,张超,等.基于面部视频的血流信号提取方法的比较及适用性研究[J].生物医学工程学杂志,2017(2):278-289.
[17]路雪,卫兵,吴小培.基于连续眨眼的人_机交互新模式[J].计算机工程与应用,2018,54(20):263-270.
[18]Wei B,He X,Zhang C,et al.Non-contact,synchronous dynamic measurement of respiratory rate and heart rate based on dual sensitive regions[J].Biomedical Engineering Online,2017,16(1):17.
[19]Wei B,Zhang C,Wu X.Comprehensive comparison study on different ICA/BSS methods in IPPG techniques for obtaining high-quality BVP signal[C]//Proceedings of International Conference on Intelligent Information Processing,2016:48.
[20]Bell A,Sejnowski T.An information-maximization approach to blind separation and blind deconvolution[J].Neural Comput,1995,7(6):1129-1159.
[21]Belouchrani A,Abedmeraim K,Cardoso J F,et al.Ablind source separation technique using second-order statistics[J].IEEE Transactions on Signal Processing,1997,45(2):434-444.