Continuous Vigilance Estimation Using LSTM Neural Networks

详细信息    查看全文

• 关键词：EEG ; Vigilance estimation ; Multimodal ; Deep learning ; Recurrent neural network
• 刊名：Lecture Notes in Computer Science
• 出版年：2016
• 出版时间：2016
• 年：2016
• 卷：9948
• 期：1
• 页码：530-537
• 全文大小：452 KB
• 参考文献：1.Brunner, C., et al.: BNCI horizon 2020 – towards a roadmap for brain/neural computer interaction. In: Stephanidis, C., Antona, M. (eds.) UAHCI 2014, Part I. LNCS, vol. 8513, pp. 475–486. Springer, Heidelberg (2014)
  2.Lu, Y., Zheng, W.-L., Li, B., Lu, B.-L.: Combining eye movements and EEG to enhance emotion recognition. In: IJCAI 2015, pp. 1170–1176 (2015)
  3.Eoh, H.J., Chung, M.K., Kim, S.-H.: Electroencephalographic study of drowsiness in simulated driving with sleep deprivation. Int. J. Ind. Ergon. 35(4), 307–320 (2005)CrossRef
  4.Davidson, P.R., Jones, R.D., Peiris, M.T.R.: EEG-based lapse detection with high temporal resolution. IEEE Trans. Biomed. Eng. 54(5), 832–839 (2007)CrossRef
  5.Krajewski, J., Batliner, A., Golz, M.: Acoustic sleepiness detection: framework and validation of a speech-adapted pattern recognition approach. Behav. Res. Methods 41(3), 795–804 (2009)CrossRef
  6.Khushaba, R.N., Kodagoda, S., Lal, S., Dissanayake, G.: Driver drowsiness classification using fuzzy wavelet-packet-based feature extraction algorithm. IEEE Trans. Biomed. Eng. 58(1), 121–131 (2011)CrossRef
  7.Shi, L.-C., Bao-Liang, L.: EEG-based vigilance estimation using extreme learning machines. Neurocomputing 102, 135–143 (2013)CrossRef
  8.Ma, J.-X., Shi, L.-C., Lu, B.-L.: Vigilance estimation by using electrooculographic features. In: 32nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 6591–6594 (2010)
  9.Zhang, Y.-F., Gao, X.-Y., Zhu, J.-Y., Zheng, W.-L., Lu, B.-L.: A novel approach to driving fatigue detection using forehead EOG. In: 2015 7th International IEEE/EMBS Conference on Neural Engineering, pp. 707–710 (2015)
  10.Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems, pp. 3104–3112 (2014)
  11.Deng, L., Li, J., Huang, J.-T., et al.: Recent advances in deep learning for speech research at Microsoft. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 8604–8608 (2013)
  12.Shi, L.-C., Jiao, Y.-Y., Lu, B.-L.: Differential entropy feature for EEG-based vigilance estimation. In: 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 6627–6630 (2013)
  13.Gao, X.-Y., Zhang, Y.-F., Zheng, W.-L., Lu, B.-L.: Evaluating driving fatigue detection algorithms using eye tracking glasses. In: 2015 7th International IEEE/EMBS Conference on Neural Engineering, pp. 767–770 (2015)
  
• 作者单位：Nan Zhang (19)
  Wei-Long Zheng (19)
  Wei Liu (19)
  Bao-Liang Lu (19) (20) (21)
  
  19. Center for Brain-like Computing and Machine Intelligence, Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, China
  20. Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognition Engineering, Shanghai Jiao Tong University, Shanghai, China
  21. Brain Science and Technology Research Center, Shanghai Jiao Tong University, Shanghai, China
  
• 丛书名：Neural Information Processing
• ISBN：978-3-319-46672-9
• 刊物类别：Computer Science
• 刊物主题：Artificial Intelligence and Robotics
  Computer Communication Networks
  Software Engineering
  Data Encryption
  Database Management
  Computation by Abstract Devices
  Algorithm Analysis and Problem Complexity
  
• 出版者：Springer Berlin / Heidelberg
• ISSN：1611-3349
• 卷排序：9948

文摘

In this paper, we propose a novel continuous vigilance estimation approach using LSTM Neural Networks and combining Electroencephalogram (EEG) and forehead Electrooculogram (EOG) signals. We combine these two modalities to leverage their complementary information using a multimodal deep learning method. Moreover, since the change of vigilance level is a time dependent process, temporal dependency information is explored in this paper, which significantly improves the performance of vigilance estimation. We introduce two LSTM Neural Network architectures, the F-LSTM and the S-LSTM, to encode the time sequences of EEG and EOG into a high level combined representation, from which we can predict the vigilance levels. The experimental results demonstrate that both of the two LSTM multimodal structures can improve the performance of vigilance estimation models in comparison with the single modality models and non-temporal dependent models.