Improving object segmentation by using EEG signals and rapid serial visual presentation

详细信息    查看全文

• 作者：Eva Mohedano ; Graham Healy ; Kevin McGuinness…
• 关键词：Brain ; computer interfaces ; Electroencephalography ; Rapid serial visual presentation ; Object segmentation ; Interactive segmentation ; GrabCut algorithm
• 刊名：Multimedia Tools and Applications
• 出版年：2015
• 出版时间：November 2015
• 年：2015
• 卷：74
• 期：22
• 页码：10137-10159
• 全文大小：3,180 KB
• 参考文献：1.Bauer G, Gerstenbrand F, Rumpl E (1979) Varieties of the locked-in syndrome. J Neurol 221(2):77鈥?1CrossRef
  2.Bell CJ, Shenoy P, Chalodhorn R, Rao R (2008) Control of a humanoid robot by a noninvasive brain computer interface in humans. J Neural Eng 16(5):432鈥?41
  3.Bigdely-Shamlo N, Vankov A, Ramirez R, Makeig S (2008) Brain activity-based image classification from rapid serial visual presentation. IEEE Trans Neural Syst Rehabil Eng 16(5):432鈥?41CrossRef
  4.Bradski G (2000) Dr. Dobb鈥檚 Journal of Software Tools
  5.Cruse D, Chennu S, Chatelle C, Bekinschtein TA, Fern谩ndez-Espejo D, Pickard JD, Laureys S, Owen AM (2012) Bedside detection of awareness in the vegetative state: a cohort study. Lancet 378(9809):2088鈥?094CrossRef
  6.Everingham M, Van Gool L, Williams CKI, Winn J, Zisserman A (2010) The Pascal visual object classes (VOC) challenge. Int J Comput Vis 88(2):303鈥?38CrossRef
  7.Fernandez-Canellas D (2013) Modeling the temporal dependency of brain responses to rapidly presented stimuli in erp based bci. Master鈥檚 thesis, Northeastern University
  8.Healy G, Smeaton A (2011) Eye fixation related potentials in a target search task. In: Engineering in Medicine and Biology Society, EMBC, 2011 Annual International Conference of the IEEE, pp 4203鈥?206
  9.Healy G, Smeaton AF (2011) Optimising the number of channels in eeg-augmented image search. In: Proceedings of the 25th BCS conference on human-computer interaction, BCS-HCI, pp 157鈥?62
  10.Hebbalaguppe R, McGuinness K, Kuklyte J, Healy G, Connor NO, Smeaton A (2013) How Interaction Methods Affect Image Segmentation : User Experience in the Task. In: Proc. The 1st IEEE workshop on user-centred computer vision (UCCV)
  11.Hu X, Li K, Han J, Hua X, Guo L, Liu T (2012) Bridging the semantic gap via functional brain imaging. IEEE Trans Multimed 14(2):314鈥?25CrossRef
  12.Huang Y, Erdogmus D, Pavel M, Mathan S, Hild II KE (2011) A framework for rapid visual image search using single-trial brain evoked responses. Neurocomputing 74(12-13):2041鈥?051CrossRef
  13.Kapoor A, Shenoy P, Tan D (2008) Combining brain computer interfaces with vision for object categorization. In: Computer vision and pattern recognition (CVPR), pp 1鈥?
  14.Luck SJ (2005) An introduction to the event-related potential technique. MIT Press
  15.Margolin R, Zelnik-Manor L, Tal A (2014) How to evaluate foreground maps?. In: CVPR
  16.Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: ICCV, vol 2, pp 416鈥?23
  17.Mohedano E, Healy G, McGuinness K, Gir贸-i Nieto X, O鈥機onnor NE, Smeaton AF (2014) Object segmentation in images using eeg signals. In: Proceedings of the ACM international conference on multimedia, MM鈥?4. ACM, New York, NY, USA, pp 417鈥?26
  18.Motomura S, Ojima Y, Zhong N (2009) Eeg/erp meets act-r: A case study for investigating human computation mechanism. In: Zhong N, Li K, Lu S, Chen L (eds) Brain Informatics, volume 5819 of Lecture Notes in Computer Science, pp 63鈥?3
  19.Pathirage I, Khokar K, Klay E, Alqasemi R, Dubey R (2013) A vision based p300 brain computer interface for grasping using a wheelchair-mounted robotic arm. In: 2013 IEEE/ASME international conference on advanced intelligent mechatronics (AIM), pp 188鈥?93
  20.Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: Machine learning in Python. J Mach Learn Res 12:2825鈥?830MATH MathSciNet
  21.Roark B, Oken B, M F-O, Orhan U, Erdogmus D (2013) Offline analysis of context contribution to erp-based typing bci performance. J Neural Eng 10(6):432鈥?41
  22.Rother C, Kolmogorov V, Blake A (2004) 鈥淕rabCut鈥? Interactive foreground extraction using iterated graph cuts. ACM Trans Graph 23(3):309鈥?14CrossRef
  23.Sajda P, Pohlmeyer E, Wang J, Parra LC, Christoforou C, Dmochowski J, Hanna B, Bahlmann C, Singh MK, Chang S-F (2010) In a blink of an eye and a switch of a transistor: cortically coupled computer vision. Proc IEEE 98(3):462鈥?78CrossRef
  24.Spence R (2002) Rapid, Serial and Visual: a presentation technique with potential. Inf Vis 1(1):13鈥?9CrossRef
  25.Wang J, Pohlmeyer E, Hanna B, Jiang Y-G, Sajda P, Chang S-F (2009) Brain state decoding for rapid image retrieval. In: Proceedings of the 17th ACM international conference on multimedia MM 鈥?9, pp 945鈥?54
  26.Yazdani A, Vesin J-M, Izzo D, Ampatzis C, Ebrahimi T (2010) Implicit retrieval of salient images using brain computer interface. In: ICIP, pp 3169鈥?172
  
• 作者单位：Eva Mohedano (1)
  Graham Healy (1)
  Kevin McGuinness (1)
  Xavier Gir贸-i-Nieto (2)
  Noel E. O鈥機onnor (1)
  Alan F. Smeaton (1)
  
  1. Insight Center for data Analytics, Dublin City University, Dublin, Ireland
  2. Image Processing Group, Universitat Politcnica de Catalunya, Catalunya, Spain
  
• 刊物类别：Computer Science
• 刊物主题：Multimedia Information Systems
  Computer Communication Networks
  Data Structures, Cryptology and Information Theory
  Special Purpose and Application-Based Systems
  
• 出版者：Springer Netherlands
• ISSN：1573-7721

文摘

This paper extends our previous work on the potential of EEG-based brain computer interfaces to segment salient objects in images. The proposed system analyzes the Event Related Potentials (ERP) generated by the rapid serial visual presentation of windows on the image. The detection of the P300 signal allows estimating a saliency map of the image, which is used to seed a semi-supervised object segmentation algorithm. Thanks to the new contributions presented in this work, the average Jaccard index was improved from 0.47 to 0.66 when processed in our publicly available dataset of images, object masks and captured EEG signals. This work also studies alternative architectures to the original one, the impact of object occupation in each image window, and a more robust evaluation based on statistical analysis and a weighted F-score. Keywords Brain-computer interfaces Electroencephalography Rapid serial visual presentation Object segmentation Interactive segmentation GrabCut algorithm