基于数字拼写的视—听联合刺激诱发ERP研究
|
摘要
与感觉、认知密切相关的事件相关电位(Event Related Potential,ERP)是是目前脑-机接口(Brain Computer Interface, BCI)系统中较为常用的提取大脑思维特征信息载体,其中主要内源性成分为P300。但长期以来ERP的可识别性与诱发速度间存在着相互制约的瓶颈,影响着BCI信号特征提取准确率与信息转化速度。为解决这一难题,本文引入了一种新型的视-听联合刺激模式替代传统的单一视觉刺激模式以诱发产生跨感觉通路的ERP信号,并研究了该信号特征,探讨了兼顾提高识别效率和传输速度的可能途径。
本课题在充分调研ERP信号产生和P300成分特点及刺激诱发ERP现状的基础上,设计了基于视-听联合刺激选择数字的跨感觉诱发ERP实验方案,对传统单一视觉诱发P300 speller刺激范式做了实质性的改进,在视觉数字Oddball序列(简化为1～9数字)中加入了相应数字语义的汉语声音刺激以实施跨感觉交叉刺激的脑-机交互任务。本文使用NeuroScan4.3数字脑电采集(64导)系统分别进行了经典的单一视觉刺激范式和上述视-听联合刺激范式的两组实验,并对其诱发产生ERP信号和P300成分特征与识别方法做了较细致的比较分析。研究中首先将经实验得到的脑电信号数据采用神经电生理学常规分析方法进行处理。结果显示,视-听联合刺激范式诱发的P300成份较单一视觉刺激范式具有更高的波幅和更短的潜伏期,与文献报道吻合,表明跨感觉通路刺激模式更适于高速脑-机信息交互。本文进而采用新的约束独立分量分析(constrained Independent Component Analysis, cICA)方法对视-听交叉刺激诱发的ERP信号进行了特征提取,获得了与实验前预设参考信号关系最密切的非高斯分量特征;其后以部分导联的P300降采样波形为基本特征,再结合前述cICA提取的分量特征,最后采用支持向量机分类算法进行了脑-机交互任务识别。对多次重复刺激情况下的靶刺激进行识别时以集成判别函数的多重输出作为最终判决依据。结果表明,视-听联合刺激较单一视觉刺激诱发的ERP信号特征更强,不但有更好的识别效率,且可在较高识别率情况下降低所需重复刺激次数以缓解识别效率和传输速度相互制约的困难;采用跨感觉通路联合刺激作为脑-机交互范式有利于摆脱单一通路模式的困境、有效地提高脑-机信息交互能力,值得进一步深入研究。
Event Related Potential (ERP) which closely correlated with the sense perceptive cognitions is an important neurophysiology paradigm for Brain Computer Interface (BCI). And in ERP, the P300 component is one of the most widely used endogenous components for BCI studies. Unfortunately, there is a difficulty of restricting condition each other between ERP recognition and evoking rate, which affects the accuracy of ERP feature extraction and its information transmitting rate badly. To try to improve the rate of ERP information transmitting and the accuracy of ERP pattern classification simultaneously, in this thesis, the visual-auditory evoking model was used instead of visual evoking model and the features of the signals were analyzed.
Based on thorough investigation about the generation of ERP、the feature of P300 components and the present situation of ERP stimulation, the traditional P300 speller stimulating pattern was improved in this thesis, which was a cross model evoking experiment for brain computer interface providing corresponding semantic auditory stimulating simultaneously when visual stimulation (number 1-9) is proved. The NeuroScan 4.3 digital sampled system was used to acquire the signals we need. The subjects were divided into two groups, the first group used the traditional P300 speller pattern, while, the second group used the new visual-auditory stimulation pattern we designed.
The EEG signals acquired in this system were analyzed by conventional neurophysiological analysis. Results showed that the P300 components in visual-auditory cross model processing had higher amplitudes and shorter latency compared with unimodal visual evoking, which was consistent with existing research and proves that the cross model pattern was more suitable for high-speed brain machine interface. The constrained Independent Component Analysis (cICA)was used in feature extracting to extract the most non-Gaussian components which was closely related to the reference signal. Then the resampled P300 features of some important channels combined with the independent component extracted by cICA were used as the feature vector for recognition tasks. The multiple output decision values of support vector machine were applied to build the classification algorithm, which was used to identify the target number.
Results showed that visual-auditory cross stimulation can evoke better EEG feature, which can lead to a higher classification rate and lower the number of the repeated trials. Thus, the accuracy of identify the target number and its information transmitting rate can be improved simultaneously. Studies showed that using the visual-auditory cross stimulation as the model of the brain machine interface can solve the difficulty existed in unimodal stimulation and effectively improve the ability of brain machine interface.
本课题在充分调研ERP信号产生和P300成分特点及刺激诱发ERP现状的基础上,设计了基于视-听联合刺激选择数字的跨感觉诱发ERP实验方案,对传统单一视觉诱发P300 speller刺激范式做了实质性的改进,在视觉数字Oddball序列(简化为1～9数字)中加入了相应数字语义的汉语声音刺激以实施跨感觉交叉刺激的脑-机交互任务。本文使用NeuroScan4.3数字脑电采集(64导)系统分别进行了经典的单一视觉刺激范式和上述视-听联合刺激范式的两组实验,并对其诱发产生ERP信号和P300成分特征与识别方法做了较细致的比较分析。研究中首先将经实验得到的脑电信号数据采用神经电生理学常规分析方法进行处理。结果显示,视-听联合刺激范式诱发的P300成份较单一视觉刺激范式具有更高的波幅和更短的潜伏期,与文献报道吻合,表明跨感觉通路刺激模式更适于高速脑-机信息交互。本文进而采用新的约束独立分量分析(constrained Independent Component Analysis, cICA)方法对视-听交叉刺激诱发的ERP信号进行了特征提取,获得了与实验前预设参考信号关系最密切的非高斯分量特征;其后以部分导联的P300降采样波形为基本特征,再结合前述cICA提取的分量特征,最后采用支持向量机分类算法进行了脑-机交互任务识别。对多次重复刺激情况下的靶刺激进行识别时以集成判别函数的多重输出作为最终判决依据。结果表明,视-听联合刺激较单一视觉刺激诱发的ERP信号特征更强,不但有更好的识别效率,且可在较高识别率情况下降低所需重复刺激次数以缓解识别效率和传输速度相互制约的困难;采用跨感觉通路联合刺激作为脑-机交互范式有利于摆脱单一通路模式的困境、有效地提高脑-机信息交互能力,值得进一步深入研究。
Event Related Potential (ERP) which closely correlated with the sense perceptive cognitions is an important neurophysiology paradigm for Brain Computer Interface (BCI). And in ERP, the P300 component is one of the most widely used endogenous components for BCI studies. Unfortunately, there is a difficulty of restricting condition each other between ERP recognition and evoking rate, which affects the accuracy of ERP feature extraction and its information transmitting rate badly. To try to improve the rate of ERP information transmitting and the accuracy of ERP pattern classification simultaneously, in this thesis, the visual-auditory evoking model was used instead of visual evoking model and the features of the signals were analyzed.
Based on thorough investigation about the generation of ERP、the feature of P300 components and the present situation of ERP stimulation, the traditional P300 speller stimulating pattern was improved in this thesis, which was a cross model evoking experiment for brain computer interface providing corresponding semantic auditory stimulating simultaneously when visual stimulation (number 1-9) is proved. The NeuroScan 4.3 digital sampled system was used to acquire the signals we need. The subjects were divided into two groups, the first group used the traditional P300 speller pattern, while, the second group used the new visual-auditory stimulation pattern we designed.
The EEG signals acquired in this system were analyzed by conventional neurophysiological analysis. Results showed that the P300 components in visual-auditory cross model processing had higher amplitudes and shorter latency compared with unimodal visual evoking, which was consistent with existing research and proves that the cross model pattern was more suitable for high-speed brain machine interface. The constrained Independent Component Analysis (cICA)was used in feature extracting to extract the most non-Gaussian components which was closely related to the reference signal. Then the resampled P300 features of some important channels combined with the independent component extracted by cICA were used as the feature vector for recognition tasks. The multiple output decision values of support vector machine were applied to build the classification algorithm, which was used to identify the target number.
Results showed that visual-auditory cross stimulation can evoke better EEG feature, which can lead to a higher classification rate and lower the number of the repeated trials. Thus, the accuracy of identify the target number and its information transmitting rate can be improved simultaneously. Studies showed that using the visual-auditory cross stimulation as the model of the brain machine interface can solve the difficulty existed in unimodal stimulation and effectively improve the ability of brain machine interface.
引文
[1] Wolpaw JR, Birbaumer N, Heetderks WJ, et al., Brain-computer interface technology: a review of the first international meeting, IEEE Trans, Rehabil, Eng., 2000, 8(2): 222~225
[2]万柏坤,高扬,赵丽等,脑-机接口:大脑对外信息交流的新途径,国外医学·生物医学工程分册,2005,28(1):4~9
[3] Luck SJ. An Introduction to the Event-Related Potential Technique. The MIT Press, 2005
[4]何庆华,吴宝明,王禾等,不同视觉刺激模式对视觉诱发电位的影响,中国康复医学杂志,2007,22(2):144~146
[5]刘忠国,张晓静,刘伯强等,视觉刺激间隔对大脑诱发电位的影响,山东大学学报(工学版),2006,36(3):34~38
[6]欧阳天斌,听觉事件相关电位测试技术的进展,听力学及言语疾病杂志,2007,15(2):167~170
[7] Walhovd KB, Fjell AM. Two-and three-stimuli auditory oddball ERP tasks and neuropsychological measures in aging. Neuroreport, 2001, 12(14): 3149~53
[8]张晓君,高玉红,赵毅等,三音听觉刺激事件相关电位P300的初步研究,临床神经电生理学杂志,2001,10(2):71~75
[9]胡晓晴,唐娜(译者),体感诱发电位的基本原理,国外医学(物理医学与康复学分册),2005,25(2):53~55
[10]俞黎平,多感觉整合效应及感觉皮层的跨模式可塑性,生物学教学,2006,31(8):7~9
[11] Guo J, Guo A. Cross-modal interactions between olfactory and visual learning in Drosophila. Science, 2005, 309 (5732 ): 307~310
[12]李凌,尧德中,刘铁军等,刺激前后脑电α波相位重排现象研究,电子科技大学学报,2006,35(1):118~121
[13] Lindsley DB. Psychological phenomena and the electroencephalogram. Electroencephalography and clinical neurophysiology, 1952, 4(4): 443~456
[14]王明时,刘瑾,诸强等,在单一模式和双模式刺激下gamma节律与ERP的同步效应,生物医学工程学杂志,2005,22 (5):884~889
[15]赵丽娜,周群,刘铁军等,视觉听觉同时刺激模式下ERP的同步性研究,临床神经电生理学杂志,2005,14(1):4~6
[16]赵晨,杨华海,张侃,跨通道的内源性选择注意,心理学报,1999,31(2): 148~153
[17] Shams L, Kamitani Y, Shimojo S, Illusions: What you see is what you hear, Nature, 2000, 408(6814), 788.
[18] Zampini M, et al., Auditory–somatosensory multisensory interactions in front and rear space. Neuropsychologia, 2007, 45(8): 1869~1877
[19] Ana TJ, Norimichi K, et al., Auditory-somatosensory multisensory interactions are spatially modulated by stimulated body surface and acoustic spectra. Neuropsychologia, 2009, 47 (1): 195~203
[20]吴健辉,罗跃嘉,盲人的跨感觉通道重组,心理科学进展,2005,13(4):406~412
[21] Driver J, Noesselt T. Multisensory interplay reveals crossmodal influences on 'sensory-specific' brain regions, neural responses, and judgments. Neuron 2008, 57(1): 11~23
[22] Wolpaw A, Schloegl C, Neuper G, et al., The BCI competition 2003: Progress and perspectives in detection and discrimination of EEG single trials. IEEE Transactions on biomedical engineering, 2004,51(6): 1044~1051
[23] Blankertz B, Mueller K-R, Krusienski D, et al., The BCI competition iii: Validating alternative approaches to actual BCI problems. IEEE Trans. Neural Syst. Rehabil. Eng., 2006, 14(2): 153~159
[24] Alain R, Vincent G, BCI CompetitionⅢ: DatasetⅡ-Ensemble of SVMs for BCI P300 Speller, IEEE Transactions on biomedical engineering, 2008, 55(3): 1147~1154
[25]赵仑,ERP实验教程,天津:天津社会科学院出版社,2004,26~28
[26]庄天戈,计算机在生物医学中的应用,北京:科学出版社,2000,79~120
[27]马建仓,牛弈龙,陈海洋,盲信号处理,北京:国防工业出版社, 2006
[28] Aapo H, Erkki O, Independent Component Analysis: Algorithms and Applications, Neural Networks, 2000, 13(4-5): 411~430
[29] Lu W, Rajapakse JC, Constrained independent component analysis, Advances in Neural Information Processing Systems, MIT Process, 2000,13: 570~576
[30] James, C.J. and Gibson, O.J, Temporally constrained ICA: an application to artifact rejection in electromagnetic brain signal analysis. IEEE Transactions on Biomedical Engineering, 2003, 50(9),: 1108~1116
[31] Lu W, Rajapakse JC, Approach and applications of constrained ICA, IEEE Trans Neural Netw, 2005, 16(1): 203-12
[32] Krusienski DJ, Sellers EW, McFarland DJ, et al., Toward enhanced P300 speller performance, Journal of Neuroscience Methods, 2008, 167(1), 15~21
[33] Nello C, John ST原著,支持向量机导论(李国正,王猛,曾华军翻译),北京:电子工业出版社,2004,1~150
[34] Vapnik VN原著,统计学习理论(许建华,张学工翻译),北京:电子工业出版社,2004,1~420
[35] Joachims T, Text categorization with SVM: learning with many relevant features, Proceeding of ECML-98, 10th European Conference on Machine Learning, 1998: 137~142
[36] Brown M, Grundy W, Lin D, et al., Knowledge-based analysis of microarray gene expression data using support vector machines, Technical report, University of Californian in Santa Cruz, 2000, 97(1): 262~267
[37] Vikramjit M, Chia-Jiu W, Satarupa B, Text classification: A least square support vector machine approach, Applied Soft Computering, 2007, 7(3): 908~914
[38] Ying W, Yue L, Pengfei S, Handwritten Bangla numeral recognition system and its application to postal automation, Pattern Recognition, 2007, 40(1): 99~107
[39] Vapnik VN, The Nature of Statistical Learning Theory, New York, Springer-Verlag, 1995
[40] Vapnik VN, Levin E, LeCun Y, Measuring the VC-dimension of a learning machine, Neural Computation, 1994, 6(5): 851~876
[41] Vapnik VN, Statistical Learning Theory, John Wiley & Sons, Inc, 1998
[42]王振,基于SVM的BCI系统中P300信号识别技术研究:[硕士学位论文],天津;天津大学,2007
[43]边肇祺,张学工,模式识别(第二版),北京:清华大学出版社,2000
[44]周仲兴,复合下肢想象动作电位的特征识别新技术研究:[博士学位论文],天津;天津大学,2009
[2]万柏坤,高扬,赵丽等,脑-机接口:大脑对外信息交流的新途径,国外医学·生物医学工程分册,2005,28(1):4~9
[3] Luck SJ. An Introduction to the Event-Related Potential Technique. The MIT Press, 2005
[4]何庆华,吴宝明,王禾等,不同视觉刺激模式对视觉诱发电位的影响,中国康复医学杂志,2007,22(2):144~146
[5]刘忠国,张晓静,刘伯强等,视觉刺激间隔对大脑诱发电位的影响,山东大学学报(工学版),2006,36(3):34~38
[6]欧阳天斌,听觉事件相关电位测试技术的进展,听力学及言语疾病杂志,2007,15(2):167~170
[7] Walhovd KB, Fjell AM. Two-and three-stimuli auditory oddball ERP tasks and neuropsychological measures in aging. Neuroreport, 2001, 12(14): 3149~53
[8]张晓君,高玉红,赵毅等,三音听觉刺激事件相关电位P300的初步研究,临床神经电生理学杂志,2001,10(2):71~75
[9]胡晓晴,唐娜(译者),体感诱发电位的基本原理,国外医学(物理医学与康复学分册),2005,25(2):53~55
[10]俞黎平,多感觉整合效应及感觉皮层的跨模式可塑性,生物学教学,2006,31(8):7~9
[11] Guo J, Guo A. Cross-modal interactions between olfactory and visual learning in Drosophila. Science, 2005, 309 (5732 ): 307~310
[12]李凌,尧德中,刘铁军等,刺激前后脑电α波相位重排现象研究,电子科技大学学报,2006,35(1):118~121
[13] Lindsley DB. Psychological phenomena and the electroencephalogram. Electroencephalography and clinical neurophysiology, 1952, 4(4): 443~456
[14]王明时,刘瑾,诸强等,在单一模式和双模式刺激下gamma节律与ERP的同步效应,生物医学工程学杂志,2005,22 (5):884~889
[15]赵丽娜,周群,刘铁军等,视觉听觉同时刺激模式下ERP的同步性研究,临床神经电生理学杂志,2005,14(1):4~6
[16]赵晨,杨华海,张侃,跨通道的内源性选择注意,心理学报,1999,31(2): 148~153
[17] Shams L, Kamitani Y, Shimojo S, Illusions: What you see is what you hear, Nature, 2000, 408(6814), 788.
[18] Zampini M, et al., Auditory–somatosensory multisensory interactions in front and rear space. Neuropsychologia, 2007, 45(8): 1869~1877
[19] Ana TJ, Norimichi K, et al., Auditory-somatosensory multisensory interactions are spatially modulated by stimulated body surface and acoustic spectra. Neuropsychologia, 2009, 47 (1): 195~203
[20]吴健辉,罗跃嘉,盲人的跨感觉通道重组,心理科学进展,2005,13(4):406~412
[21] Driver J, Noesselt T. Multisensory interplay reveals crossmodal influences on 'sensory-specific' brain regions, neural responses, and judgments. Neuron 2008, 57(1): 11~23
[22] Wolpaw A, Schloegl C, Neuper G, et al., The BCI competition 2003: Progress and perspectives in detection and discrimination of EEG single trials. IEEE Transactions on biomedical engineering, 2004,51(6): 1044~1051
[23] Blankertz B, Mueller K-R, Krusienski D, et al., The BCI competition iii: Validating alternative approaches to actual BCI problems. IEEE Trans. Neural Syst. Rehabil. Eng., 2006, 14(2): 153~159
[24] Alain R, Vincent G, BCI CompetitionⅢ: DatasetⅡ-Ensemble of SVMs for BCI P300 Speller, IEEE Transactions on biomedical engineering, 2008, 55(3): 1147~1154
[25]赵仑,ERP实验教程,天津:天津社会科学院出版社,2004,26~28
[26]庄天戈,计算机在生物医学中的应用,北京:科学出版社,2000,79~120
[27]马建仓,牛弈龙,陈海洋,盲信号处理,北京:国防工业出版社, 2006
[28] Aapo H, Erkki O, Independent Component Analysis: Algorithms and Applications, Neural Networks, 2000, 13(4-5): 411~430
[29] Lu W, Rajapakse JC, Constrained independent component analysis, Advances in Neural Information Processing Systems, MIT Process, 2000,13: 570~576
[30] James, C.J. and Gibson, O.J, Temporally constrained ICA: an application to artifact rejection in electromagnetic brain signal analysis. IEEE Transactions on Biomedical Engineering, 2003, 50(9),: 1108~1116
[31] Lu W, Rajapakse JC, Approach and applications of constrained ICA, IEEE Trans Neural Netw, 2005, 16(1): 203-12
[32] Krusienski DJ, Sellers EW, McFarland DJ, et al., Toward enhanced P300 speller performance, Journal of Neuroscience Methods, 2008, 167(1), 15~21
[33] Nello C, John ST原著,支持向量机导论(李国正,王猛,曾华军翻译),北京:电子工业出版社,2004,1~150
[34] Vapnik VN原著,统计学习理论(许建华,张学工翻译),北京:电子工业出版社,2004,1~420
[35] Joachims T, Text categorization with SVM: learning with many relevant features, Proceeding of ECML-98, 10th European Conference on Machine Learning, 1998: 137~142
[36] Brown M, Grundy W, Lin D, et al., Knowledge-based analysis of microarray gene expression data using support vector machines, Technical report, University of Californian in Santa Cruz, 2000, 97(1): 262~267
[37] Vikramjit M, Chia-Jiu W, Satarupa B, Text classification: A least square support vector machine approach, Applied Soft Computering, 2007, 7(3): 908~914
[38] Ying W, Yue L, Pengfei S, Handwritten Bangla numeral recognition system and its application to postal automation, Pattern Recognition, 2007, 40(1): 99~107
[39] Vapnik VN, The Nature of Statistical Learning Theory, New York, Springer-Verlag, 1995
[40] Vapnik VN, Levin E, LeCun Y, Measuring the VC-dimension of a learning machine, Neural Computation, 1994, 6(5): 851~876
[41] Vapnik VN, Statistical Learning Theory, John Wiley & Sons, Inc, 1998
[42]王振,基于SVM的BCI系统中P300信号识别技术研究:[硕士学位论文],天津;天津大学,2007
[43]边肇祺,张学工,模式识别(第二版),北京:清华大学出版社,2000
[44]周仲兴,复合下肢想象动作电位的特征识别新技术研究:[博士学位论文],天津;天津大学,2009