结合EOG和EEG的人机交互系统的研究与实现
摘要
以生物电为信息载体的人机交互(Human-Computer Interaction, HCI)技术,是当前计算机应用和信息处理领域的一个重要研究方向,相关的研究工作具有重要的科学意义和广阔的应用前景。现有的基于生物电信号的人机交互系统一般以人体的某一种生物电信号作为其输入,例如眼电(EOG)信号或者脑电(EEG)信号等。本文设计并实现了结合EOG和EEG的人机交互系统,主要工作如下：
1、从生物电信号的种类、信号的传输方式、信号数据的处理算法等方面分析研究了现有的基于EOG和EEG的人机交互系统的设计原理。本文对EOG和EEG的处理工作包括：依据波形特征从EOG中识别眨眼动作并累积连续眨眼的次数,其中眨眼动作的识别主要通过信号的中分位点,波峰、波谷值以及波峰波谷间距等特征值来确定,是否为一组连续眨眼的判断,主要依据是两次眨眼信号之间的波峰间距；依据快速傅里叶变换从EEG中检测alpha波,主要方法是系统每接收到一秒的数据,就对其进行一次快速傅里叶变换,得到这一秒钟内人脑波频率的峰值,若其介于8-12Hz之间,则判定产生alpha波。
2、设计并实现了结合EOG和EEG的多媒体控制人机交互系统(包括无线蓝牙和有线两种信号传输方式)。具体工作包括：阐述了两个系统的系统流程和功能结构；两个系统的模块都划分为以下三个部分,信号采集与传输模块,生物电信号处理模块和受控命令生成模块；信号采集与传输模块主要阐述了信号采集与传输的流程；生物电信号处理模块主要阐述了对EOG和EEG的分析处理方法；受控命令生成模块主要实现的功能是将生物电信号处理模块所得到的结果转化为控制多媒体播放器的控制命令
本文将眼电信号和脑电信号结合起来作为人机交互系统的输入,使得生物电在人机交互技术中的应用更加合理完善。
Taking bioelectricity as the information carrier, human-computer interaction (HCI) technology is an important research direction in the field of computer application and information processing, related research work has great scientific significance and application prospect. The past HCI systems based on bioelectrical technology mostly rely on a certain kind of bioelectricity as a means of achieving HCI, such as EOG signal or EEG signal. This thesis designed and implemented two HCI systems based on the combination of EOG and EEG. The main work is as follows:
1、 Analysis and research of the design principles of the HCI systems based on EOG and EEG from some aspects like the types of bioelectrical signals, the transmission ways of signals and the processing algorithms of signal data. Processing work to EOG and EEG include:The first, identifying the blinking movements and accumulating the continuous blinking according to the waveform features of EOG, the blinking identification is determined by the half points of the signals peak, peak and trough values, as well as the spaces between peaks and valleys and other characteristic values, we determine whether there exists a set of continuous blinking based primarily on the peak space between the two blinking signals. The second, detecting alpha waves in EEG according to FFT, the main method is once the system received data of one second, it will apply FFT to them, and the alpha waves will be generated if the peak value of brainwave frequency at this second is between8and12.
2、Design and implement the multimedia control system based on the combination of EOG and EEG (including wireless Bluetooth and wired signal transmission modes). The concrete work includes:expounding the system flow and function chat; dividing the two systems'modular into three parts, they are signal acquisition and transmission modular, bioelectrical processing modular and the control commands generated modular; The signal acquisition and transmission modular expounds the working flow of the signal acquisition and transmission; the bioelectrical processing modular expounds the methods to analyze and research the EOG and EEG; the control commands generated modular mainly implement the transformation from the outputs of the biological electricity processing modular into the control commands of multimedia player.
The thesis makes the EOG and EEG signal combined to be the import of HCI systems, so as to makes the use of biological electric more reasonable.
1、从生物电信号的种类、信号的传输方式、信号数据的处理算法等方面分析研究了现有的基于EOG和EEG的人机交互系统的设计原理。本文对EOG和EEG的处理工作包括：依据波形特征从EOG中识别眨眼动作并累积连续眨眼的次数,其中眨眼动作的识别主要通过信号的中分位点,波峰、波谷值以及波峰波谷间距等特征值来确定,是否为一组连续眨眼的判断,主要依据是两次眨眼信号之间的波峰间距；依据快速傅里叶变换从EEG中检测alpha波,主要方法是系统每接收到一秒的数据,就对其进行一次快速傅里叶变换,得到这一秒钟内人脑波频率的峰值,若其介于8-12Hz之间,则判定产生alpha波。
2、设计并实现了结合EOG和EEG的多媒体控制人机交互系统(包括无线蓝牙和有线两种信号传输方式)。具体工作包括：阐述了两个系统的系统流程和功能结构；两个系统的模块都划分为以下三个部分,信号采集与传输模块,生物电信号处理模块和受控命令生成模块；信号采集与传输模块主要阐述了信号采集与传输的流程；生物电信号处理模块主要阐述了对EOG和EEG的分析处理方法；受控命令生成模块主要实现的功能是将生物电信号处理模块所得到的结果转化为控制多媒体播放器的控制命令
本文将眼电信号和脑电信号结合起来作为人机交互系统的输入,使得生物电在人机交互技术中的应用更加合理完善。
Taking bioelectricity as the information carrier, human-computer interaction (HCI) technology is an important research direction in the field of computer application and information processing, related research work has great scientific significance and application prospect. The past HCI systems based on bioelectrical technology mostly rely on a certain kind of bioelectricity as a means of achieving HCI, such as EOG signal or EEG signal. This thesis designed and implemented two HCI systems based on the combination of EOG and EEG. The main work is as follows:
1、 Analysis and research of the design principles of the HCI systems based on EOG and EEG from some aspects like the types of bioelectrical signals, the transmission ways of signals and the processing algorithms of signal data. Processing work to EOG and EEG include:The first, identifying the blinking movements and accumulating the continuous blinking according to the waveform features of EOG, the blinking identification is determined by the half points of the signals peak, peak and trough values, as well as the spaces between peaks and valleys and other characteristic values, we determine whether there exists a set of continuous blinking based primarily on the peak space between the two blinking signals. The second, detecting alpha waves in EEG according to FFT, the main method is once the system received data of one second, it will apply FFT to them, and the alpha waves will be generated if the peak value of brainwave frequency at this second is between8and12.
2、Design and implement the multimedia control system based on the combination of EOG and EEG (including wireless Bluetooth and wired signal transmission modes). The concrete work includes:expounding the system flow and function chat; dividing the two systems'modular into three parts, they are signal acquisition and transmission modular, bioelectrical processing modular and the control commands generated modular; The signal acquisition and transmission modular expounds the working flow of the signal acquisition and transmission; the bioelectrical processing modular expounds the methods to analyze and research the EOG and EEG; the control commands generated modular mainly implement the transformation from the outputs of the biological electricity processing modular into the control commands of multimedia player.
The thesis makes the EOG and EEG signal combined to be the import of HCI systems, so as to makes the use of biological electric more reasonable.
引文
[1]董士海,王衡.人机交互[M].北京,北京大学出版社,2004.
[2]柳忠起,袁修干,刘伟,王睿.在模拟飞机降落过程中的眼动分析[J].北京航空航天大学学报,2002,28(6)：703-706
[3]赵翔.智能轮椅的BCI人—机接口技术研究[D].中国优秀硕士学位论文全文数据库,2008,(11).
[4]蔡利栋,方思行,周继鹏,张庆丰.人机交互(第三版)[M].北京,电子工业出版社,2006.
[5]张娴.基于肌电信号的人机交互技术研究[D].南京：东南大学,2006
[6]Kenji Y, Junichi H, Michio M. Development of EOG-Based Communication System Controlled by Eight-Directional Eye Movements[C]. Engineering in Medicine and Biology Society, EMBS'06,28th Annual International Conference of the IEEE,2006:2574～2577.
[7]赵予晗.基于脑电信号的意识任务的识别方法研究[D].甘肃：兰州理工大学,2005.
[8]Li Lian, Wu XiaoPei. Design and implementation of multimedia control system based bluetooth and electrooculogram (EOG)[C].5th International Conference on Bioinformatics and Biomedical Engineering, iCBBE 2011.
[9]李听.基于眼电的无障碍人机交互技术研究[D].杭州：浙江大学,2010.
[10]张磊.基于脑电信号的脑-机接口研究[D].合肥：安徽大学,2009.
[11]黄漫玲,吴平东,殷罡,刘莹。基于稳态视觉诱发电位的脑机接口实验研究[J].北京理工大学学报,2008,(11).
[12]S. Venkataramanan, Pranay Prabhat, Shubhodeep Roy Choudhury, Harshal B. Nemade, J.S. Sahambi, Biomedical Instrumentation based on EOG Signal Processing and Application to a Hospital Alarm System[C], proc.of IEEE ICISIP 2005:535～540, Chennai, India.
[13]Kenji Y, Junichi H, Michio M. Development of EOG-Based Communication System Controlled by Eight-Directional Eye Movements[C]. Engineering in Medicine and Biology Society, EMBS'06,28th Annual International Conference of the IEEE,2006:2574-2577.
[14]张莉,何传红,何为.典型相关分析去除脑电信号中眼电伪迹的研究[J].计算机工程与应用,2009,45(31)：218-220.
[15]张霖,王行愚,姚晓东,邹俊忠.眼球运动相关EOG信号的波形分析与特征提取[J].上海生物医学工程,2003,24(3)：19-22.
[16]钱志鸿,杨帆,周求湛.蓝牙技术原理、开发与应用[M].北京,北京航空航天大学出版社,2006.
[17]朱刚,谈振辉,周贤伟.蓝牙技术原理与协议[M].北京,清华大学出版社,2002.
[18]Fatourechi M, Bashashati A, Ward R K, et al. Birch. EMG and EOG artifacts in brain computer interface systems:A survey[J]. Clinical Neurophysiology, 2007,118:480～494.
[19]G. Norris, E. Wilson, The eye mouse, an eye communication device[C], in Trans. 1997 IEEE SMC.
[20]Barea R, Boquete L, Mazo M, et al. System for assisted mobility using eye movements based on electrooculography[J]. IEEE Transactions on Neural System and Rehabilitation Engineering,2002,10(4):209～218.
[21]Ayush Bhandari, Mrinal Trikha, Vijay Khare and Sneh Anand, Wavelet Based Novel Technique for Signal Conditioning of EOG Signals[C], Proc. of IEEE Indicon 2006.
[22]李卫娜,侯文生,郑小林,等Electrooculogram的眼动信息识别[J].仪器仪表学报,2007,28(8)：1428-1433.
[23]Yoshitsugu Yasui, A Brainwave Signal Measurement and Data Processing Technique for Daily Life Applications[J]. Physiological Anthropology,2009: 145～150.
[24]王兆云.基于眼电信号的人机交互系统的设计与实现[D].合肥：安徽大学,2009.
[25]徐栋君.脑电信号采集系统与研究[D].江苏：江苏大学,2006.
[26]Genaro Rebolledo-Mendez, Ian Dunwell and Sara de Freitas, Assessing NuroSky's usability to detect attention levels in an assessment exercise[C]. Human-Computer Interaction, Part1, HCII 2009, LNCS 5610:149～158.
[27]Sitaram, R., fMRI Brain-computer Interfaces[J]. IEEE Signal Processing Magazine 25(1),2008:95～106.
[28]H. Harun, W. Mansor.EOG Signal Detection for Home Appliances Activation[J], IEEE Conferences,2009(5):195～197./3.
[29]Sheng-Tun Li, Shu-Ching Chen, Mei-Ling Shyu, A live TV-quality distant learing multimedia presentation syaytem for education[C],2001 IEEE HCISS.,2001:9.
[30]D. Kumar, E. Poole, Classification of EOG for Human Computer Interface[C], in Proc.2002 Proceedings of the Second Joint EMBS/BMES Conference Houston, TX. USA.
[31]方淼滨.基于EOG信号的医疗护理机器人轨迹规划研究[D].上海：华东理工大学,2004.
[32]黄煜.基于脑电的脑-计算机接口在线异步系统研究[D].成都：电子科技大学,2010.
[33]卫兵,吴小培,吕钊.基于诱发脑电的人机交互系统的设计与实现.[J].工业控制计算机,2009,(02)：25-26.
[34]Jiro Hasegawa, Takashi Baba, Ryoko Ishikura, Effects of Metoclopramide Hydrochloride, a D2-Selective Dopamine Receptor Antagonist, on the Fast Oscillation of the Electrooculogram[J], Yonago Acta medica,2005:83～92.
[35]翟义然,尧德钟.基于真实头模型的EEG参考电极标准化技术[J].中国生物医学工程学报,2004.12,Vo1.23,No.6：523-528.
[36]张磊,张道信,吴小培.基于脑—机接口中的脑电信号前置放大电路设计[J].中国科技论文在线,2009,(02)：411-417.
[37]高扬.基于脑电Alpha波的脑-机接口系统设计[D].中国优秀硕士学位论文全文数据库,2004,(07).
[38]R. BAREA, L. BOQUETE, M. MAZO and E. LOPEZ. Wheelchair Guidance Strategies Using EOG[J]. Journal of Intelligent and Robotic Systems 2002,34: 279～299.
[39]Ramesh Farzanfar, Sophie Frishkopf, Robert Friedman, Kevin Ludena. Evaluating an automated mental health care system:making meaning of human-computer interaction. [J].Computers in Human Behavior 23 (2007) 1167～1182
[40]Yoshitsugu Yasui. A brainwave signal measurement and data processing technique for daily life applications[J]. Physiol Anthropol,2009,28:145～150.
[41]吴湛微,姚晓东,邹俊忠,王行愚,王蓓,从EOG扫视信号中提取眼球位置信息[J],华东理工大学学报(自然科学版),vo1.31 No.2,2005
[42]Rafael Barea, Luciano Boquete, Manuel Mazo, Elena Lbpez, L.M. Bergasa. E.O.G. guidance of a wheelchair using neural networks[C]. IEEE.2000
[43]Xiaoxiang Zheng, Xin Li, Jun Liu, Weidong Chen, Yaoyao Hao. A portable wireless eye movement-controlled Human-Computer Interface for the Disabled[C],IEEE.2009
[44]Thum Chia Chieh,Mohd.Marzuki Mustafa,Aini Hussain.Seyed Farshad Hendi.Development of Vehicle Driver Drowsiness Detection System Using Electrooculogram (EOG)[C],IEEE 2005
[45]S. G. Mason, A. Bashashati, M. Fatourechi, K. F. Navarro, and G. E. Birch, A Comprehensive Survey of Brain Interface Technology Designs [J], Annals of Biomedical Engineering, vol.35 No.2, February 2007:137～169.
[2]柳忠起,袁修干,刘伟,王睿.在模拟飞机降落过程中的眼动分析[J].北京航空航天大学学报,2002,28(6)：703-706
[3]赵翔.智能轮椅的BCI人—机接口技术研究[D].中国优秀硕士学位论文全文数据库,2008,(11).
[4]蔡利栋,方思行,周继鹏,张庆丰.人机交互(第三版)[M].北京,电子工业出版社,2006.
[5]张娴.基于肌电信号的人机交互技术研究[D].南京：东南大学,2006
[6]Kenji Y, Junichi H, Michio M. Development of EOG-Based Communication System Controlled by Eight-Directional Eye Movements[C]. Engineering in Medicine and Biology Society, EMBS'06,28th Annual International Conference of the IEEE,2006:2574～2577.
[7]赵予晗.基于脑电信号的意识任务的识别方法研究[D].甘肃：兰州理工大学,2005.
[8]Li Lian, Wu XiaoPei. Design and implementation of multimedia control system based bluetooth and electrooculogram (EOG)[C].5th International Conference on Bioinformatics and Biomedical Engineering, iCBBE 2011.
[9]李听.基于眼电的无障碍人机交互技术研究[D].杭州：浙江大学,2010.
[10]张磊.基于脑电信号的脑-机接口研究[D].合肥：安徽大学,2009.
[11]黄漫玲,吴平东,殷罡,刘莹。基于稳态视觉诱发电位的脑机接口实验研究[J].北京理工大学学报,2008,(11).
[12]S. Venkataramanan, Pranay Prabhat, Shubhodeep Roy Choudhury, Harshal B. Nemade, J.S. Sahambi, Biomedical Instrumentation based on EOG Signal Processing and Application to a Hospital Alarm System[C], proc.of IEEE ICISIP 2005:535～540, Chennai, India.
[13]Kenji Y, Junichi H, Michio M. Development of EOG-Based Communication System Controlled by Eight-Directional Eye Movements[C]. Engineering in Medicine and Biology Society, EMBS'06,28th Annual International Conference of the IEEE,2006:2574-2577.
[14]张莉,何传红,何为.典型相关分析去除脑电信号中眼电伪迹的研究[J].计算机工程与应用,2009,45(31)：218-220.
[15]张霖,王行愚,姚晓东,邹俊忠.眼球运动相关EOG信号的波形分析与特征提取[J].上海生物医学工程,2003,24(3)：19-22.
[16]钱志鸿,杨帆,周求湛.蓝牙技术原理、开发与应用[M].北京,北京航空航天大学出版社,2006.
[17]朱刚,谈振辉,周贤伟.蓝牙技术原理与协议[M].北京,清华大学出版社,2002.
[18]Fatourechi M, Bashashati A, Ward R K, et al. Birch. EMG and EOG artifacts in brain computer interface systems:A survey[J]. Clinical Neurophysiology, 2007,118:480～494.
[19]G. Norris, E. Wilson, The eye mouse, an eye communication device[C], in Trans. 1997 IEEE SMC.
[20]Barea R, Boquete L, Mazo M, et al. System for assisted mobility using eye movements based on electrooculography[J]. IEEE Transactions on Neural System and Rehabilitation Engineering,2002,10(4):209～218.
[21]Ayush Bhandari, Mrinal Trikha, Vijay Khare and Sneh Anand, Wavelet Based Novel Technique for Signal Conditioning of EOG Signals[C], Proc. of IEEE Indicon 2006.
[22]李卫娜,侯文生,郑小林,等Electrooculogram的眼动信息识别[J].仪器仪表学报,2007,28(8)：1428-1433.
[23]Yoshitsugu Yasui, A Brainwave Signal Measurement and Data Processing Technique for Daily Life Applications[J]. Physiological Anthropology,2009: 145～150.
[24]王兆云.基于眼电信号的人机交互系统的设计与实现[D].合肥：安徽大学,2009.
[25]徐栋君.脑电信号采集系统与研究[D].江苏：江苏大学,2006.
[26]Genaro Rebolledo-Mendez, Ian Dunwell and Sara de Freitas, Assessing NuroSky's usability to detect attention levels in an assessment exercise[C]. Human-Computer Interaction, Part1, HCII 2009, LNCS 5610:149～158.
[27]Sitaram, R., fMRI Brain-computer Interfaces[J]. IEEE Signal Processing Magazine 25(1),2008:95～106.
[28]H. Harun, W. Mansor.EOG Signal Detection for Home Appliances Activation[J], IEEE Conferences,2009(5):195～197./3.
[29]Sheng-Tun Li, Shu-Ching Chen, Mei-Ling Shyu, A live TV-quality distant learing multimedia presentation syaytem for education[C],2001 IEEE HCISS.,2001:9.
[30]D. Kumar, E. Poole, Classification of EOG for Human Computer Interface[C], in Proc.2002 Proceedings of the Second Joint EMBS/BMES Conference Houston, TX. USA.
[31]方淼滨.基于EOG信号的医疗护理机器人轨迹规划研究[D].上海：华东理工大学,2004.
[32]黄煜.基于脑电的脑-计算机接口在线异步系统研究[D].成都：电子科技大学,2010.
[33]卫兵,吴小培,吕钊.基于诱发脑电的人机交互系统的设计与实现.[J].工业控制计算机,2009,(02)：25-26.
[34]Jiro Hasegawa, Takashi Baba, Ryoko Ishikura, Effects of Metoclopramide Hydrochloride, a D2-Selective Dopamine Receptor Antagonist, on the Fast Oscillation of the Electrooculogram[J], Yonago Acta medica,2005:83～92.
[35]翟义然,尧德钟.基于真实头模型的EEG参考电极标准化技术[J].中国生物医学工程学报,2004.12,Vo1.23,No.6：523-528.
[36]张磊,张道信,吴小培.基于脑—机接口中的脑电信号前置放大电路设计[J].中国科技论文在线,2009,(02)：411-417.
[37]高扬.基于脑电Alpha波的脑-机接口系统设计[D].中国优秀硕士学位论文全文数据库,2004,(07).
[38]R. BAREA, L. BOQUETE, M. MAZO and E. LOPEZ. Wheelchair Guidance Strategies Using EOG[J]. Journal of Intelligent and Robotic Systems 2002,34: 279～299.
[39]Ramesh Farzanfar, Sophie Frishkopf, Robert Friedman, Kevin Ludena. Evaluating an automated mental health care system:making meaning of human-computer interaction. [J].Computers in Human Behavior 23 (2007) 1167～1182
[40]Yoshitsugu Yasui. A brainwave signal measurement and data processing technique for daily life applications[J]. Physiol Anthropol,2009,28:145～150.
[41]吴湛微,姚晓东,邹俊忠,王行愚,王蓓,从EOG扫视信号中提取眼球位置信息[J],华东理工大学学报(自然科学版),vo1.31 No.2,2005
[42]Rafael Barea, Luciano Boquete, Manuel Mazo, Elena Lbpez, L.M. Bergasa. E.O.G. guidance of a wheelchair using neural networks[C]. IEEE.2000
[43]Xiaoxiang Zheng, Xin Li, Jun Liu, Weidong Chen, Yaoyao Hao. A portable wireless eye movement-controlled Human-Computer Interface for the Disabled[C],IEEE.2009
[44]Thum Chia Chieh,Mohd.Marzuki Mustafa,Aini Hussain.Seyed Farshad Hendi.Development of Vehicle Driver Drowsiness Detection System Using Electrooculogram (EOG)[C],IEEE 2005
[45]S. G. Mason, A. Bashashati, M. Fatourechi, K. F. Navarro, and G. E. Birch, A Comprehensive Survey of Brain Interface Technology Designs [J], Annals of Biomedical Engineering, vol.35 No.2, February 2007:137～169.