基于混合视线-脑机接口与共享控制的人-机器人交互系统
|
摘要
设计了一种基于混合视线一脑机接口与共享控制的人一机器人交互系统,以使得用户可通过视线和意念对机器人末端在2维空间进行连续的运动控制,并在避障和趋近目标的任务中获得机器智能的辅助.首先,按照用户运动意念的强度对机器人末端的运动速度大小进行等比例连续调节,以提高用户对机器人的控制感以及完成任务的参与性.然后,提出了机器人末端运动方向的一种共享控制策略,动态地融合基于视线追踪技术所得到的用户方向控制指令以及由机器人避障和趋近目标的行为设定所得到的机器人系统方向控制指令,自适应地调整机器人系统对用户的辅助力度,以减轻用户脑力负荷,提高任务完成成功率.最后,针对搭建的基于混合视线-脑机接口和共享控制的人-机器人交互平台,通过实验验证了所提系统的有效性.
A human-robot interaction system is designed with the hybrid gaze brain-machine interface and the shared control strategy, to make the user continuously control the robot end-effector in 2 D space with his/her gaze and thought,meanwhile obtaining timely assistance from the machine intelligence in the task of avoiding obstacles and reaching target objects. Firstly, the movement speed of the robot end-effector is adjusted continuously and proportionally by the motor imagery strength of the user, in order to increase the user's sense of control and his/her engagement in the task. Next, a shared control strategy is proposed for controlling the movement direction of the robot end-effector, which dynamically fuses the direction commands from the user obtained by gaze tracking and from the robotic system for avoiding obstacles and reaching target objects. Such a shared control strategy adaptively adjusts the assist level for the user,so as to reduce the mental workload of the user and improve the success rate of completing the task. Finally, with respect to the constructed human-robot interaction system based on the hybrid gaze brain-machine interface and the shared control, experiments are conducted to verify its effectiveness.
A human-robot interaction system is designed with the hybrid gaze brain-machine interface and the shared control strategy, to make the user continuously control the robot end-effector in 2 D space with his/her gaze and thought,meanwhile obtaining timely assistance from the machine intelligence in the task of avoiding obstacles and reaching target objects. Firstly, the movement speed of the robot end-effector is adjusted continuously and proportionally by the motor imagery strength of the user, in order to increase the user's sense of control and his/her engagement in the task. Next, a shared control strategy is proposed for controlling the movement direction of the robot end-effector, which dynamically fuses the direction commands from the user obtained by gaze tracking and from the robotic system for avoiding obstacles and reaching target objects. Such a shared control strategy adaptively adjusts the assist level for the user,so as to reduce the mental workload of the user and improve the success rate of completing the task. Finally, with respect to the constructed human-robot interaction system based on the hybrid gaze brain-machine interface and the shared control, experiments are conducted to verify its effectiveness.
引文
[1]Millán J D R,Rupp R,M(u|¨)ller-Putz G R,et al.Combining braincomputer interfaces and assistive technologies:State-of-the-art and challenges[J].Frontiers in Neuroscience,2010,4:No.161.
[2]Hochberg L R,Bacher D,Jarosiewicz B,et al.Reach and grasp by people with tetraplegia using a neurally controlled robotic arm[J].Nature,2012,485(7398):372.
[3]Morgante L,Morgante F,Moro E,et al.How many parkinsonian patients are suitable candidates for deep brain stimulation of subthalamic nucleus?Results of a questionnaire[J].Parkinsonism and Related Disorders,2007,13(8):528-531.
[4]Gudayol-FerréE,Per6-Cebollero M,Gonzdlez-Garrido A A,et al.Changes in brain connectivity related to the treatment of depression measured through fMRI:A systematic review[J].Frontiers in Human Neuroscience,2015,9:No.582.
[5]Naseer N,Hong K S.Corrigendum:fNIRS-based braincomputer interfaces:A review[J].Frontiers in Human Neuroscience,2015,9:No.3.
[6]Fukuma R,Yanagisawa T,Saitoh Y,et al.Corrigendum:Realtime control of a neuroprosthetic hand by magnetoencephalographic signals from paralysed patients[J].Scientific Reports,2016,6:No.21781.
[7]Moghimi S,Kushki A,Guerguerian A M,et al.A review of EEG-based brain-computer interfaces as access pathways for individuals with severe disabilities[J].Assistive Technology,2013,25(2):99-110.
[8]Carlson T,Millan J D R.Brain-controlled wheelchairs:A robotic architecture[J].IEEE Robotics and Automation Magazine,2013,20(1):65-73.
[9]Li Z J,Zhao S N,Duan J D,et al.Human cooperative wheelchair with brain-machine interaction based on shared control strategy[J].IEEE/ASME Transactions on Mechatronics,2017,22(1):185-195.
[10]He S H,Zhang R,Wang Q H,et al.A P300-based threshold-free brain switch and its application in wheelchair control[J].IEEE Transactions on Neural Systems and Rehabilitation Engineering,2017,25(6):715-725.
[11]Chen X G,Wang Y J,Nakanishi M,et al.High-speed spelling with a noninvasive brain-computer interface[J].Proceedings of the National Academy of Sciences of the United States of America,2015,112(44):E6058-E6067.
[12]Chen Y Q,Ke Y F,Meng G F,et al.Enhancing performance of P300-Speller under mental workload by incorporating dualtask data during classifier training[J].Computer Methods and Programs in Biomedicine,2017,152:35-43.
[13]Zhao X G,Chu Y Q,Han J D,et al.SSVEP-based braincomputer interface controlled functional electrical stimulation system for upper extremity rehabilitation[J].IEEE Transactions on Systems,Man,and Cybernetics:Systems,2016,46(7):947-956.
[14]徐宝国,彭思,宋爱国.基于运动想象脑电的上肢康复机器人[J].机器人,2011,33(3):307-313.Xu B G,Peng S,Song A G.Upper-limb rehabilitation robot based on motor imagery EEG[J].Robot,2011,33(3):307-313.
[15]俞建成,张进,李伟.基于事件相关电位的水下机械手脑电波控制[J].机器人,2017,39(4):395-404.Yu J C,Zhang J,Li W.Controlling an underwater manipulator via event-related potentials of brainwaves[J].Robot,2017,39(4):395-404.
[16]李敏,徐光华,谢俊,等.脑卒中意念控制的主被动运动康复技术[J].机器人,2017,39(5):759-768.Lin M,Xu G H,Xie J,et al.Motor rehabilitation with control based on human intent for stroke survivors[J].Robot,2017,39(5):759-768.
[17]Wang Y X,Zeng H,and Liu J.Low-cost eye-tracking glasses with real-time head rotation compensation[C]//10th International Conference on Sensing Technology.Piscataway,USA:IEEE,2016.
[18]Soekadar S R,Witkowski M,G6mez C,et al.Hybrid EEG/EOG-based brain/neural hand exoskeleton restores fully independent daily living activities after quadriplegia[J].Science Robotics,2016,1(1):32-96.
[19]Yuan P,Chen X G,Wang Y J,et al.Enhancing performances of SSVEP-based brain-computer interfaces via exploiting intersubject information[J].Journal of Neural Engineering,2015,12(4):No.046006.
[20]Khan M J,Hong K S.Hybrid EEG-fNIRS-based eight-command decoding for BCI:Application to quadcopter control[J].Frontiers in Neurorobotics,2017,11:No.6.
[21]Hong K S,Khan M J.Hybrid brain-computer interface techniques for improved classification accuracy and increased number of commands:A review[J].Frontiers in Neurorobotics,2017,11:No.35.
[22]Zeng H,Wang Y X,Wu C C,et al.Closed-loop hybrid gaze brain-machine interface based robotic arm control with augmented reality feedback[J].Frontiers in Neuro robotics,2017,11:No.60.
[23]McMullen D P,Hotson G,Katyal K D,et al.Demonstration of a semi-autonomous hybrid brain-machine interface using human intracranial EEG,eye tracking,and computer vision to control a robotic upper limb prosthetic[J].IEEE Transactions on Neural Systems and Rehabilitation Engineering,2014,22(4):784-796.
[24]Frisoli A,Loconsole C,Leonardis D,et al.A new gaze-BCIdriven control of an upper limb exoskeleton for rehabilitation in real-world tasks[J].IEEE Transactions on Systems,Man,and Cybernetics,Part C:Applications and Reviews,2012,42(6):1169-1179.
[25]Kim Y J,Park S W,Yeom H G,et al.A study on a robot arm driven by three-dimensional trajectories predicted from non-invasive neural signals[J].Biomedical Engineering Online,2015,14:No.81.
[26]吴朝晖,俞一鹏,潘纲,等.脑机融合系统综述[J].生命科学,2014(6):645-649.Wu C H,Yu Y P,Pan G,et al.Brain-machine integrated systems[J].Chinese Bulletin of Life Sciences,2014(6):645-649.
[2]Hochberg L R,Bacher D,Jarosiewicz B,et al.Reach and grasp by people with tetraplegia using a neurally controlled robotic arm[J].Nature,2012,485(7398):372.
[3]Morgante L,Morgante F,Moro E,et al.How many parkinsonian patients are suitable candidates for deep brain stimulation of subthalamic nucleus?Results of a questionnaire[J].Parkinsonism and Related Disorders,2007,13(8):528-531.
[4]Gudayol-FerréE,Per6-Cebollero M,Gonzdlez-Garrido A A,et al.Changes in brain connectivity related to the treatment of depression measured through fMRI:A systematic review[J].Frontiers in Human Neuroscience,2015,9:No.582.
[5]Naseer N,Hong K S.Corrigendum:fNIRS-based braincomputer interfaces:A review[J].Frontiers in Human Neuroscience,2015,9:No.3.
[6]Fukuma R,Yanagisawa T,Saitoh Y,et al.Corrigendum:Realtime control of a neuroprosthetic hand by magnetoencephalographic signals from paralysed patients[J].Scientific Reports,2016,6:No.21781.
[7]Moghimi S,Kushki A,Guerguerian A M,et al.A review of EEG-based brain-computer interfaces as access pathways for individuals with severe disabilities[J].Assistive Technology,2013,25(2):99-110.
[8]Carlson T,Millan J D R.Brain-controlled wheelchairs:A robotic architecture[J].IEEE Robotics and Automation Magazine,2013,20(1):65-73.
[9]Li Z J,Zhao S N,Duan J D,et al.Human cooperative wheelchair with brain-machine interaction based on shared control strategy[J].IEEE/ASME Transactions on Mechatronics,2017,22(1):185-195.
[10]He S H,Zhang R,Wang Q H,et al.A P300-based threshold-free brain switch and its application in wheelchair control[J].IEEE Transactions on Neural Systems and Rehabilitation Engineering,2017,25(6):715-725.
[11]Chen X G,Wang Y J,Nakanishi M,et al.High-speed spelling with a noninvasive brain-computer interface[J].Proceedings of the National Academy of Sciences of the United States of America,2015,112(44):E6058-E6067.
[12]Chen Y Q,Ke Y F,Meng G F,et al.Enhancing performance of P300-Speller under mental workload by incorporating dualtask data during classifier training[J].Computer Methods and Programs in Biomedicine,2017,152:35-43.
[13]Zhao X G,Chu Y Q,Han J D,et al.SSVEP-based braincomputer interface controlled functional electrical stimulation system for upper extremity rehabilitation[J].IEEE Transactions on Systems,Man,and Cybernetics:Systems,2016,46(7):947-956.
[14]徐宝国,彭思,宋爱国.基于运动想象脑电的上肢康复机器人[J].机器人,2011,33(3):307-313.Xu B G,Peng S,Song A G.Upper-limb rehabilitation robot based on motor imagery EEG[J].Robot,2011,33(3):307-313.
[15]俞建成,张进,李伟.基于事件相关电位的水下机械手脑电波控制[J].机器人,2017,39(4):395-404.Yu J C,Zhang J,Li W.Controlling an underwater manipulator via event-related potentials of brainwaves[J].Robot,2017,39(4):395-404.
[16]李敏,徐光华,谢俊,等.脑卒中意念控制的主被动运动康复技术[J].机器人,2017,39(5):759-768.Lin M,Xu G H,Xie J,et al.Motor rehabilitation with control based on human intent for stroke survivors[J].Robot,2017,39(5):759-768.
[17]Wang Y X,Zeng H,and Liu J.Low-cost eye-tracking glasses with real-time head rotation compensation[C]//10th International Conference on Sensing Technology.Piscataway,USA:IEEE,2016.
[18]Soekadar S R,Witkowski M,G6mez C,et al.Hybrid EEG/EOG-based brain/neural hand exoskeleton restores fully independent daily living activities after quadriplegia[J].Science Robotics,2016,1(1):32-96.
[19]Yuan P,Chen X G,Wang Y J,et al.Enhancing performances of SSVEP-based brain-computer interfaces via exploiting intersubject information[J].Journal of Neural Engineering,2015,12(4):No.046006.
[20]Khan M J,Hong K S.Hybrid EEG-fNIRS-based eight-command decoding for BCI:Application to quadcopter control[J].Frontiers in Neurorobotics,2017,11:No.6.
[21]Hong K S,Khan M J.Hybrid brain-computer interface techniques for improved classification accuracy and increased number of commands:A review[J].Frontiers in Neurorobotics,2017,11:No.35.
[22]Zeng H,Wang Y X,Wu C C,et al.Closed-loop hybrid gaze brain-machine interface based robotic arm control with augmented reality feedback[J].Frontiers in Neuro robotics,2017,11:No.60.
[23]McMullen D P,Hotson G,Katyal K D,et al.Demonstration of a semi-autonomous hybrid brain-machine interface using human intracranial EEG,eye tracking,and computer vision to control a robotic upper limb prosthetic[J].IEEE Transactions on Neural Systems and Rehabilitation Engineering,2014,22(4):784-796.
[24]Frisoli A,Loconsole C,Leonardis D,et al.A new gaze-BCIdriven control of an upper limb exoskeleton for rehabilitation in real-world tasks[J].IEEE Transactions on Systems,Man,and Cybernetics,Part C:Applications and Reviews,2012,42(6):1169-1179.
[25]Kim Y J,Park S W,Yeom H G,et al.A study on a robot arm driven by three-dimensional trajectories predicted from non-invasive neural signals[J].Biomedical Engineering Online,2015,14:No.81.
[26]吴朝晖,俞一鹏,潘纲,等.脑机融合系统综述[J].生命科学,2014(6):645-649.Wu C H,Yu Y P,Pan G,et al.Brain-machine integrated systems[J].Chinese Bulletin of Life Sciences,2014(6):645-649.