基于模糊强化学习的微创外科手术机械臂人机交互方法
|
摘要
为实现微创外科手术机器人的手术姿态调整,提出一种基于模糊强化学习的变导纳人机力交互模型.通过在线学习的方式将人的操作特性考虑到人机力交互过程之中,并能够自适应地调整导纳控制模型以响应操作者的控制意图.通过自行研制的微创外科手术机器人样机进行相关的实验验证,实验结果表明基于模糊Sarsa(λ)学习的变导纳控制模型可实现柔顺自然的机械臂摆位操作,能够满足力交互过程中各阶段的阻尼变化需求,具有较高的可控性和稳定性.
In order to achieve the surgical gesture adjustment of the minimally invasive surgical robot, a variable admittance model based on fuzzy reinforcement learning for physical human-robot interaction is proposed. The manipulation characteristics of the operator are taken into account in the physical human-robot interaction process by an online learning method, which can adaptively modify the admittance model to respond to the operator's control intention. An experimental verification is carried out on a self-developed minimally invasive surgical robot, and the experiment results show that the pose adjustment of manipulator can be implemented naturally and smoothly by the variable admittance model based on fuzzy Sarsa(λ) learning. The proposed control strategy can meet the requirements of damping change in each stage of the physical human-robot interaction, and has high controllability and stability.
In order to achieve the surgical gesture adjustment of the minimally invasive surgical robot, a variable admittance model based on fuzzy reinforcement learning for physical human-robot interaction is proposed. The manipulation characteristics of the operator are taken into account in the physical human-robot interaction process by an online learning method, which can adaptively modify the admittance model to respond to the operator's control intention. An experimental verification is carried out on a self-developed minimally invasive surgical robot, and the experiment results show that the pose adjustment of manipulator can be implemented naturally and smoothly by the variable admittance model based on fuzzy Sarsa(λ) learning. The proposed control strategy can meet the requirements of damping change in each stage of the physical human-robot interaction, and has high controllability and stability.
引文
[1]Murphy D A,Miller J S,Langford D A,et al.Endoscopic robotic mitral valve surgery[J].Journal of Thoracic and Cardiovascular Surgery,2006,132(4):776-781.
[2]Meireles O,Horgan S.Applications of surgical robotics in general surgery[M]//Surgical Robotics.New York,USA:Springer,2011:791-812.
[3]Vanderborght B,Albu-Sch¨affer A,Bicchi A,et al.Variable impedance actuators:A review[J].Robotics and Autonomous Systems,2013,61(12):1601-1614.
[4]Haddadin S,Albu-Sch¨affer A,Hirzinger G.Safe physical human-robot interaction:Measurements,analysis and new insights[C]//13th International Symposium on Robotics Research.Berlin,Germany:Springer,2010:395-407.
[5]游有鹏,张宇,李成刚.面向直接示教的机器人零力控制[J].机械工程学报,2014,50(3):10-17.You Y P,Zhang Y,Li C G.Force-free control for the direct teaching of robots[J].Journal of Mechanical Engineering,2014,50(3):10-17.
[6]Rosenberg L B.Virtual fixtures:Perceptual tools for telerobotic manipulation[C]//Virtual Reality Annual International Symposium.Piscataway,USA:IEEE,1993:76-82.
[7]Kosuge K,Fujisawa Y,Fukuda T.Control of robot directly maneuvered by operator[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems.Piscataway,USA:IEEE,1993:49-54.
[8]Mitchell B,Koo J,Iordachita I,et al.Development and application of a new steady-hand manipulator for retinal surgery[C]//IEEE International Conference on Robotics and Automation.Piscataway,USA:IEEE,2007:623-629.
[9]Infante M L,Kyrki V.Usability of force-based controllers in physical human-robot interaction[C]//6th ACM/IEEE International Conference on Human-Robot Interaction.Piscataway,USA:IEEE,2011:355-362.
[10]Kushida D,Nakamura M,Goto S,et al.Human direct teaching of industrial articulated robot arms based on force-free control[J].Artificial Life and Robotics,2001,5(1):26-32.
[11]Goto S,Usui T,Kyura N,et al.Forcefree control with independent compensation for industrial articulated robot arms[J].Control Engineering Practice,2007,15(6):627-638.
[12]Pallegedara A,Matsuda Y,Egashira N,et al.Simulation and analysis of dynamics of the force-free control for industrial robot arms[C]//12th International Conference on Control,Automation and Systems.Piscataway,USA:IEEE,2012:733-738.
[13]Hogan N.Impedance control:An approach to manipulation:Part I theory;Part II implementation;Part III applications[J].Journal of Dynamic Systems,Measurement,and Control,1985,107(1):1-24.
[14]Ikeura R,Inooka H.Variable impedance control of a robot for cooperation with a human[C]//IEEE International Conference on Robotics and Automation.Piscataway,USA:IEEE,1995:3097-3102.
[15]Erden M S,Mari′c B.Assisting manual welding with robot[J].Robotics and Computer-Integrated Manufacturing,2011,27(4):818-828.
[16]Duchaine V,Gosselin C M.General model of human-robot cooperation using a novel velocity based variable impedance control[C]//2nd Joint Euro Haptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.Piscataway,USA:IEEE,2007:446-451.
[17]Tsumugiwa T,Yokogawa R,Hara K.Variable impedance control based on estimation of human arm stiffness for humanrobot cooperative calligraphic task[C]//IEEE International Conference on Robotics and Automation.Piscataway,USA:IEEE,2002:644-650.
[18]Lecours A,St-Onge B M,Gosselin C.Variable admittance control of a four-degree-of-freedom intelligent assist device[C]//IEEE International Conference on Robotics and Automation.Piscataway,USA:IEEE,2012:3903-3908.
[19]Duchaine V,St-Onge B M,Gao D,et al.Stable and intuitive control of an intelligent assist device[J].IEEE Transactions on Haptics,2012,5(2):148-159.
[20]Ficuciello F,Villani L,Siciliano B.Variable impedance control of redundant manipulators for intuitive human-robot physical interaction[J].IEEE Transactions on Robotics,2015,31(4):850-863.
[21]Flash T,Hogan N.The coordination of arm movements:An experimentally confirmed mathematical model[J].Journal of Neuroscience,1985,5(7):1688-1703.
[22]Maeda Y,Hara T,Arai T.Human-robot cooperative manipulation with motion estimation[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems.Piscataway,USA:IEEE,2001:2240-2245.
[23]Corteville B,Aertbeli¨en E,Bruyninckx H,et al.Humaninspired robot assistant for fast point-to-point movements[C]//IEEE International Conference on Robotics and Automation.Piscataway,USA:IEEE,2007:3639-3644.
[24]Dimeas F,Aspragathos N.Fuzzy learning variable admittance control for human-robot cooperation[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems.Piscataway,USA:IEEE,2014:4770-4775.
[25]Duchaine V,Gosselin C M.Investigation of human-robot interaction stability using Lyapunov theory[C]//IEEE International Conference on Robotics and Automation.Piscataway,USA:IEEE,2008:2189-2194.
[26]Ficuciello F,Villani L,Siciliano B.Variable impedance control of redundant manipulators for intuitive human-robot physical interaction[J].IEEE Transactions on Robotics,2015,31(4):850-863.
[27]Ikeura R,Monden H,Inooka H.Cooperative motion control of a robot and a human[C]//3rd IEEE International Workshop on Robot and Human Communication.Piscataway,USA:IEEE,1994:112-117.
[28]Sutton R S,Barto A G.Reinforcement learning:An introduction[M].Cambridge,USA:MIT Press,1998.
[29]Jouffe L.Fuzzy inference system learning by reinforcement methods[J].IEEE Transactions on Systems,Man,and Cybernetics,Part C:Applications and Reviews,1998,28(3):338-355.
[30]Emmerich C,Nordmann A,Swadzba A,et al.Assisted gravity compensation to cope with the complexity of kinesthetic teaching on redundant robots[C]//IEEE International Conference on Robotics and Automation.Piscataway,USA:IEEE,2013:4322-4328.
[2]Meireles O,Horgan S.Applications of surgical robotics in general surgery[M]//Surgical Robotics.New York,USA:Springer,2011:791-812.
[3]Vanderborght B,Albu-Sch¨affer A,Bicchi A,et al.Variable impedance actuators:A review[J].Robotics and Autonomous Systems,2013,61(12):1601-1614.
[4]Haddadin S,Albu-Sch¨affer A,Hirzinger G.Safe physical human-robot interaction:Measurements,analysis and new insights[C]//13th International Symposium on Robotics Research.Berlin,Germany:Springer,2010:395-407.
[5]游有鹏,张宇,李成刚.面向直接示教的机器人零力控制[J].机械工程学报,2014,50(3):10-17.You Y P,Zhang Y,Li C G.Force-free control for the direct teaching of robots[J].Journal of Mechanical Engineering,2014,50(3):10-17.
[6]Rosenberg L B.Virtual fixtures:Perceptual tools for telerobotic manipulation[C]//Virtual Reality Annual International Symposium.Piscataway,USA:IEEE,1993:76-82.
[7]Kosuge K,Fujisawa Y,Fukuda T.Control of robot directly maneuvered by operator[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems.Piscataway,USA:IEEE,1993:49-54.
[8]Mitchell B,Koo J,Iordachita I,et al.Development and application of a new steady-hand manipulator for retinal surgery[C]//IEEE International Conference on Robotics and Automation.Piscataway,USA:IEEE,2007:623-629.
[9]Infante M L,Kyrki V.Usability of force-based controllers in physical human-robot interaction[C]//6th ACM/IEEE International Conference on Human-Robot Interaction.Piscataway,USA:IEEE,2011:355-362.
[10]Kushida D,Nakamura M,Goto S,et al.Human direct teaching of industrial articulated robot arms based on force-free control[J].Artificial Life and Robotics,2001,5(1):26-32.
[11]Goto S,Usui T,Kyura N,et al.Forcefree control with independent compensation for industrial articulated robot arms[J].Control Engineering Practice,2007,15(6):627-638.
[12]Pallegedara A,Matsuda Y,Egashira N,et al.Simulation and analysis of dynamics of the force-free control for industrial robot arms[C]//12th International Conference on Control,Automation and Systems.Piscataway,USA:IEEE,2012:733-738.
[13]Hogan N.Impedance control:An approach to manipulation:Part I theory;Part II implementation;Part III applications[J].Journal of Dynamic Systems,Measurement,and Control,1985,107(1):1-24.
[14]Ikeura R,Inooka H.Variable impedance control of a robot for cooperation with a human[C]//IEEE International Conference on Robotics and Automation.Piscataway,USA:IEEE,1995:3097-3102.
[15]Erden M S,Mari′c B.Assisting manual welding with robot[J].Robotics and Computer-Integrated Manufacturing,2011,27(4):818-828.
[16]Duchaine V,Gosselin C M.General model of human-robot cooperation using a novel velocity based variable impedance control[C]//2nd Joint Euro Haptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.Piscataway,USA:IEEE,2007:446-451.
[17]Tsumugiwa T,Yokogawa R,Hara K.Variable impedance control based on estimation of human arm stiffness for humanrobot cooperative calligraphic task[C]//IEEE International Conference on Robotics and Automation.Piscataway,USA:IEEE,2002:644-650.
[18]Lecours A,St-Onge B M,Gosselin C.Variable admittance control of a four-degree-of-freedom intelligent assist device[C]//IEEE International Conference on Robotics and Automation.Piscataway,USA:IEEE,2012:3903-3908.
[19]Duchaine V,St-Onge B M,Gao D,et al.Stable and intuitive control of an intelligent assist device[J].IEEE Transactions on Haptics,2012,5(2):148-159.
[20]Ficuciello F,Villani L,Siciliano B.Variable impedance control of redundant manipulators for intuitive human-robot physical interaction[J].IEEE Transactions on Robotics,2015,31(4):850-863.
[21]Flash T,Hogan N.The coordination of arm movements:An experimentally confirmed mathematical model[J].Journal of Neuroscience,1985,5(7):1688-1703.
[22]Maeda Y,Hara T,Arai T.Human-robot cooperative manipulation with motion estimation[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems.Piscataway,USA:IEEE,2001:2240-2245.
[23]Corteville B,Aertbeli¨en E,Bruyninckx H,et al.Humaninspired robot assistant for fast point-to-point movements[C]//IEEE International Conference on Robotics and Automation.Piscataway,USA:IEEE,2007:3639-3644.
[24]Dimeas F,Aspragathos N.Fuzzy learning variable admittance control for human-robot cooperation[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems.Piscataway,USA:IEEE,2014:4770-4775.
[25]Duchaine V,Gosselin C M.Investigation of human-robot interaction stability using Lyapunov theory[C]//IEEE International Conference on Robotics and Automation.Piscataway,USA:IEEE,2008:2189-2194.
[26]Ficuciello F,Villani L,Siciliano B.Variable impedance control of redundant manipulators for intuitive human-robot physical interaction[J].IEEE Transactions on Robotics,2015,31(4):850-863.
[27]Ikeura R,Monden H,Inooka H.Cooperative motion control of a robot and a human[C]//3rd IEEE International Workshop on Robot and Human Communication.Piscataway,USA:IEEE,1994:112-117.
[28]Sutton R S,Barto A G.Reinforcement learning:An introduction[M].Cambridge,USA:MIT Press,1998.
[29]Jouffe L.Fuzzy inference system learning by reinforcement methods[J].IEEE Transactions on Systems,Man,and Cybernetics,Part C:Applications and Reviews,1998,28(3):338-355.
[30]Emmerich C,Nordmann A,Swadzba A,et al.Assisted gravity compensation to cope with the complexity of kinesthetic teaching on redundant robots[C]//IEEE International Conference on Robotics and Automation.Piscataway,USA:IEEE,2013:4322-4328.