基于共享控制的人机灵巧力触觉交互系统设计与实现
|
摘要
人能够在复杂感知的相互协调下做出决策并完成灵巧操作。将视听觉和力触觉等不同模态相融合,可以提高操作者对虚拟场景或远程环境的感知,进而改善操作效果。如何协调人的操作和虚拟环境中目标机器人系统的自主性,则是人机交互中的一个关键问题。为此,本文基于Unity3D引擎建立了虚拟场景,并实现了omega.7力反馈平台与虚拟场景之间的通信。进而,创建了基于碰撞检测的虚拟距离传感器,实现了虚拟场景中目标机器人对周围环境的探测。然后,设计实现了人机力触觉交互共享控制算法,将人的智能决策与灵巧操作优势同目标机器人的自主算法相融合,使得目标机器人系统的控制权限可以在人手操作和机器人自主算法之间连续平滑转移。由此,实现了人与虚拟环境和目标机器人之间功能多样、场景逼真的视听觉和灵巧力触觉多模态交互,从而为医学、康复、军事、娱乐等领域中的人机交互提供了一种解决方案。
Human beings can accomplish decision-making and dexterous manipulation with their complex sensory perception and motor control capabilities. The fusion of visual and auditory feedback together with haptic feedback,enables close resemblance of manipulation in the virtual scenarios with that in the physical world. Further,it is challenging to coordinate the human intervention and the autonomy of the robot in the virtual scenario. Here,a virtual scenario is built up with Unity3 D,taking advantages of its powerful rendering capabilities. Then,bidirectional communication between Unity3 D and the omega. 7 haptic platform is realized. After that,virtual range sensors based on collision detection are designed and implemented for environment perception of the avatar in the virtual scenario.Further,a human-robot haptic shared control strategy is designed and realized,enabling the avatar to move under the shared control of the dexterous operation by the human and the autonomous algorithms. The control authority can shift smoothly between the human operation and robot autonomy. Thus,a dexterous haptic interaction solution with Unity3 D and the omega. 7 haptic interface is established,which can find promising applications in rehabilitation,teleoperation,military missions and also the entertainment industry.
Human beings can accomplish decision-making and dexterous manipulation with their complex sensory perception and motor control capabilities. The fusion of visual and auditory feedback together with haptic feedback,enables close resemblance of manipulation in the virtual scenarios with that in the physical world. Further,it is challenging to coordinate the human intervention and the autonomy of the robot in the virtual scenario. Here,a virtual scenario is built up with Unity3 D,taking advantages of its powerful rendering capabilities. Then,bidirectional communication between Unity3 D and the omega. 7 haptic platform is realized. After that,virtual range sensors based on collision detection are designed and implemented for environment perception of the avatar in the virtual scenario.Further,a human-robot haptic shared control strategy is designed and realized,enabling the avatar to move under the shared control of the dexterous operation by the human and the autonomous algorithms. The control authority can shift smoothly between the human operation and robot autonomy. Thus,a dexterous haptic interaction solution with Unity3 D and the omega. 7 haptic interface is established,which can find promising applications in rehabilitation,teleoperation,military missions and also the entertainment industry.
引文
[1]张小瑞,孙伟,宋爱国,等.双通道力/触觉交互的虚拟肺手术仿真系统[J].仪器仪表学报,2012,33(2):421-428.ZHANG X R,SUN W,SONG AI G,et al.Virtual lung surgery simulation system based on double-channel haptic interaction[J].Chinese Journal of Scientific Instrument,2012,33(2):421-428.
[2]董士海.人机交互的进展及面临的挑战[J].计算机辅助设计与图形学学报,2004,16(1):1-13.DONG SH H.Progress and challenge of human-computer interaction[J].Journal of Computer-Aided Design&Computer Graphics,2004,16(1):1-13.
[3]冯志全,杨波,徐涛,等.基于自然手势跟踪的直接操作型三维人机交互范式[J].计算机学报,2014,37(6):1309-1323.FENG ZH Q,YANG B,XU T,et al.3D direct humancomputer interface paradigm based on free hand tracking[J].Chinese Journal of Computers,2014,37(6):1309-1323.
[4]METZGER J C,LAMBERCY O,CALIFFI A,et al.Neurocognitive robot-assisted therapy of hand function[J].IEEE Transactions on Haptics,2014,37(6):140-149.
[5]CHANUBOL R,WONGPHAET P,CHAVANICH N,et al.A randomized controlled trial of cognitive sensory motor training therapy on the recovery of arm function in acute stroke patients[J].Clinical Rehabilitation,2012,26(12):1096-1104.
[6]王爱民,戴金桥.虚拟现实的力触觉交互技术发展现状与趋势[J].测控技术,2007,26(10):1-3.WANG AI M,DAI J Q.Current development situation&the future trend of haptic interaction in virtual reality[J].Measurement&Control Technology,2007,26(10):1-3.
[7]GAO M,OBERLANDER J,SCHAMM T,et al.Contextual task-aware shared autonomy for assistive mobile robot teleoperation[C].IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS),2014:3311-3318.
[8]李敏,韩丰.虚拟现实技术综述[J].软件导刊,2010,9(6):142-144.LI M,HAN F.A survey on virtual reality[J].Software Guide,2010,9(6):142-144.
[9]温雅,赵子云.虚拟现实技术在教育类游戏中的应用[J].电脑知识与技术:学术交流,2010,6(7):5334-5336.WEN Y,ZHAO Z Y.The application of virtual reality in educational game[J].Computer Knowledge and Technology,2010,6(7):5334-5336.
[10]孔丽文,薛召军,陈龙,等.基于虚拟现实环境的脑机接口技术研究进展[J].电子测量与仪器学报,2015,29(3):317-327.KONG L W,XUE ZH J,CHEN L,et al.Review of brain-computer interface technology based on virtual reality environment[J].Journal of Electronic Measurement and Instrumentation,2015,29(3):317-327.
[11]张李杰,孙文磊,赵群,等.基于虚拟现实技术的军事训练场地布局和模拟[J].四川兵工学报,2013,34(4):91-92.ZHANG L J,SUN W L,ZHAO Q,et al.Military training site layout and simulation based on virtual reality technology[J].Journal of Sichuan Ordnance,2013,34(4):91-92.
[12]RAUTARAY S S,AGRAWAL A.Vision based hand gesture recognition for human computer interaction:A survey[J].Artificial Intelligence Review,2015,43(1):1-54.
[13]马良,闫继宏,赵杰,等.基于虚拟环境的多操作者多机器人协作遥操作系统[J].机器人,2011,33(2):169-173.MA L,YAN J H,ZHAO J,et al.Virtual environmentbased multi-operator multi-robot cooperative teleoperation system[J].Robot,2011,33(2):169-173.
[14]SANFILIPPO F,WEUSTINK P B,PETTERSEN K Y.A coupling library for the force dimension haptic devices and the 20-sim modelling and simulation environment[C].IEEE Industrial Electronics Society(IECON),2015,35d3(1):1451.
[15]王星捷,李春花.基于Unity3D平台的三维虚拟城市研究与应用[J].计算机技术与发展,2013,23(4):241-244.WANG X J,LI CH H.Research and application of 3D virtual city based on Unity3D[J].Computer Technology and Development,2013,23(4):241-244.
[16]焦卫东,杨世锡,吴昭同.用盲源分离技术创建独立的虚拟传感器[J].仪器仪表学报,2004,25(4):424-427.JIAO W D,YANG SH X,WU ZH T.Creating independent virtual sensors with blind source separation[J].Chinese Journal of Scientific Instrument,2004,25(4):424-427.
[17]GULREZ T,KAVAKLI M.Precision position tracking in virtual reality environments using sensor networks[C].IEEE International Symposium on Industrial Electronics(ISIE),2007:1997-2003.
[18]赵春霞,盛安冬.虚拟传感器的建模与实现[J].机器人,2000,22(2):154-160.ZHAO CH X,SHENG AN D.Modeling and implementation of virtual sensors[J].Robot,2000,22(2):154-160.
[19]YU N,WANG K,LIU J.Dexterous haptic interaction for functional rehabilitation and assessm ent of the upper limb[C].IEEE International Conference on Robotics and Biomimetics(ROBIO),2014:1351-1355.
[20]YU N,WANG K,LI Y,et al.A haptic shared control algorithm for flexible human assistance to semiautonomous robots[C].IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS),2015:5241-5246.
[2]董士海.人机交互的进展及面临的挑战[J].计算机辅助设计与图形学学报,2004,16(1):1-13.DONG SH H.Progress and challenge of human-computer interaction[J].Journal of Computer-Aided Design&Computer Graphics,2004,16(1):1-13.
[3]冯志全,杨波,徐涛,等.基于自然手势跟踪的直接操作型三维人机交互范式[J].计算机学报,2014,37(6):1309-1323.FENG ZH Q,YANG B,XU T,et al.3D direct humancomputer interface paradigm based on free hand tracking[J].Chinese Journal of Computers,2014,37(6):1309-1323.
[4]METZGER J C,LAMBERCY O,CALIFFI A,et al.Neurocognitive robot-assisted therapy of hand function[J].IEEE Transactions on Haptics,2014,37(6):140-149.
[5]CHANUBOL R,WONGPHAET P,CHAVANICH N,et al.A randomized controlled trial of cognitive sensory motor training therapy on the recovery of arm function in acute stroke patients[J].Clinical Rehabilitation,2012,26(12):1096-1104.
[6]王爱民,戴金桥.虚拟现实的力触觉交互技术发展现状与趋势[J].测控技术,2007,26(10):1-3.WANG AI M,DAI J Q.Current development situation&the future trend of haptic interaction in virtual reality[J].Measurement&Control Technology,2007,26(10):1-3.
[7]GAO M,OBERLANDER J,SCHAMM T,et al.Contextual task-aware shared autonomy for assistive mobile robot teleoperation[C].IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS),2014:3311-3318.
[8]李敏,韩丰.虚拟现实技术综述[J].软件导刊,2010,9(6):142-144.LI M,HAN F.A survey on virtual reality[J].Software Guide,2010,9(6):142-144.
[9]温雅,赵子云.虚拟现实技术在教育类游戏中的应用[J].电脑知识与技术:学术交流,2010,6(7):5334-5336.WEN Y,ZHAO Z Y.The application of virtual reality in educational game[J].Computer Knowledge and Technology,2010,6(7):5334-5336.
[10]孔丽文,薛召军,陈龙,等.基于虚拟现实环境的脑机接口技术研究进展[J].电子测量与仪器学报,2015,29(3):317-327.KONG L W,XUE ZH J,CHEN L,et al.Review of brain-computer interface technology based on virtual reality environment[J].Journal of Electronic Measurement and Instrumentation,2015,29(3):317-327.
[11]张李杰,孙文磊,赵群,等.基于虚拟现实技术的军事训练场地布局和模拟[J].四川兵工学报,2013,34(4):91-92.ZHANG L J,SUN W L,ZHAO Q,et al.Military training site layout and simulation based on virtual reality technology[J].Journal of Sichuan Ordnance,2013,34(4):91-92.
[12]RAUTARAY S S,AGRAWAL A.Vision based hand gesture recognition for human computer interaction:A survey[J].Artificial Intelligence Review,2015,43(1):1-54.
[13]马良,闫继宏,赵杰,等.基于虚拟环境的多操作者多机器人协作遥操作系统[J].机器人,2011,33(2):169-173.MA L,YAN J H,ZHAO J,et al.Virtual environmentbased multi-operator multi-robot cooperative teleoperation system[J].Robot,2011,33(2):169-173.
[14]SANFILIPPO F,WEUSTINK P B,PETTERSEN K Y.A coupling library for the force dimension haptic devices and the 20-sim modelling and simulation environment[C].IEEE Industrial Electronics Society(IECON),2015,35d3(1):1451.
[15]王星捷,李春花.基于Unity3D平台的三维虚拟城市研究与应用[J].计算机技术与发展,2013,23(4):241-244.WANG X J,LI CH H.Research and application of 3D virtual city based on Unity3D[J].Computer Technology and Development,2013,23(4):241-244.
[16]焦卫东,杨世锡,吴昭同.用盲源分离技术创建独立的虚拟传感器[J].仪器仪表学报,2004,25(4):424-427.JIAO W D,YANG SH X,WU ZH T.Creating independent virtual sensors with blind source separation[J].Chinese Journal of Scientific Instrument,2004,25(4):424-427.
[17]GULREZ T,KAVAKLI M.Precision position tracking in virtual reality environments using sensor networks[C].IEEE International Symposium on Industrial Electronics(ISIE),2007:1997-2003.
[18]赵春霞,盛安冬.虚拟传感器的建模与实现[J].机器人,2000,22(2):154-160.ZHAO CH X,SHENG AN D.Modeling and implementation of virtual sensors[J].Robot,2000,22(2):154-160.
[19]YU N,WANG K,LIU J.Dexterous haptic interaction for functional rehabilitation and assessm ent of the upper limb[C].IEEE International Conference on Robotics and Biomimetics(ROBIO),2014:1351-1355.
[20]YU N,WANG K,LI Y,et al.A haptic shared control algorithm for flexible human assistance to semiautonomous robots[C].IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS),2015:5241-5246.