面向虚拟维修训练的自抓取虚拟手交互行为构建
|
摘要
针对当前维护训练模拟器(MTD)不能很好地向受训者展示飞机维修过程中的具体维修动作的问题,构建一种自抓取的虚拟手与相应的交互行为,以提高机务培训质量。通过分析维修的基本动作要素,定义与构建基于蒙皮骨骼的虚拟手动作库;在场景漫游阶段,实时获取视角位置来驱动虚拟手运动,以减轻场景漫游下的虚拟手控制负荷;在具体维修操作阶段,利用所提的交互行为算法,将虚拟手的位置自调节与骨骼动画osgCal相结合,通过调取动作库中的相应动作来完成具体的维修任务。实验结果表明,该方法能较好地展现具体的维修操作,对于虚拟维修训练具有良好的可用性。
Focusing on the problem that the current maintenance training device( MTD) can't perfectly reflects the specific maintenance actions during the aircraft maintenance for trainees,this paper established a self-fetching virtual hand and the interaction behaviors to improve the training quality. By analyzing the basic elements of maintenance operations to define and built the virtual hand operation library that based on skinned mesh. In the scene roaming stage,in order to reduce the control load,it drove the virtual hand by the viewpoint position which was acquired in real time. In the specific maintenance operation stage,it used the interactive behavior algorithm to combine the position self-adjustment of the virtual hand with the skeleton animation osgCal,by calling the relevant actions in the action library to complete the specific maintenance tasks. The experimental results show that the proposed method can reflect the specific maintenance operations and have a good usability for virtual maintenance training.
Focusing on the problem that the current maintenance training device( MTD) can't perfectly reflects the specific maintenance actions during the aircraft maintenance for trainees,this paper established a self-fetching virtual hand and the interaction behaviors to improve the training quality. By analyzing the basic elements of maintenance operations to define and built the virtual hand operation library that based on skinned mesh. In the scene roaming stage,in order to reduce the control load,it drove the virtual hand by the viewpoint position which was acquired in real time. In the specific maintenance operation stage,it used the interactive behavior algorithm to combine the position self-adjustment of the virtual hand with the skeleton animation osgCal,by calling the relevant actions in the action library to complete the specific maintenance tasks. The experimental results show that the proposed method can reflect the specific maintenance operations and have a good usability for virtual maintenance training.
引文
[1]Liao Jingtang,Zhao Gang,Tan Dunming,et al.Real-time virtual hand controlling using Open SceneGraph[C]//Proc of IEEE International Conference on System Science,Engineering Design and Manufacturing Informatization.Piscataway,NJ:IEEE Press,2012:264-268.
[2]汤志波,张志利,梁丰,等.虚拟维修训练中的手势识别[J].计算机工程与设计,2014,35(12):4278-4283.(Tang Zhibo,Zhang Zhili,Liang Feng,et al.Hand recognition in virtual maintenance training[J].Computer Engineering&Design,2014,35(12):4278-4283.)
[3]周忠,周颐,肖江剑.虚拟现实增强技术综述[J].中国科学:信息科学,2015,45(2):157-180.(Zhou Zhong,Zhou Yi,Xiao Jiangjian.Survey on augmented virtual environment and augmented reality[J].Scientia Sinica Informationis,2015,45(2):157-180.)
[4]高源,刘越,程德文,等.头盔显示器发展综述[J].计算机辅助设计与图形学学报,2016,28(6):896-904.(Gao Yuan,Liu Yue,Cheng Dewen,et al.A review on development of head mounted display[J].Journal of Computer-Aided Design&Computer Graphics,2016,28(6):896-904.)
[5]陈文锋.灵巧虚拟手自然交互研究[D].杭州:浙江大学,2014.(Chen Wenfeng.Study on the natural interaction of flexible virtual hand[D].Hangzhou:Zhejiang University,2014.)
[6]Kamper D G,Cruz E G,Siegel M P.Stereotypical fingertip trajectories during grasp[J].Journal of Neurophysiology,2003,90(6):3702-3710.
[7]蔡娴娟,程成,Marwah A,等.虚拟制造环境中虚拟手的行为构造[J].计算机辅助设计与图形学学报,2015,27(3):499-507.(Cai Xianjuan,Cheng Cheng,Marwah A,et al.Research on behavior simulation of virtual hand in virtual manufacturing environment[J].Journal of Computer-Aided Design and Computer Graphics,2015,27(3):499-507.)
[8]Aslandere T,Dreyer D,Pankratz F.Virtual hand-button interaction in a generic virtual reality flight simulator[C]//Proc of IEEE Aerospace Conference.Piscataway,NJ:IEEE Press,2015:1-8.
[9]Xia Pingjun,Lopes A M,Restivo M T,et al.A new type haptics-based virtual environment system for assembly training of complex products[J].International Journal of Advanced Manufacturing Technology,2012,58(1-4):379-396.
[10]Liu Xinhua,Cui Xiaolong,Song Guomin,et al.Development of a virtual maintenance system with virtual hand[J].International Journal of Advanced Manufacturing Technology,2014,70(9-12):2241-2247.
[11]Lu Gan,Shark L K,Hall G,et al.Dynamic hand gesture tracking and recognition for real-time immersive virtual object manipulation[C]//Proc of International Conference on Cyberworlds.Piscataway,NJ:IEEEPress,2009:29-35.
[12]Li Jingrong,Xu Yuhua,Ni Jianlong,et al.Glove-based virtual hand grasping for virtual mechanical assembly[J].Assembly Automation,2016,36(4):349-361.
[13]Kumar V,Todorov E.Mu Jo Co HAPTIX:a virtual reality system for hand manipulation[C]//Proc of the 15th IEEE-RAS International Conference on Humanoid Robots.Piscataway,NJ:IEEE Press,2015:657-663.
[14]Kim J S,Park J M.Physics-based hand interaction with virtual objects[C]//Proc of IEEE International Conference on Robotics and Automation.Piscataway,NJ:IEEE Press,2015:3814-3819.
[15]胡晨,张学聃,马惠敏.适应物体形状的虚拟手抓取[J].中国图象图形学报,2017,22(5):663-670.(Hu Chen,Zhang Xuedan,Ma Huimin.Shape-adaptive virtual hand-grasping method[J].Journal of Image&Graphics,2017,22(5):663-670.)
[16]杨晓文,张志纯,况立群,等.基于虚拟手的人机交互关键技术[J].计算机应用,2015,35(10):2945-2949.(Yang Xiaowen,Zhang Zhichun,Kuang Liqun,et al.Key technologies of human-computer interaction based on virtual hand[J].Journal of Computer Applications,2015,35(10):2945-2949.)
[17]李懿,罗建勋,陈卫东,等.基于蒙皮骨骼的虚拟手交互碰撞模拟方法[J].浙江大学学报:工学版,2014,48(1):105-112.(Li Yi,Luo Jianxun,Chen Weidong,et al.Method for collision simulation in virtual hand interaction based on skeletal models[J].Journal of Zhejiang University:Engineering Science,2014,48(1):105-112.)
[18]胡弘,晁建刚,林万洪,等.基于虚拟手交互的航天员虚拟装配训练仿真方法[J].计算机应用,2015,35(S2):200-203.(Hu Hong,Chao Jiangang,Lin Wanghong,et al.Astronaut virtual assembly training simulation based on virtual hand interaction[J].Journal of Computer Applications,2015,35(S2):200-203.)
[19]顾岩,何其昌,范秀敏,等.狭小空间虚拟人手臂装配运动规划及智能寻优[J].计算机集成制造系统,2016,22(6):1447-1455.(Gu Yan,He Qichang,Fan Xiumin,et al.Motion planning and intelligent optimization of virtual human arm for narrow space assembly simulation task[J].Computer Integrated Manufacturing Systems,2016,22(6):1447-1455.)
[20]刘汝强,黄海明,刘金刚.一种适用的角色控制系统的设计与实现[J].计算机应用研究,2009,26(12):4631-4634.(Liu Ruqiang,Huang Haiming,Liu Jingang.Design and realization of flexible role of control system[J].Application Research of Computers,2009,26(12):4631-4634.)
[21]Sagayam K M,Hemanth D J.Hand posture and gesture recognition techniques for virtual reality applications:a survey[J].Virtual Reality,2016,21(2):1-17.
[22]Chen Ning,Li Wenwen,Nie Leixin,et al.Research on the simulation technology of virtual hand interventional manipulation in the virtual environment[J].Applied Mechanics&Materials,2014,665:698-705.
[23]Tang Yong,Yang Bin,Gu Hongbin.Creating a realistic virtual hand for head-mounted displays[J].Electronics World,2013,119(1925):36-39.
[24]Chen Wenfeng,Zhu Shujing,Wan Huagen,et al.Dual quaternion based virtual hand interaction modeling[J].Science China Information Sciences,2013,56(3):1-11.
[25]Wang K R,Xiao Bingjia,Xia J Y,et al.A real-time vision-based hand gesture interaction system for virtual EAST[J].Fusion Engineering&Design,2016,112(11):829-834.
[26]熊巍,王清辉,李静蓉.面向虚拟装配的层次化交互手势技术[J].华南理工大学学报:自然科学版,2016,44(1):78-84.(Xiong Wei,Wang Qinghui,Li Jingrong.A hierarchical method of interaction hand gesture for virtual assembly[J].Journal of South China University of Technology:Natural Science Edition,2016,44(1):78-84.)
[27]Lin Wanhong,Du L,Harris-Adamson C,et al.Design of hand gestures for manipulating objects in virtual reality[C]//Proc of the 19th International Conference on Human-Computer Interaction.Cham:Springer,2017:584-592.
[28]Hirota K,Tagawa K.Interaction with virtual object using deformable hand[C]//Proc of IEEE Virtual Reality.Piscataway,NJ:IEEEPress,2016:49-56.
[29]黄俊,景红.基于Leap Motion的手势识别在虚拟交互中的研究[J].计算机应用研究,2017,34(4):1231-1234.(Huang Jun,Jing Hong.Research on gesture recognition in virtual interaction based on Leap Motion[J].Application Research of Computers,2017,34(4):1231-1234.)
[2]汤志波,张志利,梁丰,等.虚拟维修训练中的手势识别[J].计算机工程与设计,2014,35(12):4278-4283.(Tang Zhibo,Zhang Zhili,Liang Feng,et al.Hand recognition in virtual maintenance training[J].Computer Engineering&Design,2014,35(12):4278-4283.)
[3]周忠,周颐,肖江剑.虚拟现实增强技术综述[J].中国科学:信息科学,2015,45(2):157-180.(Zhou Zhong,Zhou Yi,Xiao Jiangjian.Survey on augmented virtual environment and augmented reality[J].Scientia Sinica Informationis,2015,45(2):157-180.)
[4]高源,刘越,程德文,等.头盔显示器发展综述[J].计算机辅助设计与图形学学报,2016,28(6):896-904.(Gao Yuan,Liu Yue,Cheng Dewen,et al.A review on development of head mounted display[J].Journal of Computer-Aided Design&Computer Graphics,2016,28(6):896-904.)
[5]陈文锋.灵巧虚拟手自然交互研究[D].杭州:浙江大学,2014.(Chen Wenfeng.Study on the natural interaction of flexible virtual hand[D].Hangzhou:Zhejiang University,2014.)
[6]Kamper D G,Cruz E G,Siegel M P.Stereotypical fingertip trajectories during grasp[J].Journal of Neurophysiology,2003,90(6):3702-3710.
[7]蔡娴娟,程成,Marwah A,等.虚拟制造环境中虚拟手的行为构造[J].计算机辅助设计与图形学学报,2015,27(3):499-507.(Cai Xianjuan,Cheng Cheng,Marwah A,et al.Research on behavior simulation of virtual hand in virtual manufacturing environment[J].Journal of Computer-Aided Design and Computer Graphics,2015,27(3):499-507.)
[8]Aslandere T,Dreyer D,Pankratz F.Virtual hand-button interaction in a generic virtual reality flight simulator[C]//Proc of IEEE Aerospace Conference.Piscataway,NJ:IEEE Press,2015:1-8.
[9]Xia Pingjun,Lopes A M,Restivo M T,et al.A new type haptics-based virtual environment system for assembly training of complex products[J].International Journal of Advanced Manufacturing Technology,2012,58(1-4):379-396.
[10]Liu Xinhua,Cui Xiaolong,Song Guomin,et al.Development of a virtual maintenance system with virtual hand[J].International Journal of Advanced Manufacturing Technology,2014,70(9-12):2241-2247.
[11]Lu Gan,Shark L K,Hall G,et al.Dynamic hand gesture tracking and recognition for real-time immersive virtual object manipulation[C]//Proc of International Conference on Cyberworlds.Piscataway,NJ:IEEEPress,2009:29-35.
[12]Li Jingrong,Xu Yuhua,Ni Jianlong,et al.Glove-based virtual hand grasping for virtual mechanical assembly[J].Assembly Automation,2016,36(4):349-361.
[13]Kumar V,Todorov E.Mu Jo Co HAPTIX:a virtual reality system for hand manipulation[C]//Proc of the 15th IEEE-RAS International Conference on Humanoid Robots.Piscataway,NJ:IEEE Press,2015:657-663.
[14]Kim J S,Park J M.Physics-based hand interaction with virtual objects[C]//Proc of IEEE International Conference on Robotics and Automation.Piscataway,NJ:IEEE Press,2015:3814-3819.
[15]胡晨,张学聃,马惠敏.适应物体形状的虚拟手抓取[J].中国图象图形学报,2017,22(5):663-670.(Hu Chen,Zhang Xuedan,Ma Huimin.Shape-adaptive virtual hand-grasping method[J].Journal of Image&Graphics,2017,22(5):663-670.)
[16]杨晓文,张志纯,况立群,等.基于虚拟手的人机交互关键技术[J].计算机应用,2015,35(10):2945-2949.(Yang Xiaowen,Zhang Zhichun,Kuang Liqun,et al.Key technologies of human-computer interaction based on virtual hand[J].Journal of Computer Applications,2015,35(10):2945-2949.)
[17]李懿,罗建勋,陈卫东,等.基于蒙皮骨骼的虚拟手交互碰撞模拟方法[J].浙江大学学报:工学版,2014,48(1):105-112.(Li Yi,Luo Jianxun,Chen Weidong,et al.Method for collision simulation in virtual hand interaction based on skeletal models[J].Journal of Zhejiang University:Engineering Science,2014,48(1):105-112.)
[18]胡弘,晁建刚,林万洪,等.基于虚拟手交互的航天员虚拟装配训练仿真方法[J].计算机应用,2015,35(S2):200-203.(Hu Hong,Chao Jiangang,Lin Wanghong,et al.Astronaut virtual assembly training simulation based on virtual hand interaction[J].Journal of Computer Applications,2015,35(S2):200-203.)
[19]顾岩,何其昌,范秀敏,等.狭小空间虚拟人手臂装配运动规划及智能寻优[J].计算机集成制造系统,2016,22(6):1447-1455.(Gu Yan,He Qichang,Fan Xiumin,et al.Motion planning and intelligent optimization of virtual human arm for narrow space assembly simulation task[J].Computer Integrated Manufacturing Systems,2016,22(6):1447-1455.)
[20]刘汝强,黄海明,刘金刚.一种适用的角色控制系统的设计与实现[J].计算机应用研究,2009,26(12):4631-4634.(Liu Ruqiang,Huang Haiming,Liu Jingang.Design and realization of flexible role of control system[J].Application Research of Computers,2009,26(12):4631-4634.)
[21]Sagayam K M,Hemanth D J.Hand posture and gesture recognition techniques for virtual reality applications:a survey[J].Virtual Reality,2016,21(2):1-17.
[22]Chen Ning,Li Wenwen,Nie Leixin,et al.Research on the simulation technology of virtual hand interventional manipulation in the virtual environment[J].Applied Mechanics&Materials,2014,665:698-705.
[23]Tang Yong,Yang Bin,Gu Hongbin.Creating a realistic virtual hand for head-mounted displays[J].Electronics World,2013,119(1925):36-39.
[24]Chen Wenfeng,Zhu Shujing,Wan Huagen,et al.Dual quaternion based virtual hand interaction modeling[J].Science China Information Sciences,2013,56(3):1-11.
[25]Wang K R,Xiao Bingjia,Xia J Y,et al.A real-time vision-based hand gesture interaction system for virtual EAST[J].Fusion Engineering&Design,2016,112(11):829-834.
[26]熊巍,王清辉,李静蓉.面向虚拟装配的层次化交互手势技术[J].华南理工大学学报:自然科学版,2016,44(1):78-84.(Xiong Wei,Wang Qinghui,Li Jingrong.A hierarchical method of interaction hand gesture for virtual assembly[J].Journal of South China University of Technology:Natural Science Edition,2016,44(1):78-84.)
[27]Lin Wanhong,Du L,Harris-Adamson C,et al.Design of hand gestures for manipulating objects in virtual reality[C]//Proc of the 19th International Conference on Human-Computer Interaction.Cham:Springer,2017:584-592.
[28]Hirota K,Tagawa K.Interaction with virtual object using deformable hand[C]//Proc of IEEE Virtual Reality.Piscataway,NJ:IEEEPress,2016:49-56.
[29]黄俊,景红.基于Leap Motion的手势识别在虚拟交互中的研究[J].计算机应用研究,2017,34(4):1231-1234.(Huang Jun,Jing Hong.Research on gesture recognition in virtual interaction based on Leap Motion[J].Application Research of Computers,2017,34(4):1231-1234.)