面向手势动作捕捉的传感器设计及主从手运动映射
|
摘要
本文针对人手和机器人灵巧手在运动学上尺寸的不一致现象带来的主从手指尖运动空间映射的不一致性进行研究.基于聚氨酯拉伸应变传感器构建了测量手指关节弯曲角度的数据手套.通过建立主从手的运动学模型,基于旋转矩阵理论及正向运动学提出了一种指尖运动轨迹计算方法及手势动作捕捉算法.基于正向运动学及逆向运动学的指尖运动映射算法建立了主从手的指尖运动空间轮廓.建立虚拟实验场景,分别针对关节角度映射算法、手势动作捕捉算法及指尖运动映射算法进行了一系列试验.通过实验得出基于聚氨酯的拉伸应变传感器具有良好的时间响应特性及电学稳定性,基于手势动作捕捉算法能够获取主从手的指尖运动空间轮廓,指尖运动轨迹的计算误差控制在2.8 mm以内.结果证明了基于手势动作捕捉算法以及指尖运动映射算法能够实现主从手指尖运动空间的一致性.
This paper focuses on the inconsistency in motion spatial mapping of the master-slave fingertip caused by the inconsistent dimensions phenomenon of kinematics between human hand and robot dexterous hand. A data glove based on polyurethane strain sensors is made as data interface to measure the joint angles of the fingers. A fingertip motion trajectory calculation method and a hand gestures capturing algorithm are proposed based on rotation matrix theory and forward kinematics by establishing a master-slave kinematic model. The motion workspace boundaries of the master-slave fingertip are established by a fingertip motion mapping algorithm based on forward and inverse kinematics. A virtual experiment platform is established, and a series of experiments are performed for the joint angle mapping algorithm, the gesture motion capturing algorithm, and the fingertip motion mapping algorithm. The experiments show that the strain sensor based on polyurethane has good time response characteristic and electrical stability. The motion workspace boundaries of the master-slave fingertip are acquired based on the gesture motion capturing algorithm, and the calculation error of the fingertip motion trajectory is controlled within 2.8 mm. The results show that the combination of the hand gesture motion capturing algorithm and the fingertips motion mapping algorithm can achieve the consistency of master-slave fingertip motion space.
This paper focuses on the inconsistency in motion spatial mapping of the master-slave fingertip caused by the inconsistent dimensions phenomenon of kinematics between human hand and robot dexterous hand. A data glove based on polyurethane strain sensors is made as data interface to measure the joint angles of the fingers. A fingertip motion trajectory calculation method and a hand gestures capturing algorithm are proposed based on rotation matrix theory and forward kinematics by establishing a master-slave kinematic model. The motion workspace boundaries of the master-slave fingertip are established by a fingertip motion mapping algorithm based on forward and inverse kinematics. A virtual experiment platform is established, and a series of experiments are performed for the joint angle mapping algorithm, the gesture motion capturing algorithm, and the fingertip motion mapping algorithm. The experiments show that the strain sensor based on polyurethane has good time response characteristic and electrical stability. The motion workspace boundaries of the master-slave fingertip are acquired based on the gesture motion capturing algorithm, and the calculation error of the fingertip motion trajectory is controlled within 2.8 mm. The results show that the combination of the hand gesture motion capturing algorithm and the fingertips motion mapping algorithm can achieve the consistency of master-slave fingertip motion space.
引文
[1]Xu Z Q, Li H D, He J L. A novel medical human-computer interaction platform based on Kinect depth image analysis refactoring and neural network[J]. International Journal of Biomedical Engineering Technology, 2017, 23(2/3/4):315-334.
[2]Li P. Construction and simulation on intelligent medical equipment system based on virtual reality technology and human-computer interaction model[J]. International Journal of Biomedical Engineering Technology, 2017, 23(2/3/4):261-280.
[3]秦超龙,宋爱国,吴常铖,等.基于Unity3D与Kinect的康复训练机器人情景交互系统[J].仪器仪表学报,2017,38(3):530-536.Qin C L, Song A G, Wu C C, et al. Scenario interaction system of rehabilitation training robot based on Unity3D and Kinect[J].Chinese Journal of Scientific Instrument, 2017, 38(3):530-536.
[4]Robins B, Dautenhahn K, Ferrari E, et al. Scenarios of robotassisted play for children with cognitive and physical disabilities[J]. Interaction Studies, 2012, 13(2):189-199.
[5]Yoshida K, Yamada H, Kato R, et al. Development of five-finger robotic hand using master-slave control for hand-assisted laparoscopic surgery[C]//38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.Piscataway, USA:IEEE, 2016. DOI:10.1109/EMBC.2016.7591880.
[6]陈一民,张云华.基于手势识别的机器人人机交互技术研究[J].机器人,2009,31(4):351-356.Chen Y M, Zhang Y H. Research on human-robot interaction technique based on hand gesture recognition[J]. Robot, 2009,31(4):351-356.
[7]Wen G H, Wan Y, Cao J, et al. Master-slave synchronization of heterogeneous systems under scheduling communication[J].IEEE Transactions on Systems, Man, and Cybernetics:Systems, 2018, 48(3):473-484.
[8]Ferrone A, Maita F, Maiolo L, et al. Wearable band for hand gesture recognition based on strain sensors[C]//IEEE International Conference on Biomedical Robotics and Biomechatronics. Piscataway, USA:IEEE, 2016:1319-1322.
[9]齐静,徐坤,丁希仑.机器人视觉手势交互技术研究进展[J].机器人,2017,39(4):565-584.Qi J, Xu K, Ding X L. Vision-based hand gesture recognition for human-robot interaction:A review[J]. Robot, 2017, 39(4):565-584.
[10]Dipietro L, Sabatini A M, Dario P. A survey of glove-based systems and their applications[J]. IEEE Transactions on Systems,Man, and Cybernetics, Part C, 2008, 38(4):461-482.
[11]Chen W F, Zhu S J, Wan H G, et al. Dual quaternion based virtual hand interaction modeling[J]. Science China:Information Sciences, 2013, 56(3):1-11.
[12]Wang K R, Xiao B J, Xia J Y, et al. A real-time vision-based hand gesture interaction system for virtual EAST[J]. Fusion Engineering&Design, 2016, 112:829-834.
[13]Fang B, Guo D, Sun F C, et al. A robotic hand-arm teleoperation system using human arm/hand with a novel data glove[C]//IEEE International Conference on Robotics and Biomimetics. Piscataway, USA:IEEE, 2016:2483-2488.
[14]Beecks C, Hassani M, Brenger B, et al. Efficient query processing in 3D motion capture gesture databases[J]. International Journal of Semantic Computing, 2016, 10(1):5-25.
[15]Gong D X, Zhao J J, Yu J, et al. Motion mapping of the heterogeneous master-slave system for intuitive telemanipulation[J]. International Journal of Advanced Robotic Systems,2018, 15(1). DOI:10.1177/1729881417748134.
[16]Wei T, Xiao N F. Control system of dexterous hand based on data glove[J]. Computer Engineering&Design, 2009, 30(7):1707-1711.
[17]Yu G, Sun Z J, Huang W Q, et al. Master-slave motion mapping method for dexterous hand driven by ultrasonic motors based on time-varying coefficients[C]//International Conference on Electronics, Communications and Control. Piscataway, USA:IEEE,2011:890-893.
[18]Liu J, Zhang Y R. Mapping human hand motion to dexterous robotic hand[C]//IEEE International Conference on Robotics and Biomimetics. Piscataway, USA:IEEE, 2008:829-834.
[19]Zhao Y P, Fei F, Yang D H, et al. Research on data glove for hand rehabilitation[J]. Chinese Journal of Scientific Instrument,2017, 38(10):2415-2421.
[20]Kobayashi F, Kitabayashi K, Kai S, et al. Human motion caption with vision and inertial sensors for hand/arm robot teleoperation[J]. International Journal of Applied Electromagnetics&Mechanics, 2016, 52(3):1-8.
[21]Fang B, Sun F C, Liu H P, et al. 3D human gesture capturing and recognition by the IMMU-based data glove[J]. Neurocomputing, 2018, 277:198-207.
[22]张阳阳,黄英,郝超,等.基于织物拉伸传感器的手势映射系统[J].仪器仪表学报,2017,38(10):2422-2429.Zhang Y Y, Huang Y, Hao C, et al. Gestures mapping system based on fabric strain sensor[J]. Chinese Journal of Scientific Instrument, 2017, 38(10):2422-2429.
[23]Huang Y, He X Y, Gao L, et al. Pressure-sensitive carbon black/graphene nanoplatelets-silicone rubber hybrid conductive composites based on a three-dimensional polydopaminemodified polyurethane sponge[J]. Journal of Materials Science–Materials in Electronics, 2017, 28(13):9495-9504.
[24]Nakatani H, Iwakura K, Hamadate M, et al. Interface adhesion properties of syndiotactic polypropylene/cellulose group composite:Relationship between chemical structure of coupling agent and reactivity for cellulose group[J]. Journal of Applied Polymer Science, 2011, 122(4):2798-2806.
[25]Liu Z, Jones F R, Zhai Z, et al. Interfacial adhesion of polyamide66 fibres to an aqueous polyurethane-acrylic hybrid polymer adhesive[J]. Journal of Adhesion, 2014, 90(4):310-326.
[26]Kanan S M, Tze W T Y, Tripp C P. Method to double the surface concentration and control the orientation of adsorbed(3-aminopropyl)dimethylethoxysilane on silica powders and glass slides[J]. Langmuir, 2002, 18(17):6623-6627.
[27]Liu Z, Zhao F, Jones F R. Optimising the interfacial response of glass fibre composites with a functional nanoscale plasma polymer coating[J]. Composites Science&Technology, 2008,68(15):3161-3170.
[28]Vidya Shree G, Sharanabasappa. Non-contact water level monitoring using Labview with Arduino and ultrasonic sensor[C]//Third International Conference on Current Trends in Engineering Science and Technology. 2017:570-574. DOI:10.21647/ICCTEST/2017/49024.
[29]Zhu C C, Song A G. Design of a 3-DOF parallel handcontroller[J]. Journal of Sensors, 2017. DOI:10.1155/2017/8162512.
[30]Ahmad F, Tohidi E, Carrasco J A. A parameterized multi-step Newton method for solving systems of nonlinear equations[J].Numerical Algorithms, 2016, 71(3):1-23.
[31]Li B, Jin J, Li W, M et al. Method of rotation angle measurement in machine vision based on calibration pattern with spot array[J]. Applied Optics, 2010, 49(6):1001-1006.
[32]吴森纪.多用与专用胶粘剂[J].粘接,1982(1):42-47.Wu S J. Multi-purpose and special adhesive[J]. Adhesion,1982(1):42-47.
[2]Li P. Construction and simulation on intelligent medical equipment system based on virtual reality technology and human-computer interaction model[J]. International Journal of Biomedical Engineering Technology, 2017, 23(2/3/4):261-280.
[3]秦超龙,宋爱国,吴常铖,等.基于Unity3D与Kinect的康复训练机器人情景交互系统[J].仪器仪表学报,2017,38(3):530-536.Qin C L, Song A G, Wu C C, et al. Scenario interaction system of rehabilitation training robot based on Unity3D and Kinect[J].Chinese Journal of Scientific Instrument, 2017, 38(3):530-536.
[4]Robins B, Dautenhahn K, Ferrari E, et al. Scenarios of robotassisted play for children with cognitive and physical disabilities[J]. Interaction Studies, 2012, 13(2):189-199.
[5]Yoshida K, Yamada H, Kato R, et al. Development of five-finger robotic hand using master-slave control for hand-assisted laparoscopic surgery[C]//38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.Piscataway, USA:IEEE, 2016. DOI:10.1109/EMBC.2016.7591880.
[6]陈一民,张云华.基于手势识别的机器人人机交互技术研究[J].机器人,2009,31(4):351-356.Chen Y M, Zhang Y H. Research on human-robot interaction technique based on hand gesture recognition[J]. Robot, 2009,31(4):351-356.
[7]Wen G H, Wan Y, Cao J, et al. Master-slave synchronization of heterogeneous systems under scheduling communication[J].IEEE Transactions on Systems, Man, and Cybernetics:Systems, 2018, 48(3):473-484.
[8]Ferrone A, Maita F, Maiolo L, et al. Wearable band for hand gesture recognition based on strain sensors[C]//IEEE International Conference on Biomedical Robotics and Biomechatronics. Piscataway, USA:IEEE, 2016:1319-1322.
[9]齐静,徐坤,丁希仑.机器人视觉手势交互技术研究进展[J].机器人,2017,39(4):565-584.Qi J, Xu K, Ding X L. Vision-based hand gesture recognition for human-robot interaction:A review[J]. Robot, 2017, 39(4):565-584.
[10]Dipietro L, Sabatini A M, Dario P. A survey of glove-based systems and their applications[J]. IEEE Transactions on Systems,Man, and Cybernetics, Part C, 2008, 38(4):461-482.
[11]Chen W F, Zhu S J, Wan H G, et al. Dual quaternion based virtual hand interaction modeling[J]. Science China:Information Sciences, 2013, 56(3):1-11.
[12]Wang K R, Xiao B J, Xia J Y, et al. A real-time vision-based hand gesture interaction system for virtual EAST[J]. Fusion Engineering&Design, 2016, 112:829-834.
[13]Fang B, Guo D, Sun F C, et al. A robotic hand-arm teleoperation system using human arm/hand with a novel data glove[C]//IEEE International Conference on Robotics and Biomimetics. Piscataway, USA:IEEE, 2016:2483-2488.
[14]Beecks C, Hassani M, Brenger B, et al. Efficient query processing in 3D motion capture gesture databases[J]. International Journal of Semantic Computing, 2016, 10(1):5-25.
[15]Gong D X, Zhao J J, Yu J, et al. Motion mapping of the heterogeneous master-slave system for intuitive telemanipulation[J]. International Journal of Advanced Robotic Systems,2018, 15(1). DOI:10.1177/1729881417748134.
[16]Wei T, Xiao N F. Control system of dexterous hand based on data glove[J]. Computer Engineering&Design, 2009, 30(7):1707-1711.
[17]Yu G, Sun Z J, Huang W Q, et al. Master-slave motion mapping method for dexterous hand driven by ultrasonic motors based on time-varying coefficients[C]//International Conference on Electronics, Communications and Control. Piscataway, USA:IEEE,2011:890-893.
[18]Liu J, Zhang Y R. Mapping human hand motion to dexterous robotic hand[C]//IEEE International Conference on Robotics and Biomimetics. Piscataway, USA:IEEE, 2008:829-834.
[19]Zhao Y P, Fei F, Yang D H, et al. Research on data glove for hand rehabilitation[J]. Chinese Journal of Scientific Instrument,2017, 38(10):2415-2421.
[20]Kobayashi F, Kitabayashi K, Kai S, et al. Human motion caption with vision and inertial sensors for hand/arm robot teleoperation[J]. International Journal of Applied Electromagnetics&Mechanics, 2016, 52(3):1-8.
[21]Fang B, Sun F C, Liu H P, et al. 3D human gesture capturing and recognition by the IMMU-based data glove[J]. Neurocomputing, 2018, 277:198-207.
[22]张阳阳,黄英,郝超,等.基于织物拉伸传感器的手势映射系统[J].仪器仪表学报,2017,38(10):2422-2429.Zhang Y Y, Huang Y, Hao C, et al. Gestures mapping system based on fabric strain sensor[J]. Chinese Journal of Scientific Instrument, 2017, 38(10):2422-2429.
[23]Huang Y, He X Y, Gao L, et al. Pressure-sensitive carbon black/graphene nanoplatelets-silicone rubber hybrid conductive composites based on a three-dimensional polydopaminemodified polyurethane sponge[J]. Journal of Materials Science–Materials in Electronics, 2017, 28(13):9495-9504.
[24]Nakatani H, Iwakura K, Hamadate M, et al. Interface adhesion properties of syndiotactic polypropylene/cellulose group composite:Relationship between chemical structure of coupling agent and reactivity for cellulose group[J]. Journal of Applied Polymer Science, 2011, 122(4):2798-2806.
[25]Liu Z, Jones F R, Zhai Z, et al. Interfacial adhesion of polyamide66 fibres to an aqueous polyurethane-acrylic hybrid polymer adhesive[J]. Journal of Adhesion, 2014, 90(4):310-326.
[26]Kanan S M, Tze W T Y, Tripp C P. Method to double the surface concentration and control the orientation of adsorbed(3-aminopropyl)dimethylethoxysilane on silica powders and glass slides[J]. Langmuir, 2002, 18(17):6623-6627.
[27]Liu Z, Zhao F, Jones F R. Optimising the interfacial response of glass fibre composites with a functional nanoscale plasma polymer coating[J]. Composites Science&Technology, 2008,68(15):3161-3170.
[28]Vidya Shree G, Sharanabasappa. Non-contact water level monitoring using Labview with Arduino and ultrasonic sensor[C]//Third International Conference on Current Trends in Engineering Science and Technology. 2017:570-574. DOI:10.21647/ICCTEST/2017/49024.
[29]Zhu C C, Song A G. Design of a 3-DOF parallel handcontroller[J]. Journal of Sensors, 2017. DOI:10.1155/2017/8162512.
[30]Ahmad F, Tohidi E, Carrasco J A. A parameterized multi-step Newton method for solving systems of nonlinear equations[J].Numerical Algorithms, 2016, 71(3):1-23.
[31]Li B, Jin J, Li W, M et al. Method of rotation angle measurement in machine vision based on calibration pattern with spot array[J]. Applied Optics, 2010, 49(6):1001-1006.
[32]吴森纪.多用与专用胶粘剂[J].粘接,1982(1):42-47.Wu S J. Multi-purpose and special adhesive[J]. Adhesion,1982(1):42-47.