一种视觉引导的作业型飞行机器人设计
|
摘要
针对作业型飞行机器人执行抓取、投放等作业任务时飞行机器人与被抓取目标之间难以相对定位的问题,提出了一种视觉引导的作业型飞行机器人设计方法.首先,介绍作业型飞行机器人系统的整体机构设计,建立飞行器和空中作业装置的运动学和动力学模型.然后,根据针孔成像模型,在ArUco标记尺寸已知的前提下,通过机载的单目摄像头检测被抓目标上的ArUco标记,利用n点透视(PnP)算法解算摄像头位姿,进而利用摄像头位姿信息对飞行器和作业装置进行分级控制.最后,通过静止实验和户外悬停实验验证了位姿估计算法的有效性,并通过自主抓取直径2 cm、质量100 g的管状物体进一步验证视觉引导的有效性和合理性.
Aiming at the difficulty of relative positioning between the aerial robot and the grabbed target when the aerial manipulator system performs the tasks of grabbing, dropping, etc., a design method for vision-guided aerial manipulators is presented. Firstly, the overall mechanism design of the aerial manipulator system is introduced, and kinematics and dynamics models are developed for the aircraft and the aerial operation device. Secondly, the on-board monocular camera is utilized to detect the ArUco markers on the grabbed target according to the pinhole imaging model on the premise that the marker size is known, and the perspective-n-point(PnP) algorithm is used to calculate the camera pose. Then the flying robot and the operation device are controlled hierarchically based on the camera pose information. Finally, the effectiveness of the pose estimation algorithm is validated by the stationary and outdoor hover experiments, and the effectiveness and rationality of visual guidance are further verified by autonomously grabbing a tubular object with a diameter of 2 cm and mass of 100 g.
Aiming at the difficulty of relative positioning between the aerial robot and the grabbed target when the aerial manipulator system performs the tasks of grabbing, dropping, etc., a design method for vision-guided aerial manipulators is presented. Firstly, the overall mechanism design of the aerial manipulator system is introduced, and kinematics and dynamics models are developed for the aircraft and the aerial operation device. Secondly, the on-board monocular camera is utilized to detect the ArUco markers on the grabbed target according to the pinhole imaging model on the premise that the marker size is known, and the perspective-n-point(PnP) algorithm is used to calculate the camera pose. Then the flying robot and the operation device are controlled hierarchically based on the camera pose information. Finally, the effectiveness of the pose estimation algorithm is validated by the stationary and outdoor hover experiments, and the effectiveness and rationality of visual guidance are further verified by autonomously grabbing a tubular object with a diameter of 2 cm and mass of 100 g.
引文
[1]杨斌,何玉庆,韩建达,等.作业型飞行机器人研究现状与展望[J].机器人,2015,37(5):628-640.Yang B, He Y Q, Han J D, et al. Survey on aerial manipulator systems[J]. Robot, 2015, 37(5):628-640.
[2] Thomas J, Loianno G, Daniilidis K, et al. The role of vision in perching and grasping for MAVs[C]//Micro-and Nanotechnology Sensors, Systems, and Applications Ⅷ. Bellingham, USA:SPIE, 2016:No.98361S.
[3] Bisgaard M, la Cour-Harbo A, Bendtsen J D. Adaptive control system for autonomous helicopter slung load operations[J].Control Engineering Practice, 2010, 18(7):800-811.
[4] Lindsey Q, Mellinger D, Kumar V. Construction of cubic structures with quadrotor teams[C]//Robotics:Science and Systems Ⅶ. 2011. DOI:10.15607/RSS.2011.Ⅶ.025.
[5] Mellinger D, Lindsey Q, Shomin M, et al. Design, modeling,estimation and control for aerial grasping and manipulation[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway, USA:IEEE, 2011:2668-2673.
[6] Pounds P E I, Bersak D R, Dollar A M. Grasping from the air:Hovering capture and load stability[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2011:2491-2498.
[7] Thomas J, Loianno G, Sreenath K, et al. Toward image based visual servoing for aerial grasping and perching[C]//IEEE International Conference on Robotics and Automation. Piscataway,USA:IEEE, 2014:2113-2118.
[8] Kim S, Seo H, Choi S, et al. Vision-guided aerial manipulation using a multirotor with a robotic arm[J]. IEEE/ASME Transactions on Mechatronics, 2016, 21(4):1912-1923.
[9] Kim S, Seo H, Kim H J. Operating an unknown drawer using an aerial manipulator[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2015:5503-5508.
[10] Seo H, Kim S, Kim H J. Aerial grasping of cylindrical object using visual servoing based on stochastic model predictive control[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2017:6362-6368.
[11]钟杭,王耀南,李玲,等.旋翼飞行机械臂建模及动态重心补偿控制[J].控制理论与应用,2016,33(3):311-320.Zhong H, Wang Y N, Li L, et al. Rotor-flying manipulator modeling and control with dynamic compensation for gravity offset[J]. Control Theory&Applications, 2016, 33(3):311-320.
[12]宋大雷,孟祥冬,齐俊桐,等.3自由度旋翼飞行机械臂系统动力学建模与预测控制方法[J].机器人,2015,37(2):152-160.Song D L, Meng X D, Qi J T, et al. Strategy of dynamic modeling and predictive control on 3-DoF rotorcraft aerial manipulator system[J]. Robot, 2015, 37(2):152-160.
[13] Garrido-Jurado S, Munoz-Salinas R, Madrid-Cuevas F J, et al.Automatic generation and detection of highly reliable fiducial markers under occlusion[J]. Pattern Recognition, 2014, 47(6):2280-2292.
[14]孙富春,朱纪洪,刘国栋,等.机器人学导论——分析、控制及应用[M].2版.北京:电子工业出版社,2013:27-73.Sun F C, Zhu J H, Liu G D, et al. Introduction to robotics:Analysis, control, application[M]. 2nd ed. Beijing:Publishing House of Electronics Industry, 2013:27-73.
[15] Mahony R, Kumar V, Corke P. Multirotor aerial vehicles:Modeling, estimation, and control of quadrotor[J]. IEEE Robotics and Automation Magazine, 2012, 19(3):20-32.
[16] Alexis K, Nikolakopoulos G, Tzes A. Model predictive quadrotor control:Attitude, altitude and position experimental studies[J]. IET Control Theory and Applications, 2012, 6(12):1812-1827.
[17] Wang P, Sun C K, Zhang Z M. Linear pose estimation with a monocular vision system[J]. Chinese Journal of Scientific Instrument, 2011, 32(5):1126-1131.
[18] Baggio D L, Emami S, Escriva D M, et al. Mastering OpenCV with practical computer vision projects[M]. Birmingham, UK:Packt Publishing Ltd, 2012:47-82.
[19] Zhang Z Y. A flexible new technique for camera calibration[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11):1330-1334.
[20] Otsu N. A threshold selection method from gray-level histograms[J]. IEEE Transactions on Systems, Man, and Cybernetics, 1979, 9(1):62-66.
[2] Thomas J, Loianno G, Daniilidis K, et al. The role of vision in perching and grasping for MAVs[C]//Micro-and Nanotechnology Sensors, Systems, and Applications Ⅷ. Bellingham, USA:SPIE, 2016:No.98361S.
[3] Bisgaard M, la Cour-Harbo A, Bendtsen J D. Adaptive control system for autonomous helicopter slung load operations[J].Control Engineering Practice, 2010, 18(7):800-811.
[4] Lindsey Q, Mellinger D, Kumar V. Construction of cubic structures with quadrotor teams[C]//Robotics:Science and Systems Ⅶ. 2011. DOI:10.15607/RSS.2011.Ⅶ.025.
[5] Mellinger D, Lindsey Q, Shomin M, et al. Design, modeling,estimation and control for aerial grasping and manipulation[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway, USA:IEEE, 2011:2668-2673.
[6] Pounds P E I, Bersak D R, Dollar A M. Grasping from the air:Hovering capture and load stability[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2011:2491-2498.
[7] Thomas J, Loianno G, Sreenath K, et al. Toward image based visual servoing for aerial grasping and perching[C]//IEEE International Conference on Robotics and Automation. Piscataway,USA:IEEE, 2014:2113-2118.
[8] Kim S, Seo H, Choi S, et al. Vision-guided aerial manipulation using a multirotor with a robotic arm[J]. IEEE/ASME Transactions on Mechatronics, 2016, 21(4):1912-1923.
[9] Kim S, Seo H, Kim H J. Operating an unknown drawer using an aerial manipulator[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2015:5503-5508.
[10] Seo H, Kim S, Kim H J. Aerial grasping of cylindrical object using visual servoing based on stochastic model predictive control[C]//IEEE International Conference on Robotics and Automation. Piscataway, USA:IEEE, 2017:6362-6368.
[11]钟杭,王耀南,李玲,等.旋翼飞行机械臂建模及动态重心补偿控制[J].控制理论与应用,2016,33(3):311-320.Zhong H, Wang Y N, Li L, et al. Rotor-flying manipulator modeling and control with dynamic compensation for gravity offset[J]. Control Theory&Applications, 2016, 33(3):311-320.
[12]宋大雷,孟祥冬,齐俊桐,等.3自由度旋翼飞行机械臂系统动力学建模与预测控制方法[J].机器人,2015,37(2):152-160.Song D L, Meng X D, Qi J T, et al. Strategy of dynamic modeling and predictive control on 3-DoF rotorcraft aerial manipulator system[J]. Robot, 2015, 37(2):152-160.
[13] Garrido-Jurado S, Munoz-Salinas R, Madrid-Cuevas F J, et al.Automatic generation and detection of highly reliable fiducial markers under occlusion[J]. Pattern Recognition, 2014, 47(6):2280-2292.
[14]孙富春,朱纪洪,刘国栋,等.机器人学导论——分析、控制及应用[M].2版.北京:电子工业出版社,2013:27-73.Sun F C, Zhu J H, Liu G D, et al. Introduction to robotics:Analysis, control, application[M]. 2nd ed. Beijing:Publishing House of Electronics Industry, 2013:27-73.
[15] Mahony R, Kumar V, Corke P. Multirotor aerial vehicles:Modeling, estimation, and control of quadrotor[J]. IEEE Robotics and Automation Magazine, 2012, 19(3):20-32.
[16] Alexis K, Nikolakopoulos G, Tzes A. Model predictive quadrotor control:Attitude, altitude and position experimental studies[J]. IET Control Theory and Applications, 2012, 6(12):1812-1827.
[17] Wang P, Sun C K, Zhang Z M. Linear pose estimation with a monocular vision system[J]. Chinese Journal of Scientific Instrument, 2011, 32(5):1126-1131.
[18] Baggio D L, Emami S, Escriva D M, et al. Mastering OpenCV with practical computer vision projects[M]. Birmingham, UK:Packt Publishing Ltd, 2012:47-82.
[19] Zhang Z Y. A flexible new technique for camera calibration[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11):1330-1334.
[20] Otsu N. A threshold selection method from gray-level histograms[J]. IEEE Transactions on Systems, Man, and Cybernetics, 1979, 9(1):62-66.