双目家政机器人的目标物识别及抓取研究
|
摘要
本文的研究紧紧依托于华南理工大学智能机器人实验室的家政机器人平台,针对家政机器人进行了详细的分析,获得了一些较重要的研究成果和结论。
首先构建了包括摄像机标定、目标物图像的获取和识别、获取目标点的深度信息、空间坐标变换、机器人运动学模型分析与建立、以及基于位置的视觉伺服在内的双目机器人视觉伺服控制系统的框架结构。在理论上建立了基于D-H表示法的家政机器人手臂的运动学模型,对其进行了逆运动学求解,并探讨了动态的机器人手眼协调关系。
针对SIFT目标识别算法的特点及其在机器人目标识别应用中的不足,提出了结合Snake模型的SIFT目标位置及轮廓识别算法,解决了家政机器人目标识别过程中目标物轮廓信息获取不足的问题,并将相关的目标信息结合到机器人手臂末端执行器对目标物的抓取过程中。利用Visual Studio 2005开发平台实现了相关算法。
研究了机器人的视觉伺服系统,对传统的基于位置的视觉伺服系统的结构及相关技术进行了分析,根据其在机器人目标物抓取中的不足,结合雅可比矩阵在机器人微分运动学中的关系,提出了改进的基于位置的视觉伺服系统控制结构。该控制结构的主要思想是通过运动学逆解使家政机器人手臂达到目标物的大致位置,接下来在线计算机器人的雅克比矩阵以控制手臂的微分运动,最后使其能够精确的达到目标物位置并抓取目标物。并通过实验证明了本文中以上理论的有效性。
This research closely relies on the household robot platform in the intelligent robot Lab of South China University of Technology, analyzing the household robot particularly. This study gets access to some important research results and conclusions.
First it builds binocular robot vision servo control system’s framework, including camera calibration, target image acquisition and processing, getting the depth information of the target point, space coordinate transformation, kinematics modeling and analysis, and visual servo control system. Then we establish the kinematics model for our household robot based on the D-H method, and the inverse kinematics was carried out. We research the relationship of the dynamic robot hand-eye coordinate transformation.
This study particularly analyses the characteristics of SIFT object recognition algorithm and its application in robot target recognition, we propose a method which combines Snake model and SIFT target recognition algorithm to solve the issue that we can’t obtain enough target contour information from the traditional robot target recognition process, and it integrate the target information to the target capture process using robot arm end-effector. We use Visual Studio 2005 development platform to achieve the correlation algorithm.
This paper researches the robot visual servo system, and analyses the related techniques of the traditional position-based visual servo system. According to the traditional method’s deficiency in the target capture process with robot hand, and the relationship of the Jacobian matrix in robot differential kinematics, we propose the improved position-based visual servo system control method. The main idea of this control method is to control the robot hand to reach the target approximate location using inverse kinematics, then to calculate the Jacobian matrix of the robot hand on line, and finally to enable robot hand to reach the target precise position and grasp the target object. We Proved the effectiveness of this theory by experiment.
首先构建了包括摄像机标定、目标物图像的获取和识别、获取目标点的深度信息、空间坐标变换、机器人运动学模型分析与建立、以及基于位置的视觉伺服在内的双目机器人视觉伺服控制系统的框架结构。在理论上建立了基于D-H表示法的家政机器人手臂的运动学模型,对其进行了逆运动学求解,并探讨了动态的机器人手眼协调关系。
针对SIFT目标识别算法的特点及其在机器人目标识别应用中的不足,提出了结合Snake模型的SIFT目标位置及轮廓识别算法,解决了家政机器人目标识别过程中目标物轮廓信息获取不足的问题,并将相关的目标信息结合到机器人手臂末端执行器对目标物的抓取过程中。利用Visual Studio 2005开发平台实现了相关算法。
研究了机器人的视觉伺服系统,对传统的基于位置的视觉伺服系统的结构及相关技术进行了分析,根据其在机器人目标物抓取中的不足,结合雅可比矩阵在机器人微分运动学中的关系,提出了改进的基于位置的视觉伺服系统控制结构。该控制结构的主要思想是通过运动学逆解使家政机器人手臂达到目标物的大致位置,接下来在线计算机器人的雅克比矩阵以控制手臂的微分运动,最后使其能够精确的达到目标物位置并抓取目标物。并通过实验证明了本文中以上理论的有效性。
This research closely relies on the household robot platform in the intelligent robot Lab of South China University of Technology, analyzing the household robot particularly. This study gets access to some important research results and conclusions.
First it builds binocular robot vision servo control system’s framework, including camera calibration, target image acquisition and processing, getting the depth information of the target point, space coordinate transformation, kinematics modeling and analysis, and visual servo control system. Then we establish the kinematics model for our household robot based on the D-H method, and the inverse kinematics was carried out. We research the relationship of the dynamic robot hand-eye coordinate transformation.
This study particularly analyses the characteristics of SIFT object recognition algorithm and its application in robot target recognition, we propose a method which combines Snake model and SIFT target recognition algorithm to solve the issue that we can’t obtain enough target contour information from the traditional robot target recognition process, and it integrate the target information to the target capture process using robot arm end-effector. We use Visual Studio 2005 development platform to achieve the correlation algorithm.
This paper researches the robot visual servo system, and analyses the related techniques of the traditional position-based visual servo system. According to the traditional method’s deficiency in the target capture process with robot hand, and the relationship of the Jacobian matrix in robot differential kinematics, we propose the improved position-based visual servo system control method. The main idea of this control method is to control the robot hand to reach the target approximate location using inverse kinematics, then to calculate the Jacobian matrix of the robot hand on line, and finally to enable robot hand to reach the target precise position and grasp the target object. We Proved the effectiveness of this theory by experiment.
引文
[1] Y. Shirai and H. Inoue,“Guiding a robot by visual feedback assembling tasks”Pattern Recognition, vol. 5, pp. 99-108, 1973.
[2] M.Arlotti and M.Grnaieri,“A Perception technique for a 3D robotic stereo eye-in-hand Vision system”The Fifth International Conference on Advanced Robotics,“Robots in Unstructured Environments”1991,pp.1626-1629.
[3] E.Malis, F.Chaumette and S.Boudet,“Multi-camera visual servoing”IEEE International Conference on Robotics and Automation, 2000, vol 3, pp. 3183-3188.
[4] R.Andersson,“Understanding and applying a robot ping-pong player’s expert controller”, IEEE International Conference on Robotics and Automation, 1989, vol 3, pp. 1284-1289
[5] R.Andersson,“Dynamic sensing in ping-pong playing robot”, IEEE Transactions on Robotics and Automation, 1989 ,5(6):728-739.
[6] G.Hager,“Calibration-free visual control using projective invariance”International Conference on Computer Vision,1995, pp, 1009-1015.
[7] G.Hager, W.Chang and A.Morse“Robot hand-eye coordination based on stereo vision”IEEE Control System Magazine, 1995, 15(1), 30-39.
[8] G.Hager,“A modular system for robust positioning using feedback from stereo vision”IEEE Transactions on Robotics and Automation, 1997 ,13(4):582-595.
[9] Reyes F. and R. Kelly,“Experimental Evaluation of Fixed-Camera Direct Visual Controllers on a Direct-Drive Robot”. IEEE International Conference on Robotics & Automation. 1998, pp. 2327-2332
[10] Malis E. and S. Benhimane,“A Unified Approach to Visual Tracking and Servoing”. Robotics and Autonomous Systems, Vol. 52, 2005, pp. 39-52
[11] Park J. and Y.J. Lee,“Robust Visual Servoing for Motion Control of the Ball on a Plate”. Mechatronics, Vol. 13, 2003, pp. 723-738.
[12] Jaime Cid M. and Fernando Reyes C.“Visual servoing controller for robot manipulators”. IEEE International Conference on Electrical, 2009,pp 155-18.
[13] Mario Prats, Philippe Martinet and Angel P.“Robotic execution of everyday tasks by means of external vision/force control”Intel Serv Robotics, 2008, pp 253-266.
[14] R. S. Hartenberg and J. Denavit,“A kinematic notation for lower pair mechanisms based on matrices,”Journal of Applied Mechanics, vol. 77, pp. 215–221, June 1955.
[15] C. S. G. Lee,“Robot arm kinematics, dynamics and control,”IEEE Computer, vol. 15, pp. 62–80, Dec. 1982.
[16] J.J.Craig,“Introduction to Robotics”. Third Edition. 2004
[17] M. Spong and M. Vidyasagar,“Robot Dynamics and Control”. John Wiley and Sons, 1989.
[18] Hutchinson, G. Hager, and P. Corke,“A tutorial on visual servo control,”IEEE Transactions on Robotics and Automation, vol. 12, pp. 651–670, Oct. 1996.
[19]张广军.机器视觉.北京:科学出版社, 2005. pp 69-98.
[20] Z.Zhnag.“A flexible new technique for camera calibration”. Technical Report MSR-TR-98-71, Microsoft Research,Dee.1998.
[21] Juyang Weng, Paul Cohen and Marc Herniou.“Calibration of stereo Cameras Using a Non-linear Distortion Model”. Proc. International Conference on Pattern Recognition, 1990. Pp 245-253.
[22]马颂德,张正友.计算机视觉:计算理论与算法基础.科学出版社, 1998. pp 52-70.
[23] R.Y.Tsai.“A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses”. IEEE J. of Robotics Automation,1987,3(4) pp 323-344.
[24] Juyang Weng, Paul Cohen and Marc Herniou.“Camera Calibration with Distortion Models and Accuracy Evaluation”. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1992, 14(10) pp 965-980.
[25] R. P. Paul, Robot Manipulators: Mathematics, Programming, and Control. Cambridge, Massachusetts: MIT Press, 1981.
[26]高文,陈熙霖.计算机视觉:算法与系统原理.清华大学出版社,1999.
[27] Barnard.S.T, Fischer.M.A.“Computational stereo”. Computaing Surveys. 1982.14 pp 553-572
[28] Levine.M.D.“Computer determination of depth maps”. Computer Graphics and Image Process, 1972. 2 pp 131-150.
[29] Moravec.H.D.“Towards automatic visual obstacle avoidance”. proc. 5th IJCAI, 1977.
[30]章毓晋.图像工程上册:图象处理和分析,清华大学出版社,1999
[31]肖南峰,智能机器人.华南理工大学出版社.2008
[32]吴立得.计算机视觉.上海:复旦大学出版社.1993
[33] NGUGENM C.S“intuition-oriented behavior-based stereo vision to gain robustness and adaptation in manipulator control”[A]. Proc Of SPIE[C].Massachusetts, 1999(3837)
[34]赵清杰,连广宇,孙增昕.机器人视觉伺服综述[J].控制与决策, 2001
[35]马香峰.机器人机构学.机械工业出版社. 1991
[36]李现勇. Visual C++串口通信技术与工程实践.人民邮电出版社. 2004
[37]梶田秀司,管贻生.仿人机器人.清华大学出版社. 2007
[38]钟玉琢,乔秉新,李树青.机器人视觉技术.国防工业出版社. 1994
[39]蔡自兴.机器人学.清华大学出版社. 2000
[40]何斌,马天予.Visual C++数字图像处理.北京人民邮电出版社. 2002
[41] David G. Lowe.“Distinctive image features from scale-invariant keypoints”. International Journal of Computer Vision. 2004.
[42] Tony Linderberg.“Scale-Space for Discrete signals”[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, VOL.12, NO.3. 1990.
[43] Moore A W.”An Introductory Tutorial on KD trees”[R].UK:University of Cambridge, 1991
[44]厉茂海,洪炳镕,罗荣华.“基于单目视觉的移动机器人全局定位”[J].机器人,2007, 29(2):140-144.
[45] Kass M, Witkin A, Terzopoulouse D.“Snakes:active contour model”[J]. International Journal of Computer Vision. 1987, 1(4) pp 321-331.
[46] Saeed B.Niku著.孙富春等译.机器人学导论-分析、系统及应用.北京:电子工业出版社. 2004.1.
[47] Ganghua Sun, Brian Scassellati.“A fast and efficient model for learning to reach”. International Journal of Humanoid Robotics. Vol. 2, No. 4 (2005) pp 391–413
[48] Dietmar Hildenbrand, Julio Zamora and Eduardo Bayro-Corrochano.“Inverse Kinematics Computation in Computer Graphics and Robotics Using Conformal Geometric Algebra”. Adv. appl. Clifford alg. Online First. 2008. DOI 10.1007/s00006-008-0096-5
[49]郝宗波.家庭移动服务机器人的若干关键技术研究:[博士学位论文].哈尔滨工业大学计算机科学与技术学院. 2006.
[50] Mario Prats, Pedro J. Sanz, Angel P. Del Pobil.“A framework for compliant physical interaction-The grasp meets the task”. Auton Robot (2010) 28: pp 89–111.
[51] William J. Wilson, Williams Hulls, Graham S. Bell.“Relative End-Effector Control Using Cartesian Position Based Visual Servoing”. IEEE Transactions on Robotics and Automation, VOL 12, NO. 5, 1996.
[52] Mario Prats, Pedro J. Sanz, Angel P. Del Pobil.“Task-Oriented Grasping using Hand Preshapes and Task Frames”. IEEE International Conference on Robotics and Automation. 2007. pp 10-14.
[53] Radu Horaud, Fadi Dornaika, Bernard Espiau.“Visually Guided Object Grasping”. IEEE Transactions on Robotics and Automation. VOL. 14, NO. 4, AUGUST 1998.
[54] Luis Gracia, Carlos Perez-Vidal.“A New Control Scheme for Visual Servoing”. International Journal of Control, Automation, and Systems (2009) 7(5):764-776.
[55] J. Fuentes‐ Pacheco, J. Ruiz‐Ascencio, J. M. Rendón‐ Mancha.“Binocular visual tracking and grasping of a moving object with a 3D trajectory predictor”. Journal of Applied Research and Technology, Vol.7 No. 3:255-274, 2009.
[56] Rigas Kouskouridas, Efthimios Badekas, and Antonios Gasteratos.“Simultaneous Visual Object Recognition and Position Estimation Using SIFT”. ICIRA 2009, LNAI 5928, pp:866-875, 2009.
[57]张毅,孙虎元,孙立娟,孙晓光.“边缘检测和Snake Model结合的轮廓识别”[J].计算机工程与应用. 2009,45(26). ????????
[2] M.Arlotti and M.Grnaieri,“A Perception technique for a 3D robotic stereo eye-in-hand Vision system”The Fifth International Conference on Advanced Robotics,“Robots in Unstructured Environments”1991,pp.1626-1629.
[3] E.Malis, F.Chaumette and S.Boudet,“Multi-camera visual servoing”IEEE International Conference on Robotics and Automation, 2000, vol 3, pp. 3183-3188.
[4] R.Andersson,“Understanding and applying a robot ping-pong player’s expert controller”, IEEE International Conference on Robotics and Automation, 1989, vol 3, pp. 1284-1289
[5] R.Andersson,“Dynamic sensing in ping-pong playing robot”, IEEE Transactions on Robotics and Automation, 1989 ,5(6):728-739.
[6] G.Hager,“Calibration-free visual control using projective invariance”International Conference on Computer Vision,1995, pp, 1009-1015.
[7] G.Hager, W.Chang and A.Morse“Robot hand-eye coordination based on stereo vision”IEEE Control System Magazine, 1995, 15(1), 30-39.
[8] G.Hager,“A modular system for robust positioning using feedback from stereo vision”IEEE Transactions on Robotics and Automation, 1997 ,13(4):582-595.
[9] Reyes F. and R. Kelly,“Experimental Evaluation of Fixed-Camera Direct Visual Controllers on a Direct-Drive Robot”. IEEE International Conference on Robotics & Automation. 1998, pp. 2327-2332
[10] Malis E. and S. Benhimane,“A Unified Approach to Visual Tracking and Servoing”. Robotics and Autonomous Systems, Vol. 52, 2005, pp. 39-52
[11] Park J. and Y.J. Lee,“Robust Visual Servoing for Motion Control of the Ball on a Plate”. Mechatronics, Vol. 13, 2003, pp. 723-738.
[12] Jaime Cid M. and Fernando Reyes C.“Visual servoing controller for robot manipulators”. IEEE International Conference on Electrical, 2009,pp 155-18.
[13] Mario Prats, Philippe Martinet and Angel P.“Robotic execution of everyday tasks by means of external vision/force control”Intel Serv Robotics, 2008, pp 253-266.
[14] R. S. Hartenberg and J. Denavit,“A kinematic notation for lower pair mechanisms based on matrices,”Journal of Applied Mechanics, vol. 77, pp. 215–221, June 1955.
[15] C. S. G. Lee,“Robot arm kinematics, dynamics and control,”IEEE Computer, vol. 15, pp. 62–80, Dec. 1982.
[16] J.J.Craig,“Introduction to Robotics”. Third Edition. 2004
[17] M. Spong and M. Vidyasagar,“Robot Dynamics and Control”. John Wiley and Sons, 1989.
[18] Hutchinson, G. Hager, and P. Corke,“A tutorial on visual servo control,”IEEE Transactions on Robotics and Automation, vol. 12, pp. 651–670, Oct. 1996.
[19]张广军.机器视觉.北京:科学出版社, 2005. pp 69-98.
[20] Z.Zhnag.“A flexible new technique for camera calibration”. Technical Report MSR-TR-98-71, Microsoft Research,Dee.1998.
[21] Juyang Weng, Paul Cohen and Marc Herniou.“Calibration of stereo Cameras Using a Non-linear Distortion Model”. Proc. International Conference on Pattern Recognition, 1990. Pp 245-253.
[22]马颂德,张正友.计算机视觉:计算理论与算法基础.科学出版社, 1998. pp 52-70.
[23] R.Y.Tsai.“A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses”. IEEE J. of Robotics Automation,1987,3(4) pp 323-344.
[24] Juyang Weng, Paul Cohen and Marc Herniou.“Camera Calibration with Distortion Models and Accuracy Evaluation”. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1992, 14(10) pp 965-980.
[25] R. P. Paul, Robot Manipulators: Mathematics, Programming, and Control. Cambridge, Massachusetts: MIT Press, 1981.
[26]高文,陈熙霖.计算机视觉:算法与系统原理.清华大学出版社,1999.
[27] Barnard.S.T, Fischer.M.A.“Computational stereo”. Computaing Surveys. 1982.14 pp 553-572
[28] Levine.M.D.“Computer determination of depth maps”. Computer Graphics and Image Process, 1972. 2 pp 131-150.
[29] Moravec.H.D.“Towards automatic visual obstacle avoidance”. proc. 5th IJCAI, 1977.
[30]章毓晋.图像工程上册:图象处理和分析,清华大学出版社,1999
[31]肖南峰,智能机器人.华南理工大学出版社.2008
[32]吴立得.计算机视觉.上海:复旦大学出版社.1993
[33] NGUGENM C.S“intuition-oriented behavior-based stereo vision to gain robustness and adaptation in manipulator control”[A]. Proc Of SPIE[C].Massachusetts, 1999(3837)
[34]赵清杰,连广宇,孙增昕.机器人视觉伺服综述[J].控制与决策, 2001
[35]马香峰.机器人机构学.机械工业出版社. 1991
[36]李现勇. Visual C++串口通信技术与工程实践.人民邮电出版社. 2004
[37]梶田秀司,管贻生.仿人机器人.清华大学出版社. 2007
[38]钟玉琢,乔秉新,李树青.机器人视觉技术.国防工业出版社. 1994
[39]蔡自兴.机器人学.清华大学出版社. 2000
[40]何斌,马天予.Visual C++数字图像处理.北京人民邮电出版社. 2002
[41] David G. Lowe.“Distinctive image features from scale-invariant keypoints”. International Journal of Computer Vision. 2004.
[42] Tony Linderberg.“Scale-Space for Discrete signals”[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, VOL.12, NO.3. 1990.
[43] Moore A W.”An Introductory Tutorial on KD trees”[R].UK:University of Cambridge, 1991
[44]厉茂海,洪炳镕,罗荣华.“基于单目视觉的移动机器人全局定位”[J].机器人,2007, 29(2):140-144.
[45] Kass M, Witkin A, Terzopoulouse D.“Snakes:active contour model”[J]. International Journal of Computer Vision. 1987, 1(4) pp 321-331.
[46] Saeed B.Niku著.孙富春等译.机器人学导论-分析、系统及应用.北京:电子工业出版社. 2004.1.
[47] Ganghua Sun, Brian Scassellati.“A fast and efficient model for learning to reach”. International Journal of Humanoid Robotics. Vol. 2, No. 4 (2005) pp 391–413
[48] Dietmar Hildenbrand, Julio Zamora and Eduardo Bayro-Corrochano.“Inverse Kinematics Computation in Computer Graphics and Robotics Using Conformal Geometric Algebra”. Adv. appl. Clifford alg. Online First. 2008. DOI 10.1007/s00006-008-0096-5
[49]郝宗波.家庭移动服务机器人的若干关键技术研究:[博士学位论文].哈尔滨工业大学计算机科学与技术学院. 2006.
[50] Mario Prats, Pedro J. Sanz, Angel P. Del Pobil.“A framework for compliant physical interaction-The grasp meets the task”. Auton Robot (2010) 28: pp 89–111.
[51] William J. Wilson, Williams Hulls, Graham S. Bell.“Relative End-Effector Control Using Cartesian Position Based Visual Servoing”. IEEE Transactions on Robotics and Automation, VOL 12, NO. 5, 1996.
[52] Mario Prats, Pedro J. Sanz, Angel P. Del Pobil.“Task-Oriented Grasping using Hand Preshapes and Task Frames”. IEEE International Conference on Robotics and Automation. 2007. pp 10-14.
[53] Radu Horaud, Fadi Dornaika, Bernard Espiau.“Visually Guided Object Grasping”. IEEE Transactions on Robotics and Automation. VOL. 14, NO. 4, AUGUST 1998.
[54] Luis Gracia, Carlos Perez-Vidal.“A New Control Scheme for Visual Servoing”. International Journal of Control, Automation, and Systems (2009) 7(5):764-776.
[55] J. Fuentes‐ Pacheco, J. Ruiz‐Ascencio, J. M. Rendón‐ Mancha.“Binocular visual tracking and grasping of a moving object with a 3D trajectory predictor”. Journal of Applied Research and Technology, Vol.7 No. 3:255-274, 2009.
[56] Rigas Kouskouridas, Efthimios Badekas, and Antonios Gasteratos.“Simultaneous Visual Object Recognition and Position Estimation Using SIFT”. ICIRA 2009, LNAI 5928, pp:866-875, 2009.
[57]张毅,孙虎元,孙立娟,孙晓光.“边缘检测和Snake Model结合的轮廓识别”[J].计算机工程与应用. 2009,45(26). ????????