并联机器人双目主动视觉平台刀具定位及位姿检测研究
|
摘要
为了解决并联机器人工作状态的主动认知问题,引入了立体视觉技术,构建了一种基于圆形导轨的双目主动视觉监测平台,以期实时获取并联机器人加工刀具的状态信息,对提高并联机器人运动准确性,推动并联机器人实用化,具有重要的理论意义和技术价值。本文对并联机器人双目主动视觉监测平台中控制系统,刀具视觉定位及其位姿检测进行了分析和研究。
首先,从计算机视觉的基本理论开始,对双目主动视觉监测平台上摄像机模型进行分析,阐述了图像坐标、摄像机坐标、世界坐标之间的转换关系;研究了极线约束原理以及点、线、曲线的三维重建原理。
其次,对双目主动视觉监测平台电气控制原理进行了分析,对平台上伺服电机运动模块、数字云台控制模块和摄像机控制模块及它们之间的协作进行研究,实现了视觉服务系统与电气控制系统之间的交互,是刀具视觉定位时摄像机姿态调整的控制基础。
再次,为了得到质量较好的刀具图像,给出了刀具的视觉定位方法,以调整摄像机姿态,使刀具形心位于其光轴。根据图像中的刀具的特征,在图像中搜寻刀具的形心位置,在此基础上,得到摄像机摆角和俯仰角的调整角度,选取调整后刀具所在的区域作为位姿检测的原图像。
最后,在得到刀具图像子区域的基础上,研究了加工刀具的位姿检测方法。提取刀具角点作为特征点,特征点所在极线两侧的区域为特征区域,根据对极几何的立体成像关系,对特征区域进行方向校正,通过相似性求取同名点,并对其三维重建。实验结果表明,此方法可提高图像匹配精度和速度,能够精确地对加工刀具位姿进行检测。
To solve the state initiative recognizing problem of the parallel robot, stereo vision technology is introduced into it. And we built a circular guide rail based binocular active visual monitoring platform (BAVMP) to acquire the cutting tool state information in real time. The BAVMP is of significant theory importance and technology value in parallel robot accuracy and practicality. In this paper, we analyze and study the control system and the the cutting tool's image collection and position and pose detection based on the BAVMP.
Firstly, basic computer vision theory was introduced. And the camera model of BAVMP was analyzed. Then image coordinate system, camera coordinate system and world coordinate system were expatiated. Then we studied the epipolar constraint theory and three-dimensional reconstruction theory on point, line and curve.
Secondly, the electric control function and the interface between the vision service system and it were analyzed. We studied the servo motor sports module, the digital PTZ control module, the camera control module and the collaboration among them. The communications between the electric control function and the vision service system were completed. So did the image data collection.
Thirdly, to obstain the better cutting tool image, we give a vision location method. We make the cutting tool centroid on the optic axis by adjusting the camera pose. Centroid position is searched according to the cutting tool feature. Then the adjusted swing angle and pitching angle is computed. And then the cutting tool-in area is transfered.
Finally, based the cutting tool-in small area, cutting tool's position and pose detection method was studied. The inflection points extracted from the cutting tool edge images are regarded as the characteristic points. Based on epipolar geometry theory, the local regions whose center is characteristic points is corrected in the orientation and size. Then the space position is computed by mean of 3D reconstruction.In this paper,the experimental results show that this method can improve matching accuracy and speed, and is able to detect position and pose of cutting tool more accurately.
首先,从计算机视觉的基本理论开始,对双目主动视觉监测平台上摄像机模型进行分析,阐述了图像坐标、摄像机坐标、世界坐标之间的转换关系;研究了极线约束原理以及点、线、曲线的三维重建原理。
其次,对双目主动视觉监测平台电气控制原理进行了分析,对平台上伺服电机运动模块、数字云台控制模块和摄像机控制模块及它们之间的协作进行研究,实现了视觉服务系统与电气控制系统之间的交互,是刀具视觉定位时摄像机姿态调整的控制基础。
再次,为了得到质量较好的刀具图像,给出了刀具的视觉定位方法,以调整摄像机姿态,使刀具形心位于其光轴。根据图像中的刀具的特征,在图像中搜寻刀具的形心位置,在此基础上,得到摄像机摆角和俯仰角的调整角度,选取调整后刀具所在的区域作为位姿检测的原图像。
最后,在得到刀具图像子区域的基础上,研究了加工刀具的位姿检测方法。提取刀具角点作为特征点,特征点所在极线两侧的区域为特征区域,根据对极几何的立体成像关系,对特征区域进行方向校正,通过相似性求取同名点,并对其三维重建。实验结果表明,此方法可提高图像匹配精度和速度,能够精确地对加工刀具位姿进行检测。
To solve the state initiative recognizing problem of the parallel robot, stereo vision technology is introduced into it. And we built a circular guide rail based binocular active visual monitoring platform (BAVMP) to acquire the cutting tool state information in real time. The BAVMP is of significant theory importance and technology value in parallel robot accuracy and practicality. In this paper, we analyze and study the control system and the the cutting tool's image collection and position and pose detection based on the BAVMP.
Firstly, basic computer vision theory was introduced. And the camera model of BAVMP was analyzed. Then image coordinate system, camera coordinate system and world coordinate system were expatiated. Then we studied the epipolar constraint theory and three-dimensional reconstruction theory on point, line and curve.
Secondly, the electric control function and the interface between the vision service system and it were analyzed. We studied the servo motor sports module, the digital PTZ control module, the camera control module and the collaboration among them. The communications between the electric control function and the vision service system were completed. So did the image data collection.
Thirdly, to obstain the better cutting tool image, we give a vision location method. We make the cutting tool centroid on the optic axis by adjusting the camera pose. Centroid position is searched according to the cutting tool feature. Then the adjusted swing angle and pitching angle is computed. And then the cutting tool-in area is transfered.
Finally, based the cutting tool-in small area, cutting tool's position and pose detection method was studied. The inflection points extracted from the cutting tool edge images are regarded as the characteristic points. Based on epipolar geometry theory, the local regions whose center is characteristic points is corrected in the orientation and size. Then the space position is computed by mean of 3D reconstruction.In this paper,the experimental results show that this method can improve matching accuracy and speed, and is able to detect position and pose of cutting tool more accurately.
引文
1张世辉,孔令富.一种新型6-PUS并联机构雕刻机.机器人,2005,27(4):313~318
2李立新,潘春萍,卢颖.六轴并联机器人控制软件的设计与实现.长春理工大学学报, 2004,27(1):51~53
3李强,闫洪波,张玉宝.并联机床发展的历史、研究现状与展望.机床与液压, 2007,35(3):206~209
4 Kong Lingfu, Zhang Shihui. A Novel Parallel Engraving Machine Based on 6-PUS M- echanism and Related Technologies. V.Kordic, A.Lazinica, M.Merdan.Cutting Edge R- obotics, Austria, 2005. Austria, Pro Literatur Verlag,Germany ARS,2005:629~648
5张世辉.并联机器人汉字雕刻技术的研究. [燕山大学工学博士学位论文].2005:10 ~120
6张世辉,孔令富,刘天洋.一种面向并联机构的雕刻刀具系统设计.机器人, 2007, 29(3):244~249
7 P.Renaud, N.Andreff. Vision-based Kinematic Calibration of A H4 Parallel Mechanics- m, In Proceedings of The 2003 IEEE International Conference on Robotics and Auto- mation. Taipei, Taiwan. September 2003:1191~1196
8 P. Renaud, N. Andreff. Kinematic Calibration of Parallel Mechanism: A Novel Approa- ch Using Legs Observatio. IEEE Transactions on Robotics, August 2005 21(4):529~53 8
9 N. Andreff, P. Martinet. Vision Servoing of a Gough-Stewart Parallel Robot without P- roprioceptive Sensors. Fifth International Workshop on Robot Motion and Control, Ju- ne 23-25, 2005:225~230
10陈梅兰.机器视觉关键技术与应用实例分析. MODERN COMPUTER. 2006, 1:21~ 25
11 Z. Huang, Y. Cao. Property Identification of The Singularity Loci of A Class of Gough- Stewart Manipulators. International Journal of Robotics Research, 2005,24(8):675~685
12 Marr O., Vision, W. H. Freeman and Company,San Francisco.中译本:视觉计算理论.姚国正,刘磊,汪云九译.北京:科学出版社, 1988
13 R. Bajcsy, Active Perception vs. Passive Perception, Proceedings Third IEEE Work- shop on Computer Vision. Bellair, MI, October 1985:55~59
14余洪山,王耀南.主动立体双目视觉平台的设计与实现.工业仪器与自动化装置,2004,(1):61~63.
15岁波,都东,陈强.基于双目视觉的工业机器人运动轨迹准确度检测.机械工程学报,2003,39(5):88~91.
16刘江华,陈佳品,程君实.双目视觉平台的研究.机器人技术与应用,2002,(1):36~40.
17 R. Peters,M. Bishay. Centering Peripheral Features in an Indoor Environment Using a Binocular Log-polar 4 DOF Camera Head. Journal of Robotics and Autonomous Systems, 1996, 18:271~282
18 D. H. Ballard. Animate Vision, Artificial Intelligence, 1991, 48:57-86
19高庆吉,洪炳熔,阮玉峰.基于异构双目视觉的全自主足球机器人导航.哈尔滨工业大学学报, 2003, 35(9):1029~1032
20 K. Pahlavan, J. O. Eklundh. Heads, Eyes, and Head-eye Systems, Proc SPIE on applications of AIX, Machine Vision and Robotics, Orlando, 1992:14~25
21 D. H. Ballard. Animate Vision, Artificial Intelligence, 1991, 48:57~86
22 T.S. Huang, A.R. Netravali. Motion and Structure from Feature Correspondences: A Review, Proceedings of the IEEE, 1994, 82(2):252~268
23 S.D. Blostein, T.S. Huang. Error Analysis in Stereo Determination of 3-D Point Positions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1987, 9(6):752~765
24 Z. Zhang, O. Faugeras. Estimation of Displacements from Two 3-D Frames Obtained from Stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1999, 14(12):1141~1156
25 J. Batista, P. Peixoto, H. Araujo. Real-Time Visual Behaviors with a Binocular Active Vision System, Proc of the 1996 IEEE/SICE/RSJ International Conference on Multi-se- nsor Fusion and Integration for Intelligent Systems, 1996:663~670
26 X. Roca, J. Vitria, M. Vanrell, J.J.Villanueva. Gaze Control in Binocular Robot Systems. Emerging Technologies and Factory Automation, 1999. Proceedings. ETFA '99. 1999:479~485
27 M. Asada, T. Tanaka, K. Hosoda. Adaptive Binocular Visual Servoing for Independen- tly Moving Target Tracking, Proc of the 2000 IEEE International Conference on Rob- otics & Automation, San Francisco, 2000:2076~2081
28 Asada. M, Tanaka.T. Hosada. K. Visual Tracking of Unknown Moving Object By Ada- ptive Binocular Visual Servoing. Proceeding. IEEE. International Conference on Mul- tisensor Fusion and Intelligent Systems, 1999:249~254
29 Kei Okada, Masayuki Inaba. Integration of Real-Time Binocular Stereo Vision And Whole Body Information For Dynamic Walking Navigation of Humanoid Robot. IEEE conference on Multisensor Fusion and Intergration for Intelligent Systems 2003:131~ 136
30 Clark F, Olson, Habib Abi-Rached, Ming Ye, Jonathan P. Hendrich. Wide-Baseline Stereo Vision For Mars Rovers. Proceedings of The 2003 IEEE/RSJ Intl. Conference On Intelligent Robots And Systerms. October 2003(2):1302~1307
31 O. Avni, F. Borrelli, G. Katzir, E. Rivlin, H. Rotstein. Scanning the Environment with Two Independent Cameras - Biologically Motivated Approach, In Proc. IEEE/RSJ Inte. Conf. on Intelligent Robots and Systems, Beijing, China, 2006:5297~5302
32 E. Samson, D. Laurendeau, M. Parizeau1 etc. The Agile Stereo Pair for Active Vision. Machine Vision and Applications, Published by Springer-Verlag, 2006, 17(1):32~50
33赵晓光,谭民,汪建华.一种五自由度立体视觉监控装置:中国, 200410068998.5, 2004.07.15
34孔令富,王月明,赵立强.雕刻并联机器人刀具导向期望加工位置研究.高技术通讯, 2007, 17(10):1039~1043
35孔令富,王月明,赵立强.并联机器人双目主动视觉目标定位的研究.计算机集成制造系统, 2007, 13(11):2284~2288
36马颂德,张正友.计算机视觉---计算理论与算法基础.北京:科学出版社, 2003:52~78
37 Weng J, Cohen P, Herniou M. Camera Calibration with Distortion Models And Accuracy Evaluation. IEEE Trails on PAMI, 1992, 14(10):965~980.
38 Wang J H, Shi F H, Zhang J, eta1. A New Calibration Model And Method of Camera Lens distortion, Proc of IEEE lnt Conf Oil Intdligent Robot and Systems. Bering:IEEE Press, 2006, 5713~5718
39 Hu M K.Visual pattern recognition by moment invariants. IEEE Transactions on Information Theory. 1962,8(2):179~187.
40 M.K.Hu. Pattern Recognition by Moments Invariants. Proc. IRE, 1961, vol.49:1427~ 1428.
41 Xiaolong Dai and Siamak Khorram, A feature- Based Image Registration Algorithm Using Improved Chain-Code Representation Combined with Invariant Moments, IEEE Trans. on Geoscience and Remote Sensing, 1999, 37(5), 2351~2362.
42窦燕,孔令富.一种基于视觉注意机制的刀具检测方法.中国机械工程,2008,1 9(17):2024~2027.
43 Marr D,Hildreth E C. Theory of Edge Detection. Proc R Soc Lond, 1980,B(207):18 7~217
44 H.Freeman. On the encoding of arbitrary geometric configurations. IRE Trans, Electron Comput. 1961 EC-10,:250~268
45罗翔,席文明,颜景平.一种双目主动立体视觉系统的目标定位算法.东南大学学报, 2002,32(1)59~63
46 Shirin Mahmoudi Barmas,Shohreh Kasaei. Contourlet-Based Edge Extraction for Image Registration. Iranian Journal of Electrical and Electronic Engineering. 2008, 4(1-2):17~34.
47李立春,张小虎,傅丹.基于极线局部校正的特征匹配方法.光学技术, 2008, 34(2):285~288.
48 RA YMOND V E, CLIFTON M S. Unconstrainted Stereoscopic Matching of Lines. Vision Research. 2000(40):151~162.
49 KANADE T, OKUTOMI M. A Stereo Matching Algorighm with An Adaptive Windo- w. Theory And Experiment,1994,16(9):920~931
50李德广,李科杰,高丽丽.基于多尺度多方向相位匹配的立体视觉方法.仪器仪表学报, 2004, 25(4):600~602
2李立新,潘春萍,卢颖.六轴并联机器人控制软件的设计与实现.长春理工大学学报, 2004,27(1):51~53
3李强,闫洪波,张玉宝.并联机床发展的历史、研究现状与展望.机床与液压, 2007,35(3):206~209
4 Kong Lingfu, Zhang Shihui. A Novel Parallel Engraving Machine Based on 6-PUS M- echanism and Related Technologies. V.Kordic, A.Lazinica, M.Merdan.Cutting Edge R- obotics, Austria, 2005. Austria, Pro Literatur Verlag,Germany ARS,2005:629~648
5张世辉.并联机器人汉字雕刻技术的研究. [燕山大学工学博士学位论文].2005:10 ~120
6张世辉,孔令富,刘天洋.一种面向并联机构的雕刻刀具系统设计.机器人, 2007, 29(3):244~249
7 P.Renaud, N.Andreff. Vision-based Kinematic Calibration of A H4 Parallel Mechanics- m, In Proceedings of The 2003 IEEE International Conference on Robotics and Auto- mation. Taipei, Taiwan. September 2003:1191~1196
8 P. Renaud, N. Andreff. Kinematic Calibration of Parallel Mechanism: A Novel Approa- ch Using Legs Observatio. IEEE Transactions on Robotics, August 2005 21(4):529~53 8
9 N. Andreff, P. Martinet. Vision Servoing of a Gough-Stewart Parallel Robot without P- roprioceptive Sensors. Fifth International Workshop on Robot Motion and Control, Ju- ne 23-25, 2005:225~230
10陈梅兰.机器视觉关键技术与应用实例分析. MODERN COMPUTER. 2006, 1:21~ 25
11 Z. Huang, Y. Cao. Property Identification of The Singularity Loci of A Class of Gough- Stewart Manipulators. International Journal of Robotics Research, 2005,24(8):675~685
12 Marr O., Vision, W. H. Freeman and Company,San Francisco.中译本:视觉计算理论.姚国正,刘磊,汪云九译.北京:科学出版社, 1988
13 R. Bajcsy, Active Perception vs. Passive Perception, Proceedings Third IEEE Work- shop on Computer Vision. Bellair, MI, October 1985:55~59
14余洪山,王耀南.主动立体双目视觉平台的设计与实现.工业仪器与自动化装置,2004,(1):61~63.
15岁波,都东,陈强.基于双目视觉的工业机器人运动轨迹准确度检测.机械工程学报,2003,39(5):88~91.
16刘江华,陈佳品,程君实.双目视觉平台的研究.机器人技术与应用,2002,(1):36~40.
17 R. Peters,M. Bishay. Centering Peripheral Features in an Indoor Environment Using a Binocular Log-polar 4 DOF Camera Head. Journal of Robotics and Autonomous Systems, 1996, 18:271~282
18 D. H. Ballard. Animate Vision, Artificial Intelligence, 1991, 48:57-86
19高庆吉,洪炳熔,阮玉峰.基于异构双目视觉的全自主足球机器人导航.哈尔滨工业大学学报, 2003, 35(9):1029~1032
20 K. Pahlavan, J. O. Eklundh. Heads, Eyes, and Head-eye Systems, Proc SPIE on applications of AIX, Machine Vision and Robotics, Orlando, 1992:14~25
21 D. H. Ballard. Animate Vision, Artificial Intelligence, 1991, 48:57~86
22 T.S. Huang, A.R. Netravali. Motion and Structure from Feature Correspondences: A Review, Proceedings of the IEEE, 1994, 82(2):252~268
23 S.D. Blostein, T.S. Huang. Error Analysis in Stereo Determination of 3-D Point Positions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1987, 9(6):752~765
24 Z. Zhang, O. Faugeras. Estimation of Displacements from Two 3-D Frames Obtained from Stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1999, 14(12):1141~1156
25 J. Batista, P. Peixoto, H. Araujo. Real-Time Visual Behaviors with a Binocular Active Vision System, Proc of the 1996 IEEE/SICE/RSJ International Conference on Multi-se- nsor Fusion and Integration for Intelligent Systems, 1996:663~670
26 X. Roca, J. Vitria, M. Vanrell, J.J.Villanueva. Gaze Control in Binocular Robot Systems. Emerging Technologies and Factory Automation, 1999. Proceedings. ETFA '99. 1999:479~485
27 M. Asada, T. Tanaka, K. Hosoda. Adaptive Binocular Visual Servoing for Independen- tly Moving Target Tracking, Proc of the 2000 IEEE International Conference on Rob- otics & Automation, San Francisco, 2000:2076~2081
28 Asada. M, Tanaka.T. Hosada. K. Visual Tracking of Unknown Moving Object By Ada- ptive Binocular Visual Servoing. Proceeding. IEEE. International Conference on Mul- tisensor Fusion and Intelligent Systems, 1999:249~254
29 Kei Okada, Masayuki Inaba. Integration of Real-Time Binocular Stereo Vision And Whole Body Information For Dynamic Walking Navigation of Humanoid Robot. IEEE conference on Multisensor Fusion and Intergration for Intelligent Systems 2003:131~ 136
30 Clark F, Olson, Habib Abi-Rached, Ming Ye, Jonathan P. Hendrich. Wide-Baseline Stereo Vision For Mars Rovers. Proceedings of The 2003 IEEE/RSJ Intl. Conference On Intelligent Robots And Systerms. October 2003(2):1302~1307
31 O. Avni, F. Borrelli, G. Katzir, E. Rivlin, H. Rotstein. Scanning the Environment with Two Independent Cameras - Biologically Motivated Approach, In Proc. IEEE/RSJ Inte. Conf. on Intelligent Robots and Systems, Beijing, China, 2006:5297~5302
32 E. Samson, D. Laurendeau, M. Parizeau1 etc. The Agile Stereo Pair for Active Vision. Machine Vision and Applications, Published by Springer-Verlag, 2006, 17(1):32~50
33赵晓光,谭民,汪建华.一种五自由度立体视觉监控装置:中国, 200410068998.5, 2004.07.15
34孔令富,王月明,赵立强.雕刻并联机器人刀具导向期望加工位置研究.高技术通讯, 2007, 17(10):1039~1043
35孔令富,王月明,赵立强.并联机器人双目主动视觉目标定位的研究.计算机集成制造系统, 2007, 13(11):2284~2288
36马颂德,张正友.计算机视觉---计算理论与算法基础.北京:科学出版社, 2003:52~78
37 Weng J, Cohen P, Herniou M. Camera Calibration with Distortion Models And Accuracy Evaluation. IEEE Trails on PAMI, 1992, 14(10):965~980.
38 Wang J H, Shi F H, Zhang J, eta1. A New Calibration Model And Method of Camera Lens distortion, Proc of IEEE lnt Conf Oil Intdligent Robot and Systems. Bering:IEEE Press, 2006, 5713~5718
39 Hu M K.Visual pattern recognition by moment invariants. IEEE Transactions on Information Theory. 1962,8(2):179~187.
40 M.K.Hu. Pattern Recognition by Moments Invariants. Proc. IRE, 1961, vol.49:1427~ 1428.
41 Xiaolong Dai and Siamak Khorram, A feature- Based Image Registration Algorithm Using Improved Chain-Code Representation Combined with Invariant Moments, IEEE Trans. on Geoscience and Remote Sensing, 1999, 37(5), 2351~2362.
42窦燕,孔令富.一种基于视觉注意机制的刀具检测方法.中国机械工程,2008,1 9(17):2024~2027.
43 Marr D,Hildreth E C. Theory of Edge Detection. Proc R Soc Lond, 1980,B(207):18 7~217
44 H.Freeman. On the encoding of arbitrary geometric configurations. IRE Trans, Electron Comput. 1961 EC-10,:250~268
45罗翔,席文明,颜景平.一种双目主动立体视觉系统的目标定位算法.东南大学学报, 2002,32(1)59~63
46 Shirin Mahmoudi Barmas,Shohreh Kasaei. Contourlet-Based Edge Extraction for Image Registration. Iranian Journal of Electrical and Electronic Engineering. 2008, 4(1-2):17~34.
47李立春,张小虎,傅丹.基于极线局部校正的特征匹配方法.光学技术, 2008, 34(2):285~288.
48 RA YMOND V E, CLIFTON M S. Unconstrainted Stereoscopic Matching of Lines. Vision Research. 2000(40):151~162.
49 KANADE T, OKUTOMI M. A Stereo Matching Algorighm with An Adaptive Windo- w. Theory And Experiment,1994,16(9):920~931
50李德广,李科杰,高丽丽.基于多尺度多方向相位匹配的立体视觉方法.仪器仪表学报, 2004, 25(4):600~602