采棉机器人视觉系统的关键技术研究
|
摘要
采棉机器人是未来机械采棉的发展方向，具有广阔的应用前景。开展采棉机器人的研究对于市场需求、提高经济效率和降低劳动强度都有着重要的现实意义。自然环境下成熟棉花的识别与定位技术是采棉机器人视觉系统的关键技术，本研究以田间成熟期的棉花为研究对象，主要的研究内容包括：
在成熟棉花识别方面，本研究提出了一种基于YCbCr颜色空间下利用Fisher判别分析的成熟棉花的分割策略。利用摄像机拍摄不同光照下的棉花图像，对比分析了在RGB、HSV和YCbCr颜色空间下利用Fisher判别分析的分割效果，并与目前已提出的几种棉花分割策略做了对比分析。为了保证成熟棉花的边界信息，本章采用贴标签的处理方法很好的将成熟棉花区域分割出来。实验证明本章提出的方法有效避免了阳光直射和阴影的干扰，分割准确率是90.44%。
在成熟棉花定位方面，本研究对双目立体视觉模型、摄像机成像模型和标定理论进行了研究，采用张正友标定方法对本研究的双目立体视觉系统进行了标定。提出了多目标情况下确定各个采摘点的方法，采用了基于区域相似匹配和约束条件的方法进行匹配。本研究根据三角测量方法恢复特征点三维坐标，并对该方法进行了分析和实验，实验结果表明该方法能够满足采棉机器人视觉系统的需求。由于SIFT算法的复杂度高，提取的关键点不是角点，本研究先采用Harris算子提取分割后的棉花图像的特征点，然后计算每个特征点的SIFT向量的描述符，后面使用SIFT特征匹配方法匹配，实验表明本方法得到的匹配点更适合恢复棉花的形态。
The application of cotton-picking robot will be a development trend of agriculturalmechanization and has a great prospect. Researches in cotton-picking robot have greatsignificance for reducing working intension, increasing economy efficiency and adaptingrequirement of market. The cotton-picking robot, the cotton identification andlocation technology in the natural environment is the key technology. The maturity cotton in thenatural environment was the research object. The two point in the research were as follow:
In recognition of the mature cotton, the study proposed a mature cotton segmention strategybased on the YCbCr color space and Fisher discriminant analysis. First the research shoted thescene under different illumination use CCD camera, then contrasted analysis of segmentationresult in RGB,HSV, and YCbCr color space using Fisher discriminant analysis, and comparedwith proposed segmentation strategy. In order to ensure the boundaries information of the maturecotton, the research used the method of labeling to denoise. The simulation result that the cottoncould be separated exactly from background by the above algorithm whether the cotton wasexposed to the sunlight or the shadow with an accuracy of90.44%.
In location of the mature cotton, the research studied the stereo vision theory, cameraimaging model and calibration theory, and used Zhang Zhengyou calibrating method to calibratethe binocular stereo vision system. The research proposed a method to determine picking points,used region-match and specific constraints to match them, accorded to the triangulation method torestore the three-dimensional coordinates. The experimental results that the method the methodcan meet the demand for cotton-picking robot vision system. Due to the key points of the SIFTalgorithm is not corners, so the research used Harris method to extract corners in segmented cottonimage, and then calculated the SIFT descriptor vector of each feature point, and finally used theSIFT matching method to match them. The experiments showed that the matching point obtainedby this method was more suitable for the restoration of the morphology of the cotton.
在成熟棉花识别方面，本研究提出了一种基于YCbCr颜色空间下利用Fisher判别分析的成熟棉花的分割策略。利用摄像机拍摄不同光照下的棉花图像，对比分析了在RGB、HSV和YCbCr颜色空间下利用Fisher判别分析的分割效果，并与目前已提出的几种棉花分割策略做了对比分析。为了保证成熟棉花的边界信息，本章采用贴标签的处理方法很好的将成熟棉花区域分割出来。实验证明本章提出的方法有效避免了阳光直射和阴影的干扰，分割准确率是90.44%。
在成熟棉花定位方面，本研究对双目立体视觉模型、摄像机成像模型和标定理论进行了研究，采用张正友标定方法对本研究的双目立体视觉系统进行了标定。提出了多目标情况下确定各个采摘点的方法，采用了基于区域相似匹配和约束条件的方法进行匹配。本研究根据三角测量方法恢复特征点三维坐标，并对该方法进行了分析和实验，实验结果表明该方法能够满足采棉机器人视觉系统的需求。由于SIFT算法的复杂度高，提取的关键点不是角点，本研究先采用Harris算子提取分割后的棉花图像的特征点，然后计算每个特征点的SIFT向量的描述符，后面使用SIFT特征匹配方法匹配，实验表明本方法得到的匹配点更适合恢复棉花的形态。
The application of cotton-picking robot will be a development trend of agriculturalmechanization and has a great prospect. Researches in cotton-picking robot have greatsignificance for reducing working intension, increasing economy efficiency and adaptingrequirement of market. The cotton-picking robot, the cotton identification andlocation technology in the natural environment is the key technology. The maturity cotton in thenatural environment was the research object. The two point in the research were as follow:
In recognition of the mature cotton, the study proposed a mature cotton segmention strategybased on the YCbCr color space and Fisher discriminant analysis. First the research shoted thescene under different illumination use CCD camera, then contrasted analysis of segmentationresult in RGB,HSV, and YCbCr color space using Fisher discriminant analysis, and comparedwith proposed segmentation strategy. In order to ensure the boundaries information of the maturecotton, the research used the method of labeling to denoise. The simulation result that the cottoncould be separated exactly from background by the above algorithm whether the cotton wasexposed to the sunlight or the shadow with an accuracy of90.44%.
In location of the mature cotton, the research studied the stereo vision theory, cameraimaging model and calibration theory, and used Zhang Zhengyou calibrating method to calibratethe binocular stereo vision system. The research proposed a method to determine picking points,used region-match and specific constraints to match them, accorded to the triangulation method torestore the three-dimensional coordinates. The experimental results that the method the methodcan meet the demand for cotton-picking robot vision system. Due to the key points of the SIFTalgorithm is not corners, so the research used Harris method to extract corners in segmented cottonimage, and then calculated the SIFT descriptor vector of each feature point, and finally used theSIFT matching method to match them. The experiments showed that the matching point obtainedby this method was more suitable for the restoration of the morphology of the cotton.
引文
[1] Schertz C.E.,Brown G.k..Basic considerations in mechanizing citrus harvest[J].Transactions of the ASAE,1968,11(2):343-346.
[2] Grand D’Esnon A.,Rabatel G.,Pellenc R..A self-propelled robot to pick apples[J].ASAE paper,1987,46(3):353-358.
[3] Rabatel G.A vision system for”magali”,the fruit picking robot[J].Int.Conf.Agricultural Engineering,1988:1-10.
[4] Tillett R.D,Batchelor B.J..An algorithm for locating mushrooms in a growing bed[J].Computers and Electronics in Agriculture,1991,(6):191-200.
[5] Slaughter D.C,Harrel R. Discriminating fruit for robotic harvest using color innatural outdoor scenes [J].Transactions of the ASAE,1989,32(32):757-763.
[6] Alessio P.,Giorgio G..Localization of spherical fruits for robotic harvesting[J].Machine Vision and Applications,2001,13(2):70-79.
[7] Van Henten E J,Hemming J,Van Tuijl B A J,Kornet J G,Meuleman J,Bontsema J,Van OsEA.Field Test of an Autonomous Cucmber Picking Robot [J].BiosystemsEnginerring,2003,86(3):305-313.
[8] Haysshi S,Ganno K,Ishii Y,et al. Robotic harvesting system for eggplants[J].JapanAgricultural Research Quarterly,2002,36(3):163-168.
[9]Kassay L.,Slaughter D.C..Hungarian robotic apple harvest
[A].St.Joseph.Mich:ASAE,1992,ASAE Paper No.922047.
[10] Kondo N,Nishitsuji Y,Ling P P.Visual feedback guided robotic cherry tomatoharvesting [J].Transactions of the ASAE,1996,39(6)2331-2338.
[11]Takahashi T,Zhang S,Sun M,et al.New method of image processing for distancemeasurement by a passive stereo vision[A].St.Joseph,Mich:ASAE,1998,ASAE PaperNo.983031.
[12]Takahashi T,Zhang S,Fukuchi H,et al.Binocular stereo vision system for measuringdistance of apples in orchard (Part1)-Method due to composition of left and rightimages[J].Journal of the Japanese Society of Agricultural Machinery,2000,62(1):89-99．
[13]Takahashi T,Zhang S,Fukuchi H,et al.Binocular stereo vision system for measuringdistance of apples in orchard (Part2)-Analysis of and solution to the correspondenceproblem[J].Journal of the Japanese Society of AgriculturalMachinery,2000,62(3):94-102.
[14]Takahashi T, Zhang Shuhuai,Fukuchi H.Measurement of3-D locations of fruit bybinocular stereo vision for apple harvesting in anorchard[A].St.Joseph,Mich.:ASAE,2002,ASAE Paper No.021102.
[15] Kondo N,Ninomiya K,Hayashi S,et al.A new challenge of robot for harvestingstrawberry grown on table top culture[A].St.Joseph,Mich.:ASAE,2005,ASAE PaperNo.053138.
[16] Rostam Affendi Hamzah,Afifah Maheran Abdul Hamid,Sani Irwan Md Salim.The Solutionof Stereo Correspondence Problem Using Block Matching Algorithm in Stereo Vision MobileRobot[A]. Computer Research and Development[C],2010:733-737.
[17] Jose Prado,Luis Santos,Jorge Dias.A Technique for Dynamic Background Segmentationusing a Robotic Stereo Vision Head[A].Proceedings of Robot and Human InteractiveCommunication[C],2009:1035-1040.
[18] Rajendra Peter, Mitsutaka Kurita,Kazunori Ninomiya, Gosei Okazaki, ShigehikoHayashi. Shading compensation methods for robots to harvest strawberries in tabletopculture[J].System Integration (SII),2011,20(22):172-177.
[19]刘兆祥,刘刚,乔军.苹果采摘机器人三维视觉传感器设计[J].农业机械学报,2010,41(2):171-175.
[20]刘坤.基于改进随机Hough变换的棉桃识别技术[J].农业机械学报,201041(8):160-165.
[21] Rong Xiang, Yibin Ying,Huanyu Jiang, Coll.Research on image segmentation methodsof tomato in natural conditions[J].Image and Signal Processing,2011,15(17):1268-1272.
[22]赵杰文,刘木华,杨国彬.基于HIS颜色特征的田间成熟番茄识别技术[J].农业机械学报,2004,35(9):122-124,135.
[23]徐惠荣,叶尊忠,应义斌.基于彩色信息的树上柑橘识别研究[J].农业工程学报,2005,21(5):98-101.
[24]王勇,沈明霞,姬长英.采摘期成熟棉花不同部位颜色识别分析[J].农业工程学报,2007,23(4):183-185.
[25]韦皆顶,费树岷,汪木兰,袁建宁.基于HSV彩色模型的自然场景下棉花图像分割策略研究[J].棉花学报,2008,20(1):34-38.
[26]王玲,姬长英,刘善军,陈兵林,王萍.基于竞争学习网络的田间籽棉图像分割[J].农业工程学报,2008,24(10):156-160.
[27]蒋焕煜,彭永石,申川,应义斌.基于双目立体视觉技术的成熟番茄识别与定位[J].农业工程学报,2008,24(8):279-283.
[28]熊锋.基于双目立体视觉的杂草定位技术研究[D].江苏:江苏大学,2008.
[29]郑小东.基于计算机视觉的番茄识别与定位技术研究[J].江苏:江苏大学,2004.
[30]张铁中,陈利兵,宋健基于图像的草莓重心位置和采摘点的确定[J].中国农业大学学报,2005,10(1):48-51.
[31]刘菘,戚小平,钟双英.CCD摄像机原理及应用.中国有限电视[J],2005,(14):1417-1419.
[32] G Wei，S Ma Implicit and explicit camera calibration：Theory and experiments[J]IEEETransactions on Pattern Analysis and Machine Intelligence，1994，16(5)：469—480．
[33] Frederic Deverna，Ol ivier Faugera．Straight l ines have to be straight[J]．MachineVision and Applications，2001，13(1)：14—24．
[34] Heikkil，0Silvn．A four—step camera calibration procedure with implicit imagecorrection[J]．IEEE Computer Society Conference on Computer Vision and PatternRecognition，San Juan，Puerto Rico，1997，1106一1112．
[35]杨珍,孙军华,吴子彦,张广军.基于双一维靶标的摄像机标定方法[J].2010,21(3):411-414.
[36]段福庆,吕科,周明全.基于空间共线点的单光心反射折射摄像机标定[J].2011,37(11):1296-1305.
[37]刘源洞,孔建益,王兴东,刘钊.双目立体视觉系统的非线性摄像机标定技术[J].计算机应用研究,2011,28(9):3398-3400.
[38] H abed A,Bou fam a B.Camera self-calibration from bivariate polynomisals derivedfrom Kruppa’s equations[J].Pattern Recognition,2008,41(8):2484-2492.
[39]张广军.机器视觉[M].北京:科学出版社,2005:2-23,33-46.
[40] Tsai R Y.A versatile camera calibration technique for high accuracy3D machinevision metrology using off-the-shelf TV cameras and lenses[J].IEEE Journal of Roboticsand Automation,1987,3(4):323-344.
[41] Z.Zhang.A flexible new technique for camera calibration[J]. In IEEE Transactionson Pattern Analysis and Machine Intelligence,2000,22(11):1330-1334.
[42]章毓晋.图像工程[M].北京：清华大学出版社,2000.
[43]谢志勇,张铁中.基于RGB彩色模型的草莓图像色调分配算法[J].中国农业大学学报,2006,11(1):84-86.
[44] Fisher R A.The use of multiple measurements in taxonomic problems [J].AnnalsEugenics,1936,7:179-188.
[45] Duchene J,Leclercq S.An optimal transformation for discriminant and principalcomponent analysis[J].IEEE Trans Pattern Analysis Machine Intelligence,1988,10:978-983
[46] Milan Sonic,Vaclav Hlavac,Roger Boyle.Image Processing,Analysis and MachineVision[M].北京:人民邮电出版社,2002.
[47] Bulanon D.M.,Kataoka T.,Okamoto H.,Hata S..Development of a Real-time MachineVision System for the Apple Harvesting Robot[C].SICE AnnualConference.Sapporo:2004,595-598.
[48] Kimiko Gotou,Tateshi Fujiura,Yoshifumi Nishiura,Hideo Ikeda.3-D Vision System ofTomato Production Robot[C]. International Conference on Advanced IntelligentMeehatronics.2003,1210-1215．
[49] Mitsuji Monta,Kazuhiko Namba.Three-dimension Sensing System for AgriculturalRobot[C].International Conference on Advanced IntelligentMeehatronics.2003,1216-1221.
[50] M.Monta,N.Kondo,Y.Shibano. Agricultural Robot in Grape Production System[J].IEEEInternational Conference on Robotics and Automation.1995:2504-2509.
[51] Alessio Plebe,Giorgio Grasso.Localization of spherical fruits for roboticharvesting[J].Machine Vision and Applications,2001,13(2):70-79.
[52] D.M.Bulanon,T.Kataoka,Y.Ota,T.Hiroma. A Segmentation Algorithm for the AutomaticRecognition of Fuji Apples at Harvest[J].Biosystems Engineering,2002,83(4):405-512.
[53]曹其新,吕恬生,永田雅辉,王红永.草莓拣选机器人的开发.上海交通大学学报.1999,33(7):880-884.
[54]王沈辉.机器人采摘番茄中的双目立体技术研究[D].南京:江苏大学,2006年.
[2] Grand D’Esnon A.,Rabatel G.,Pellenc R..A self-propelled robot to pick apples[J].ASAE paper,1987,46(3):353-358.
[3] Rabatel G.A vision system for”magali”,the fruit picking robot[J].Int.Conf.Agricultural Engineering,1988:1-10.
[4] Tillett R.D,Batchelor B.J..An algorithm for locating mushrooms in a growing bed[J].Computers and Electronics in Agriculture,1991,(6):191-200.
[5] Slaughter D.C,Harrel R. Discriminating fruit for robotic harvest using color innatural outdoor scenes [J].Transactions of the ASAE,1989,32(32):757-763.
[6] Alessio P.,Giorgio G..Localization of spherical fruits for robotic harvesting[J].Machine Vision and Applications,2001,13(2):70-79.
[7] Van Henten E J,Hemming J,Van Tuijl B A J,Kornet J G,Meuleman J,Bontsema J,Van OsEA.Field Test of an Autonomous Cucmber Picking Robot [J].BiosystemsEnginerring,2003,86(3):305-313.
[8] Haysshi S,Ganno K,Ishii Y,et al. Robotic harvesting system for eggplants[J].JapanAgricultural Research Quarterly,2002,36(3):163-168.
[9]Kassay L.,Slaughter D.C..Hungarian robotic apple harvest
[A].St.Joseph.Mich:ASAE,1992,ASAE Paper No.922047.
[10] Kondo N,Nishitsuji Y,Ling P P.Visual feedback guided robotic cherry tomatoharvesting [J].Transactions of the ASAE,1996,39(6)2331-2338.
[11]Takahashi T,Zhang S,Sun M,et al.New method of image processing for distancemeasurement by a passive stereo vision[A].St.Joseph,Mich:ASAE,1998,ASAE PaperNo.983031.
[12]Takahashi T,Zhang S,Fukuchi H,et al.Binocular stereo vision system for measuringdistance of apples in orchard (Part1)-Method due to composition of left and rightimages[J].Journal of the Japanese Society of Agricultural Machinery,2000,62(1):89-99．
[13]Takahashi T,Zhang S,Fukuchi H,et al.Binocular stereo vision system for measuringdistance of apples in orchard (Part2)-Analysis of and solution to the correspondenceproblem[J].Journal of the Japanese Society of AgriculturalMachinery,2000,62(3):94-102.
[14]Takahashi T, Zhang Shuhuai,Fukuchi H.Measurement of3-D locations of fruit bybinocular stereo vision for apple harvesting in anorchard[A].St.Joseph,Mich.:ASAE,2002,ASAE Paper No.021102.
[15] Kondo N,Ninomiya K,Hayashi S,et al.A new challenge of robot for harvestingstrawberry grown on table top culture[A].St.Joseph,Mich.:ASAE,2005,ASAE PaperNo.053138.
[16] Rostam Affendi Hamzah,Afifah Maheran Abdul Hamid,Sani Irwan Md Salim.The Solutionof Stereo Correspondence Problem Using Block Matching Algorithm in Stereo Vision MobileRobot[A]. Computer Research and Development[C],2010:733-737.
[17] Jose Prado,Luis Santos,Jorge Dias.A Technique for Dynamic Background Segmentationusing a Robotic Stereo Vision Head[A].Proceedings of Robot and Human InteractiveCommunication[C],2009:1035-1040.
[18] Rajendra Peter, Mitsutaka Kurita,Kazunori Ninomiya, Gosei Okazaki, ShigehikoHayashi. Shading compensation methods for robots to harvest strawberries in tabletopculture[J].System Integration (SII),2011,20(22):172-177.
[19]刘兆祥,刘刚,乔军.苹果采摘机器人三维视觉传感器设计[J].农业机械学报,2010,41(2):171-175.
[20]刘坤.基于改进随机Hough变换的棉桃识别技术[J].农业机械学报,201041(8):160-165.
[21] Rong Xiang, Yibin Ying,Huanyu Jiang, Coll.Research on image segmentation methodsof tomato in natural conditions[J].Image and Signal Processing,2011,15(17):1268-1272.
[22]赵杰文,刘木华,杨国彬.基于HIS颜色特征的田间成熟番茄识别技术[J].农业机械学报,2004,35(9):122-124,135.
[23]徐惠荣,叶尊忠,应义斌.基于彩色信息的树上柑橘识别研究[J].农业工程学报,2005,21(5):98-101.
[24]王勇,沈明霞,姬长英.采摘期成熟棉花不同部位颜色识别分析[J].农业工程学报,2007,23(4):183-185.
[25]韦皆顶,费树岷,汪木兰,袁建宁.基于HSV彩色模型的自然场景下棉花图像分割策略研究[J].棉花学报,2008,20(1):34-38.
[26]王玲,姬长英,刘善军,陈兵林,王萍.基于竞争学习网络的田间籽棉图像分割[J].农业工程学报,2008,24(10):156-160.
[27]蒋焕煜,彭永石,申川,应义斌.基于双目立体视觉技术的成熟番茄识别与定位[J].农业工程学报,2008,24(8):279-283.
[28]熊锋.基于双目立体视觉的杂草定位技术研究[D].江苏:江苏大学,2008.
[29]郑小东.基于计算机视觉的番茄识别与定位技术研究[J].江苏:江苏大学,2004.
[30]张铁中,陈利兵,宋健基于图像的草莓重心位置和采摘点的确定[J].中国农业大学学报,2005,10(1):48-51.
[31]刘菘,戚小平,钟双英.CCD摄像机原理及应用.中国有限电视[J],2005,(14):1417-1419.
[32] G Wei，S Ma Implicit and explicit camera calibration：Theory and experiments[J]IEEETransactions on Pattern Analysis and Machine Intelligence，1994，16(5)：469—480．
[33] Frederic Deverna，Ol ivier Faugera．Straight l ines have to be straight[J]．MachineVision and Applications，2001，13(1)：14—24．
[34] Heikkil，0Silvn．A four—step camera calibration procedure with implicit imagecorrection[J]．IEEE Computer Society Conference on Computer Vision and PatternRecognition，San Juan，Puerto Rico，1997，1106一1112．
[35]杨珍,孙军华,吴子彦,张广军.基于双一维靶标的摄像机标定方法[J].2010,21(3):411-414.
[36]段福庆,吕科,周明全.基于空间共线点的单光心反射折射摄像机标定[J].2011,37(11):1296-1305.
[37]刘源洞,孔建益,王兴东,刘钊.双目立体视觉系统的非线性摄像机标定技术[J].计算机应用研究,2011,28(9):3398-3400.
[38] H abed A,Bou fam a B.Camera self-calibration from bivariate polynomisals derivedfrom Kruppa’s equations[J].Pattern Recognition,2008,41(8):2484-2492.
[39]张广军.机器视觉[M].北京:科学出版社,2005:2-23,33-46.
[40] Tsai R Y.A versatile camera calibration technique for high accuracy3D machinevision metrology using off-the-shelf TV cameras and lenses[J].IEEE Journal of Roboticsand Automation,1987,3(4):323-344.
[41] Z.Zhang.A flexible new technique for camera calibration[J]. In IEEE Transactionson Pattern Analysis and Machine Intelligence,2000,22(11):1330-1334.
[42]章毓晋.图像工程[M].北京：清华大学出版社,2000.
[43]谢志勇,张铁中.基于RGB彩色模型的草莓图像色调分配算法[J].中国农业大学学报,2006,11(1):84-86.
[44] Fisher R A.The use of multiple measurements in taxonomic problems [J].AnnalsEugenics,1936,7:179-188.
[45] Duchene J,Leclercq S.An optimal transformation for discriminant and principalcomponent analysis[J].IEEE Trans Pattern Analysis Machine Intelligence,1988,10:978-983
[46] Milan Sonic,Vaclav Hlavac,Roger Boyle.Image Processing,Analysis and MachineVision[M].北京:人民邮电出版社,2002.
[47] Bulanon D.M.,Kataoka T.,Okamoto H.,Hata S..Development of a Real-time MachineVision System for the Apple Harvesting Robot[C].SICE AnnualConference.Sapporo:2004,595-598.
[48] Kimiko Gotou,Tateshi Fujiura,Yoshifumi Nishiura,Hideo Ikeda.3-D Vision System ofTomato Production Robot[C]. International Conference on Advanced IntelligentMeehatronics.2003,1210-1215．
[49] Mitsuji Monta,Kazuhiko Namba.Three-dimension Sensing System for AgriculturalRobot[C].International Conference on Advanced IntelligentMeehatronics.2003,1216-1221.
[50] M.Monta,N.Kondo,Y.Shibano. Agricultural Robot in Grape Production System[J].IEEEInternational Conference on Robotics and Automation.1995:2504-2509.
[51] Alessio Plebe,Giorgio Grasso.Localization of spherical fruits for roboticharvesting[J].Machine Vision and Applications,2001,13(2):70-79.
[52] D.M.Bulanon,T.Kataoka,Y.Ota,T.Hiroma. A Segmentation Algorithm for the AutomaticRecognition of Fuji Apples at Harvest[J].Biosystems Engineering,2002,83(4):405-512.
[53]曹其新,吕恬生,永田雅辉,王红永.草莓拣选机器人的开发.上海交通大学学报.1999,33(7):880-884.
[54]王沈辉.机器人采摘番茄中的双目立体技术研究[D].南京:江苏大学,2006年.