自然环境下果实作业机器人幼果期苹果侦测方法
|
摘要
为实现机器人幼果期自动化果实作业,以果园幼小青苹果为对象,研究了自然环境下幼果期苹果的机器侦测方法。首先,采用自适应G-B色差法对初始图像计算,获得色差灰度图,使用迭代阈值分割法提取果实兴趣区;其次,对经形态学处理后的兴趣区图像进行Blob分析,计算每个Blob的离心率和像素面积,去除明显偏离果实形状特点的Blob;最后,应用改进圆形Hough变换算法检测潜在类圆形果实目标,最终采用融合方向梯度直方图特征和网格搜索优化支持向量机的判别模型进一步去除虚假果实目标,提升苹果目标的侦测精确度。试验结果显示,该方法对果园自然环境下幼小青苹果的侦测正确率为88.51%,漏报率和误报率分别为11.49%和4.84%,算法模型综合性能指标为90.29%,表明该方法对幼果期苹果目标具有较强的侦测能力和较好的鲁棒性,该结果为果实作业机器人幼果期的自动化果实侦测提供参考。
In order to realize automatically managing fruit production by robot during young fruit period,this paper took young green apples in orchard as object and studied the detection method of young green apples by machine under natural environment. Firstly,adaptive green and blue chromatic aberration(AGBCA) map was designed and combined with the iterative threshold segmentation(ITS) algorithm to detect region of interest(ROI) contained potential apple fruits pixels. Then,potential fruits were identified by an improved circular hough transformation(CHT) after morphological operation and Blob analysis of the results obtained from AGBCA and ITS,which kept many potential apple fruits pixels as possible. Finally,a kernel support vector machine(SVM) classifier,optimized by grid search optimal algorithm,was built to remove false fruit objects based on histogram of oriented gradient(HOG) feature descriptor. The experimental results showed that the true positive rate of proposed method was88.51%,false negative rate and false positive rate were 11. 49% and 4. 84%,respectively. And the F1-Measure of proposed model was 90. 29%,indicating the proposed method had better detection ability and robustness for young green apples detection. The results provided references to fruit robot for automatic detection during young fruit stage.
In order to realize automatically managing fruit production by robot during young fruit period,this paper took young green apples in orchard as object and studied the detection method of young green apples by machine under natural environment. Firstly,adaptive green and blue chromatic aberration(AGBCA) map was designed and combined with the iterative threshold segmentation(ITS) algorithm to detect region of interest(ROI) contained potential apple fruits pixels. Then,potential fruits were identified by an improved circular hough transformation(CHT) after morphological operation and Blob analysis of the results obtained from AGBCA and ITS,which kept many potential apple fruits pixels as possible. Finally,a kernel support vector machine(SVM) classifier,optimized by grid search optimal algorithm,was built to remove false fruit objects based on histogram of oriented gradient(HOG) feature descriptor. The experimental results showed that the true positive rate of proposed method was88.51%,false negative rate and false positive rate were 11. 49% and 4. 84%,respectively. And the F1-Measure of proposed model was 90. 29%,indicating the proposed method had better detection ability and robustness for young green apples detection. The results provided references to fruit robot for automatic detection during young fruit stage.
引文
[1]中华人民共和国国家统计局.国家数据[EB/OL].http://data.stats.gov.cn/easyquery.htm?cn=C01.
[2]杨慧莲,刘军弟,时卫平,等.世界苹果主产国生产、加工、贸易与消费状况分析[J].北方园艺,2015(10):166-169.Yang H L,Liu J D,Shi W P,et al..Analysis on apple production,processing,trade and consumption situation in major apple producing countries[J].Northern Hortic.,2015(10):166-169.
[3]张强强,霍学喜,刘军弟,等.世界苹果产销格局及市场动态预测分析[J].世界农业,2016(7):147-152.Zhang Q Q,Huo X X,Liu J D,et al..The analysis and prediction of world apple production pattern and market dynamic[J].World Agric.,2016(7):147-152.
[4]夏雪,丘耘,王健,等.果园环境下苹果侦测与定位方法研究现状与展望[J].中国农业科技导报,2017,19(2):65-74.Xia X,Qiu Y,Wang J,et al..Research status and prospect of apple detection and localization methods under orchard environment[J].J.Agric.Sci.Technol.,2017,19(2):65-74.
[5]周国民.我国数字果园的研究与发展[J].农业网络信息,2012(1):10-12.Zhou G M.Research and development of digital orchard in China[J].Agric.Network Inform.,2012(1):10-12.
[6]周国民.数字果园研究现状与应用前景展望[J].农业展望,2015(5):61-63.Zhou G M.Application situation and outlook of digital orchard[J].Agric.Outlook,2015(5):61-63.
[7]Stajnko D,Lakota M,Hoˇcevar M.Estimation of number and diameter of apple fruits in an orchard during the growing season by thermal imaging[J].Comput.Electron.Agric.,2004,42(1):31-42.
[8]张树槐,高桥照夫,福地博,等.果树园作业の自动化に关する研究(第1报)树园作业パタ一ン认识にするリンゴの俭出[J].农业机械学会志,1996,58(1):9-16.
[9]Safren O,Alchanatis V,Ostrovsky V,et al..Detection of green apples in hyperspectral images of apple-tree foliage using machine vision[J].Trans.ASABE,2007,50(6):2303-2313.
[10]Kim Y,Reid J.Apple yield mapping using a multispectral imaging sensor[A].In:Proceedings of International Scientific Conference on Agricultural Engineering[C].Belgium:Leuven,2004.
[11]Parish E,Goksel A K.Pictorial pattern recognition applied to fruit harvesting[J].Trans.ASAE,1977,20(5):822-827.
[12]D'Esnon G,Rabate G,Pellene R.A self-propelled robot to pick apples[J].Trans.ASAE,1987(7):1032-1037.
[13]Li L,Zhang Q,Huang D.A review of imaging techniques for plant phenotyping[J].Sensors,2014,14(11):20078-20111.
[14]Bulanon D M,Burks T F,Alchanatis V.Study on temporal variation in citrus canopy using thermal imaging for citrus fruit detection[J].Biosyst.Engin.,2008,101(2):161-171.
[15]Manolakis D,Marden D,Shaw G A.Hyperspectral image processing for automatic target detection applications[J].Lincoln Lab.J.,2003,14(1):79-116.
[16]Okamoto H,Lee W S.Green citrus detection using hyperspectral imaging[J].Comput.Electron.Agric.,2009,66(2):201-208.
[17]Zhao J,Tow J,Katupitiya J.On-tree fruit recognition using texture properties and color data[A].In:Proceedings of IEEE/RSJ International Conference on Intelligent Robots&Systems[C].IEEE,2005,263-268.
[18]Yamamoto K,Guo W,Yoshioka Y,et al..On plant detection of intact tomato fruits using image analysis and machine learning methods[J].Sensors,2014,14(7):12191-12206.
[19]Chaivivatrakul S,Dailey M N.Texture-based fruit detection[J].Precision Agric.,2014,15(6):662-683.
[20]Longsheng F,Bin W,Yongjie C,et al..Kiwifruit recognition at nighttime using artificial lighting based on machine vision[J].Int.J.Agric.Biol.Engin.,2015,8(4):52-59.
[21]Mc Cool C,Sa I,Dayoub F,et al..Visual detection of occluded crop:For automated harvesting[A].In:Proceedings of EEE/RSJ International Conference on Intelligent Robots&Systems[C].IEEE,2016,2506-2512.
[22]Bulanon D M,Kataoka T,Ota Y,et al..A segmentation algorithm for the automatic recognition of Fuji apples at harvest[J].Biosyst.Engin.,2002,83(4):405-412.
[23]Huang L,He D,Yang S X.Segmentation on ripe Fuji apple with fuzzy 2D entropy based on 2D histogram and GA optimization[J].Intell.Autom.Soft Comput.,2013,19(3):239-251.
[24]Kim D,Choi H,Choi J,et al..A novel red apple detection algorithm based on Ada Boost learning[J].IEIE Trans.Smart Process.Comput.,2015,4(4):265-271.
[25]钱建平,杨信廷,吴晓明,等.自然场景下基于混合颜色空间的成熟期苹果识别方法[J].农业工程学报,2012,28(17):137-142.Qian J P,Yang X T,Wu X M,et al..Mature apple recognition based on hybrid color space in natural scene[J].Trans.Chin.Soc.Agric.Engin.,2012,28(17):137-142.
[26]Rakun J,Stajnko D,Zazula D.Detecting fruits in natural scenes by using spatial-frequency based texture analysis and multiview geometry[J].Comput.Electron.Agric.,2011,76(1):80-88.
[27]Linker R,Cohen O,Naor A.Determination of the number of green apples in RGB images recorded in orchards[J].Comput.Electron.Agric.,2012,81:45-57.
[28]张春龙,张楫,张俊雄,等.近色背景中树上绿色苹果识别方法[J].农业机械学报,2014,45(10):277-281.Zhang C L,Zhang J,Zhang J X,et al..Recognition of green apple in similar background[J].Trans.Chin.Soc.Agric.Machin.,2014,45(10):277-281.
[29]司永胜,刘刚,高瑞.基于K-均值聚类的绿色苹果识别技术[J].农业机械学报,2009,40(S1):100-104.Si Y S,Liu G,Gao R.Segmentation algorithm for green apples recognition based on K-means algorithm[J].Trans.Chin.Soc.Agric.Machin.,2009,40(S1):100-104.
[30]崔永杰,苏帅,王霞霞,等.基于机器视觉的自然环境中猕猴桃识别与特征提取[J].农业机械学报,2013,44(5):247-252.Cui Y J,Su S,Wang X X,et al..Recognition and feature extraction of kiwifruit in natural environment based on machine vision[J].Trans.Chin.Soc.Agric.Machin.,2013,44(5):247-252.
[31]Drever L,Roa W,Mc Ewan A,et al..Iterative threshold segmentation for PET target volume delineation[J].Med.Phys.,2007,34(4):1253-1265.
[32]Sengupta S,Lee W S.Identification and determination of the number of immature green citrus fruit in a canopy under different ambient light conditions[J].Biosyst.Engin.,2014,117:51-61.
[33]Smereka M,Duleba I.Circular object detection using a modified Hough transform[J].Int.J.Appl.Math.Comput.Sci.,2008,18(1):85-91.
[34]Atherton T J,Kerbyson D J.Size invariant circle detection[J].Image Vision Comput.,1999,17(11):795-803.
[35]Davies E R.Computer and Machine Vision:Theory,Algorithms,Practicalities[M].USA:Academic Press,2012.
[36]Dalal N,Triggs B.Histograms of oriented gradients for human detection[A].In:Proceedings of EEE/RSJ International Conference on Intelligent Robots&Systems[C].IEEE,2005,1:886-893.
[37]胡庆新,吕鹏.基于多特征融合的红外图像行人检测[J].计算机应用,2016,36(s1):157-160.Hu Q X,Lv P,Wang X X et al..Pedestrian detection in infrared images based on multi-feature fusion[J].J.Comput.Appl.,2013,36(s1):157-160.
[38]Gavriilidis A,Stahlschmidt C,Velten J,et al..Feature recycling cascaded SVM classifier based on feature selection of HOGs for pedestrian detection[A].In:Proceedings of International Conference on Multimedia Communications,Services and Security[C].Springer Berlin Heidelberg,2013,82-94.
[39]麦春艳,郑立华,肖昌一,等.自然光照条件下苹果识别方法对比研究[J].中国农业大学学报,2016,21(11):43-50.Mai C Y,Zheng L H,Xiao C Y,et al..Comparison of apple recognition methods under natural light[J].J.Chin.Agric.Univ.,2016,21(11):43-50.
[40]Vpnik V N,Chervonenkis A Ja.Theory of Pattern Recognition[M].New York:Springer,1974.
[41]王璨,李志伟.利用融合高度与单目图像特征的支持向量机模型识别杂草[J].农业工程学报,2016,32(15):165-174.Wang C,Li Z W.Weed recognition using SVM model with fusion height and monocular image features[J].Trans.Chin.Soc.Agric.Engin.,2016,32(15):165-174.
[42]黄辰,费继友.基于图像特征融合的苹果在线分级方法[J].农业工程学报,2017,33(1):285-291.Huang C,Fei J Y.Online apple grading based on decision fusion of image features[J].Trans.Chin.Soc.Agric.Engin.,2017,33(1):285-291.
[43]Sa I,Ge Z,Dayoub F,et al..Deepfruits:A fruit detection system using deep neural networks[J].Sensors,2016,16(8):1222-1244.
[2]杨慧莲,刘军弟,时卫平,等.世界苹果主产国生产、加工、贸易与消费状况分析[J].北方园艺,2015(10):166-169.Yang H L,Liu J D,Shi W P,et al..Analysis on apple production,processing,trade and consumption situation in major apple producing countries[J].Northern Hortic.,2015(10):166-169.
[3]张强强,霍学喜,刘军弟,等.世界苹果产销格局及市场动态预测分析[J].世界农业,2016(7):147-152.Zhang Q Q,Huo X X,Liu J D,et al..The analysis and prediction of world apple production pattern and market dynamic[J].World Agric.,2016(7):147-152.
[4]夏雪,丘耘,王健,等.果园环境下苹果侦测与定位方法研究现状与展望[J].中国农业科技导报,2017,19(2):65-74.Xia X,Qiu Y,Wang J,et al..Research status and prospect of apple detection and localization methods under orchard environment[J].J.Agric.Sci.Technol.,2017,19(2):65-74.
[5]周国民.我国数字果园的研究与发展[J].农业网络信息,2012(1):10-12.Zhou G M.Research and development of digital orchard in China[J].Agric.Network Inform.,2012(1):10-12.
[6]周国民.数字果园研究现状与应用前景展望[J].农业展望,2015(5):61-63.Zhou G M.Application situation and outlook of digital orchard[J].Agric.Outlook,2015(5):61-63.
[7]Stajnko D,Lakota M,Hoˇcevar M.Estimation of number and diameter of apple fruits in an orchard during the growing season by thermal imaging[J].Comput.Electron.Agric.,2004,42(1):31-42.
[8]张树槐,高桥照夫,福地博,等.果树园作业の自动化に关する研究(第1报)树园作业パタ一ン认识にするリンゴの俭出[J].农业机械学会志,1996,58(1):9-16.
[9]Safren O,Alchanatis V,Ostrovsky V,et al..Detection of green apples in hyperspectral images of apple-tree foliage using machine vision[J].Trans.ASABE,2007,50(6):2303-2313.
[10]Kim Y,Reid J.Apple yield mapping using a multispectral imaging sensor[A].In:Proceedings of International Scientific Conference on Agricultural Engineering[C].Belgium:Leuven,2004.
[11]Parish E,Goksel A K.Pictorial pattern recognition applied to fruit harvesting[J].Trans.ASAE,1977,20(5):822-827.
[12]D'Esnon G,Rabate G,Pellene R.A self-propelled robot to pick apples[J].Trans.ASAE,1987(7):1032-1037.
[13]Li L,Zhang Q,Huang D.A review of imaging techniques for plant phenotyping[J].Sensors,2014,14(11):20078-20111.
[14]Bulanon D M,Burks T F,Alchanatis V.Study on temporal variation in citrus canopy using thermal imaging for citrus fruit detection[J].Biosyst.Engin.,2008,101(2):161-171.
[15]Manolakis D,Marden D,Shaw G A.Hyperspectral image processing for automatic target detection applications[J].Lincoln Lab.J.,2003,14(1):79-116.
[16]Okamoto H,Lee W S.Green citrus detection using hyperspectral imaging[J].Comput.Electron.Agric.,2009,66(2):201-208.
[17]Zhao J,Tow J,Katupitiya J.On-tree fruit recognition using texture properties and color data[A].In:Proceedings of IEEE/RSJ International Conference on Intelligent Robots&Systems[C].IEEE,2005,263-268.
[18]Yamamoto K,Guo W,Yoshioka Y,et al..On plant detection of intact tomato fruits using image analysis and machine learning methods[J].Sensors,2014,14(7):12191-12206.
[19]Chaivivatrakul S,Dailey M N.Texture-based fruit detection[J].Precision Agric.,2014,15(6):662-683.
[20]Longsheng F,Bin W,Yongjie C,et al..Kiwifruit recognition at nighttime using artificial lighting based on machine vision[J].Int.J.Agric.Biol.Engin.,2015,8(4):52-59.
[21]Mc Cool C,Sa I,Dayoub F,et al..Visual detection of occluded crop:For automated harvesting[A].In:Proceedings of EEE/RSJ International Conference on Intelligent Robots&Systems[C].IEEE,2016,2506-2512.
[22]Bulanon D M,Kataoka T,Ota Y,et al..A segmentation algorithm for the automatic recognition of Fuji apples at harvest[J].Biosyst.Engin.,2002,83(4):405-412.
[23]Huang L,He D,Yang S X.Segmentation on ripe Fuji apple with fuzzy 2D entropy based on 2D histogram and GA optimization[J].Intell.Autom.Soft Comput.,2013,19(3):239-251.
[24]Kim D,Choi H,Choi J,et al..A novel red apple detection algorithm based on Ada Boost learning[J].IEIE Trans.Smart Process.Comput.,2015,4(4):265-271.
[25]钱建平,杨信廷,吴晓明,等.自然场景下基于混合颜色空间的成熟期苹果识别方法[J].农业工程学报,2012,28(17):137-142.Qian J P,Yang X T,Wu X M,et al..Mature apple recognition based on hybrid color space in natural scene[J].Trans.Chin.Soc.Agric.Engin.,2012,28(17):137-142.
[26]Rakun J,Stajnko D,Zazula D.Detecting fruits in natural scenes by using spatial-frequency based texture analysis and multiview geometry[J].Comput.Electron.Agric.,2011,76(1):80-88.
[27]Linker R,Cohen O,Naor A.Determination of the number of green apples in RGB images recorded in orchards[J].Comput.Electron.Agric.,2012,81:45-57.
[28]张春龙,张楫,张俊雄,等.近色背景中树上绿色苹果识别方法[J].农业机械学报,2014,45(10):277-281.Zhang C L,Zhang J,Zhang J X,et al..Recognition of green apple in similar background[J].Trans.Chin.Soc.Agric.Machin.,2014,45(10):277-281.
[29]司永胜,刘刚,高瑞.基于K-均值聚类的绿色苹果识别技术[J].农业机械学报,2009,40(S1):100-104.Si Y S,Liu G,Gao R.Segmentation algorithm for green apples recognition based on K-means algorithm[J].Trans.Chin.Soc.Agric.Machin.,2009,40(S1):100-104.
[30]崔永杰,苏帅,王霞霞,等.基于机器视觉的自然环境中猕猴桃识别与特征提取[J].农业机械学报,2013,44(5):247-252.Cui Y J,Su S,Wang X X,et al..Recognition and feature extraction of kiwifruit in natural environment based on machine vision[J].Trans.Chin.Soc.Agric.Machin.,2013,44(5):247-252.
[31]Drever L,Roa W,Mc Ewan A,et al..Iterative threshold segmentation for PET target volume delineation[J].Med.Phys.,2007,34(4):1253-1265.
[32]Sengupta S,Lee W S.Identification and determination of the number of immature green citrus fruit in a canopy under different ambient light conditions[J].Biosyst.Engin.,2014,117:51-61.
[33]Smereka M,Duleba I.Circular object detection using a modified Hough transform[J].Int.J.Appl.Math.Comput.Sci.,2008,18(1):85-91.
[34]Atherton T J,Kerbyson D J.Size invariant circle detection[J].Image Vision Comput.,1999,17(11):795-803.
[35]Davies E R.Computer and Machine Vision:Theory,Algorithms,Practicalities[M].USA:Academic Press,2012.
[36]Dalal N,Triggs B.Histograms of oriented gradients for human detection[A].In:Proceedings of EEE/RSJ International Conference on Intelligent Robots&Systems[C].IEEE,2005,1:886-893.
[37]胡庆新,吕鹏.基于多特征融合的红外图像行人检测[J].计算机应用,2016,36(s1):157-160.Hu Q X,Lv P,Wang X X et al..Pedestrian detection in infrared images based on multi-feature fusion[J].J.Comput.Appl.,2013,36(s1):157-160.
[38]Gavriilidis A,Stahlschmidt C,Velten J,et al..Feature recycling cascaded SVM classifier based on feature selection of HOGs for pedestrian detection[A].In:Proceedings of International Conference on Multimedia Communications,Services and Security[C].Springer Berlin Heidelberg,2013,82-94.
[39]麦春艳,郑立华,肖昌一,等.自然光照条件下苹果识别方法对比研究[J].中国农业大学学报,2016,21(11):43-50.Mai C Y,Zheng L H,Xiao C Y,et al..Comparison of apple recognition methods under natural light[J].J.Chin.Agric.Univ.,2016,21(11):43-50.
[40]Vpnik V N,Chervonenkis A Ja.Theory of Pattern Recognition[M].New York:Springer,1974.
[41]王璨,李志伟.利用融合高度与单目图像特征的支持向量机模型识别杂草[J].农业工程学报,2016,32(15):165-174.Wang C,Li Z W.Weed recognition using SVM model with fusion height and monocular image features[J].Trans.Chin.Soc.Agric.Engin.,2016,32(15):165-174.
[42]黄辰,费继友.基于图像特征融合的苹果在线分级方法[J].农业工程学报,2017,33(1):285-291.Huang C,Fei J Y.Online apple grading based on decision fusion of image features[J].Trans.Chin.Soc.Agric.Engin.,2017,33(1):285-291.
[43]Sa I,Ge Z,Dayoub F,et al..Deepfruits:A fruit detection system using deep neural networks[J].Sensors,2016,16(8):1222-1244.