基于人类视觉特性的自然光照条件下成熟石榴检测
|
摘要
农业收获机器人的正常工作有赖于对作业对象的正确识别,因而要实现对水果的采摘,前提是从果树中准确识别出水果。在本课题的研究中,提出了基于人类视觉特性的自然光照条件下水果识别方法,并用于树上成熟石榴的检测试验。本文主要研究内容和方法如下：
(1)首先,基于人类视觉系统对颜色感知特性,在HSI空间对石榴采摘图像进行特征研究；通过试验分析光照对饱和度和色调的影响,并基于人类视觉特性提出解决光照问题设想。结合颜色恒常性理论,提出了一种将全局特征和局部特征相联合的带颜色恢复因子的多尺度Retinex(MSRCR)算法。实验表明该算法实现了动态范围压缩、色彩恒常、颜色高保真等特点的自动图像增强。
(2)针对果实图像的灰度和颜色特点,利用色调直方图呈明显双峰状,采用OSTU自动阈值分割,有效去除大部分背景。对二值图像首先采用形态学滤波去除残留小面积噪声；八邻域标记并用种子填充算法进行区域填充；然后统计各标记区域面积,根据区域面积和外接矩形长宽比作阈值,将不符合参考值的区域作为伪目标去除,从而使分割效果得到进一步改善；采用Sobel算子进行轮廓提取；利用优化后的广义Hough变换,有效地恢复果形并准确提取果实半径及形心。
(3)利用图像的不变矩,并结合形态、灰度和变换域信息,通过构造综合特征来进行图像匹配,并将其应用于目标检测。即：在图像预处理时,利用SUSAN算子检测图像目标的边缘和角点,来实现初定位与分割；引入不变矩理论,通过实验证明面矩和线矩都具有平移、比例放缩和旋转不变性,基于线矩的特征矢量计算在时间上明显优于基于面矩的计算,有利于提高图像处理的实时性。然后基于线矩提取局部显著性特征,并结合面积周长比、长宽比、占空比、灰度信息等综合特征进行图像匹配,实现了成熟石榴识别,并给出具体实现过程及结论。
The normal work of agricultural harvesting robot depends on correct recognition of manipulating object, the fruit detected accurately in a tree is the prerequisite of the successful harvest. The recognition method of fruits in natural lighting conditions based on human visual characteristics is proposed.In this research, the detection experiment of mature pomegranate was conducted in nature. The main research contents and methods are as follows:
(1)Firstly, characteristics of picking pomegranate image were studied based on color perceptual characteristics of human visual system in HSI space model. The illumination influences about saturation and color were analyzed by experiments, at the same time, solving illumination theoretical assumption was proposed based on human visual system. Secondly, a multi-scale Retinex algorithm with color healing factor(MSRCR) composed global and local characteristics was studied. Combined with color constancy theory, the experiments showed that the algorithm has realized the automatic image enhancement with different characteristics, such as dynamic range compression, color constancy and color high fidelity.
(2) According to gray and color characteristics in fruit image, the image histogram of hue showed a clear bimodal shape obviously, so the OSTU automatic threshold segmentation was adopted in order to remove most of image background. Firstly, morphological filtering was used to remove residual small area noise in binary images. Secondly,labeled by eight neighborhoods,optimized seed filling algorithm was used to fill the holes. Thirdly,every labeled area was calculated out, set the threshold according to the area and the length to width ratio of minimum enclosing rectangle, and the other areas were wiped out. The edge was detected by Sobel algorithm.At last,centroids and radius were extracted by the optimized Hough transform, and the shape of fruits was recovered.
(3) The comprehensive characteristics can complete image matching based on invariant moments of combination with configuration and intensity and transform domain, and applied to the target recognition. That is, Edge and corner point detection used SUSAN operator at low level image processing, in order to realize the preliminary localization and segmentation; while the invariant moments theory were introduced, invariant moments characteristics of area moments and line moments theory were proved by mass data tests, when translated or constrained or rotated.And the running average time of line invariant moments is more superior than area invariant moments, it is helpful to improve real time of image processing. Then the local significant characteristics extracted by line moment invariants, and image matching combines with the comprehensive characteristics, for instance, area to perimeter ratio, length to width ratio, duty ratio, intensity information etc. The detection of mature pomegranate was implemented, also the realization process and conclusions of target recognition were presented in this paper.
(1)首先,基于人类视觉系统对颜色感知特性,在HSI空间对石榴采摘图像进行特征研究；通过试验分析光照对饱和度和色调的影响,并基于人类视觉特性提出解决光照问题设想。结合颜色恒常性理论,提出了一种将全局特征和局部特征相联合的带颜色恢复因子的多尺度Retinex(MSRCR)算法。实验表明该算法实现了动态范围压缩、色彩恒常、颜色高保真等特点的自动图像增强。
(2)针对果实图像的灰度和颜色特点,利用色调直方图呈明显双峰状,采用OSTU自动阈值分割,有效去除大部分背景。对二值图像首先采用形态学滤波去除残留小面积噪声；八邻域标记并用种子填充算法进行区域填充；然后统计各标记区域面积,根据区域面积和外接矩形长宽比作阈值,将不符合参考值的区域作为伪目标去除,从而使分割效果得到进一步改善；采用Sobel算子进行轮廓提取；利用优化后的广义Hough变换,有效地恢复果形并准确提取果实半径及形心。
(3)利用图像的不变矩,并结合形态、灰度和变换域信息,通过构造综合特征来进行图像匹配,并将其应用于目标检测。即：在图像预处理时,利用SUSAN算子检测图像目标的边缘和角点,来实现初定位与分割；引入不变矩理论,通过实验证明面矩和线矩都具有平移、比例放缩和旋转不变性,基于线矩的特征矢量计算在时间上明显优于基于面矩的计算,有利于提高图像处理的实时性。然后基于线矩提取局部显著性特征,并结合面积周长比、长宽比、占空比、灰度信息等综合特征进行图像匹配,实现了成熟石榴识别,并给出具体实现过程及结论。
The normal work of agricultural harvesting robot depends on correct recognition of manipulating object, the fruit detected accurately in a tree is the prerequisite of the successful harvest. The recognition method of fruits in natural lighting conditions based on human visual characteristics is proposed.In this research, the detection experiment of mature pomegranate was conducted in nature. The main research contents and methods are as follows:
(1)Firstly, characteristics of picking pomegranate image were studied based on color perceptual characteristics of human visual system in HSI space model. The illumination influences about saturation and color were analyzed by experiments, at the same time, solving illumination theoretical assumption was proposed based on human visual system. Secondly, a multi-scale Retinex algorithm with color healing factor(MSRCR) composed global and local characteristics was studied. Combined with color constancy theory, the experiments showed that the algorithm has realized the automatic image enhancement with different characteristics, such as dynamic range compression, color constancy and color high fidelity.
(2) According to gray and color characteristics in fruit image, the image histogram of hue showed a clear bimodal shape obviously, so the OSTU automatic threshold segmentation was adopted in order to remove most of image background. Firstly, morphological filtering was used to remove residual small area noise in binary images. Secondly,labeled by eight neighborhoods,optimized seed filling algorithm was used to fill the holes. Thirdly,every labeled area was calculated out, set the threshold according to the area and the length to width ratio of minimum enclosing rectangle, and the other areas were wiped out. The edge was detected by Sobel algorithm.At last,centroids and radius were extracted by the optimized Hough transform, and the shape of fruits was recovered.
(3) The comprehensive characteristics can complete image matching based on invariant moments of combination with configuration and intensity and transform domain, and applied to the target recognition. That is, Edge and corner point detection used SUSAN operator at low level image processing, in order to realize the preliminary localization and segmentation; while the invariant moments theory were introduced, invariant moments characteristics of area moments and line moments theory were proved by mass data tests, when translated or constrained or rotated.And the running average time of line invariant moments is more superior than area invariant moments, it is helpful to improve real time of image processing. Then the local significant characteristics extracted by line moment invariants, and image matching combines with the comprehensive characteristics, for instance, area to perimeter ratio, length to width ratio, duty ratio, intensity information etc. The detection of mature pomegranate was implemented, also the realization process and conclusions of target recognition were presented in this paper.
引文
[1]沈明霞,姬长英.农业机器人的开发背景及技术动向[J].农机化研究,2000(2)：31-35
[2]汤修映,张铁中.果蔬收获机器人研究综述[J].机器人,2005,27(1)：90-91
[3]Schertz.C.E,and G K. Brown Basic considerations in mechanizing citrus harvest[J].Transactions of the ASAE 1968,11 (2):343-346
[4]http://www.zjncpck.com/news/shownews.asp?id=210
[5]徐惠荣,应义斌.红外热成像在树上柑桔识别中的应用研究[J].红外与毫米波学报,2004,23(5)：353-356
[6]周云山等.计算机视觉在蘑菇采摘机器人上的应用[J].农业工程学报,1995,11(4)：27-32
[7]S.Limsiroratana,Y.Lkeda.On image analysis for harvesting tropical fruits.2002,SICE02-0824: 1336-1341
[8]孙明,凌云.基于计算机视觉的萝卜幼苗自动识别技术[J].农业机械学报,2002,33(5)：75-77
[9]Bulanon D M,KataokaT,Otay,etal A segmentation algorithmfor the autmvatic recognition of Fuji Apples at harvest[J].Bio-systems Engineering,2002,83(4):405-412
[10]张铁中,周人娟.草莓采摘机器人研究：Ⅰ.基于BP神经网络的草莓图像分割[J].中国农业大学学报,2004,9(4)：65-68
[11]齐龙,马旭,张小超.基于网络的植物病害彩色图像的分割技术[J].吉林大学学报.2006,36(2)：126-139
[12]尹建军,毛罕平等.不同生长状态下多目标番茄图像的自动分割方法[J].农业工程学报,2006,22(10)：149-153
[13]Kassay L. Hungarian robotic apple harvester[H].ASAE Paper,1992,92(2)
[14]张瑞合,姬长英等.计算机视觉技术在番茄收获中的应用[J].农业机械学报,2001,32(5)：50-53
[15]郑小冬,赵杰文.基于双目立体视觉的番茄识别与定位技术[J].计算机工程,2004,12(4)：50-53
[16]冈萨雷斯.数字图像处理[M].北京：电子工业出版社,2006：235-237,60-61,72-74
[17]安秋.农业机器人视觉导航系统及其光照问题的研究[D].南京农业大学,博士学位论文,2008
[18]陶霖密,姚国正,汪云九.人类颜色视觉的计算理论[J].心理学报,1993,25(3)：233-240
[19]Webster M A,Mollon J D.Colour constancy influenced by contrast adaptation[J].Nature, 1995,373(23):694-698
[20]Pokorny J.Vision and Visual Dysfunction[J].The Perception of Colour.London:Macmillan,1991 (6): 43-61
[21]D'Zmura M, Lennie P.Mechanisms of color constancy[J].J Opt Soc Am,1986,A3:1662-1672
[22]Sun J,Perona P Early computation of shape and reflectance in the visual system[J]. Nature,1996, 379(11):165-168
[23]Cropper S J,Derrington A M. Rapid colour-specific detection of motion in human vision[J].Nature, 1996,379(4):72-74
[24]Jin E W,Shevell S K.Color memory and color constancy[J].J Opt Soc Am 1996,A13(10):1981-1989
[25]Derefeldt G,Swartling T.Color concept retrieval by free color naming-identification of up to 30 colors without training[J].Displays,1995,16(2):69-77
[26]Brainard D H, Brunt W A, Speigle J M. Color constancy in the nearly natural image[J].2 Achromatic loci,Opt Soc Am,1998,A I 5(2):307-325
[27]Forsyth D A novel algorithm for colour constancy[J].Computer Vision 1990,5(1):5-36
[28]Finlayson G, Hordley S.Improving gamut mapping color constancy [J].IEEE Transactions on Image Processing,2000,9(10):1774-1783
[29]A Date Whittaker,G E Miles.Fruit location in a partially occluded image[J].Transactions of the ASAE,1989,30(3):591-596
[30]Finlayson G, Hordley S, Hubel P M.Color by correlation:a simple,unifying framework for color constancy[J].IEEE Trans on Pattern Analysis And Machine Intelligence,2001.23(11):1209-1221
[31]陶霖密,徐光佑.计算机视觉的二步法颜色恒常性[J].清华大学学报,自然科学版
[32]Steven A.Cover, Another look at Land's Retinex algorithm[J].IEEE proceedings of Southeastcon, April 1991:351-355
[33]http://baike.baidu.com/view/1915546.html?fromTaglist
[34]肖燕峰.基于Retinex理论的图像增强恢复算法研究[D].上海交通大学,硕士学位论文,2007
[35]Zia-ur Rahma,Daniel,Jobson,Glenn A.Woodell,Retinex Processing for automatic image enhancement[J].Journal of Electronic Imaging 13(1),January 2004:100-110
[36]陈雾.基于Retinex理论的图像增强算法研究[D].南京理工大学,硕士学位论文,2006
[37]章毓晋.图像工程(上)——图像处理[M].北京：清华大学出版社,2006：279-280
[38]章毓晋.图像工程(上)——图像处理[M].北京：清华大学出版社,2005：73-105,369-381,143-149
[39]姚敏.数字图象处理[M].北京：机械工业出版社,2006：225,245,261,281-287
[40]汤建华.番茄收获机械人中视觉目标的自动分割与识别研究[D].江苏大学,硕士学位论文,2005
[41]李玉良.基于立体视觉的遮挡柑橘识别与空间匹配研究[D].江苏大学,硕士学位论文,2007
[42]蔡健荣.基于机器视觉自然场景下成熟柑橘识别[J].农业工程学报,2008,24(1)175-178
[43]T Peng,A Balijepalli,S K Gupta,and T LeBrun.Algorithms for on-line monitoring of micro-spheres in an optical tweezers-based assembly cell[J].ASME Journal of Computing and Information Science in Engineering,7(4):330-338,2007
[44]姚立健.茄子收获机器人视觉系统和机械臂避障规划研究[D].南京农业大学,博士学位论文,2008
[45]龚声蓉,刘纯平,王强等.数字图像处理与分析[M].北京：清华大学出版社,2006：239-240
[46]范立南.图像处理与模式识别[M].北京：科学出版社,2007：113-118
[47]M K Hu.Visnal Pattern Rcognition by Moment Invariant[J].IRE Tran-sactions on Information Theory,1962,IT(8):179-187
[48]童卓,李霆.基于遗传算法的图像不变矩匹配[M].计算机工程与科学,2002,24(3)：14-17
[49]胡俊华,许守时等.基于局部自相似性的遥感图像港口舰船检测[J].中国图象图形学报,2009,14(4)：591-597
[50]翁木云,何明一.图像综合特征及其在图像检测与匹配中的应用[J].中国图象图形学报,2007,12(1)：122-124
[2]汤修映,张铁中.果蔬收获机器人研究综述[J].机器人,2005,27(1)：90-91
[3]Schertz.C.E,and G K. Brown Basic considerations in mechanizing citrus harvest[J].Transactions of the ASAE 1968,11 (2):343-346
[4]http://www.zjncpck.com/news/shownews.asp?id=210
[5]徐惠荣,应义斌.红外热成像在树上柑桔识别中的应用研究[J].红外与毫米波学报,2004,23(5)：353-356
[6]周云山等.计算机视觉在蘑菇采摘机器人上的应用[J].农业工程学报,1995,11(4)：27-32
[7]S.Limsiroratana,Y.Lkeda.On image analysis for harvesting tropical fruits.2002,SICE02-0824: 1336-1341
[8]孙明,凌云.基于计算机视觉的萝卜幼苗自动识别技术[J].农业机械学报,2002,33(5)：75-77
[9]Bulanon D M,KataokaT,Otay,etal A segmentation algorithmfor the autmvatic recognition of Fuji Apples at harvest[J].Bio-systems Engineering,2002,83(4):405-412
[10]张铁中,周人娟.草莓采摘机器人研究：Ⅰ.基于BP神经网络的草莓图像分割[J].中国农业大学学报,2004,9(4)：65-68
[11]齐龙,马旭,张小超.基于网络的植物病害彩色图像的分割技术[J].吉林大学学报.2006,36(2)：126-139
[12]尹建军,毛罕平等.不同生长状态下多目标番茄图像的自动分割方法[J].农业工程学报,2006,22(10)：149-153
[13]Kassay L. Hungarian robotic apple harvester[H].ASAE Paper,1992,92(2)
[14]张瑞合,姬长英等.计算机视觉技术在番茄收获中的应用[J].农业机械学报,2001,32(5)：50-53
[15]郑小冬,赵杰文.基于双目立体视觉的番茄识别与定位技术[J].计算机工程,2004,12(4)：50-53
[16]冈萨雷斯.数字图像处理[M].北京：电子工业出版社,2006：235-237,60-61,72-74
[17]安秋.农业机器人视觉导航系统及其光照问题的研究[D].南京农业大学,博士学位论文,2008
[18]陶霖密,姚国正,汪云九.人类颜色视觉的计算理论[J].心理学报,1993,25(3)：233-240
[19]Webster M A,Mollon J D.Colour constancy influenced by contrast adaptation[J].Nature, 1995,373(23):694-698
[20]Pokorny J.Vision and Visual Dysfunction[J].The Perception of Colour.London:Macmillan,1991 (6): 43-61
[21]D'Zmura M, Lennie P.Mechanisms of color constancy[J].J Opt Soc Am,1986,A3:1662-1672
[22]Sun J,Perona P Early computation of shape and reflectance in the visual system[J]. Nature,1996, 379(11):165-168
[23]Cropper S J,Derrington A M. Rapid colour-specific detection of motion in human vision[J].Nature, 1996,379(4):72-74
[24]Jin E W,Shevell S K.Color memory and color constancy[J].J Opt Soc Am 1996,A13(10):1981-1989
[25]Derefeldt G,Swartling T.Color concept retrieval by free color naming-identification of up to 30 colors without training[J].Displays,1995,16(2):69-77
[26]Brainard D H, Brunt W A, Speigle J M. Color constancy in the nearly natural image[J].2 Achromatic loci,Opt Soc Am,1998,A I 5(2):307-325
[27]Forsyth D A novel algorithm for colour constancy[J].Computer Vision 1990,5(1):5-36
[28]Finlayson G, Hordley S.Improving gamut mapping color constancy [J].IEEE Transactions on Image Processing,2000,9(10):1774-1783
[29]A Date Whittaker,G E Miles.Fruit location in a partially occluded image[J].Transactions of the ASAE,1989,30(3):591-596
[30]Finlayson G, Hordley S, Hubel P M.Color by correlation:a simple,unifying framework for color constancy[J].IEEE Trans on Pattern Analysis And Machine Intelligence,2001.23(11):1209-1221
[31]陶霖密,徐光佑.计算机视觉的二步法颜色恒常性[J].清华大学学报,自然科学版
[32]Steven A.Cover, Another look at Land's Retinex algorithm[J].IEEE proceedings of Southeastcon, April 1991:351-355
[33]http://baike.baidu.com/view/1915546.html?fromTaglist
[34]肖燕峰.基于Retinex理论的图像增强恢复算法研究[D].上海交通大学,硕士学位论文,2007
[35]Zia-ur Rahma,Daniel,Jobson,Glenn A.Woodell,Retinex Processing for automatic image enhancement[J].Journal of Electronic Imaging 13(1),January 2004:100-110
[36]陈雾.基于Retinex理论的图像增强算法研究[D].南京理工大学,硕士学位论文,2006
[37]章毓晋.图像工程(上)——图像处理[M].北京：清华大学出版社,2006：279-280
[38]章毓晋.图像工程(上)——图像处理[M].北京：清华大学出版社,2005：73-105,369-381,143-149
[39]姚敏.数字图象处理[M].北京：机械工业出版社,2006：225,245,261,281-287
[40]汤建华.番茄收获机械人中视觉目标的自动分割与识别研究[D].江苏大学,硕士学位论文,2005
[41]李玉良.基于立体视觉的遮挡柑橘识别与空间匹配研究[D].江苏大学,硕士学位论文,2007
[42]蔡健荣.基于机器视觉自然场景下成熟柑橘识别[J].农业工程学报,2008,24(1)175-178
[43]T Peng,A Balijepalli,S K Gupta,and T LeBrun.Algorithms for on-line monitoring of micro-spheres in an optical tweezers-based assembly cell[J].ASME Journal of Computing and Information Science in Engineering,7(4):330-338,2007
[44]姚立健.茄子收获机器人视觉系统和机械臂避障规划研究[D].南京农业大学,博士学位论文,2008
[45]龚声蓉,刘纯平,王强等.数字图像处理与分析[M].北京：清华大学出版社,2006：239-240
[46]范立南.图像处理与模式识别[M].北京：科学出版社,2007：113-118
[47]M K Hu.Visnal Pattern Rcognition by Moment Invariant[J].IRE Tran-sactions on Information Theory,1962,IT(8):179-187
[48]童卓,李霆.基于遗传算法的图像不变矩匹配[M].计算机工程与科学,2002,24(3)：14-17
[49]胡俊华,许守时等.基于局部自相似性的遥感图像港口舰船检测[J].中国图象图形学报,2009,14(4)：591-597
[50]翁木云,何明一.图像综合特征及其在图像检测与匹配中的应用[J].中国图象图形学报,2007,12(1)：122-124