基于可见光图像和卷积神经网络的冬小麦苗期长势参数估算
|
摘要
针对目前基于计算机视觉估算冬小麦苗期长势参数存在易受噪声干扰且对人工特征依赖性较强的问题,该文综合运用图像处理和深度学习技术,提出一种基于卷积神经网络(convolutional neural network, CNN)的冬小麦苗期长势参数估算方法。以冬小麦苗期冠层可见光图像作为输入,构建了适用于冬小麦苗期长势参数估算卷积神经网络模型,通过学习的方式建立冬小麦冠层可见光图像与长势参数的关系,实现了农田尺度冬小麦苗期冠层叶面积指数(leaf area index,LAI)和地上生物量(above ground biomass, AGB)的准确估算。为验证方法的有效性,该研究采用以冠层覆盖率(canopy cover, CC)作为自变量的线性回归模型和以图像特征为输入的随机森林(random forest, RF)、支持向量机回归(support vectormachinesregression,SVM)进行对比分析,采用决定系数(coefficientofdetermination,R2)和归一化均方根误差(normalized root mean square error, NRMSE)定量评价估算方法的准确率。结果表明:该方法估算准确率均优于对比方法,其中AGB估算结果的R2为0.7917,NRMSE为24.37%,LAI估算结果的R2为0.8256,NRMSE为23.33%。研究可为冬小麦苗期长势监测与田间精细管理提供参考。
Leaf area index(LAI) and above ground biomass(AGB) are two critical traits indicating the growth of winter wheat. Currently, non-destructive methods for measuring LAI and AGB heavily are subjected to limitations that the methods are susceptible to the environmental noises and greatly depend on the manual designed features. In this study, an easy-to-use growth-related traits estimation method for winter wheat at early growth stages was proposed by using digital images captured under field conditions and Convolutional Neural Network(CNN). RGB images of winter wheat canopy in 12 plots were captured at the field station of Shangqiu Academy of Agriculture and Forestry Sciences, Henan, China. The canopy images were captured by a low-cost camera at the early growth stages. Using canopy images at early growth stages as input, a CNN structure suitable for the estimation of growth related traits was explored, which was then trained to learn the relationship between the canopy images and the corresponding growth-related traits. Based on the trained CNN, the estimation of LAI and AGB of winter wheat at early growth stages was achieved. In order to compare the results of the CNN, conventionally adopted methods for estimating LAI and AGB in conjunction with a collection of color and texture feature extraction techniques were used. The conventional methods included a linear regression model using canopy cover as the predictor variable(LR-CC), Random Forest(RF) and Support Vector Machine Regression(SVR). The canopy images of winter wheat were captured at early growth stages, resulting in the existence of pixels representing non-vegetation elements in these images, such as soil. Therefore, it was necessary to perform image segmentation of vegetation for the compared methods prior to feature extraction. The segmentation was achieved by Canopeo. The linear regression was used to compare the accuracy of the methods. Normalized Root-Mean-Squared error(NRMSE) and coefficient of determination(R2) were used as the criterion for model evaluation. Results showed the CNN demonstrated superior results to the compared methods in the two metrics. Strong correlations could be observed between the actual measurements of traits to those estimated by the CNN. The estimation results of LAI had R2 equaled to 0.825 6 and NRMSE equaled to 23.33%, and the results of AGB had R2 equaled to 0.791 7 and NRMSE equals to 24.37%. Compare to the comparative methods, the CNN was a more direct method for AGB and LAI estimation. The image segmentation of vegetation was not necessary because the CNN was able to use the important features to estimate AGB and LAI and ignore the non-important features, which not only reduced the computation cost but also increased the efficiency of the estimation. In contrast, the performances of the compared estimating methods greatly depended on the results of image segmentation. Accurate segmentation results guaranteed accurate data sources to feature extraction. However, canopy images captured under real field conditions were suffering from uneven illumination and complicated background, which was a big challenge to achieve robust image segmentation of vegetation. It was revealed that robust estimation of AGB and LAI of winter wheat at early growth stages could be achieved by CNN, which can provide support to growth monitoring and field management of winter wheat.
Leaf area index(LAI) and above ground biomass(AGB) are two critical traits indicating the growth of winter wheat. Currently, non-destructive methods for measuring LAI and AGB heavily are subjected to limitations that the methods are susceptible to the environmental noises and greatly depend on the manual designed features. In this study, an easy-to-use growth-related traits estimation method for winter wheat at early growth stages was proposed by using digital images captured under field conditions and Convolutional Neural Network(CNN). RGB images of winter wheat canopy in 12 plots were captured at the field station of Shangqiu Academy of Agriculture and Forestry Sciences, Henan, China. The canopy images were captured by a low-cost camera at the early growth stages. Using canopy images at early growth stages as input, a CNN structure suitable for the estimation of growth related traits was explored, which was then trained to learn the relationship between the canopy images and the corresponding growth-related traits. Based on the trained CNN, the estimation of LAI and AGB of winter wheat at early growth stages was achieved. In order to compare the results of the CNN, conventionally adopted methods for estimating LAI and AGB in conjunction with a collection of color and texture feature extraction techniques were used. The conventional methods included a linear regression model using canopy cover as the predictor variable(LR-CC), Random Forest(RF) and Support Vector Machine Regression(SVR). The canopy images of winter wheat were captured at early growth stages, resulting in the existence of pixels representing non-vegetation elements in these images, such as soil. Therefore, it was necessary to perform image segmentation of vegetation for the compared methods prior to feature extraction. The segmentation was achieved by Canopeo. The linear regression was used to compare the accuracy of the methods. Normalized Root-Mean-Squared error(NRMSE) and coefficient of determination(R2) were used as the criterion for model evaluation. Results showed the CNN demonstrated superior results to the compared methods in the two metrics. Strong correlations could be observed between the actual measurements of traits to those estimated by the CNN. The estimation results of LAI had R2 equaled to 0.825 6 and NRMSE equaled to 23.33%, and the results of AGB had R2 equaled to 0.791 7 and NRMSE equals to 24.37%. Compare to the comparative methods, the CNN was a more direct method for AGB and LAI estimation. The image segmentation of vegetation was not necessary because the CNN was able to use the important features to estimate AGB and LAI and ignore the non-important features, which not only reduced the computation cost but also increased the efficiency of the estimation. In contrast, the performances of the compared estimating methods greatly depended on the results of image segmentation. Accurate segmentation results guaranteed accurate data sources to feature extraction. However, canopy images captured under real field conditions were suffering from uneven illumination and complicated background, which was a big challenge to achieve robust image segmentation of vegetation. It was revealed that robust estimation of AGB and LAI of winter wheat at early growth stages could be achieved by CNN, which can provide support to growth monitoring and field management of winter wheat.
引文
[1]徐旭,陈国庆,王良,等.基于敏感光谱波段图像特征的冬小麦LAI和地上部生物量监测[J].农业工程学报,2015,31(22):169-175.Xu Xu,Chen Guoqing,Wang Liang,et al.Monitoring leaf area index and biomass above ground of winter wheat based on sensitive spectral waveband and corresponding image characteristic[J].Transactions of the Chinese Society of Agricultural Engineering(Transactions of the CSAE),2015,31(22):169-175.(in Chinese with English abstract)
[2]Zhang L,Verma B,Stockwell D,et al.Density weighted connectivity of grass pixels in image frames for biomass estimation[J].Expert Systems with Applications,2018,101:213-227.
[3]Walter J,Edwards J,McDonald G,et al.Photogrammetry for the estimation of wheat biomass and harvest index[J].Field Crops Research,2018,216:165-174.
[4]陈玉青,杨玮,李民赞,等.基于Android手机平台的冬小麦叶面积指数快速测量系统[J].农业机械学报,2017,48(增刊):123-128.Chen Yuqing,Yang Wei,Li Minzan,et al.Measurement system of winter wheat LAI based on android mobile platform[J].Transactions of the Chinese Society for Agricultural Machinery,2017,48(Supp):123-128.(in Chinese with English abstract)
[5]高林,杨贵军,于海洋,等.基于无人机高光谱遥感的冬小麦叶面积指数反演[J].农业工程学报,2016,32(22):113-120.Gao Lin,Yang Guijun,Yu Haiyang,et al.Retrieving winter wheat leaf area index based on unmanned aerial vehicle hyperspectral remoter sensing[J].Transactions of the Chinese Society of Agricultural Engineering(Transactions of the CSAE),2016,32(22):113-120.(in Chinese with English abstract)
[6]Schirrmann M,Hamdorf A,Garz A,et al.Estimating wheat biomass by combining image clustering with crop height[J].Computers and Electronics in Agriculture,2016,121:374-384.
[7]Rasmussen J,Ntakos G,Nielsen J,et al.Are vegetation indices derived from consumer-grade cameras mounted on UAVs sufficiently reliable for assessing experimental plots?[J].European Journal of Agronomy,2016,74:75-92.
[8]苏伟,张明政,展郡鸽,等.基于机载Li DAR数据的农作物叶面积指数估算方法研究[J].农业机械学报,2016,47(3):272-277.Su Wei,Zhang Mingzheng,Zhan Junge,et al.Estimation method of crop leaf area index based on airborne LiDARdata[J].Transactions of the Chinese Society for Agricultural Machinery,2016,47(3):272-277.(in Chinese with English abstract)
[9]李明,张长利,房俊龙.基于图像处理技术的小麦叶面积指数的提取[J].农业工程学报,2010,26(1):205-209.Li Ming,Zhang Changli,Fang Junlong.Extraction of leaf area index of wheat based on image processing technique[J].Transactions of the Chinese Society of Agricultural Engineering(Transactions of the CSAE),2010,26(1):205-209.(in Chinese with English abstract)
[10]Hamuda E,Mc Ginley B,Glavin M,et al.Automatic crop detection under field conditions using the HSV colour space and morphological operations[J].Computers and Electronics in Agriculture,2017,133:97-107.
[11]González-Esquiva J M,Oates M J,García-Mateos G,et al.Development of a visual monitoring system for water balance estimation of horticultural crops using low cost cameras[J].Computers and Electronics in Agriculture,2017,141:15-26.
[12]Casadesús J,Villegas D.Conventional digital cameras as a tool for assessing leaf area index and biomass for cereal breeding[J].Journal of Integrative Plant Biology,2014,56(1):7-14.
[13]马浚诚,杜克明,郑飞翔,等.可见光光谱和支持向量机的温室黄瓜霜霉病图像分割[J].光谱学与光谱分析,2018,38(6):1863-1868.Ma Juncheng,Du Keming,Zheng Feixiang,et al.Asegmenting method for greenhouse cucumber downy mildew images based on visual spectral and support vector machine[J].Spectroscopy and Spectral Analysis,2018,38(6):1863-1868.(in Chinese with English abstract)
[14]Ma J,Li X,Wen H,et al.A key frame extraction method for processing greenhouse vegetables production monitoring video[J].Computers and Electronics in Agriculture,2015,111:92-102.
[15]Chung Y S,Choi S C,Silva R R,et al.Case study:Estimation of sorghum biomass using digital image analysis with Canopeo[J].Biomass and Bioenergy,2017,105:207-210.
[16]Neumann K,Klukas C,Friedel S,et al.Dissecting spatiotemporal biomass accumulation in barley under different water regimes using high-throughput image analysis[J].Plant,Cell and Environment,2015,38(10):1980-1996.
[17]Patrignani A,Ochsner T E.Canopeo:A powerful new tool for measuring fractional green canopy cover[J].Agronomy Journal,2015,107(6):2312-2320.
[18]Virlet N,Sabermanesh K,Sadeghi-Tehran P,et al.Field scanalyzer:An automated robotic field phenotyping platform for detailed crop monitoring[J].Functional Plant Biology,2017,44(1):143-153.
[19]崔日鲜,刘亚东,付金东.基于可见光光谱和BP人工神经网络的冬小麦生物量估算研究[J].光谱学与光谱分析,2015,35(9):2596-2601.Cui Rixian,Liu Yadong,Fu Jindong.Estimation of winter wheat biomass using visible spectral and bp based artificial neural network[J].Spectroscopy and Spectral Analysis,2015,35(9):2596-2601.(in Chinese with English abstract)
[20]马浚诚,杜克明,郑飞翔,等.基于卷积神经网络的温室黄瓜病害识别系统[J].农业工程学报,2018,34(12):186-192.Ma Juncheng,Du Keming,Zheng Feixiang,et al.Disease recognition system for greenhouse cucumbers based on deep convolutional neural network[J].Transactions of the Chinese Society of Agricultural Engineering(Transactions of the CSAE),2018,34(12):186-192.(in Chinese with English abstract)
[21]Ma J,Du K,Zhang L,et al.A segmentation method for greenhouse vegetable foliar disease spots images using color information and region growing[J].Computers and Electronics in Agriculture,2017,142:110-117.
[22]Krizhevsky A,Sutskever I,Hinton G E.ImageNet classification with deep convolutional neural networks[J].Advances in Neural Information Processing Systems,2012,25(2):1?9.
[23]Simonyan K,Zisserman A.Very deep convolutional networks for large-scale image recognition[C]//International Conference on Learning Representations,2014:1-14.
[24]Ghosal S,Blystone D,Singh A K,et al.An explainable deep machine vision framework for plant stress phenotyping[J].Proceedings of the National Academy of Sciences of the United States of America,2018,115(18):4613-4618.
[25]Ferreira A D S,Freitas D M,Silva G G D,et al.Weed detection in soybean crops using Conv Nets[J].Computers and Electronics in Agriculture,2017,143:314-324.
[26]Ding W,Taylor G.Automatic moth detection from trap images for pest management[J].Computers and Electronics in Agriculture,2016,123:17-28.
[27]Xiong X,Duan L,Liu L,et al.Panicle-SEG:A robust image segmentation method for rice panicles in the field based on deep learning and superpixel optimization[J].Plant Methods,2017,13(1):1-15.
[28]段凌凤,熊雄,刘谦,等.基于深度全卷积神经网络的大田稻穗分割[J].农业工程学报,2018,34(12):202-209.Duan Lingfeng,Xiong Xiong,Liu Qian,et al.Field rice panicles segmentation based on deep full convolutional neural network[J].Transactions of the Chinese Society of Agricultural Engineering(Transactions of the CSAE),2018,34(12):202-209.(in Chinese with English abstract)
[29]刘立波,程晓龙,赖军臣.基于改进全卷积网络的棉田冠层图像分割方法[J].农业工程学报,2018,34(12):193-201.Liu Libo,Cheng Xiaolong,Lai Junchen.Segmentation method for cotton canopy image based on improved fully convolutional network model[J].Transactions of the Chinese Society of Agricultural Engineering(Transactions of the CSAE),2018,34(12):193-201.(in Chinese with English abstract)
[30]刘镕源,王纪华,杨贵军,等.冬小麦叶面积指数地面测量方法的比较[J].农业工程学报,2011,27(3):220-224.Liu Rongyuan,Wang Jihua,Yang Guijun,et al.Comparison of ground-based LAI measuring methods on winter wheat[J].Transactions of the Chinese Society of Agricultural Engineering(Transactions of the CSAE),2011,27(3):220-224.(in Chinese with English abstract)
[31]Baresel J P,Rischbeck P,Hu Y,et al.Use of a digital camera as alternative method for non-destructive detection of the leaf chlorophyll content and the nitrogen nutrition status in wheat[J].Computers and Electronics in Agriculture,2017,140:25-33.
[32]Ma J,Li Y,Du K,et al.Estimating above ground biomass of winter wheat at early growth stages using digital images and deep convolutional neural network[J].European Journal of Agronomy,2019,103:117-129.
[33]Ma J,Du K,Zheng F,et al.A recognition method for cucumber diseases using leaf symptom images based on deep convolutional neural network[J].Computers and Electronics in Agriculture,2018,154:18-24.
[2]Zhang L,Verma B,Stockwell D,et al.Density weighted connectivity of grass pixels in image frames for biomass estimation[J].Expert Systems with Applications,2018,101:213-227.
[3]Walter J,Edwards J,McDonald G,et al.Photogrammetry for the estimation of wheat biomass and harvest index[J].Field Crops Research,2018,216:165-174.
[4]陈玉青,杨玮,李民赞,等.基于Android手机平台的冬小麦叶面积指数快速测量系统[J].农业机械学报,2017,48(增刊):123-128.Chen Yuqing,Yang Wei,Li Minzan,et al.Measurement system of winter wheat LAI based on android mobile platform[J].Transactions of the Chinese Society for Agricultural Machinery,2017,48(Supp):123-128.(in Chinese with English abstract)
[5]高林,杨贵军,于海洋,等.基于无人机高光谱遥感的冬小麦叶面积指数反演[J].农业工程学报,2016,32(22):113-120.Gao Lin,Yang Guijun,Yu Haiyang,et al.Retrieving winter wheat leaf area index based on unmanned aerial vehicle hyperspectral remoter sensing[J].Transactions of the Chinese Society of Agricultural Engineering(Transactions of the CSAE),2016,32(22):113-120.(in Chinese with English abstract)
[6]Schirrmann M,Hamdorf A,Garz A,et al.Estimating wheat biomass by combining image clustering with crop height[J].Computers and Electronics in Agriculture,2016,121:374-384.
[7]Rasmussen J,Ntakos G,Nielsen J,et al.Are vegetation indices derived from consumer-grade cameras mounted on UAVs sufficiently reliable for assessing experimental plots?[J].European Journal of Agronomy,2016,74:75-92.
[8]苏伟,张明政,展郡鸽,等.基于机载Li DAR数据的农作物叶面积指数估算方法研究[J].农业机械学报,2016,47(3):272-277.Su Wei,Zhang Mingzheng,Zhan Junge,et al.Estimation method of crop leaf area index based on airborne LiDARdata[J].Transactions of the Chinese Society for Agricultural Machinery,2016,47(3):272-277.(in Chinese with English abstract)
[9]李明,张长利,房俊龙.基于图像处理技术的小麦叶面积指数的提取[J].农业工程学报,2010,26(1):205-209.Li Ming,Zhang Changli,Fang Junlong.Extraction of leaf area index of wheat based on image processing technique[J].Transactions of the Chinese Society of Agricultural Engineering(Transactions of the CSAE),2010,26(1):205-209.(in Chinese with English abstract)
[10]Hamuda E,Mc Ginley B,Glavin M,et al.Automatic crop detection under field conditions using the HSV colour space and morphological operations[J].Computers and Electronics in Agriculture,2017,133:97-107.
[11]González-Esquiva J M,Oates M J,García-Mateos G,et al.Development of a visual monitoring system for water balance estimation of horticultural crops using low cost cameras[J].Computers and Electronics in Agriculture,2017,141:15-26.
[12]Casadesús J,Villegas D.Conventional digital cameras as a tool for assessing leaf area index and biomass for cereal breeding[J].Journal of Integrative Plant Biology,2014,56(1):7-14.
[13]马浚诚,杜克明,郑飞翔,等.可见光光谱和支持向量机的温室黄瓜霜霉病图像分割[J].光谱学与光谱分析,2018,38(6):1863-1868.Ma Juncheng,Du Keming,Zheng Feixiang,et al.Asegmenting method for greenhouse cucumber downy mildew images based on visual spectral and support vector machine[J].Spectroscopy and Spectral Analysis,2018,38(6):1863-1868.(in Chinese with English abstract)
[14]Ma J,Li X,Wen H,et al.A key frame extraction method for processing greenhouse vegetables production monitoring video[J].Computers and Electronics in Agriculture,2015,111:92-102.
[15]Chung Y S,Choi S C,Silva R R,et al.Case study:Estimation of sorghum biomass using digital image analysis with Canopeo[J].Biomass and Bioenergy,2017,105:207-210.
[16]Neumann K,Klukas C,Friedel S,et al.Dissecting spatiotemporal biomass accumulation in barley under different water regimes using high-throughput image analysis[J].Plant,Cell and Environment,2015,38(10):1980-1996.
[17]Patrignani A,Ochsner T E.Canopeo:A powerful new tool for measuring fractional green canopy cover[J].Agronomy Journal,2015,107(6):2312-2320.
[18]Virlet N,Sabermanesh K,Sadeghi-Tehran P,et al.Field scanalyzer:An automated robotic field phenotyping platform for detailed crop monitoring[J].Functional Plant Biology,2017,44(1):143-153.
[19]崔日鲜,刘亚东,付金东.基于可见光光谱和BP人工神经网络的冬小麦生物量估算研究[J].光谱学与光谱分析,2015,35(9):2596-2601.Cui Rixian,Liu Yadong,Fu Jindong.Estimation of winter wheat biomass using visible spectral and bp based artificial neural network[J].Spectroscopy and Spectral Analysis,2015,35(9):2596-2601.(in Chinese with English abstract)
[20]马浚诚,杜克明,郑飞翔,等.基于卷积神经网络的温室黄瓜病害识别系统[J].农业工程学报,2018,34(12):186-192.Ma Juncheng,Du Keming,Zheng Feixiang,et al.Disease recognition system for greenhouse cucumbers based on deep convolutional neural network[J].Transactions of the Chinese Society of Agricultural Engineering(Transactions of the CSAE),2018,34(12):186-192.(in Chinese with English abstract)
[21]Ma J,Du K,Zhang L,et al.A segmentation method for greenhouse vegetable foliar disease spots images using color information and region growing[J].Computers and Electronics in Agriculture,2017,142:110-117.
[22]Krizhevsky A,Sutskever I,Hinton G E.ImageNet classification with deep convolutional neural networks[J].Advances in Neural Information Processing Systems,2012,25(2):1?9.
[23]Simonyan K,Zisserman A.Very deep convolutional networks for large-scale image recognition[C]//International Conference on Learning Representations,2014:1-14.
[24]Ghosal S,Blystone D,Singh A K,et al.An explainable deep machine vision framework for plant stress phenotyping[J].Proceedings of the National Academy of Sciences of the United States of America,2018,115(18):4613-4618.
[25]Ferreira A D S,Freitas D M,Silva G G D,et al.Weed detection in soybean crops using Conv Nets[J].Computers and Electronics in Agriculture,2017,143:314-324.
[26]Ding W,Taylor G.Automatic moth detection from trap images for pest management[J].Computers and Electronics in Agriculture,2016,123:17-28.
[27]Xiong X,Duan L,Liu L,et al.Panicle-SEG:A robust image segmentation method for rice panicles in the field based on deep learning and superpixel optimization[J].Plant Methods,2017,13(1):1-15.
[28]段凌凤,熊雄,刘谦,等.基于深度全卷积神经网络的大田稻穗分割[J].农业工程学报,2018,34(12):202-209.Duan Lingfeng,Xiong Xiong,Liu Qian,et al.Field rice panicles segmentation based on deep full convolutional neural network[J].Transactions of the Chinese Society of Agricultural Engineering(Transactions of the CSAE),2018,34(12):202-209.(in Chinese with English abstract)
[29]刘立波,程晓龙,赖军臣.基于改进全卷积网络的棉田冠层图像分割方法[J].农业工程学报,2018,34(12):193-201.Liu Libo,Cheng Xiaolong,Lai Junchen.Segmentation method for cotton canopy image based on improved fully convolutional network model[J].Transactions of the Chinese Society of Agricultural Engineering(Transactions of the CSAE),2018,34(12):193-201.(in Chinese with English abstract)
[30]刘镕源,王纪华,杨贵军,等.冬小麦叶面积指数地面测量方法的比较[J].农业工程学报,2011,27(3):220-224.Liu Rongyuan,Wang Jihua,Yang Guijun,et al.Comparison of ground-based LAI measuring methods on winter wheat[J].Transactions of the Chinese Society of Agricultural Engineering(Transactions of the CSAE),2011,27(3):220-224.(in Chinese with English abstract)
[31]Baresel J P,Rischbeck P,Hu Y,et al.Use of a digital camera as alternative method for non-destructive detection of the leaf chlorophyll content and the nitrogen nutrition status in wheat[J].Computers and Electronics in Agriculture,2017,140:25-33.
[32]Ma J,Li Y,Du K,et al.Estimating above ground biomass of winter wheat at early growth stages using digital images and deep convolutional neural network[J].European Journal of Agronomy,2019,103:117-129.
[33]Ma J,Du K,Zheng F,et al.A recognition method for cucumber diseases using leaf symptom images based on deep convolutional neural network[J].Computers and Electronics in Agriculture,2018,154:18-24.