区域配对引导的光照传播视频阴影去除方法
|
摘要
传统方法在处理自由移动相机捕获视频中的阴影时,存在时空不连贯现象。为解决该问题,提出一种区域配对引导的光照传播阴影去除方法。首先,使用基于尺度不变特征变换(SIFT)特征向量的均值漂移方法分割视频,通过支持向量机(SVM)分类器检测出其中的阴影;然后,将输入视频帧分解成重叠的二维图像区域块,建立其马尔可夫随机场(MRF),通过光流引导的区域块匹配机制,为每一个阴影块找到最佳匹配的非阴影块;最后,使用局部光照传播算子恢复阴影区域块的光照,并对其进行全局光照优化。实验结果表明,与传统基于光照传播方法相比,所提方法在阴影检测综合评价指标上提升约6. 23%,像素均方根误差(RMSE)减小约30. 12%,且大幅度缩短了阴影处理时间,得到的无阴影视频结果更具时空连贯性。
In order to solve spatio-temporally incoherent problem of traditional shadow removal methods for videos captured by free moving cameras,a shadow detection and removal approach using region matching guided by illumination transfer was proposed.Firstly,the input video was segmented by using Mean Shift method based on Scale Invariant Feature Transform(SIFT),and the video shadow was detected by Support Vector Machine(SVM)classifier.Secondly,the input video was decomposed into overlapped 2D patches,and a Markov Random Field(MRF)for this video was set up,and the corresponding lit patch for every shadow patch was found via region matching guided by optical flow.Finally,in order to get spatio-temporally coherent results,each shadow patch was processed with its matched lit patch by local illumination transfer operation and global shadow removal.The experimental results show that the proposed algorithm obtains higher accuracy and lower error than the traditional methods based on illumination transfer,the comprehensive evaluation metric is improved by about 6.23%,and the Root Mean Square Error(RMSE)is reduced by about 30.12%.It can obtain better shadow removal results with more spatio-temporal coherence but much less time.
In order to solve spatio-temporally incoherent problem of traditional shadow removal methods for videos captured by free moving cameras,a shadow detection and removal approach using region matching guided by illumination transfer was proposed.Firstly,the input video was segmented by using Mean Shift method based on Scale Invariant Feature Transform(SIFT),and the video shadow was detected by Support Vector Machine(SVM)classifier.Secondly,the input video was decomposed into overlapped 2D patches,and a Markov Random Field(MRF)for this video was set up,and the corresponding lit patch for every shadow patch was found via region matching guided by optical flow.Finally,in order to get spatio-temporally coherent results,each shadow patch was processed with its matched lit patch by local illumination transfer operation and global shadow removal.The experimental results show that the proposed algorithm obtains higher accuracy and lower error than the traditional methods based on illumination transfer,the comprehensive evaluation metric is improved by about 6.23%,and the Root Mean Square Error(RMSE)is reduced by about 30.12%.It can obtain better shadow removal results with more spatio-temporal coherence but much less time.
引文
[1]GUO R,DAI Q,HOIEM D.Single-image shadow detection and removal using paired regions[C]//Proceedings of the 2011 IEEEComputer Society Conference on Computer Vision and Pattern Recognition.Washington,DC:IEEE Computer Society,2011:2033-2040.
[2]ZHENG C,SUN Z-L,WANG N,et al.Moving cast shadow removal based on texture feature and color space[C]//Proceedings of the 2018 International Symposium on Neural Networks,LNCS10878.Cham:Springer,2018:611-618.
[3]XIAO Y,TSOUGENIS E,TANG C.Shadow removal from single RGB-D images[C]//Proceedings of the 2014 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Washington,DC:IEEE Computer Society,2014:3011-3018.
[4]KHAN S H,BENNAMOUN M,SOHEL F,et al.Automatic shadow detection and removal from a single image[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(3):431-446.
[5]QU L,TIAN J,HE S,et al.DeshadowNet:a multi-context embedding deep network for shadow removal[C]//Proceedings of the2017 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Washington,DC:IEEE Computer Society,2017:2308-2316.
[6]李鹏伟,葛文英,刘国英.基于阴影概率模型的遥感影像阴影检测方法[J].计算机应用,2015,35(2):510-514.(LI P W,GEW Y,LIU G Y.Shadow detection method based on shadow probability model for remote sensing images[J].Journal of Computer Applications,2015,35(2):510-514.)
[7]JAYNES C O,WEBB S B,STEELE R M.Dynamic shadow removal from front projection displays:US7133083[P].2006-11-07.
[8]LEONE A,DISTANTE C,ANCONA N,et al.Texture analysis for shadow removing in video-surveillance systems[C]//Proceedings of the 2004 IEEE International Conference on Systems,Man and Cybernetics.Piscataway,NJ:IEEE,2004:6325-6330.
[9]JUNG C R.Efficient background subtraction and shadow removal for monochromatic video sequences[J].IEEE Transactions on Multimedia,2009,11(3):571-577.
[10]LIU Z,HUANG K,TAN T,et al.Cast shadow removal combining local and global features[C]//Proceedings of the 2007 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Washington,DC:IEEE Computer Society,2007:1-8.
[11]MARTEL-BRISSON N,ZACCARIN A.Learning and removing cast shadows through a multidistribution approach[J].IEEETransactions on Pattern Analysis and Machine Intelligence,2007,29(7):1133-1146.
[12]HUANG J B,CHEN C S.Moving cast shadow detection using physics-based features[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Washington,DC:IEEE Computer Society,2009:2310-2317.
[13]BENEDEK C,SZIRANYI T.Bayesian foreground and shadow detection in uncertain frame rate surveillance videos[J].IEEETransactions on Image Processing,2008,17(4):608-621.
[14]XIAO C,SHE R,XIAO D,et al.Fast shadow removal using adaptive multi-scale illumination transfer[J].Computer Graphics Forum,2013,32(8):207-218.
[15]AMATO A,MOZEROV M G,BAGDANOV A D,et al.Accurate moving cast shadow suppression based on local color constancy detection[J].IEEE Transactions on Image Processing,2011,20(10):2954-2966.
[16]LIU Z,HUANG K,TAN T.Cast shadow removal in a hierarchical manner using MRF[J].IEEE Transactions on Circuits&Systems for Video Technology,2012,22(1):56-66.
[17]ZHANG L,ZHANG Q,XIAO C.Shadow remover:image shadow removal based on illumination recovering optimization[J].IEEETransactions on Image Processing,2015,24(11):4623-4636.
[18]COMANICIU D,MEER P.Mean Shift:a robust approach toward feature space analysis[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2002,24(5):603-619.
[19]ADAMS A,GELFAND N,DOLSON J,et al.Gaussian KD-trees for fast high-dimensional filtering[J].ACM Transactions on Graphics,2009,28(3):Article No.21.
[20]MARTIN D R,FOWLKES C C,MALIK J.Learning to detect natural image boundaries using local brightness,color,and texture cues[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2004,26(5):530-549.
[21]NG H-F,CHEN I-C,LIAO H-Y.An illumination invariant image descriptor for color image matching[EB/OL].[2018-04-11].http://pdfs.semanticscholar.org/56cb/21b1ff1c82e1fa78ed64e4671 80b858d09c2.pdf.
[22]HUANG J-B,KANG S B,AHUJA N,et al.Temporally coherent completion of dynamic video[J].ACM Transactions on Graphics,2016,35(6):Article No.196.
[23]SHOR Y,LISCHINSKI D.The shadow meets the mask:pyramidbased shadow removal[J].Computer Graphics Forum,2008,27(2):577-586.
[24]PRATI A,MIKIC I,TRIVEDI M M,et al.Detecting moving shadows:algorithms and evaluation[J].IEEE Transactions on Pattern Analysis&Machine Intelligence,2003,25(7):918-923.
[2]ZHENG C,SUN Z-L,WANG N,et al.Moving cast shadow removal based on texture feature and color space[C]//Proceedings of the 2018 International Symposium on Neural Networks,LNCS10878.Cham:Springer,2018:611-618.
[3]XIAO Y,TSOUGENIS E,TANG C.Shadow removal from single RGB-D images[C]//Proceedings of the 2014 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Washington,DC:IEEE Computer Society,2014:3011-3018.
[4]KHAN S H,BENNAMOUN M,SOHEL F,et al.Automatic shadow detection and removal from a single image[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(3):431-446.
[5]QU L,TIAN J,HE S,et al.DeshadowNet:a multi-context embedding deep network for shadow removal[C]//Proceedings of the2017 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Washington,DC:IEEE Computer Society,2017:2308-2316.
[6]李鹏伟,葛文英,刘国英.基于阴影概率模型的遥感影像阴影检测方法[J].计算机应用,2015,35(2):510-514.(LI P W,GEW Y,LIU G Y.Shadow detection method based on shadow probability model for remote sensing images[J].Journal of Computer Applications,2015,35(2):510-514.)
[7]JAYNES C O,WEBB S B,STEELE R M.Dynamic shadow removal from front projection displays:US7133083[P].2006-11-07.
[8]LEONE A,DISTANTE C,ANCONA N,et al.Texture analysis for shadow removing in video-surveillance systems[C]//Proceedings of the 2004 IEEE International Conference on Systems,Man and Cybernetics.Piscataway,NJ:IEEE,2004:6325-6330.
[9]JUNG C R.Efficient background subtraction and shadow removal for monochromatic video sequences[J].IEEE Transactions on Multimedia,2009,11(3):571-577.
[10]LIU Z,HUANG K,TAN T,et al.Cast shadow removal combining local and global features[C]//Proceedings of the 2007 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Washington,DC:IEEE Computer Society,2007:1-8.
[11]MARTEL-BRISSON N,ZACCARIN A.Learning and removing cast shadows through a multidistribution approach[J].IEEETransactions on Pattern Analysis and Machine Intelligence,2007,29(7):1133-1146.
[12]HUANG J B,CHEN C S.Moving cast shadow detection using physics-based features[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Washington,DC:IEEE Computer Society,2009:2310-2317.
[13]BENEDEK C,SZIRANYI T.Bayesian foreground and shadow detection in uncertain frame rate surveillance videos[J].IEEETransactions on Image Processing,2008,17(4):608-621.
[14]XIAO C,SHE R,XIAO D,et al.Fast shadow removal using adaptive multi-scale illumination transfer[J].Computer Graphics Forum,2013,32(8):207-218.
[15]AMATO A,MOZEROV M G,BAGDANOV A D,et al.Accurate moving cast shadow suppression based on local color constancy detection[J].IEEE Transactions on Image Processing,2011,20(10):2954-2966.
[16]LIU Z,HUANG K,TAN T.Cast shadow removal in a hierarchical manner using MRF[J].IEEE Transactions on Circuits&Systems for Video Technology,2012,22(1):56-66.
[17]ZHANG L,ZHANG Q,XIAO C.Shadow remover:image shadow removal based on illumination recovering optimization[J].IEEETransactions on Image Processing,2015,24(11):4623-4636.
[18]COMANICIU D,MEER P.Mean Shift:a robust approach toward feature space analysis[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2002,24(5):603-619.
[19]ADAMS A,GELFAND N,DOLSON J,et al.Gaussian KD-trees for fast high-dimensional filtering[J].ACM Transactions on Graphics,2009,28(3):Article No.21.
[20]MARTIN D R,FOWLKES C C,MALIK J.Learning to detect natural image boundaries using local brightness,color,and texture cues[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2004,26(5):530-549.
[21]NG H-F,CHEN I-C,LIAO H-Y.An illumination invariant image descriptor for color image matching[EB/OL].[2018-04-11].http://pdfs.semanticscholar.org/56cb/21b1ff1c82e1fa78ed64e4671 80b858d09c2.pdf.
[22]HUANG J-B,KANG S B,AHUJA N,et al.Temporally coherent completion of dynamic video[J].ACM Transactions on Graphics,2016,35(6):Article No.196.
[23]SHOR Y,LISCHINSKI D.The shadow meets the mask:pyramidbased shadow removal[J].Computer Graphics Forum,2008,27(2):577-586.
[24]PRATI A,MIKIC I,TRIVEDI M M,et al.Detecting moving shadows:algorithms and evaluation[J].IEEE Transactions on Pattern Analysis&Machine Intelligence,2003,25(7):918-923.