基于交互式条件随机场的RGB-D图像语义分割
|
摘要
RGB-D图像语义分割是场景识别与分析的基础步骤,基于条件随机场(CRF)的图像分割方法不能有效应用于复杂多变的现实场景,因此提出一种交互式条件随机场的RGB-D图像语义分割方法。首先利用中值滤波和形态重构方法对Kinect相机拍摄的RGB-D图像进行预处理,降低图像噪声及数据缺失;其次,利用基于条件随机场的分割方法对经过预处理的图像进行自动分割,得到粗略的分割结果;最后,用户通过交互平台,将代表正确场景信息的标签反应到条件随机场模型中并进行模型更新,改善分割结果。通过多组实验验证了该算法不仅满足用户对于复杂场景分割与识别的需求,而且用户交互简单、方便、直观。相较于传统的基于条件随机场分割方法,该方法得到较高的分割精度和较好的识别效果。
RGB-D image semantic segmentation is the primary step of scene recognition and analysis,and the image segmentation method based on conditional random fields( CRF) cannot be applied in complex and volatile scenes,therefore an RGB-D image semantic segmentation method with interactive conditional random fields is proposed. Firstly,preprocess the depth and color images generated from Kinect with median filter and morphology reconstruction method,reducing the image noise and missing data. Secondly,automatically segment the preprocessed images with conditional random fields to obtain the rough segmentation. Finally,user takes the correct labels into the conditional random fields' model to update the model through an interactive platform,which can improve the segmentation results. Compared with the traditional segmentation method based on conditional random fields, the proposed method can achieve better performance in scene understanding and analysis.
RGB-D image semantic segmentation is the primary step of scene recognition and analysis,and the image segmentation method based on conditional random fields( CRF) cannot be applied in complex and volatile scenes,therefore an RGB-D image semantic segmentation method with interactive conditional random fields is proposed. Firstly,preprocess the depth and color images generated from Kinect with median filter and morphology reconstruction method,reducing the image noise and missing data. Secondly,automatically segment the preprocessed images with conditional random fields to obtain the rough segmentation. Finally,user takes the correct labels into the conditional random fields' model to update the model through an interactive platform,which can improve the segmentation results. Compared with the traditional segmentation method based on conditional random fields, the proposed method can achieve better performance in scene understanding and analysis.
引文
[1]Shao L,Han J,Kohli P,et al.Computer vision and machine learning with RGB-D sensors[M].Switzerland:Springer International Publishing,2014:3-26.
[2]Kohli P,Ladicky L,Torr P H.Robust higher order potentials for enforcing label consistency[J].International Journal of Computer Vision,2009,82(3):302-324.
[3]Lafferty J D,McCallum A,Pereira F C N.Conditional random fields:probabilistic models for segmenting and labeling sequence data[C]//Proceedings of the Eighteenth International Conference on Machine Learning,Williamstown,MA,USA.San Francisco,CA,USA:Morgan Kaufmann Publishers,2001:282-289.
[4]Boykov Y Y,Jolly M P.Interactive graph cuts for optimal boundary and region segmentation of objects in images[C]//Proceeding of the Eighth IEEE International Conference on Computer Vision,Vancouver,BC,Canada,2001:105-112.
[5]Grady L.Random walks for image segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2006,28(11):1768-1783.
[6]Noma A,Graciano A B V,Jr R M C,et al.Interactive image segmentation by matching attributed relational graphs[J].Pattern Recognition,2012,45(3):1159-1179.
[7]Hwang H,Haddad R A.Adaptive median filters:new algorithm and results[J].IEEE Transactions on Image Processing,1995,4(4):499-502.
[8]文华.基于数学形态学的图像处理算法的研究[D].哈尔滨:哈尔滨工程大学,2007.
[9]Boykov Y,Funka-Lea G.Graph cuts and efficient image segmentation[J].International Journal of Computer Vision,2006,70(2):109-131.
[10]Yuan J,Bae E,Tai X C,et al.A continuous max,low approach to Potts model[C]//11th European Conference on Computer Vision,Heraklion,Crete,Greece.Springer,2010:379-392.
[11]Boykov Y,Veksler O,Zabih R.Fast approximate energy minimization via graph cuts[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2001,23(11):1222-1239.
[12]Li Y,Sun J,Tang C K,et al.Lazy snapping[J].ACM Transactions on Graphics,2004,23(3):303-308.
[13]Chum O,Matas J.Randomized RANSAC with T(d,d)test[C]//Proceedings of the 13th British Machine Vision Conference,Cardiff,UK,2002:448-457.
[14]Rother C,Kolmogorov V,Blake A.“GrabCut”:Interactive foreground extraction using iterated graph cuts[J].ACM Transactions on Graphics,2004,23(3):309-314.
[15]Friedland G,Jantz K,Rojas R.SIOXl simple interactive object extraction in still images[C]//Proceedings of the2005 IEEE International Symposium on Multimedia,Irvine,CA,USA,2005:253-260.
[2]Kohli P,Ladicky L,Torr P H.Robust higher order potentials for enforcing label consistency[J].International Journal of Computer Vision,2009,82(3):302-324.
[3]Lafferty J D,McCallum A,Pereira F C N.Conditional random fields:probabilistic models for segmenting and labeling sequence data[C]//Proceedings of the Eighteenth International Conference on Machine Learning,Williamstown,MA,USA.San Francisco,CA,USA:Morgan Kaufmann Publishers,2001:282-289.
[4]Boykov Y Y,Jolly M P.Interactive graph cuts for optimal boundary and region segmentation of objects in images[C]//Proceeding of the Eighth IEEE International Conference on Computer Vision,Vancouver,BC,Canada,2001:105-112.
[5]Grady L.Random walks for image segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2006,28(11):1768-1783.
[6]Noma A,Graciano A B V,Jr R M C,et al.Interactive image segmentation by matching attributed relational graphs[J].Pattern Recognition,2012,45(3):1159-1179.
[7]Hwang H,Haddad R A.Adaptive median filters:new algorithm and results[J].IEEE Transactions on Image Processing,1995,4(4):499-502.
[8]文华.基于数学形态学的图像处理算法的研究[D].哈尔滨:哈尔滨工程大学,2007.
[9]Boykov Y,Funka-Lea G.Graph cuts and efficient image segmentation[J].International Journal of Computer Vision,2006,70(2):109-131.
[10]Yuan J,Bae E,Tai X C,et al.A continuous max,low approach to Potts model[C]//11th European Conference on Computer Vision,Heraklion,Crete,Greece.Springer,2010:379-392.
[11]Boykov Y,Veksler O,Zabih R.Fast approximate energy minimization via graph cuts[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2001,23(11):1222-1239.
[12]Li Y,Sun J,Tang C K,et al.Lazy snapping[J].ACM Transactions on Graphics,2004,23(3):303-308.
[13]Chum O,Matas J.Randomized RANSAC with T(d,d)test[C]//Proceedings of the 13th British Machine Vision Conference,Cardiff,UK,2002:448-457.
[14]Rother C,Kolmogorov V,Blake A.“GrabCut”:Interactive foreground extraction using iterated graph cuts[J].ACM Transactions on Graphics,2004,23(3):309-314.
[15]Friedland G,Jantz K,Rojas R.SIOXl simple interactive object extraction in still images[C]//Proceedings of the2005 IEEE International Symposium on Multimedia,Irvine,CA,USA,2005:253-260.