面向双目立体视觉的迭代式局部颜色校正
|
摘要
在双目立体视觉中,由于相机参数设置差异、环境光照变化、拍摄物体表面非理想漫反射等因素,拍摄获得的立体图像对可能存在颜色差异,进而降低视差计算的准确性.针对这一问题,提出一种面向双目立体视觉的迭代式局部颜色校正方法.首先使用Meanshift算法以不同粒度对2幅图像进行分割,并基于SIFT特征匹配、区域分布及颜色差异初步建立2幅图像中物体间的对应关系;然后使用加权局部颜色校正方法,逐区域进行颜色校正,由于双目图像中物体遮挡范围不同,初步的区域对应存在误差,因此利用校正的双目图像计算视差图,基于视差图像对之间稠密的像素对应,优化立体图像对的区域分割,建立更准确的对应关系,并再次进行颜色校正;迭代地进行立体匹配、优化图像区域对应和颜色校正,直至获得最佳的立体匹配结果.与已有颜色校正方法对基准测试图像集的处理结果表明,文中方法可以有效地提升立体图像对的颜色相似度,提高立体匹配视差结果的准确性.
Due to difference of camera parameters, variance of environmental illumination, non-diffuse reflection of object surface, there is always the color discrepancy between stereoscopic image pair in stereo matching, which will decrease the accuracy of disparity computation. To address this problem, an iterative local color correction method is proposed in this paper. First, the Mean shift algorithm is adopted to segment the stereoscopic images with different granularities respectively. Meanwhile, the SIFT features are extracted and used for local region correspondence initially based on object distributions in the images. Then the target image is corrected by using a weighted local color correction function. Because of different view angles of objects in the images, occlusions will occur and cause the initial region correspondence inaccurate. Thus, a stereo matching algorithm is adopted to generate the disparity maps. The region correspondence is then refined based on the dense feature correspondence between the disparity maps, and the weighted local color correction is performed again. The above refinement will be iteratively performed till the disparity result is convergent. Comparing with several color transfer methods on the benchmark image set, the proposed method can improve the color similarity between stereoscopic image pair and improve the accuracy of stereo matching effectively.
Due to difference of camera parameters, variance of environmental illumination, non-diffuse reflection of object surface, there is always the color discrepancy between stereoscopic image pair in stereo matching, which will decrease the accuracy of disparity computation. To address this problem, an iterative local color correction method is proposed in this paper. First, the Mean shift algorithm is adopted to segment the stereoscopic images with different granularities respectively. Meanwhile, the SIFT features are extracted and used for local region correspondence initially based on object distributions in the images. Then the target image is corrected by using a weighted local color correction function. Because of different view angles of objects in the images, occlusions will occur and cause the initial region correspondence inaccurate. Thus, a stereo matching algorithm is adopted to generate the disparity maps. The region correspondence is then refined based on the dense feature correspondence between the disparity maps, and the weighted local color correction is performed again. The above refinement will be iteratively performed till the disparity result is convergent. Comparing with several color transfer methods on the benchmark image set, the proposed method can improve the color similarity between stereoscopic image pair and improve the accuracy of stereo matching effectively.
引文
[1]Scharstein D,Szeliski R,Zabih R.A taxonomy and evaluation of dense two-frame stereo correspondence algorithms[C]//Proceedings of the IEEE Workshop on Stereo and Multi-Baseline Vision.Los Alamitos:IEEE Computer Society Press,2001:131-140
[2]Zabih R,Woodfill J.Non-parametric local transforms for computing visual correspondence[C]//Proceedings of the European Conference on Computer Vision.Heidelberg:Springer,1994:151-158
[3]Hirschmuller H.Accurate and efficient stereo processing by semi-global matching and mutual information[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision&Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2005:807-814
[4]Hirschmuller H,Scharstein D.Evaluation of cost functions for stereo matching[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEEComputer Society Press,2007:1-8
[5]Xu W,Mulligan J.Performance evaluation of color correction approaches for automatic multi-view image and video stitching[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2010:263-270
[6]Yamamoto K,Oi R.Color correction for multi-view video using energy minimization of view networks[J].International Journal of Automation and Computing,2008,5(3):234-245
[7]Jia J Y,Tang C K.Image registration with global and local luminance alignment[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2003:156-163
[8]Mouffranc C,Nozick V.Colorimetric correction for stereoscopic camera arrays[C]//Proceedings of the Asian Conference on Computer Vision.Heidelberg:Springer,2012,7728:206-217
[9]Gui Y T,Gledhill D,Taylor D,et al.Colour correction for panoramic imaging[C]//Proceedings of the 6th International Conference on Information Visualization.London:DBLP Press,2002:483-488
[10]Zhang M J,Georganas N D.Fast color correction using principal regions mapping in different color spaces[J].Real-Time Imaging,2004,10(1):23-30
[11]Ceulemans B,Lu S P,Schelkens P,et al.Globally optimized multiview video color correction using dense spatio-temporal matching[C]//Proceedings of 3DTV-Conference:the True Vision-Capture,Transmission and Display of 3D Video.Los Alamitos:IEEE Computer Society Press,2015:1-4
[12]Tai Y W,Jia J Y,Tang C K.Local color transfer via probabilistic segmentation by expectation-maximization[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2005:747-754
[13]Wang Q,Sun X,Wang Z F.A robust algorithm for color correction between two stereo images[C]//Proceedings of the Asian Conference on Computer Vision.Heidelberg:Springer,2009,5995:405-416
[14]Wang Q,Yan P K,Yuan Y,et al.Robust color correction in stereo vision[C]//Proceedings of the 18th IEEE International Conference on Image Processing.Los Alamitos:IEEE Computer Society Press,2011:965-968
[15]Ran Q,Zhao W J,Feng J Q.Robust region-wise colour correction method for stereo matching[J].IET Computer Vision,2016,10(7):641-649
[16]Lee Y H,Lee I K.Colour correction of stereo images using local correspondence[J].Electronic Letters,2014,50(16):1136-1138
[17]Zheng X H,Niu Y Z,Chen J H,et al.Color correction for stereoscopic image based on matching and optimization[C]//Proceedings of International Conference on 3D Immersion.Los Alamitos:IEEE Computer Society Press,2017:1-8
[18]Lowe D G.Distinctive image features from scale-invariant keypoints[J].International Journal Computer Vision,2004,60(2):91-110
[19]Hartley R,Zisserman A.Multiple view geometry in computer vision[M].New York:Cambridge University Press,2003
[20]Reinhard E,Adhikhmin M,Gooch B,et al.Color transfer between images[J].IEEE Computer Graphics and Applications,2001,21(5):34-41
[21]Oliveira M,Sappa A D,Santos V.Unsupervised local color correction for coarsely registered images[C]//Proceedings of Computer Vision and Pattern Recognition.Los Alamitos:IEEEComputer Society Press,2011:201-208
[22]Scharstein D,Szeliski R,Hirschmuller H.Middlebury stereo vision page[OL].[2018-06-25].http://vision.middlebury.edu/stereo/
[23]Kolmogorov V,Zabih R.Computing visual correspondence with occlusions using graph cuts[C]//Proceedings of the 8th IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2002:508-515
[2]Zabih R,Woodfill J.Non-parametric local transforms for computing visual correspondence[C]//Proceedings of the European Conference on Computer Vision.Heidelberg:Springer,1994:151-158
[3]Hirschmuller H.Accurate and efficient stereo processing by semi-global matching and mutual information[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision&Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2005:807-814
[4]Hirschmuller H,Scharstein D.Evaluation of cost functions for stereo matching[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEEComputer Society Press,2007:1-8
[5]Xu W,Mulligan J.Performance evaluation of color correction approaches for automatic multi-view image and video stitching[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2010:263-270
[6]Yamamoto K,Oi R.Color correction for multi-view video using energy minimization of view networks[J].International Journal of Automation and Computing,2008,5(3):234-245
[7]Jia J Y,Tang C K.Image registration with global and local luminance alignment[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2003:156-163
[8]Mouffranc C,Nozick V.Colorimetric correction for stereoscopic camera arrays[C]//Proceedings of the Asian Conference on Computer Vision.Heidelberg:Springer,2012,7728:206-217
[9]Gui Y T,Gledhill D,Taylor D,et al.Colour correction for panoramic imaging[C]//Proceedings of the 6th International Conference on Information Visualization.London:DBLP Press,2002:483-488
[10]Zhang M J,Georganas N D.Fast color correction using principal regions mapping in different color spaces[J].Real-Time Imaging,2004,10(1):23-30
[11]Ceulemans B,Lu S P,Schelkens P,et al.Globally optimized multiview video color correction using dense spatio-temporal matching[C]//Proceedings of 3DTV-Conference:the True Vision-Capture,Transmission and Display of 3D Video.Los Alamitos:IEEE Computer Society Press,2015:1-4
[12]Tai Y W,Jia J Y,Tang C K.Local color transfer via probabilistic segmentation by expectation-maximization[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Los Alamitos:IEEE Computer Society Press,2005:747-754
[13]Wang Q,Sun X,Wang Z F.A robust algorithm for color correction between two stereo images[C]//Proceedings of the Asian Conference on Computer Vision.Heidelberg:Springer,2009,5995:405-416
[14]Wang Q,Yan P K,Yuan Y,et al.Robust color correction in stereo vision[C]//Proceedings of the 18th IEEE International Conference on Image Processing.Los Alamitos:IEEE Computer Society Press,2011:965-968
[15]Ran Q,Zhao W J,Feng J Q.Robust region-wise colour correction method for stereo matching[J].IET Computer Vision,2016,10(7):641-649
[16]Lee Y H,Lee I K.Colour correction of stereo images using local correspondence[J].Electronic Letters,2014,50(16):1136-1138
[17]Zheng X H,Niu Y Z,Chen J H,et al.Color correction for stereoscopic image based on matching and optimization[C]//Proceedings of International Conference on 3D Immersion.Los Alamitos:IEEE Computer Society Press,2017:1-8
[18]Lowe D G.Distinctive image features from scale-invariant keypoints[J].International Journal Computer Vision,2004,60(2):91-110
[19]Hartley R,Zisserman A.Multiple view geometry in computer vision[M].New York:Cambridge University Press,2003
[20]Reinhard E,Adhikhmin M,Gooch B,et al.Color transfer between images[J].IEEE Computer Graphics and Applications,2001,21(5):34-41
[21]Oliveira M,Sappa A D,Santos V.Unsupervised local color correction for coarsely registered images[C]//Proceedings of Computer Vision and Pattern Recognition.Los Alamitos:IEEEComputer Society Press,2011:201-208
[22]Scharstein D,Szeliski R,Hirschmuller H.Middlebury stereo vision page[OL].[2018-06-25].http://vision.middlebury.edu/stereo/
[23]Kolmogorov V,Zabih R.Computing visual correspondence with occlusions using graph cuts[C]//Proceedings of the 8th IEEE International Conference on Computer Vision.Los Alamitos:IEEE Computer Society Press,2002:508-515