基于空间平面分割和投影变换的光场图像拼接算法
|
摘要
提出了一种基于空间平面分割和投影变换的光场图像拼接算法。依据光场深度信息对目标场景进行平面拟合和分割,并根据颜色信息利用马尔可夫模型对分割结果进行了优化。利用不同视点间的光流信息求解投影变换矩阵,将对应的分割区域投影变换到同一视点。通过计算最小缝合线拼接重叠区域,并采用泊松优化的方法融合拼接区域。实验结果表明,所提算法能够生成大视野的光场图像,且光场中光线对齐准确,无明显几何畸变。
A method for stitching light-field images is proposed based on spatial plane segmentation and projective transformation.The spatial plane fitting and segmentation are performed on the target scene according to the depth map,and the initial segmentation result is refined using a Markov random framework model according to the color information.The corresponding segmentation regions are projected to the same viewpoint using the projective transformation matrix,which is calculated using the optical-flow information among different viewpoints.Finally,a minimum energy seam is calculated to stitch together the corresponding overlapped regions,and a Poisson optimization technique is utilized to blend the stitched area.The experimental results show that the proposed method can generate large field of view light-field images without any ray misalignment or noticeable geometric distortion.
A method for stitching light-field images is proposed based on spatial plane segmentation and projective transformation.The spatial plane fitting and segmentation are performed on the target scene according to the depth map,and the initial segmentation result is refined using a Markov random framework model according to the color information.The corresponding segmentation regions are projected to the same viewpoint using the projective transformation matrix,which is calculated using the optical-flow information among different viewpoints.Finally,a minimum energy seam is calculated to stitch together the corresponding overlapped regions,and a Poisson optimization technique is utilized to blend the stitched area.The experimental results show that the proposed method can generate large field of view light-field images without any ray misalignment or noticeable geometric distortion.
引文
[1] Zhao T,Kang H L,Zhang Z P.Fast image mosaic algorithm based on area blocking and BRISK[J].Laser&Optoelectronics Progress,2018,55(3):031005.赵婷,康海林,张正平.结合区域分块的快速BRISK图像拼接算法[J].激光与光电子学进展,2018,55(3):031005.
[2] Cheng Y F,Jin S Y,Wang M,et al.An high accuracy image mosaicking approach of optical remote sensing satellite for multi-camera system[J].Acta Optica Sinica,2017,37(8):0828003.程宇峰,金淑英,王密,等.一种光学遥感卫星多相机成像系统的高精度影像拼接方法[J].光学学报,2017,37(8):0828003.
[3] Brown M,Lowe D G.Automaticpanoramic image stitching using invariant features[J].International Journal of Computer Vision,2007,74(1):59-73.
[4] Yang S,Xia M H,Zheng Z S. Medical image registrationalgorithmbasedonpolynomial deterministic matrix and SIFT transform[J].Laser&Optoelectronics Progress,2016,53(8):081002.杨飒,夏明华,郑志硕.基于多项式确定性矩阵的SIFT医学图像配准算法[J].激光与光电子学进展,2016,53(8):081002.
[5] Yu Z J,Wang S B.Improved PCA-SIFT algorithm for matchingstereosystem[J]. Laser&Optoelectronics Progress,2016,53(3):031501.于之靖,王韶彬.改进PCA-SIFT算法的立体匹配系统[J].激光与光电子学进展,2016,53(3):031501.
[6] Gao J H,Kim S J,Brown M S.Constructing image panoramas using dual-homography warping[C]∥2011Computer Vision andPattern Recognition(CVPR),June 20-25,2011,Colorado Springs,CO,USA.New York:IEEE,2011:49-56.
[7] Zaragoza J,Chin T J,Brown M S,et al.Asprojective-as-possible image stitching with moving DLT[C]∥2013 IEEE Conference on Computer Vision and Pattern Recognition,June 23-28,2013,Portland,OR, USA. New York:IEEE,2013:2339-2346.
[8] Chang C H,Sato Y,Chuang Y.Shape-preserving half-projective warps for image stitching[C]∥2014IEEE Conference on Computer Vision and Pattern Recognition,June 23-28,2014,Columbus, OH,USA.New York:IEEE,2014:3254-3261.
[9] Peleg S, Ben-Ezra M, Pritch Y. Omnistereo:Panoramic stereo imaging[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2001,23(3):279-290.
[10] Shum H Y,He L W.Rendering with concentric mosaics[C]∥Proceedings of the 26th annual conference on Computer graphics and interactive techniques,Los Angeles,August 8-13,1999,CA,USA.New York:ACM Press,1999:299-306.
[11] Richardt C, Pritch Y, Zimmer H,et al.Megastereo:constructinghigh-resolutionstereo panoramas[C]∥2013IEEE Conference on Computer Vision and Pattern Recognition,June 23-28,2013,Portland,OR, USA. New York:IEEE,2013:1256-1263.
[12] Birklbauer C, Bimber O. Panorama light-field imaging[J].Computer Graphics Forum,2014,33(2):43-52.
[13] Guo X Q,Yu Z,Kang S B,et al.Enhancing light fields through ray-space stitching[J]. IEEE TransactionsonVisualizationandComputer Graphics,2016,22(7):1852-1861.
[14] Buyukalp Y,Cihan A F, Mutlu M.Panoramic reconstruction from multiple light-field images[EB/OL].[2018-10-28].http:∥stanford.edu/class/ee367/Winter2016/Buyukalp_Cihan_Mutlu_Report.pdf.
[15] Taniai T,Matsushita Y,Sato Y,et al.Continuous3Dlabel stereo matching using local expansion moves[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2018,40(11):2725-2739.
[16] Jeon H G,Park J,Choe G,et al.Accurate depth map estimation from a lenslet light field camera[C]∥2015 IEEE Conference on Computer Vision and Pattern Recognition(CVPR),June 7-12,2015,Boston,MA,USA.New York:IEEE,2015:1547-1555.
[17] Feng C, Taguchi Y, Kamat V R. Fast plane extractioninorganizedpointcloudsusing agglomerative hierarchical clustering[C]∥2014IEEE International Conference on Robotics and Automation(ICRA),May 31-June 7,2014,Hong Kong,China.New York:IEEE,2014:6218-6225.
[18] Boykov Y,Jolly M P.Interactive graph cuts for optimal boundary®ion segmentation of objects in N-D images[C]∥Proceedings Eighth IEEE International Conference on Computer Vision.ICCV2001,July 7-14,2001,Vancouver,BC,Canada.New York:IEEE,2001:105-112.
[19] Bao L C,Yang Q X,Jin H L.Fast edge-preserving PatchMatch for large displacement optical flow[C]∥2014 IEEE Conference on Computer Vision and Pattern Recognition,June 23-28,2014,Columbus,OH,USA.New York:IEEE,2014:3534-3541.
[20] Fang X Y,Pan Z G,Xu D.An improved algorithm for image mosaics[J].Journal of Computer-Aided Design&Computer Graphics,2003,15(11):1362-1365.方贤勇,潘志庚,徐丹.图像拼接的改进算法[J].计算机辅助设计与图形学学报,2003,15(11):1362-1365.
[21] Pérez P,Gangnet M,Blake A.Poisson image editing[J].ACM Transactions on Graphics,2003,22(3):313-318.
[22] Dansereau D G,Pizarro O,Williams S B.Decoding,calibrationandrectificationforlenselet-based plenoptic cameras[C]∥2013IEEE Conference on Computer Vision and Pattern Recognition,June 23-28,2013,Portland,OR,USA.New York:IEEE,2013:1027-1034.
[23] Zuo C,Sun J S,Feng S J,et al.Programmable aperture microscopy:a computational method for multi-modal phase contrast and light field imaging[J].Optics and Lasers in Engineering,2016,80:24-31.
[2] Cheng Y F,Jin S Y,Wang M,et al.An high accuracy image mosaicking approach of optical remote sensing satellite for multi-camera system[J].Acta Optica Sinica,2017,37(8):0828003.程宇峰,金淑英,王密,等.一种光学遥感卫星多相机成像系统的高精度影像拼接方法[J].光学学报,2017,37(8):0828003.
[3] Brown M,Lowe D G.Automaticpanoramic image stitching using invariant features[J].International Journal of Computer Vision,2007,74(1):59-73.
[4] Yang S,Xia M H,Zheng Z S. Medical image registrationalgorithmbasedonpolynomial deterministic matrix and SIFT transform[J].Laser&Optoelectronics Progress,2016,53(8):081002.杨飒,夏明华,郑志硕.基于多项式确定性矩阵的SIFT医学图像配准算法[J].激光与光电子学进展,2016,53(8):081002.
[5] Yu Z J,Wang S B.Improved PCA-SIFT algorithm for matchingstereosystem[J]. Laser&Optoelectronics Progress,2016,53(3):031501.于之靖,王韶彬.改进PCA-SIFT算法的立体匹配系统[J].激光与光电子学进展,2016,53(3):031501.
[6] Gao J H,Kim S J,Brown M S.Constructing image panoramas using dual-homography warping[C]∥2011Computer Vision andPattern Recognition(CVPR),June 20-25,2011,Colorado Springs,CO,USA.New York:IEEE,2011:49-56.
[7] Zaragoza J,Chin T J,Brown M S,et al.Asprojective-as-possible image stitching with moving DLT[C]∥2013 IEEE Conference on Computer Vision and Pattern Recognition,June 23-28,2013,Portland,OR, USA. New York:IEEE,2013:2339-2346.
[8] Chang C H,Sato Y,Chuang Y.Shape-preserving half-projective warps for image stitching[C]∥2014IEEE Conference on Computer Vision and Pattern Recognition,June 23-28,2014,Columbus, OH,USA.New York:IEEE,2014:3254-3261.
[9] Peleg S, Ben-Ezra M, Pritch Y. Omnistereo:Panoramic stereo imaging[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2001,23(3):279-290.
[10] Shum H Y,He L W.Rendering with concentric mosaics[C]∥Proceedings of the 26th annual conference on Computer graphics and interactive techniques,Los Angeles,August 8-13,1999,CA,USA.New York:ACM Press,1999:299-306.
[11] Richardt C, Pritch Y, Zimmer H,et al.Megastereo:constructinghigh-resolutionstereo panoramas[C]∥2013IEEE Conference on Computer Vision and Pattern Recognition,June 23-28,2013,Portland,OR, USA. New York:IEEE,2013:1256-1263.
[12] Birklbauer C, Bimber O. Panorama light-field imaging[J].Computer Graphics Forum,2014,33(2):43-52.
[13] Guo X Q,Yu Z,Kang S B,et al.Enhancing light fields through ray-space stitching[J]. IEEE TransactionsonVisualizationandComputer Graphics,2016,22(7):1852-1861.
[14] Buyukalp Y,Cihan A F, Mutlu M.Panoramic reconstruction from multiple light-field images[EB/OL].[2018-10-28].http:∥stanford.edu/class/ee367/Winter2016/Buyukalp_Cihan_Mutlu_Report.pdf.
[15] Taniai T,Matsushita Y,Sato Y,et al.Continuous3Dlabel stereo matching using local expansion moves[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2018,40(11):2725-2739.
[16] Jeon H G,Park J,Choe G,et al.Accurate depth map estimation from a lenslet light field camera[C]∥2015 IEEE Conference on Computer Vision and Pattern Recognition(CVPR),June 7-12,2015,Boston,MA,USA.New York:IEEE,2015:1547-1555.
[17] Feng C, Taguchi Y, Kamat V R. Fast plane extractioninorganizedpointcloudsusing agglomerative hierarchical clustering[C]∥2014IEEE International Conference on Robotics and Automation(ICRA),May 31-June 7,2014,Hong Kong,China.New York:IEEE,2014:6218-6225.
[18] Boykov Y,Jolly M P.Interactive graph cuts for optimal boundary®ion segmentation of objects in N-D images[C]∥Proceedings Eighth IEEE International Conference on Computer Vision.ICCV2001,July 7-14,2001,Vancouver,BC,Canada.New York:IEEE,2001:105-112.
[19] Bao L C,Yang Q X,Jin H L.Fast edge-preserving PatchMatch for large displacement optical flow[C]∥2014 IEEE Conference on Computer Vision and Pattern Recognition,June 23-28,2014,Columbus,OH,USA.New York:IEEE,2014:3534-3541.
[20] Fang X Y,Pan Z G,Xu D.An improved algorithm for image mosaics[J].Journal of Computer-Aided Design&Computer Graphics,2003,15(11):1362-1365.方贤勇,潘志庚,徐丹.图像拼接的改进算法[J].计算机辅助设计与图形学学报,2003,15(11):1362-1365.
[21] Pérez P,Gangnet M,Blake A.Poisson image editing[J].ACM Transactions on Graphics,2003,22(3):313-318.
[22] Dansereau D G,Pizarro O,Williams S B.Decoding,calibrationandrectificationforlenselet-based plenoptic cameras[C]∥2013IEEE Conference on Computer Vision and Pattern Recognition,June 23-28,2013,Portland,OR,USA.New York:IEEE,2013:1027-1034.
[23] Zuo C,Sun J S,Feng S J,et al.Programmable aperture microscopy:a computational method for multi-modal phase contrast and light field imaging[J].Optics and Lasers in Engineering,2016,80:24-31.