大视场多模型图像拼接技术研究
|
摘要
本文主要研究基于多模型整合的大视场图像拼接技术。
大视场图像投影拼接首先要选择合理的模型。一般常用的模型是多项式模型和投影模型。多项式模型可以线性拟合求解,计算相对简单,但它需要大量分布均匀的同名点对才能获得比较理想的效果。投影模型最大的优点是符合相机成像机理,且只需要四个精确的同名点对就能解出模型系数。因此,本文选用投影模型,推导出了多投影模型整合求解方法。由于投影模型是非线性的,计算上比较困难,这是本文重点关注的问题。
虽然投影模型只需四个同名点对,但对其精度要求很高。为此,本文对目前性能较好的SIFT算法进行了全面分析,以确定符合求解投影模型系数需要的关键算法参数,如尺度空间采枓频率等。
大视场多幅图像投影拼接一般采用局部对齐拼接技术,其致命缺陷是积累误差大。为解决该问题,本文提出了基于投影模型的大视场多模型整合拼接方法。该方法的主要思想是,将多幅图像各自的投影模型系数统一求解,以整体校正误差最小为求解的目标,利用相机外参数变化趋势与模型系数关系,对系数解迭代校正。实验表明,该方法拼接积累误差小,效男好。
This paper mostly makes a study on image mosaic technique in large view scene based on multiple model combination.
The image projection mosaic in large view scene firstly must choose an appropriate model. Multinomial model and projection model are the prevalent models in common. For the multinomial, it is easy to calculate and can get the model coefficients with linearity fitting. However, a mass of feature points with well-proportioned distributing must be extracted from images in order to achieve perfect coefficients. For the projection model, the most excellent quality of projection model is that it accords with the theory of camera imaging. Moreover, we can obtain model coefficients in need of only four accurate feature matching points. Based on such excellence, in this paper, we raise a method of multiple projection models combination to get coefficients. On account of the nonlinearity of projection model, it makes computing become more complicated, which also becomes emphasis issue of this paper. For example, sampling in scale.
Although projection model requires only four feature matching points, it has a strict demand on precision of matching points. Consequently, this paper makes a general analysis about the SIFT operator which has the excellent capability, and obtains key SIFT operator parameters which satisfy the need of getting model coefficients, such as frequency of sampling in scale.
Local alignment has been commonly applied in more images mosaic of large view scene, whereas it easily brings a lot of accumulating error in the integrated mosaic image. For the sake of solving such problem, based on projection model, this paper presents a new technique of image mosaic of multiple model combination in large view scene. The technique firstly obtains model coefficients of every model all together, and the object is to get the minimum total error of all models. Secondly, all coefficients will be corrected by iterative algorithm, which makes using of the relation between outer parameters of camera and model coefficients. The results of experiment prove that such algorithm is efficient and brings less accumulating error.
大视场图像投影拼接首先要选择合理的模型。一般常用的模型是多项式模型和投影模型。多项式模型可以线性拟合求解,计算相对简单,但它需要大量分布均匀的同名点对才能获得比较理想的效果。投影模型最大的优点是符合相机成像机理,且只需要四个精确的同名点对就能解出模型系数。因此,本文选用投影模型,推导出了多投影模型整合求解方法。由于投影模型是非线性的,计算上比较困难,这是本文重点关注的问题。
虽然投影模型只需四个同名点对,但对其精度要求很高。为此,本文对目前性能较好的SIFT算法进行了全面分析,以确定符合求解投影模型系数需要的关键算法参数,如尺度空间采枓频率等。
大视场多幅图像投影拼接一般采用局部对齐拼接技术,其致命缺陷是积累误差大。为解决该问题,本文提出了基于投影模型的大视场多模型整合拼接方法。该方法的主要思想是,将多幅图像各自的投影模型系数统一求解,以整体校正误差最小为求解的目标,利用相机外参数变化趋势与模型系数关系,对系数解迭代校正。实验表明,该方法拼接积累误差小,效男好。
This paper mostly makes a study on image mosaic technique in large view scene based on multiple model combination.
The image projection mosaic in large view scene firstly must choose an appropriate model. Multinomial model and projection model are the prevalent models in common. For the multinomial, it is easy to calculate and can get the model coefficients with linearity fitting. However, a mass of feature points with well-proportioned distributing must be extracted from images in order to achieve perfect coefficients. For the projection model, the most excellent quality of projection model is that it accords with the theory of camera imaging. Moreover, we can obtain model coefficients in need of only four accurate feature matching points. Based on such excellence, in this paper, we raise a method of multiple projection models combination to get coefficients. On account of the nonlinearity of projection model, it makes computing become more complicated, which also becomes emphasis issue of this paper. For example, sampling in scale.
Although projection model requires only four feature matching points, it has a strict demand on precision of matching points. Consequently, this paper makes a general analysis about the SIFT operator which has the excellent capability, and obtains key SIFT operator parameters which satisfy the need of getting model coefficients, such as frequency of sampling in scale.
Local alignment has been commonly applied in more images mosaic of large view scene, whereas it easily brings a lot of accumulating error in the integrated mosaic image. For the sake of solving such problem, based on projection model, this paper presents a new technique of image mosaic of multiple model combination in large view scene. The technique firstly obtains model coefficients of every model all together, and the object is to get the minimum total error of all models. Secondly, all coefficients will be corrected by iterative algorithm, which makes using of the relation between outer parameters of camera and model coefficients. The results of experiment prove that such algorithm is efficient and brings less accumulating error.
引文
[1]王东峰,多模态和大型图像配准技术研究,中国科学院电子学研究所博士学位论文,2002年5月.
[2]BarbaraZitova,Jan Flusse.Image registration methods:a survey.Image and Vision Computing,2003,23(7):935-942.
[3]谢凯,郭恒业,张田文,图像Mosaic技术综述。电子学报,2004,32(4).
[4]Brown LG.A survey of image registration technoLOGy.ACMComputing Surveys,1992,24(4):325-376.
[5]B Srinivasa,et al.An fft-based technique for translation,rotation,and scale-Invariant image registration.IEEE Trans On Image Processiong,1996,5(8):237-252.
[6]蔡勇等,基于图像绘制的虚拟现实系统环境.软件学报,1997,8(10):721-728.
[7]Sawhney S H.The true multi-image aligment and hisapplication to mosaicing and lens distortion correction.IEEE Trans on PAMI,1999,21(3):235-243.
[8]Davis T.Mosaics of scenes with moving objects.Proc of IEEE Computer Society Conference on Computer Vision and PatternRecognition.SantaBarbara,1998.353-360.
[9]Shum H Y,Szeliski R.Systems and experiment paper:construction of panoramic image mosasics with global and local alignment.I.J of Computer Vision,2000,36(2):101-130.
[10]Savheney HS,Kunnar Retal.Video brush:experiences with consumer video mosaicing.[A].IEEE 4~(th) Workshop on Application of Computer Vision.Sarasota Florida,1998.418-423.
[11]孙即祥等,模式识别中的特征提取与计算机视觉不变量。长沙:国防工业出版社,2001.
[12]陈鹰.遥感影像的数字摄影测量。同济大学出版社,2003年6月.
[13]马颂德、张正友,计算机视觉一计算理论与算法基础,科学出版社,1998.
[14]吴立德.计算机视觉.上海:复旦大学出版社,1993.
[15]胡金辉,胡占义.同心拼图中深度的计算.计算机学报.2001,vol.24,no.6:580-587.
[16]陈义.数字摄影测量共线方程的一种新解法.同济大学学报.2003.vo32,no.5:66-71.
[17]Rafael C.Gonzalez.Richard E.Woods.Digital Image Processing Second Edition.电子工业出版社.
[18]张亮,秦永元.基于最大熵算法的图像几何与辐射校正研究.弹箭与制导学报.
[19]田金文,柳斌,程伟.基于局部最大熵的辐射校正方法.中国图像图报,1999,(12).
[20]孟晋宇,舒华忠.基于形状的二维灰度图像插值.2003.Vol 8,no.3:312-316.
[21]Shu H Z,Luo L M,Zhou J Detal.Moment-based me thuds for polygonal approximation of digitized curves.PatternRecognition,2002,35(3):421-1434.
[22]王德人.非线性方程组解法与最优化方法.人民教育出版社,1979年.
[23]李庆扬,莫孜中.非线性方程组的数值解法.科学出版社,1992年.
[24]奥特加(Ortega J.M.),莱恩博尔特(Rheinboldt W.C.)著.多元非线性方程组迭代数法.科学出版社.1983.
[25]栗塔山,彭维杰,周作益.最优化计算原理与算法程序设计.国防科技大学出版社.2001.
[26]D.Lowe.Distinctive image features from scale-invariant keypoints.In Journal of Computer Vision,60(2):91-100.
[26]D.Lowe.Distinctive image features from scale-invariant keypoints.In Journal of Computer Vision,60(2):91-100.
[27]T.Lindeberg,"Feature detection with automatic scale selection,IJCV,30(2):79-116,1998.
[28]Y.Ke and R.Sukthankar.PCA-SIFT:A more distinctive representation for local image descriptors.In IEEE Conf.on Computer Vision and Pattern Recognition,2004.
[29]孙剑,许宗本.计算机视觉中的尺度空间方法.工程数学报.2005.Vol.22,no.6:952-960
[30]陈涛,图像仿射不变特征提取方法研究,国防科技大学博士学位论文,2006年10月.
[31]孙即椎等,现代模式识别。长沙:国防科技大学出版社,2002.
[32]Mokhtarian F,Suomela R.Robust image comer detection through curvature scale space.IEEE Transactions on Pattern Analysis and Machine Intelligence,1998,20(12):1376-1381.
[33]K.Nikolajczyk and C.Schmid.A performance evaluation of local dscriptors.In Proceedings of Computer Vision and Pattern Recognition,June 2003.
[34]S.Belongie,J.Malik and J.Puzicha," Shape context:A new descriptor for shape matching and object recognition," In NIPS,pp.831-847,2000.
[35]D.G.Lowe,"Object recognition from local scale invariant features," in ICCV,pp.682-688,1999.
[36]R.Fergus,P.Perona,and A.Zisserman.Object class recognition by unsupervised scale-invariant learning.In Proceedings of Computer Vision and Pattern Recognition,June 2003.
[37]F.Schaffalitzky and A.Zisserman.Multi-view matching for unordered image sets.In Proceedings of European Conference on Computer Vision,volumel,pages414-431.Springer-Verlag,2002.
[38]L.VanGool,T.Moons,and D.Ungureanu.Affine/photometric invariants for planar intensity patterns.In Proceedings of European Conference on Computer Vision,1996.
[39]Mokhtarian F,Suomela R.Robust image corner detection through curvature scale space.IEEE Transactions on Pattern Analysis and Machine Intelligence,1998,20(12):1376-1381.
[40] Lindeberg T. Scale-space theory: A basic tool for analysing structures at different scales. Journal Applied Statistics, 1994, 21(2): 223—261.
[41] Hansen M, Anadan P, Dana K et al. Real-time Scene stabilization and mosaic construction. IEEE Computer Vision &Pattera Recognition, Monterrey, 1993.
[42] M.Hanke, Iterative Regularization Techniques in Image Reconstruction, in Proceedings of the Conference Mathematical Methods in Inverse Problems for Partial Differential Equations, M t. Holyoke: S pringer-Verlag,1998.
[43] R .J.Hanson, Fred T.Krogh, A Quadratic-Tensor Model Algorithm for Nonlinear Least-Squares Problems with Linear Constraints, ACM Transaction on Mathematical Software, Vol. 18 ,N o-2,115-133.
[44] H. B. Nielsen, Separable NonLinear Least Squares. Report IMM-REP-00-01,Department of Mathematical Modelling, DTU(2000),16 pages.
[45] Teunissen P J. G. Nonliear least squares Manuscripta Geodaetica. 1990,15:137-150
[46] Szeliski R. Image mosaicing for tele-reality applications. IEEE Computer Graphics and Applications, 1994,14(6):43-53.
[47] Sawhney HS, Ayer S, Gorkani M. Model-based 2D&3D dominant motion estimation for mosaicing and video representation. In Fifth International Conferenceon Computer Vision (ICCV'95), MIT, Cambridge, MA, IEEE Computer Society Press, 1995:583-590.
[48] Szeliski R. Video mosaics for virtual environments. IEEE Computer Graphics and Applications, 1996,16(2):22-30.
[2]BarbaraZitova,Jan Flusse.Image registration methods:a survey.Image and Vision Computing,2003,23(7):935-942.
[3]谢凯,郭恒业,张田文,图像Mosaic技术综述。电子学报,2004,32(4).
[4]Brown LG.A survey of image registration technoLOGy.ACMComputing Surveys,1992,24(4):325-376.
[5]B Srinivasa,et al.An fft-based technique for translation,rotation,and scale-Invariant image registration.IEEE Trans On Image Processiong,1996,5(8):237-252.
[6]蔡勇等,基于图像绘制的虚拟现实系统环境.软件学报,1997,8(10):721-728.
[7]Sawhney S H.The true multi-image aligment and hisapplication to mosaicing and lens distortion correction.IEEE Trans on PAMI,1999,21(3):235-243.
[8]Davis T.Mosaics of scenes with moving objects.Proc of IEEE Computer Society Conference on Computer Vision and PatternRecognition.SantaBarbara,1998.353-360.
[9]Shum H Y,Szeliski R.Systems and experiment paper:construction of panoramic image mosasics with global and local alignment.I.J of Computer Vision,2000,36(2):101-130.
[10]Savheney HS,Kunnar Retal.Video brush:experiences with consumer video mosaicing.[A].IEEE 4~(th) Workshop on Application of Computer Vision.Sarasota Florida,1998.418-423.
[11]孙即祥等,模式识别中的特征提取与计算机视觉不变量。长沙:国防工业出版社,2001.
[12]陈鹰.遥感影像的数字摄影测量。同济大学出版社,2003年6月.
[13]马颂德、张正友,计算机视觉一计算理论与算法基础,科学出版社,1998.
[14]吴立德.计算机视觉.上海:复旦大学出版社,1993.
[15]胡金辉,胡占义.同心拼图中深度的计算.计算机学报.2001,vol.24,no.6:580-587.
[16]陈义.数字摄影测量共线方程的一种新解法.同济大学学报.2003.vo32,no.5:66-71.
[17]Rafael C.Gonzalez.Richard E.Woods.Digital Image Processing Second Edition.电子工业出版社.
[18]张亮,秦永元.基于最大熵算法的图像几何与辐射校正研究.弹箭与制导学报.
[19]田金文,柳斌,程伟.基于局部最大熵的辐射校正方法.中国图像图报,1999,(12).
[20]孟晋宇,舒华忠.基于形状的二维灰度图像插值.2003.Vol 8,no.3:312-316.
[21]Shu H Z,Luo L M,Zhou J Detal.Moment-based me thuds for polygonal approximation of digitized curves.PatternRecognition,2002,35(3):421-1434.
[22]王德人.非线性方程组解法与最优化方法.人民教育出版社,1979年.
[23]李庆扬,莫孜中.非线性方程组的数值解法.科学出版社,1992年.
[24]奥特加(Ortega J.M.),莱恩博尔特(Rheinboldt W.C.)著.多元非线性方程组迭代数法.科学出版社.1983.
[25]栗塔山,彭维杰,周作益.最优化计算原理与算法程序设计.国防科技大学出版社.2001.
[26]D.Lowe.Distinctive image features from scale-invariant keypoints.In Journal of Computer Vision,60(2):91-100.
[26]D.Lowe.Distinctive image features from scale-invariant keypoints.In Journal of Computer Vision,60(2):91-100.
[27]T.Lindeberg,"Feature detection with automatic scale selection,IJCV,30(2):79-116,1998.
[28]Y.Ke and R.Sukthankar.PCA-SIFT:A more distinctive representation for local image descriptors.In IEEE Conf.on Computer Vision and Pattern Recognition,2004.
[29]孙剑,许宗本.计算机视觉中的尺度空间方法.工程数学报.2005.Vol.22,no.6:952-960
[30]陈涛,图像仿射不变特征提取方法研究,国防科技大学博士学位论文,2006年10月.
[31]孙即椎等,现代模式识别。长沙:国防科技大学出版社,2002.
[32]Mokhtarian F,Suomela R.Robust image comer detection through curvature scale space.IEEE Transactions on Pattern Analysis and Machine Intelligence,1998,20(12):1376-1381.
[33]K.Nikolajczyk and C.Schmid.A performance evaluation of local dscriptors.In Proceedings of Computer Vision and Pattern Recognition,June 2003.
[34]S.Belongie,J.Malik and J.Puzicha," Shape context:A new descriptor for shape matching and object recognition," In NIPS,pp.831-847,2000.
[35]D.G.Lowe,"Object recognition from local scale invariant features," in ICCV,pp.682-688,1999.
[36]R.Fergus,P.Perona,and A.Zisserman.Object class recognition by unsupervised scale-invariant learning.In Proceedings of Computer Vision and Pattern Recognition,June 2003.
[37]F.Schaffalitzky and A.Zisserman.Multi-view matching for unordered image sets.In Proceedings of European Conference on Computer Vision,volumel,pages414-431.Springer-Verlag,2002.
[38]L.VanGool,T.Moons,and D.Ungureanu.Affine/photometric invariants for planar intensity patterns.In Proceedings of European Conference on Computer Vision,1996.
[39]Mokhtarian F,Suomela R.Robust image corner detection through curvature scale space.IEEE Transactions on Pattern Analysis and Machine Intelligence,1998,20(12):1376-1381.
[40] Lindeberg T. Scale-space theory: A basic tool for analysing structures at different scales. Journal Applied Statistics, 1994, 21(2): 223—261.
[41] Hansen M, Anadan P, Dana K et al. Real-time Scene stabilization and mosaic construction. IEEE Computer Vision &Pattera Recognition, Monterrey, 1993.
[42] M.Hanke, Iterative Regularization Techniques in Image Reconstruction, in Proceedings of the Conference Mathematical Methods in Inverse Problems for Partial Differential Equations, M t. Holyoke: S pringer-Verlag,1998.
[43] R .J.Hanson, Fred T.Krogh, A Quadratic-Tensor Model Algorithm for Nonlinear Least-Squares Problems with Linear Constraints, ACM Transaction on Mathematical Software, Vol. 18 ,N o-2,115-133.
[44] H. B. Nielsen, Separable NonLinear Least Squares. Report IMM-REP-00-01,Department of Mathematical Modelling, DTU(2000),16 pages.
[45] Teunissen P J. G. Nonliear least squares Manuscripta Geodaetica. 1990,15:137-150
[46] Szeliski R. Image mosaicing for tele-reality applications. IEEE Computer Graphics and Applications, 1994,14(6):43-53.
[47] Sawhney HS, Ayer S, Gorkani M. Model-based 2D&3D dominant motion estimation for mosaicing and video representation. In Fifth International Conferenceon Computer Vision (ICCV'95), MIT, Cambridge, MA, IEEE Computer Society Press, 1995:583-590.
[48] Szeliski R. Video mosaics for virtual environments. IEEE Computer Graphics and Applications, 1996,16(2):22-30.