LAMOST光谱数据获取与预处理方法的改进
|
摘要
大天区面积多目标光纤光谱望远镜(简称LAMOST)是中国正在建造的一台大型光纤光谱天文望远镜,它的建成将使我国天文学在大规模光学光谱观测以及大视场天文学研究上,居于国际领先地位。
LAMOST一个观测夜将观测到上万条天体的光谱,属于海量数据,因此迫切需要一个全自动的观测运行和数据处理系统。依此为目的,本文介绍了LAMOST的观测和数据处理流程,并针对其中抽谱算法处理时间过长的缺点,对其进行了有效的改进,提高了计算效率。然后对LAMOST光谱预处理中的难点问题——光谱去噪和拟合连续谱做了详细的分析,具体做了以下几方面的工作:
1.提出了基于稀疏表示的小波去噪方法。该方法先对天体光谱进行小波变换得到不同尺度上的小波系数,通过对各个尺度上的小波系数进行稀疏化处理来去除噪声。其优点是在处理小波系数时虽然将其作为整体进行考虑,但依然能保持小波系数的局部特性不变,所以在去噪的同时很好地保持了特征谱线的信息。
2.提出了小波变换和其它方法相结合的连续谱拟合新方法,该方法在做小波变换的过程中,尽量将强谱线扣除掉,并且对所得到的连续谱进行多项式插值或样条拟合,使其尽量逼近真实的连续谱。
3.利用本文提出的去噪和拟合连续谱算法,对传统的减天光和谱线提取方法进行了合理有效的改进,实验结果表明了改进算法的优越性。
Large Sky Area Multi-Object Fiber Spectroscopy Telescope(LAMOST) is one of the National Major Scientific Projects undertaken by the Chinese Academy of Sciences. The set-up of it will make our country a leading role in the research field of large scale spectrum observation and large field astronomy.
LAMOST will obtain several ten-thousands of spectra per night. Therefore an automatically observational controlling and data processing system is in urgent need. This paper firstly introduces the process of observation and data processing as well as methods to improve the algorithm of spectrum flux extraction which solves the disadvantage of processing slowly, making the calculation more effective. Then the difficult issues—spectrum de-nosing and continuum fitting are analyzed in detail. The concrete works are described as follows:
1. A new wavelet de-noising scheme based on sparse representation is presented in this paper. This method removes noise by means of dealing with the wavelet coefficients of each scale based on sparse representation. The method not only takes the structure properties in the wavelet coefficients into consideration, but also can maintain the local characteristics of wavelet coefficients. Therefore it can effectively keep the information of featured spectral lines during the process of de-noising.
2. A combined method of wavelet transform and spine fitting is proposed for continuum fitting. This method removes strong spectral lines during the process of wavelet transform, trying to approximate it to real continuum spectrum.
3. The traditional methods for sky spectrum subtraction and spectral line extraction are improved reasonably and effectively by means of the presented algorithms for de-noising and continuum fitting. Experimental results show the superiority of our improved algorithms.
LAMOST一个观测夜将观测到上万条天体的光谱,属于海量数据,因此迫切需要一个全自动的观测运行和数据处理系统。依此为目的,本文介绍了LAMOST的观测和数据处理流程,并针对其中抽谱算法处理时间过长的缺点,对其进行了有效的改进,提高了计算效率。然后对LAMOST光谱预处理中的难点问题——光谱去噪和拟合连续谱做了详细的分析,具体做了以下几方面的工作:
1.提出了基于稀疏表示的小波去噪方法。该方法先对天体光谱进行小波变换得到不同尺度上的小波系数,通过对各个尺度上的小波系数进行稀疏化处理来去除噪声。其优点是在处理小波系数时虽然将其作为整体进行考虑,但依然能保持小波系数的局部特性不变,所以在去噪的同时很好地保持了特征谱线的信息。
2.提出了小波变换和其它方法相结合的连续谱拟合新方法,该方法在做小波变换的过程中,尽量将强谱线扣除掉,并且对所得到的连续谱进行多项式插值或样条拟合,使其尽量逼近真实的连续谱。
3.利用本文提出的去噪和拟合连续谱算法,对传统的减天光和谱线提取方法进行了合理有效的改进,实验结果表明了改进算法的优越性。
Large Sky Area Multi-Object Fiber Spectroscopy Telescope(LAMOST) is one of the National Major Scientific Projects undertaken by the Chinese Academy of Sciences. The set-up of it will make our country a leading role in the research field of large scale spectrum observation and large field astronomy.
LAMOST will obtain several ten-thousands of spectra per night. Therefore an automatically observational controlling and data processing system is in urgent need. This paper firstly introduces the process of observation and data processing as well as methods to improve the algorithm of spectrum flux extraction which solves the disadvantage of processing slowly, making the calculation more effective. Then the difficult issues—spectrum de-nosing and continuum fitting are analyzed in detail. The concrete works are described as follows:
1. A new wavelet de-noising scheme based on sparse representation is presented in this paper. This method removes noise by means of dealing with the wavelet coefficients of each scale based on sparse representation. The method not only takes the structure properties in the wavelet coefficients into consideration, but also can maintain the local characteristics of wavelet coefficients. Therefore it can effectively keep the information of featured spectral lines during the process of de-noising.
2. A combined method of wavelet transform and spine fitting is proposed for continuum fitting. This method removes strong spectral lines during the process of wavelet transform, trying to approximate it to real continuum spectrum.
3. The traditional methods for sky spectrum subtraction and spectral line extraction are improved reasonably and effectively by means of the presented algorithms for de-noising and continuum fitting. Experimental results show the superiority of our improved algorithms.
引文
[1]中国科学院。LAMOST项目计划建议书[R].1995
[2]LAMOST二维光纤数据模拟软件1.0版技术报告
[3]LAMOST二维光纤光谱处理程序1.0版技术报告
[4]LAMOST二维光纤光谱处理程序2.0版技术报告
[5]邱波.天体光谱数据自动处理和算法研究.中国科学院自动化研究所,2002
[6]贾冬梅.天体光谱信号的自动识别方法研究.中国科学院自动化研究所,2003
[7]Horne Keith,An optimal extraction algorithm for CCD spectroscopy,PASP,1986,98:609-617
[8]J.G.Robertson,Optimal extraction of single-object spectra from observations with two-dimensional detectors,PASP,1986,98:1220-1231
[9]M.Roth,A.Kelz,T.Fechner,et al.The Potsdam Multi-Aperture Spectrophotometer,PASP,2005,117:620-642
[10]A.Bershady,R.Andersen,et al.A Formatted Fiber Field Unit for the WIYN Telescope Bench Spectrograph,PASP,2004,116:565-590
[11]A.Kelz,A.Verheijen,et al.The Potsdam Multi-Aperture Spectrophotometer,PASP,2006,118:129-145
[12]S.M.Kent,Sloan Digital Sky Survey,Astrophysics and Space Science,1994,217:27-30
[13]F.Watson,A.Offer,I.Lewis,et al.Fiber sky-subtraction revisited,Fiber Optics in Astronomy Ⅲ of ASPC,152,1998,50-59
[14]D.York,J.Adelman,J.Anderson,et al.The Sloan Digital Sky Survey:Technical Summary,AJ,120,2000,1579-1587
[15]J.Gunn,W.Siegmund,E.Mannery,et al.The 2.5 m Telescope of the Sloan Digital Sky Survey,AJ,131,2006,2332-2359
[16]白仲瑞,张垒,叶中付.一种基于主分量分析的减天光方法.天文学报.2006,48(4):507-514
[17]Vivienne Wild,Paul C Hewett,MNRAS,2005,358:1083
[18]段福庆,吴福超,罗阿理,赵永恒.基于均值漂移的非发射线天体光谱的谱线自动提取.光谱学与光谱分析.2005,25(11):1844-1888
[19]刘中田,吴福超,罗阿理,赵永恒.基于卷积型小波变换的谱线提取方法.光谱学与光谱分析.2006,26(2):372-376
[20]刘蓉,段福庆,刘三阳.基于特征约束的谱线自动提取.光谱学与光谱分析.2007,27(8):1648-1652
[21]赵瑞珍,罗阿理.天体光谱信号的连续谱归一化新方法.光谱学与光谱分析.2006,26(3):587-590
[22]徐晨,赵瑞珍,甘小冰.小波分析·应用算法.北京:科学出版社,2004,5:64-87
[23]D.L.Donoho,IEEE Trans.Inform.Theory,2006,52(4),1289
[24]E.J.Candes,J.Romberg and T.Tao,IEEE Trans.Inform.Theory,2006,52(2):489
[25]Martin Vetterli,Pina Marziliano,Thierry Blu.Sampling Signals With Finite Rate of Innovation.IEEE Trans on.SP,2002,(20):1417-1428
[26]Jeol A.Tropp,Anna C,Gilbert,Signal Recovery from Partial Information.Department of Mathematics,April 2005
[27]D.L.Donoho,I.M.Johnstone.Ideal spatial adaptation via wavelet shrinkage.Biometrika,Apr.,1994,81(2):425-455
[28]R.G.Baraniuk,IEEE Signal Processing Magazine,July 2007,118
[29]J.Haupt,R.Nowak,IEEE Trans.Inform.Theory,2006,52(9):4036
[30]R.Zhao,C-C.Li,X.Liu and et al,34th Northeast Bioengineering Conference,Brown University,April 4-6,2008,312
[31]E.J.Candes and T.Tao,IEEE Trans.Inform.Theory,2006,52(12):5406
[32]A.Pizurica,W.Philips,I.Lemahieu and M.Acheeroy,IEEE Trans.on IP,2002,11(5):545
[33]罗阿理.光谱自动分析技术中的模式识别方法研究.中国科学院国家天文台,2002
[34]Faghhih F,ith M.IEEE Trans.on IP,2002,11(9):1062.
[35]Daubechies L.Ten Lectures on Wavelets.Philadelphia:SIAM,1992
[36]Mallat S,Hwang W L.IEEE Trans.on IT,1992,38(2):617.
[37]白仲瑞.LAMOST减天光方法研究.中国科学技术大学,2007
[38]J.Jackson.A User's Guide to Principal Components,New York:A Wiley-Interscience Publication,1992
[39]I.Jolliffe.Principal Component Analysis,New York:Springer,1986
[40]R.Johnson,D.Wichern著,陆漩译.Applied Multivariate Statistical Analysis,北京:清华大学出版社,2001
[41]William Press,Saul Teukolsky,William Vetterling,et al.Numerical recipes in C:the art of scientific computing.2nd ed.London:Cambridge University Press,1992,469-475
[42]罗阿理,赵永恒.使用小波技术自动搜索天体谱线.光谱学与光谱分析.2000,20(4):427-436
[43]赵瑞珍,胡占义.谱线自动提取的小波变换零交叉点方法.光谱学与光谱分析.2005,25(1):153-156
[44]刘蓉,刘三阳,赵瑞珍.正常星系光谱的一种谱线自动提取方法.光谱学与光谱分析.2006,26(3):583-586
[45]周虹,黄凌云.一种基于Hough变换和神经网络的分层类星体识别方法.电子科学学刊.2000,22(4):529-535
[2]LAMOST二维光纤数据模拟软件1.0版技术报告
[3]LAMOST二维光纤光谱处理程序1.0版技术报告
[4]LAMOST二维光纤光谱处理程序2.0版技术报告
[5]邱波.天体光谱数据自动处理和算法研究.中国科学院自动化研究所,2002
[6]贾冬梅.天体光谱信号的自动识别方法研究.中国科学院自动化研究所,2003
[7]Horne Keith,An optimal extraction algorithm for CCD spectroscopy,PASP,1986,98:609-617
[8]J.G.Robertson,Optimal extraction of single-object spectra from observations with two-dimensional detectors,PASP,1986,98:1220-1231
[9]M.Roth,A.Kelz,T.Fechner,et al.The Potsdam Multi-Aperture Spectrophotometer,PASP,2005,117:620-642
[10]A.Bershady,R.Andersen,et al.A Formatted Fiber Field Unit for the WIYN Telescope Bench Spectrograph,PASP,2004,116:565-590
[11]A.Kelz,A.Verheijen,et al.The Potsdam Multi-Aperture Spectrophotometer,PASP,2006,118:129-145
[12]S.M.Kent,Sloan Digital Sky Survey,Astrophysics and Space Science,1994,217:27-30
[13]F.Watson,A.Offer,I.Lewis,et al.Fiber sky-subtraction revisited,Fiber Optics in Astronomy Ⅲ of ASPC,152,1998,50-59
[14]D.York,J.Adelman,J.Anderson,et al.The Sloan Digital Sky Survey:Technical Summary,AJ,120,2000,1579-1587
[15]J.Gunn,W.Siegmund,E.Mannery,et al.The 2.5 m Telescope of the Sloan Digital Sky Survey,AJ,131,2006,2332-2359
[16]白仲瑞,张垒,叶中付.一种基于主分量分析的减天光方法.天文学报.2006,48(4):507-514
[17]Vivienne Wild,Paul C Hewett,MNRAS,2005,358:1083
[18]段福庆,吴福超,罗阿理,赵永恒.基于均值漂移的非发射线天体光谱的谱线自动提取.光谱学与光谱分析.2005,25(11):1844-1888
[19]刘中田,吴福超,罗阿理,赵永恒.基于卷积型小波变换的谱线提取方法.光谱学与光谱分析.2006,26(2):372-376
[20]刘蓉,段福庆,刘三阳.基于特征约束的谱线自动提取.光谱学与光谱分析.2007,27(8):1648-1652
[21]赵瑞珍,罗阿理.天体光谱信号的连续谱归一化新方法.光谱学与光谱分析.2006,26(3):587-590
[22]徐晨,赵瑞珍,甘小冰.小波分析·应用算法.北京:科学出版社,2004,5:64-87
[23]D.L.Donoho,IEEE Trans.Inform.Theory,2006,52(4),1289
[24]E.J.Candes,J.Romberg and T.Tao,IEEE Trans.Inform.Theory,2006,52(2):489
[25]Martin Vetterli,Pina Marziliano,Thierry Blu.Sampling Signals With Finite Rate of Innovation.IEEE Trans on.SP,2002,(20):1417-1428
[26]Jeol A.Tropp,Anna C,Gilbert,Signal Recovery from Partial Information.Department of Mathematics,April 2005
[27]D.L.Donoho,I.M.Johnstone.Ideal spatial adaptation via wavelet shrinkage.Biometrika,Apr.,1994,81(2):425-455
[28]R.G.Baraniuk,IEEE Signal Processing Magazine,July 2007,118
[29]J.Haupt,R.Nowak,IEEE Trans.Inform.Theory,2006,52(9):4036
[30]R.Zhao,C-C.Li,X.Liu and et al,34th Northeast Bioengineering Conference,Brown University,April 4-6,2008,312
[31]E.J.Candes and T.Tao,IEEE Trans.Inform.Theory,2006,52(12):5406
[32]A.Pizurica,W.Philips,I.Lemahieu and M.Acheeroy,IEEE Trans.on IP,2002,11(5):545
[33]罗阿理.光谱自动分析技术中的模式识别方法研究.中国科学院国家天文台,2002
[34]Faghhih F,ith M.IEEE Trans.on IP,2002,11(9):1062.
[35]Daubechies L.Ten Lectures on Wavelets.Philadelphia:SIAM,1992
[36]Mallat S,Hwang W L.IEEE Trans.on IT,1992,38(2):617.
[37]白仲瑞.LAMOST减天光方法研究.中国科学技术大学,2007
[38]J.Jackson.A User's Guide to Principal Components,New York:A Wiley-Interscience Publication,1992
[39]I.Jolliffe.Principal Component Analysis,New York:Springer,1986
[40]R.Johnson,D.Wichern著,陆漩译.Applied Multivariate Statistical Analysis,北京:清华大学出版社,2001
[41]William Press,Saul Teukolsky,William Vetterling,et al.Numerical recipes in C:the art of scientific computing.2nd ed.London:Cambridge University Press,1992,469-475
[42]罗阿理,赵永恒.使用小波技术自动搜索天体谱线.光谱学与光谱分析.2000,20(4):427-436
[43]赵瑞珍,胡占义.谱线自动提取的小波变换零交叉点方法.光谱学与光谱分析.2005,25(1):153-156
[44]刘蓉,刘三阳,赵瑞珍.正常星系光谱的一种谱线自动提取方法.光谱学与光谱分析.2006,26(3):583-586
[45]周虹,黄凌云.一种基于Hough变换和神经网络的分层类星体识别方法.电子科学学刊.2000,22(4):529-535