模型诊断用于近红外光谱建模校正集中奇异样本的识别
详细信息 查看官网全文
- 英文论文题名：Outlier Detection for Multivariate Calibration in Near Infrared Spectroscopic Analysis by Model Diagnostics
- 论文作者：李正风 ; 徐广晋 ; 王家俊 ; 杜国荣 ; 蔡文生 ; 邵学广
- 英文论文作者：LI Zheng-Feng ; XU Guang-Jin ; WANG Jia-Jun ; DU Guo-Rong ; CAI Wen-Sheng ; SHAO Xue-Guang ; R&D Center ; China Tobacco Yunnan Industrial Co.Ltd. ; Research Center for Analytical Sciences ; College of Chemistry ; Nankai University ; College of Chemistry and Environmental Science ; Kashgar University
- 年：2016
- 作者机构：云南中烟工业有限责任公司技术中心;南开大学化学学院分析科学研究中心;喀什大学化学与环境科学学院;
- 论文关键词：多元校正 ; 奇异样本检测 ; 偏最小二乘 ; 近红外光谱 ; 定量分析
- 英文论文关键词：Multivariate calibration ; Outlier detection ; Partial least squares ; Near infrared spectroscopy ; Quantitative analysis
- 会议召开时间：2016-12-01
- 会议录名称：中国烟草学会2016年度优秀论文汇编——烟草工业主题
- 语种：中文
- 分类号：TS255.44;O657.33
- 学会代码：ZGYG
- 学会名称：中国烟草学会
- 页数：5
- 文件大小：339k
- 原文格式：D
摘要
由于校正集样本的质量决定校正模型的质量,校正集中奇异样本的检测在多元校正建模中具有非常重要的意义。本研究建立了一种用于近红外光谱多元校正建模时校正集中奇异样本的检测方法。本方法基于奇异样本的定义和偏最小二乘方法的原理,通过考察每个校正集样本在模型的每个因子(或主成分)中对模型的贡献,将与多数样本表现不同的样本识别为奇异样本。采用218个橘汁样本构成的近红外光谱数据进行了分析,结果表明,校正集中存在6个奇异样本,扣除奇异样本后,校正集的交叉验证均方根误差由16.870减小为4.809,预测集的均方根误差从3.688减小为3.332。
Outlier detection is an important task in multivariate calibration because the quality of a calibration model is determined by that of the calibration data.An outlier detection method is proposed for near infrared(NIR) spectral analysis.The method is based on the definition of outlier and the principle of partial least squares(PLS) regression,i.e.,an outlier in a dataset behaves differently from the rest,and the prediction result of a PLS model is an accumulation of several independent latent variables.Therefore,the proposed method builds a PLS model with a calibration dataset,and then the contribution of each latent variable is investigated.Outliers can be detected by comparing these contributions.An NIR spectral dataset of orange juice samples is adopted for testing the method.Six outliers are detected in the calibration set.The root mean squared error of cross validation(RMSECV) becomes to 4.809 from 16.870 and the root mean squared error of prediction(RMSEP) becomes to 3.332 from 3.688 after the removal of the outliers.Compared with a robust regression method,the result of the proposed method seems more reasonable.
Outlier detection is an important task in multivariate calibration because the quality of a calibration model is determined by that of the calibration data.An outlier detection method is proposed for near infrared(NIR) spectral analysis.The method is based on the definition of outlier and the principle of partial least squares(PLS) regression,i.e.,an outlier in a dataset behaves differently from the rest,and the prediction result of a PLS model is an accumulation of several independent latent variables.Therefore,the proposed method builds a PLS model with a calibration dataset,and then the contribution of each latent variable is investigated.Outliers can be detected by comparing these contributions.An NIR spectral dataset of orange juice samples is adopted for testing the method.Six outliers are detected in the calibration set.The root mean squared error of cross validation(RMSECV) becomes to 4.809 from 16.870 and the root mean squared error of prediction(RMSEP) becomes to 3.332 from 3.688 after the removal of the outliers.Compared with a robust regression method,the result of the proposed method seems more reasonable.
引文
1 Wold S,Ruhe A,Wold H,Dunn W J.SIAM J.Sci.Stat.Comput.,1984,5(3):735-743
2 LIANG Miao,CAI Jia-Yue,YANG Kai,SHU Ru-Xin,ZHAO Long-Lian,ZHANG Lu-Da,LI Jun-Hui.Chinese J.Anal.Chem.,2014,42(11):1687-1691梁淼,蔡嘉月,杨凯,束茹欣,赵龙莲,张录达,李军会.分析化学,2014,42(11):1687-1691
3 ZHANG Lu-Da,SU Shi-Guang,WANG Lai-Sheng,LI Jun-Hui,YANG Li-Ming.Spectroscopy and Spectral Analysis,2005,25(1):33-35张录达,苏时光,王来生,李军会,杨丽明.光谱学与光谱分析,2005,25(1):33-35
4 Li Y K,Shao X G,Cai W S.Talanta,2007,72(1):217-222
5 LIN Hao,ZHAO Jie-Wen,CHEN Quan-Sheng,CAI Jian-Rong,ZHOU Ping.Spectroscopy and Spectral Analysis,2010,30(4):929-932林颢,赵杰文,陈全胜,蔡健荣,周平.光谱学与光谱分析,2010,30(4):929-932
6 Shao X G,Bian X H,Liu J J,Zhang M,Cai W S.Anal.Methods,2010,2(11):1662-1666
7 Wold S,Antti H,Lindgren F,Ohman J.Chemom.Intell.Lab.Syst.,1998,44(1-2):175-185
8 Shao X G,Leung A K M,Chau F T.Acc.Chem.Res.,2003,36(4):276-283
9 Norgaard L,Saudland A,Wagner J,Wagner J,Nielsen J P,Munk L,Engelsen S B.Appl.Spectrosc.,2000,54(3):413-419
10 Centner V,Massart D L,de Noord 0 E,de Jong S,Vandeginste M B,Sterna C.Anal.Chem.,1996,68(21):3851-3858
11 Cai W S,Li Y K,Shao X G.Chemom.Intell.Lab.Syst.,2008,90(2):188-194
12 Li H D,Liang Y Z,Xu Q S,Cao D S.Anal.Chim.Acta,2009,648(1):77-84
13 Araujo M C U,Saldanha T C B,Galvao R K H,Yoneyama T,Chame H C,Visani V.Chemom.Intell.Lab.Syst.,2001,57(2):65-73
14 Xu H,Liu Z C,Cai W S,Shao X G.Chemom.Intell.Lab.Syst.,2009,97(1):189-193
15 Liang Y Z,Kvalheim O M.Chemom.Intell.Lab.Syst.,1996,32(1):1-10
16 Pierna J A F,Jin L,Daszykowski M,Wahl F,Massart D L.Chemom.Intell.Lab.Syst.,2003,68(1-2):17-28
17 Bian X H,Cai W S,Shao X G,Chen D,Grant E R.Analyst,2010,135(11):2841-2847
18 Pierna J A F,Wahl F,de Noord O E,Massart D L.Chemom.Intell.Lab.Syst.,2002,63(1):27-39
19 Walczak B,Massart D L.Chemom.Intell.Lab.Syst.,1998,41(1):1-15
20 Hubert M,Vanden Branden K.J.Chemom.,2003,17(10):537-549
21 Liu Z C,Cai W S,Shao X G.Sci.China Ser B-Chem.,2008,51(8):751-759
22 Liu Z C,Ma X,Wen Y D,Wang Y,Cai W S,Shao X G.Sci.China Ser B-Chem.,2009,52(7):1021-1027
23 Breunig M M,Kriegel H P,Ng R T,Sander J.Sigmod.Rec.,2000,29(2):93-104
24 Li W,Goovaerts P,Meurens M.J.Arg.Food Chem.,19%,44(8):2252-2259
2 LIANG Miao,CAI Jia-Yue,YANG Kai,SHU Ru-Xin,ZHAO Long-Lian,ZHANG Lu-Da,LI Jun-Hui.Chinese J.Anal.Chem.,2014,42(11):1687-1691梁淼,蔡嘉月,杨凯,束茹欣,赵龙莲,张录达,李军会.分析化学,2014,42(11):1687-1691
3 ZHANG Lu-Da,SU Shi-Guang,WANG Lai-Sheng,LI Jun-Hui,YANG Li-Ming.Spectroscopy and Spectral Analysis,2005,25(1):33-35张录达,苏时光,王来生,李军会,杨丽明.光谱学与光谱分析,2005,25(1):33-35
4 Li Y K,Shao X G,Cai W S.Talanta,2007,72(1):217-222
5 LIN Hao,ZHAO Jie-Wen,CHEN Quan-Sheng,CAI Jian-Rong,ZHOU Ping.Spectroscopy and Spectral Analysis,2010,30(4):929-932林颢,赵杰文,陈全胜,蔡健荣,周平.光谱学与光谱分析,2010,30(4):929-932
6 Shao X G,Bian X H,Liu J J,Zhang M,Cai W S.Anal.Methods,2010,2(11):1662-1666
7 Wold S,Antti H,Lindgren F,Ohman J.Chemom.Intell.Lab.Syst.,1998,44(1-2):175-185
8 Shao X G,Leung A K M,Chau F T.Acc.Chem.Res.,2003,36(4):276-283
9 Norgaard L,Saudland A,Wagner J,Wagner J,Nielsen J P,Munk L,Engelsen S B.Appl.Spectrosc.,2000,54(3):413-419
10 Centner V,Massart D L,de Noord 0 E,de Jong S,Vandeginste M B,Sterna C.Anal.Chem.,1996,68(21):3851-3858
11 Cai W S,Li Y K,Shao X G.Chemom.Intell.Lab.Syst.,2008,90(2):188-194
12 Li H D,Liang Y Z,Xu Q S,Cao D S.Anal.Chim.Acta,2009,648(1):77-84
13 Araujo M C U,Saldanha T C B,Galvao R K H,Yoneyama T,Chame H C,Visani V.Chemom.Intell.Lab.Syst.,2001,57(2):65-73
14 Xu H,Liu Z C,Cai W S,Shao X G.Chemom.Intell.Lab.Syst.,2009,97(1):189-193
15 Liang Y Z,Kvalheim O M.Chemom.Intell.Lab.Syst.,1996,32(1):1-10
16 Pierna J A F,Jin L,Daszykowski M,Wahl F,Massart D L.Chemom.Intell.Lab.Syst.,2003,68(1-2):17-28
17 Bian X H,Cai W S,Shao X G,Chen D,Grant E R.Analyst,2010,135(11):2841-2847
18 Pierna J A F,Wahl F,de Noord O E,Massart D L.Chemom.Intell.Lab.Syst.,2002,63(1):27-39
19 Walczak B,Massart D L.Chemom.Intell.Lab.Syst.,1998,41(1):1-15
20 Hubert M,Vanden Branden K.J.Chemom.,2003,17(10):537-549
21 Liu Z C,Cai W S,Shao X G.Sci.China Ser B-Chem.,2008,51(8):751-759
22 Liu Z C,Ma X,Wen Y D,Wang Y,Cai W S,Shao X G.Sci.China Ser B-Chem.,2009,52(7):1021-1027
23 Breunig M M,Kriegel H P,Ng R T,Sander J.Sigmod.Rec.,2000,29(2):93-104
24 Li W,Goovaerts P,Meurens M.J.Arg.Food Chem.,19%,44(8):2252-2259