基于LS_SVM建立发酵过程动态模型的研究及软件实现
|
摘要
微生物发酵涉及到制药、食品等多个工业领域,与经济发展和人民生活密切相关。高成本和高能耗是微生物发酵生产的特征,为了提高发酵单位,降低生产成本,实现对微生物发酵过程的优化控制就成为了一个重要课题。
由于微生物发酵过程是一类非常复杂的生化反应过程,人类尚未完全弄清楚它的机理。并且现有的在线生物传感器的测量精度难以保证,生物参数主要通过离线分析得到,这往往存在较大的滞后,无法及时反馈控制信息。所以,建立高精度的发酵过程产物预估模型,就成为优化控制需要研究的核心内容。
本文针对发酵过程时变性、非线性、不可逆、多变量耦合等特点,通过对现有发酵过程建模方法的对比研究,提出了基于动态时间弯曲距离(DTW)的最小二乘支持向量机(LS_SVM)在线建立发酵过程局部模型的方法。主要有以下几个方面的研究成果:
1、基于DTW的在线构造相似训练样本集的方法:该方法首先将当前批次滑动时间窗内数据作为查询序列,以DTW作为判断时间序列相似性的标准,从历史批次数据库中搜索与之相似度最高的数据区间,组成在线训练样本集。
2、模型输入变量的选取方法以及超参数敏感度分析:通过仿真实验,分析了不同输入变量以及核函数对模型均方误差(MSE)的影响,选择了适合发酵过程的RBF核函数,分析了模型精度对γ、σ2的敏感程度,确定了最优超参数的取值范围。
3、基于粒子群交叉验证(PSO-CV)的在线超参数优化方法:通过对交叉验证确定模型超参数方法的分析,提出了以最小化K-CV均方误差为目标的PSO超参数优化算法,在保证模型精度的情况下,兼顾了模型的泛化能力,与网格搜索法相比,有更好的性能。
4、在线建模软件的开发:利用VC++6.0开发了Windows系统下的发酵过程在线建模软件,实现了通过OPC方式读取组态软件中新采集到的数据,通过ADO方式读取数据库中历史批次的数据,对实际发酵过程建立基于DTW的LS_SVM在线局部模型,同时绘制模型预估输出以及主要可测变量的动态实时曲线,有助于实现发酵过程的优化控制。
The microbial fermentation involves the pharmaceutical, food and other industries, and is closely related to economic development and people's everyday life. High cost and energy consumption are the characteristics of microbial fermentation. Therefore, in order to improve the fermentation unit and reduce production cost, it has become an important issue to realize optimal control of fermentation process.
Since the fermentation process is a kind of very complex biochemical reaction process, the human have not fully made clear of its mechanism yet. What is more, the existing online biosensor is difficult to ensure measurement accuracy, the measurement of biomechanical parameters is mainly obtained through off-line analysis, it usually has a big lag which can’t feedback control information in time. Therefore, the establishment of high-precision prediction model of fermentation process products becomes core content to be researched in optimal control.
In this paper, we aim at the characteristics of fermentation process, including variability、nonlinear、irreversible and multivariable coupling. By comparing existing fermentation process modeling methods, a new online local modeling method is proposed for fed-batch fermentation processes based on dynamic time warping (DTW) and least squares support vector machine (LS_SVM). The major research findings and innovation are as follows:
1) Online constructing similar training sample set based on DTW:
A set of data within the sliding window is set as a query sequence in the current batch, and then using DTW as standard to judge the similarity of time series, we search for the most similar sub-sequence from the historical batch database to form the training set.
2) Selection method of model input variables and parameters sensitivity analysis:
By analyzing the influence on model MSE of different input variables and the kernel function through simulation experiments, I select RBF kernel function for the fermentation process and analyze sensitivity degree of the model accuracy onγandσ2, and determine the optimal range of super parameters.
3) Online super parameters optimization method based on PSO-CV:
Through analysis of cross-validation method used to determine the model’s super parameters, a new online super parameters optimization method based on Particle Swarm Optimization(PSO)is proposed to minimize the K-CV error of for the model, it not only ensure accuracy of the model, but also take into account the model generalization ability. Results show that PSO-CV method has better performance compared with the grid search method.
4) Online modeling software development:
A set of software for modeling and optimization of the fermentation process has been designed by VC++ 6.0 in Windows system. The software can read current and historical data through OPC and ADO to establish the DTW LS_SVM online local model of actual fermentation process, at the same time the dynamic real-time curve of model predictive output and the major measurable variables can be draw when the new data arriving. The development of the software is helpful to realize the optimal control of the fermentation process.
由于微生物发酵过程是一类非常复杂的生化反应过程,人类尚未完全弄清楚它的机理。并且现有的在线生物传感器的测量精度难以保证,生物参数主要通过离线分析得到,这往往存在较大的滞后,无法及时反馈控制信息。所以,建立高精度的发酵过程产物预估模型,就成为优化控制需要研究的核心内容。
本文针对发酵过程时变性、非线性、不可逆、多变量耦合等特点,通过对现有发酵过程建模方法的对比研究,提出了基于动态时间弯曲距离(DTW)的最小二乘支持向量机(LS_SVM)在线建立发酵过程局部模型的方法。主要有以下几个方面的研究成果:
1、基于DTW的在线构造相似训练样本集的方法:该方法首先将当前批次滑动时间窗内数据作为查询序列,以DTW作为判断时间序列相似性的标准,从历史批次数据库中搜索与之相似度最高的数据区间,组成在线训练样本集。
2、模型输入变量的选取方法以及超参数敏感度分析:通过仿真实验,分析了不同输入变量以及核函数对模型均方误差(MSE)的影响,选择了适合发酵过程的RBF核函数,分析了模型精度对γ、σ2的敏感程度,确定了最优超参数的取值范围。
3、基于粒子群交叉验证(PSO-CV)的在线超参数优化方法:通过对交叉验证确定模型超参数方法的分析,提出了以最小化K-CV均方误差为目标的PSO超参数优化算法,在保证模型精度的情况下,兼顾了模型的泛化能力,与网格搜索法相比,有更好的性能。
4、在线建模软件的开发:利用VC++6.0开发了Windows系统下的发酵过程在线建模软件,实现了通过OPC方式读取组态软件中新采集到的数据,通过ADO方式读取数据库中历史批次的数据,对实际发酵过程建立基于DTW的LS_SVM在线局部模型,同时绘制模型预估输出以及主要可测变量的动态实时曲线,有助于实现发酵过程的优化控制。
The microbial fermentation involves the pharmaceutical, food and other industries, and is closely related to economic development and people's everyday life. High cost and energy consumption are the characteristics of microbial fermentation. Therefore, in order to improve the fermentation unit and reduce production cost, it has become an important issue to realize optimal control of fermentation process.
Since the fermentation process is a kind of very complex biochemical reaction process, the human have not fully made clear of its mechanism yet. What is more, the existing online biosensor is difficult to ensure measurement accuracy, the measurement of biomechanical parameters is mainly obtained through off-line analysis, it usually has a big lag which can’t feedback control information in time. Therefore, the establishment of high-precision prediction model of fermentation process products becomes core content to be researched in optimal control.
In this paper, we aim at the characteristics of fermentation process, including variability、nonlinear、irreversible and multivariable coupling. By comparing existing fermentation process modeling methods, a new online local modeling method is proposed for fed-batch fermentation processes based on dynamic time warping (DTW) and least squares support vector machine (LS_SVM). The major research findings and innovation are as follows:
1) Online constructing similar training sample set based on DTW:
A set of data within the sliding window is set as a query sequence in the current batch, and then using DTW as standard to judge the similarity of time series, we search for the most similar sub-sequence from the historical batch database to form the training set.
2) Selection method of model input variables and parameters sensitivity analysis:
By analyzing the influence on model MSE of different input variables and the kernel function through simulation experiments, I select RBF kernel function for the fermentation process and analyze sensitivity degree of the model accuracy onγandσ2, and determine the optimal range of super parameters.
3) Online super parameters optimization method based on PSO-CV:
Through analysis of cross-validation method used to determine the model’s super parameters, a new online super parameters optimization method based on Particle Swarm Optimization(PSO)is proposed to minimize the K-CV error of for the model, it not only ensure accuracy of the model, but also take into account the model generalization ability. Results show that PSO-CV method has better performance compared with the grid search method.
4) Online modeling software development:
A set of software for modeling and optimization of the fermentation process has been designed by VC++ 6.0 in Windows system. The software can read current and historical data through OPC and ADO to establish the DTW LS_SVM online local model of actual fermentation process, at the same time the dynamic real-time curve of model predictive output and the major measurable variables can be draw when the new data arriving. The development of the software is helpful to realize the optimal control of the fermentation process.
引文
1潘丰.补料分批发酵过程优化控制.自动化仪表, 2004, 25(8):51~54
2黄丽,孙玉坤,黄永红,薛力红.补料分批发酵过程控制系统研究.微计算机信息, 2006, 22(9):123~125
3姚志湘,伍时华,陈宁,张克旭.发酵过程动力学仿真研究.计算机与应用化学, 2003, 20(3):305~308
4张嗣良.发酵过程多水平问题及其生物反应器装置技术研究——基于过程参数相关的发酵过程优化与放大技术.中国工程科学,2001,3(8):37~44
5 Ratkov A I,Georgiev Tz,Ivanova V,et al. Investigation of L-lysine fed-batch fermentation process with droppings by mathematical modeling. Information Technology Interfaces, 2004, 10(10):543~547
6 Zhang H W,Lennox B. Integrated condition monitoring and control of fed-batch fermentation process. Journal of Process Control, 2004, 14(1):41~50
7 Massimo C D,Montague G A,Willis M J,et al. Towards improved penicillin fermentation via artificial neural networks. Computers and Chemical Engineering, 1992, 16(4):283~291
8 Chen B H, Woodley J M. Wavelet shrinkage data processing for neural networks in bioprocess modeling. Computers and Chemical Engineering, 2002, 26(11):1611~1620
9 Vapnik V N. The nature of statistical learning theory. Springer, 1995:152~166
10 Feng R, Shen W, Shao H. A soft sensor modeling approach using support vector machines. Proceedings of the American Control Conference, 2003:3702~3707
11许光,俞欢军,陈德钊.基于支持向量机的柠檬酸发酵过程统计建模.化学反应工程与工艺, 2004, 20(1):59~63
12刘毅,王海清.采用最小二乘支持向量机的青霉素发酵过程建模研究.生物工程学报, 2006, 22(1):144~149
13桑海峰,王福利,何大阔.基于多支持向量机的诺西肽发酵中菌体浓度软测量.系统仿真学报, 2006, 18(7):1983~1986
14高学金,王普,孙崇正,易建强,张亚庭,张会清.基于动态ε-SVM的发酵过程建模.仪器仪表学报, 2006, 27(11):1497~1500
15于乃功,阮晓刚.细胞自动机及其在菌体生长建模仿真中的应用.系统仿真学报, 2004, 16(12):2651~2654
16张成燕,郜培,段作营.利用代谢网络模型和线性规则优法在线预测谷氨酸发酵中产物浓度.食品与生物技术学报, 2005, 24(4):31~37
17 Zuo K, Wu W T. Semi-real time optimization and control of a fed-batch fermentation system. Computers and Chemical Engineering, 2000, 24(2):1105~1109
18 Berkholz R, Rehlig D, Guthke R. Data and knowledge based experimental design for fermentation process optimization. Enzyme and Microbial Technology, 2000, 27(10):784~788
19 Tholudur A, Ramirez W F, McMillan J D. Interpolated parameter functions for neural network models. Computers and Chemical Engineering, 2000, 24(11):2545~2553
20许光,俞欢军,陶少辉.与机理杂交的支持向量机为发酵过程建模.化工学报, 2005, 56(4):653~658
21于湘莉,苗志奇,元英进.基于模糊理论的酵母发酵动力学模型.无锡轻工大学学报, 2002, 21(6):638～644
22 Vapnik V N. The nature of statistical learning theory. Springer Verlag, 1995:79~87
23李晋,秦琳琳,吴刚,苑媛,岳大志.基于加权最小二乘支持向量机的温室小气候建模与仿真.系统仿真学报, 2008, 20(16):4232~4236
24 Suykens J.A.K, Vandewalle J. Least Squares Support Vector Machine Classifiers. Neural Processing Letters, 1999, 9(3):293~300
25 Suykens J.A.K, Van Gestel T, De Brabanter J, et al. Least Squares Support Vector Machines. World Scientific Singapore, 2002:358~423
26 Suykens J.A.K, Lukas L, Vandwealle J. Sparse Least Squares Support Vector Machines Classifiers, European Symposium on Artificial Neural Networks, 2000, 8(4):37~42
27 Suykens J.A.K, Lukas L, Vandewalle J. Sparse approximation using least squares support vector machines. Proc of IEEE Int Symposium on Circuits and System. Piscataway, 2000:757~760
28 Kruif B J, Vries J A. Pruning error minimization in least squares support vector machines. IEEE Trans on Neural Networks, 2003, 14(3):696~702
29 Hoegaerts L, Suykens J.A.K, Vandewalle J, et al. Acomparison of pruning algorithms for sparse least squares support vector machines. Proc of Int Conf on Neural Information Processing, 2004:1247~ 1253
30 Platt J, et al. Fast training of support vector machines using sequential minimal optimization. Proc of Advances in Kernel Methods-Support Vector Machine,1999:185~208
31 Keerthi S S, Shevade S K. SMO algorithm for least squares SVM. Proc of the Int Joint Conf on Neural Networks, 2003:2088~2093
32 Zeng X Y, Chen X W. SMO-based pruning methods for sparse least squares support vector machines. IEEE Trans on Neural Networks, 2005, 16(6):1541~1546
33 Jiao L C, Bo L F, Wang L. Fast sparse approximation for least squares support vector machine. IEEE Trans on Neural Networks, 2007, 18(3):685~697
34张浩然,汪晓东.回归最小二乘支持向量机的增量和在线式学习算法.计算机学报, 2006, 29(3):400~406
35 Suykens J.A.K, De Brabanter J, Lukas L, et al. Weighted least Squares Support Vector Machines: robustness and sparse approximation. Neuro computing, 2002, 48(1):85~105
36刘毅,王海清. Pensim仿真平台在青霉素发酵过程的应用研究.系统仿真学报, 2006, 18(12):3524~3527
37刘毅,王海清,李平.局部最小二乘支持向量机回归在线建模方法及其在间歇过程的应用.化工学报, 2007, 58(11):2846~2851
38潘天红,李少远.基于即时学习的非线性系统优化控制.控制与决策, 2007, 22(1):25~29
39 Gao J B, Shi D, Liu X M. Significant vector learning to construct sparse kernel regression models. Neural Networks, 2007, 20(7):791~798
40徐晔,杜文莉,钱锋.基于核主元分析和最小二乘支持向量机的软测量建模.系统仿真学报, 2007,. 19(17):3873~3876
41李勇,邵诚.一种新的灰关联分析算法在软测量中的应用.自动化学报, 2006, 32(2):311~317
42 Ge Z, Song Z. Online monitoring of nonlinear multiple mode processes based on adaptive local model approach. Control Engineering Practice, 2008, 16(12):1427~1437
43江浩.面向相似性的时间序列表示与搜索方法研究.华中科技大学,硕士学位论文, 2004:18~19
44 Keogh, E.C.A. Ratanamahatana, Exact indexing of dynamic time warping. Knowledge and Information Systems, 2005, 7(3):358~386
45 Baudat G, Anouar F. Generalized discriminant analysis using a kernel function. Neural Computing, 2000, 12(10):2385~2404
46 Smola A.J. Learning with kernels. Technical University of Berlin,1998:344~368
47 Michael W.Browne. Cross-Validation Method. Mathematical Psychology, 2000, 44(3):108~132
48 OPC Foundation. OPC data Access Custom Interface Standard Version 2.04. 2000:23~65
49刘廷瑞,于瀛洁,陈明仪.基于Matcom4.5的VC++与Matlab通用接口程序编制.计算机应用与软件, 2004, 21(11):34~35
2黄丽,孙玉坤,黄永红,薛力红.补料分批发酵过程控制系统研究.微计算机信息, 2006, 22(9):123~125
3姚志湘,伍时华,陈宁,张克旭.发酵过程动力学仿真研究.计算机与应用化学, 2003, 20(3):305~308
4张嗣良.发酵过程多水平问题及其生物反应器装置技术研究——基于过程参数相关的发酵过程优化与放大技术.中国工程科学,2001,3(8):37~44
5 Ratkov A I,Georgiev Tz,Ivanova V,et al. Investigation of L-lysine fed-batch fermentation process with droppings by mathematical modeling. Information Technology Interfaces, 2004, 10(10):543~547
6 Zhang H W,Lennox B. Integrated condition monitoring and control of fed-batch fermentation process. Journal of Process Control, 2004, 14(1):41~50
7 Massimo C D,Montague G A,Willis M J,et al. Towards improved penicillin fermentation via artificial neural networks. Computers and Chemical Engineering, 1992, 16(4):283~291
8 Chen B H, Woodley J M. Wavelet shrinkage data processing for neural networks in bioprocess modeling. Computers and Chemical Engineering, 2002, 26(11):1611~1620
9 Vapnik V N. The nature of statistical learning theory. Springer, 1995:152~166
10 Feng R, Shen W, Shao H. A soft sensor modeling approach using support vector machines. Proceedings of the American Control Conference, 2003:3702~3707
11许光,俞欢军,陈德钊.基于支持向量机的柠檬酸发酵过程统计建模.化学反应工程与工艺, 2004, 20(1):59~63
12刘毅,王海清.采用最小二乘支持向量机的青霉素发酵过程建模研究.生物工程学报, 2006, 22(1):144~149
13桑海峰,王福利,何大阔.基于多支持向量机的诺西肽发酵中菌体浓度软测量.系统仿真学报, 2006, 18(7):1983~1986
14高学金,王普,孙崇正,易建强,张亚庭,张会清.基于动态ε-SVM的发酵过程建模.仪器仪表学报, 2006, 27(11):1497~1500
15于乃功,阮晓刚.细胞自动机及其在菌体生长建模仿真中的应用.系统仿真学报, 2004, 16(12):2651~2654
16张成燕,郜培,段作营.利用代谢网络模型和线性规则优法在线预测谷氨酸发酵中产物浓度.食品与生物技术学报, 2005, 24(4):31~37
17 Zuo K, Wu W T. Semi-real time optimization and control of a fed-batch fermentation system. Computers and Chemical Engineering, 2000, 24(2):1105~1109
18 Berkholz R, Rehlig D, Guthke R. Data and knowledge based experimental design for fermentation process optimization. Enzyme and Microbial Technology, 2000, 27(10):784~788
19 Tholudur A, Ramirez W F, McMillan J D. Interpolated parameter functions for neural network models. Computers and Chemical Engineering, 2000, 24(11):2545~2553
20许光,俞欢军,陶少辉.与机理杂交的支持向量机为发酵过程建模.化工学报, 2005, 56(4):653~658
21于湘莉,苗志奇,元英进.基于模糊理论的酵母发酵动力学模型.无锡轻工大学学报, 2002, 21(6):638～644
22 Vapnik V N. The nature of statistical learning theory. Springer Verlag, 1995:79~87
23李晋,秦琳琳,吴刚,苑媛,岳大志.基于加权最小二乘支持向量机的温室小气候建模与仿真.系统仿真学报, 2008, 20(16):4232~4236
24 Suykens J.A.K, Vandewalle J. Least Squares Support Vector Machine Classifiers. Neural Processing Letters, 1999, 9(3):293~300
25 Suykens J.A.K, Van Gestel T, De Brabanter J, et al. Least Squares Support Vector Machines. World Scientific Singapore, 2002:358~423
26 Suykens J.A.K, Lukas L, Vandwealle J. Sparse Least Squares Support Vector Machines Classifiers, European Symposium on Artificial Neural Networks, 2000, 8(4):37~42
27 Suykens J.A.K, Lukas L, Vandewalle J. Sparse approximation using least squares support vector machines. Proc of IEEE Int Symposium on Circuits and System. Piscataway, 2000:757~760
28 Kruif B J, Vries J A. Pruning error minimization in least squares support vector machines. IEEE Trans on Neural Networks, 2003, 14(3):696~702
29 Hoegaerts L, Suykens J.A.K, Vandewalle J, et al. Acomparison of pruning algorithms for sparse least squares support vector machines. Proc of Int Conf on Neural Information Processing, 2004:1247~ 1253
30 Platt J, et al. Fast training of support vector machines using sequential minimal optimization. Proc of Advances in Kernel Methods-Support Vector Machine,1999:185~208
31 Keerthi S S, Shevade S K. SMO algorithm for least squares SVM. Proc of the Int Joint Conf on Neural Networks, 2003:2088~2093
32 Zeng X Y, Chen X W. SMO-based pruning methods for sparse least squares support vector machines. IEEE Trans on Neural Networks, 2005, 16(6):1541~1546
33 Jiao L C, Bo L F, Wang L. Fast sparse approximation for least squares support vector machine. IEEE Trans on Neural Networks, 2007, 18(3):685~697
34张浩然,汪晓东.回归最小二乘支持向量机的增量和在线式学习算法.计算机学报, 2006, 29(3):400~406
35 Suykens J.A.K, De Brabanter J, Lukas L, et al. Weighted least Squares Support Vector Machines: robustness and sparse approximation. Neuro computing, 2002, 48(1):85~105
36刘毅,王海清. Pensim仿真平台在青霉素发酵过程的应用研究.系统仿真学报, 2006, 18(12):3524~3527
37刘毅,王海清,李平.局部最小二乘支持向量机回归在线建模方法及其在间歇过程的应用.化工学报, 2007, 58(11):2846~2851
38潘天红,李少远.基于即时学习的非线性系统优化控制.控制与决策, 2007, 22(1):25~29
39 Gao J B, Shi D, Liu X M. Significant vector learning to construct sparse kernel regression models. Neural Networks, 2007, 20(7):791~798
40徐晔,杜文莉,钱锋.基于核主元分析和最小二乘支持向量机的软测量建模.系统仿真学报, 2007,. 19(17):3873~3876
41李勇,邵诚.一种新的灰关联分析算法在软测量中的应用.自动化学报, 2006, 32(2):311~317
42 Ge Z, Song Z. Online monitoring of nonlinear multiple mode processes based on adaptive local model approach. Control Engineering Practice, 2008, 16(12):1427~1437
43江浩.面向相似性的时间序列表示与搜索方法研究.华中科技大学,硕士学位论文, 2004:18~19
44 Keogh, E.C.A. Ratanamahatana, Exact indexing of dynamic time warping. Knowledge and Information Systems, 2005, 7(3):358~386
45 Baudat G, Anouar F. Generalized discriminant analysis using a kernel function. Neural Computing, 2000, 12(10):2385~2404
46 Smola A.J. Learning with kernels. Technical University of Berlin,1998:344~368
47 Michael W.Browne. Cross-Validation Method. Mathematical Psychology, 2000, 44(3):108~132
48 OPC Foundation. OPC data Access Custom Interface Standard Version 2.04. 2000:23~65
49刘廷瑞,于瀛洁,陈明仪.基于Matcom4.5的VC++与Matlab通用接口程序编制.计算机应用与软件, 2004, 21(11):34~35