递推与迭代学习辨识算法及其应用研究
|
摘要
系统辨识是高性能自动化技术(如适应控制与学习控制)中的重要内容。随着人们生活水平和工业生产水平的提高,人们对工业产品质量的要求越来越高,生产工艺对自动化技术的依赖程度也随之提高,对系统辨识理论与方法提出了更高的要求。许多实际系统动力学特性中的参数随时间变化,这要求辨识方法具有跟踪能力,能够估计时变参数。在实际中也存在着受突发异常干扰的辨识环境,因此,有必要研究鲁棒辨识算法。针对这样两个方面,本文的研究工作主要包括以下几部分:
1.讨论时变遗忘因子和加权配合的递推辨识算法的算法性质和确定性收敛性。
2.针对一类在有限时间区间上重复运行的离散时变系统,讨论用于系统时变参数估计的迭代学习辨识算法。文中进一步讨论迭代学习投影算法、迭代学习最小二乘算法的理论性质,并给出数值仿真结果;推导得到两种迭代学习辨识算法:迭代学习贝叶斯法和迭代学习随机牛顿法。不同于常规跟踪算法的有界收敛性,迭代学习辨识算法可在有限时间区间上实现对时变参数的完全估计,使得沿整个区间参数估计误差为零。
3.将近似最小l_l模递推算法应用于回波消除,分析了这种算法的收敛性能。分析表明,这种算法一旦收敛,对于有界干扰具有强鲁棒性。这一性能有利于提高回波消除质量。双讲时在不增加系统硬件(对讲检测器)情形下,算法仍具有良好的收敛性能,避免了在双讲情况下设置双方对讲监测器。
4.依据非线性PCA准则,以迭代加权最小二乘算法极小化目标函数,近似一类l_p模指标下的盲分离递推算法,文中给出了算法性质。和RLS算法相比,所提算法未显著增加计算复杂度,对幅度较大的非正态分布或统计特性未知的噪声有较强的抑止能力。
5.设计实现冰球式蓄冷空调系统。用时变系统来近似实际系统,采用遗忘和加权配合的辨识算法来预测能量。在系统实现上,利用组态王实现数据采集、友好人机界面设计,利用MATLAB完成复杂的预测算法,充分发挥组态软件在工业控制上组态方便、开发周期短和MATLAB强大数值计算功能的优势,两者之间通过DDE技术实现数据实时通信。
System identification plays an important role in high performance automation technologies such as adaptive control and learning control. More high quality of products is required, in nowadays plants, to meet the needs of people's production and living demands. It should be challenged for system identification when high control performance is pursued. The parameters in dynamics of many real systems are time-dependent, which requires that identification techniques are of the ability of tracking, and can estimate time-varying parameters. We also need robust identification algorithms owing to the presence of abrupt disturbances. Take into account the issues above mentioned, this thesis focuses on the following aspects:
1. The properties and deterministic convergence of recursive identification algorithm are considered, in which both a forgetting factor and weights have been incorporated.
2. Iterative learning identification algorithms are presented for estimating time-varying parameters of a class of discrete time-varying systems over a finite time interval. The theoretic properties of iterative learning projection and least squares algorithms are further discussed, and corresponding numerical simulations are presented. Two prototype algorithms of iterative learning identification, iterative learning Bayes and stochastic Newton algorithms, are proposed with detail. Different from the bounded convergence performance obtained by conventional tracking algorithms, perfect estimation for the time-varying unknowns is achieved through iterative learning, and the parameter estimation error is guaranteed to converge to zero over the entire time interval.
3. The approximate least l_1-norm recursive algorithm is presented for adaptive acoustic echo cancellation. The convergence performance of the algorithm is analyzed, by which the algorithm exhibits robustness with respect to bounded disturbances at steady-state. It is in turn ? useful for performance improvement of the acoustic echo cancellation. During duplex talk, the proposed algorithm works well without requiring Double-Talk Detector and without requiring more hardware in the system.
4. Based on the nonlinear PCA criterion, an iterative weighted least squares algorithm is presented for blind source separation. The iterative algorithm is for the purpose to approximate the l_p-norm index, and the theoretical properties of the algorithm are derived. In comparison with the conventional RLS algorithm, the proposed algorithm has comparable computational complexity, and is robust with respect to non-Gaussian disturbances or those with unknown statistical distribution.
5. An ice storage air condition system is designed and implemented. The real system is modeled as a time-varying system, and the energy is predicted by the identification algorithm with both forgetting and weighted factors. For the implementation, KingView is adopted for data acquisition, with friendly human machine interface design, and MATLAB is used to carry out the on-line prediction computing. In the ice storage system, DDE technique is applied for the real time data exchange between the configuration software and MATLAB.
1.讨论时变遗忘因子和加权配合的递推辨识算法的算法性质和确定性收敛性。
2.针对一类在有限时间区间上重复运行的离散时变系统,讨论用于系统时变参数估计的迭代学习辨识算法。文中进一步讨论迭代学习投影算法、迭代学习最小二乘算法的理论性质,并给出数值仿真结果;推导得到两种迭代学习辨识算法:迭代学习贝叶斯法和迭代学习随机牛顿法。不同于常规跟踪算法的有界收敛性,迭代学习辨识算法可在有限时间区间上实现对时变参数的完全估计,使得沿整个区间参数估计误差为零。
3.将近似最小l_l模递推算法应用于回波消除,分析了这种算法的收敛性能。分析表明,这种算法一旦收敛,对于有界干扰具有强鲁棒性。这一性能有利于提高回波消除质量。双讲时在不增加系统硬件(对讲检测器)情形下,算法仍具有良好的收敛性能,避免了在双讲情况下设置双方对讲监测器。
4.依据非线性PCA准则,以迭代加权最小二乘算法极小化目标函数,近似一类l_p模指标下的盲分离递推算法,文中给出了算法性质。和RLS算法相比,所提算法未显著增加计算复杂度,对幅度较大的非正态分布或统计特性未知的噪声有较强的抑止能力。
5.设计实现冰球式蓄冷空调系统。用时变系统来近似实际系统,采用遗忘和加权配合的辨识算法来预测能量。在系统实现上,利用组态王实现数据采集、友好人机界面设计,利用MATLAB完成复杂的预测算法,充分发挥组态软件在工业控制上组态方便、开发周期短和MATLAB强大数值计算功能的优势,两者之间通过DDE技术实现数据实时通信。
System identification plays an important role in high performance automation technologies such as adaptive control and learning control. More high quality of products is required, in nowadays plants, to meet the needs of people's production and living demands. It should be challenged for system identification when high control performance is pursued. The parameters in dynamics of many real systems are time-dependent, which requires that identification techniques are of the ability of tracking, and can estimate time-varying parameters. We also need robust identification algorithms owing to the presence of abrupt disturbances. Take into account the issues above mentioned, this thesis focuses on the following aspects:
1. The properties and deterministic convergence of recursive identification algorithm are considered, in which both a forgetting factor and weights have been incorporated.
2. Iterative learning identification algorithms are presented for estimating time-varying parameters of a class of discrete time-varying systems over a finite time interval. The theoretic properties of iterative learning projection and least squares algorithms are further discussed, and corresponding numerical simulations are presented. Two prototype algorithms of iterative learning identification, iterative learning Bayes and stochastic Newton algorithms, are proposed with detail. Different from the bounded convergence performance obtained by conventional tracking algorithms, perfect estimation for the time-varying unknowns is achieved through iterative learning, and the parameter estimation error is guaranteed to converge to zero over the entire time interval.
3. The approximate least l_1-norm recursive algorithm is presented for adaptive acoustic echo cancellation. The convergence performance of the algorithm is analyzed, by which the algorithm exhibits robustness with respect to bounded disturbances at steady-state. It is in turn ? useful for performance improvement of the acoustic echo cancellation. During duplex talk, the proposed algorithm works well without requiring Double-Talk Detector and without requiring more hardware in the system.
4. Based on the nonlinear PCA criterion, an iterative weighted least squares algorithm is presented for blind source separation. The iterative algorithm is for the purpose to approximate the l_p-norm index, and the theoretical properties of the algorithm are derived. In comparison with the conventional RLS algorithm, the proposed algorithm has comparable computational complexity, and is robust with respect to non-Gaussian disturbances or those with unknown statistical distribution.
5. An ice storage air condition system is designed and implemented. The real system is modeled as a time-varying system, and the energy is predicted by the identification algorithm with both forgetting and weighted factors. For the implementation, KingView is adopted for data acquisition, with friendly human machine interface design, and MATLAB is used to carry out the on-line prediction computing. In the ice storage system, DDE technique is applied for the real time data exchange between the configuration software and MATLAB.
引文
[1]Astrom K J,EykhoffP.System Identification-A Survey[J].Automatica,1971,7(2):123-162.
[2]Goodwin G C,Sin K S.Adaptive filtering prediction and control[M].Prentice-hall,INC.,Englewood Cliffs,New Jersey,1984.
[3]Chen H F,Guo L.Identification and Stochastic Adaptive Control[M].Boston:Birkhauser,1991.
[4]Zadeh LA.From Circuit Theory to System Theory[J].Proc,IRE,1962,50(5):856-865.
[5]Ljung L.System Identification--Theory for the User,Second Edition[M].New Jersey:Prentice Hall,1999.
[6]方崇智,萧德云.过程辨识[M].北京:清华大学出版社,2001.
[7]Sorenson H W.Least-squares Estimation:from Gauss to Kalman[J].IEEE Spectrum,1970,7:63-68.
[8]Weiner N.Extrapolation,Interpolation and Smoothing of Stationary Time Series,with Engineering Applications[M].MA:MIT Press,1949.
[9]Kailath T.A view of three decades of linear filtering theory[J].IEEE Transactions of Information Theory,1974,20:146-181.
[10]Van Trees H L.Detection,Estimation,and Modulation Theory,Part Ⅰ[J].IEEE Transactions of Information Theory,1968,14:612-613.
[11]Van Trees H L.Detection,Estimation,and Modulation Theory,Part Ⅱ[J].IEEE Transactions of Information Theory,1972,18:650-651.
[12]Stiffler J J.Theory of Synchronous Communication[J].IEEE Transactions of Information Theory,1972,18:218-219.
[13]Lindsey W C.Synchronization Systems in Communication and Control[M].Prentice Hall:Englewood Cliffs,NJ,1972.
[14]Kalman R E.A New Approach to Linear Filtering and Prediction Problems[J].Journal of Basic Engineering,1960,82:34-45.
[15]Kalman R E.On the General Theory of Control System[J].Proceedings of First International Congress,IFAC,Moscow,USSR,1960,481-492.
[16]Haykin S.Adaptive Filter Theory,3rd ed[M].Prentice-Hall,Upper Saddle River,N.J.,1996.
[17]Willsky A.Digital Signal Processing and Control and Estimation Theory:Points of Tangency,Areas of Intersection,and Parallel Directions[M].MIT Press,Cambridge,Mass,1979.
[18]谢胜利,何昭水,高鹰.信号处理的自适应理论[M].北京:科学出版社,2006.
[19]Niedzwiecki M.Identification of time-varying processes[M].New York:Wiley,2000.
[20]Robbins H,Monro S.A stochastic approximation method[J].Annals of Mathematical Statistics,1951,22:400-407.
[21]Tsypkin Y Z.Adaption and Learning in Automatic Systems[M].Academic Press,New York,1971.
[22]Tsypkin Y Z.Foundation of the Theory of Learning Systems[M].Academic Press,New York,1973.
[23]Solo V.The limit behavior of LMS[J].IEEE Transactions on Acoustics,Speech,and Signal Processing, 1989,37(12):1909-1922.
[24]Widrow B,Hoff M E.Adaptive switching circuits[R].IRE WESCON Convention Record Part Ⅳ,1960,96-104.
[25]Ljung L,Gunnarsson S.Adaptation and tracking in system identification-a survey[J].Automatica,1990,26(1):7-21.
[26]Benveniste A.Design of adaptive algorithms for the tracking of time-varying systems[J].International Journal of Adaptive Control and Signal Processing,1987,1(1):3-29.
[27]Widrow B,Stearns S.Adaptive Signal Processing[M].Prentice-Hall,Englewood Cliffs,NJ,1985.
[28]Feuer A,Weinstein E.Convergence analysis of LMS filters with uneorrelated Gaussian data[J].IEEE Transactions on Acoustics,Speech,Signal Processing,1985,33:222-230.
[29]Weiss A,Mitra D.Digital adaptive filters:Conditions for convergence,rotes of convergence,effects of noise and errors arising from the implementation[J].IEEE Transaction on Information Theory,1979,25(6):637-652.
[30]Guo L,Meyn S P.Adaptive control for time-varying systems:A combination of Martingale and Markov chain techniques[J].International Journal of Adaptive Control and Signal Processing,1989,3(1):1-14.
[31]Guo L.On adaptive stabilization of time-varying stochastic systems[J].SIAM Journal on Control and Optimization,1990,28(6):1432-1451.
[32]Chen H F,Caines P E.On the adaptive control of a class of systems with random parameters and disturbances[J].Automatiea,1985,21(6):737-741.
[33]Ljung L.Analysis ofrecttrsive stochastic algorithms[J].IEEE Transactions on Automatic Control,1997,22(4):551-575.
[34]Macchi O,Eweda E.Second-order convergence analysis of stochastic adaptive linear filtering[J].IEEE Transactions on Automatic Control,1983,28(1):76-85.
[35]Eweda E,Macehi O.Tracking error bounds of adaptive nonstationary filtering[J].Automatiea,1985,21(3):293-302.
[36]Macchi O.Optimization of adaptive identification for time-varying filters[J].IEEE Transaction on Automatic Control,1986,31(3):283-287.
[37]Eleftheriou E,Falconer D D.Traeldng properties and steady-state performance of RLS adaptive filter algorithms[J].IEEE Transactions on Acoustics,Speech,and Signal Processing,1986,34(5):1097-1110.
[38]Fortesque T R,Kershenbaum L S,Ydstie B F.Implementation of self-tuning regulators with variable forgetting factors[J].Automatica,1981,17:831-835.
[39]Salgado M E,Goodwin G C,Middletton R H.Modified least squares algorithm incorporating exponential resetting and forgetting[J].International Journal of Control,1988,47(2):477-491.
[40]Widrow B,et al.Stationary and nonstationary learning characteristics of the LMS adaptive filter[J].IEEE Proceedings,1976,64(8):1151-1162.
[41]Eweda E,Macchi O.Convergence of the RLS and LMS adaptive filters[J].IEEE Transactions on Circuits and Systems,1987,34(7):799-803.
[42]Niedzwiecki M.On tracking characteristics of weighted least squares estimation applied to nonstationary system identification[J].IEEE Transaction on Automatic Control,1988,33(1):96-98.
[43]Guo L,Ljung L,Priouret P.Performance analysis of forgetting factor RLS algorithms[J].International Journal of Adaptive Control and Signal Processing,1993,7:525-537.
[44]Widrow B,Wallach E.On the statistical efficiency of the LMS algorithm with nonstationary inputs[J]. IEEE Transaction on Information Theory,1984,30(2):211-221.
[45]He Y C.On the stochastic approximation method and optimal filtering theory[J].Journal of Mathematical Analysis and Applications,1963,6(1):152-154.
[46]Astrom K J,Eykhoff P.System Identification-A Survey[J].Automatica,7(2):123-162,1971.
[47]Guo L.Stability of Recursive Stochastic Tracking Algorithms[J].SIAM Journal on Control and Optimization,1994,32(5):1195-1225.
[48]Ravikanth R,Meyn S P.Bounds on achievable performance in the identification and adaptive control of time-varying systems[J].IEEE Transaction on Automatic Control,1999,44(4):670-682.
[49]Niedzwiecki M.On the lower smoothing bound in identification of time-varying systems[J].Automatica,2008,44(2):459-464.
[50]Shen M F,Song R,Ting K H,Chart F H Y.A method for identifying non-Gaussian parametric model with time-varying coefficients[C].2003 IEEE International Conference on Acoustics,Speech,and Signal Processing,2003,6:Ⅵ629-Ⅵ632.
[51]Momen A,Masajedian S,Miar Y,Chime J D.A new method in blind estimation of fast time-varying channels based on subspace method[C].2004 Canadian Conference on Electrical and Computer Engineering,2004,4:1957-1961.
[52]Tugnait J K,Luo W.Blind identification of time-varying channels using multistep linear predictors[J].IEEE Transactions on Signal Processing,2004,52(6):1739-1749.
[53]Tugnait J K,Luo W.Blind space-time multiuser channel estimation in time-varying DS-CDMA systems [J].IEEE Transactions on Vehicular Technology,2006,55(1):207-218.
[54]Niedzwiecki M.On 'Cheap Smoothing' Opportunities in Identification of Time-varying Systems[R].Technical Report # ETI-2/2007.
[55]Niedzwiecki M,Kaczmarek P.Generalized adaptive notch and comb filters for identification of quasi-periodically varying systems[J].IEEE Transaction on Signal Processing,2005,53(12):4599-4609.
[56]Niedzwiecki M,Kaczmarek P.Estimation and tracking of complex-valued quasi-periodically varying systems[J].Auyomatiea,2005,41(9):1503-1516.
[57]Niedzwiecki M,Kaczmarek P.Tracking analysis of a generalized adaptive notch filter[J].IEEE Transaction on Signal Processing,2006,54(1):304-314.
[58]Niedzwiecki M,Kaczmarek P.Generalized adaptive notch filter with a self-optimization capability[J].IEEE Transaction on Signal Processing,2006,54(11):4185-4193.
[59]Niedzwiecki M,Sobocinski.A simple way of increasing estimation accuracy of generalized adaptive notch filters[J].IEEE Signal Processing Letters,2007,14(3):217-220.
[60]Niedzwiecki M,Soboeinski.On Tracking Properties of Real-Valued generalized Adaptive Notch Filters [J].IEEE Transaction on Signal Processing,2007,55(5):1688-1695.
[61]Sanyal S,Kukreja S L,Perreault E J,Westwiek D T.Identification of Linear Time Varying Systems using Basis Pursuit[C].Proceedings of the 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference,Shanghai,China,2005:22-25.
[62]Vipin B V,Parthasarathy H.Parameter estimation for time varying nonlinear circuit from state analysis and simulation[J].Communications m Nonlinear Science and Numerical Simulation,2008,13(4):839-846.
[63]Rajagopalan V,Chakraborty S,Ray A.Estimation of slowly varying parameters in nonlinear systems via symbolic dynamic filtering[J].Signal Processing,2008,88(2):339-348.
[64]Rajagopalan V,Ray A.Symbolic time series analysis via wavelet-based partitioning[J].Signal Processing,2006,86(11):3309-3320.
[65]Wang G Y.The Most General Time-Varying Filter Bank and Time-Varying Lapped Transforms[J].IEEE Transaction on Signal Processing,2006,54(10):3775-3789.
[66]Alamo T,Bravo J M,Camacho E F.Guaranteed state estimation by zonotopes[J].Automatica,2005,41(6):1035-1043.
[67]Bravo J M,Alamo T,Camaeho E F.Bounded error identification of systems with time-varying parameters[J].IEEE Transaction on Automatic Control,2006,51(7):1144-1150.
[68]程乾生.信号数字处理的数学原理[M].北京:石油工业出版社,1993.
[69]Huber P J.Robust Statistics[M].New York:Wiley,1981.
[70]Ozeki K,Umeda T.An adaptive filtering using an orthogonal projection to an affine subspace and its properties[J].IECE,1984,J67-A(2):126-132.
[71]Cioffi J M,Kailath T.Windowed fast transversal filters for recursive least squares adaptive filtering[J].IEEE Transactions on Acoustics,Speech,Signal Processing,1985,33(3):607-625.
[72]Sun W,Li H B.Blind channel estimation and detection for space-time coded CDMA in ISI channels[J].Digital Signal Processing,2007,17(1):280-296.
[73]Hoyer P O,Hyvarinen A.Independent component analysis applied to feature extraction from colour and stereo images[J].Network:Computation in Neural Systems,2000,11(3):191-210.
[74]Park H J,Lee T W.Capturing nonlinear dependencies in natural images using ICA and mixture of Laplacian distribution[J].Neurocomputing,2006,69(13):1513-1528.
[75]Barros A K,Ohnishi N.Single channel speech enhancement by efficient coding[J].Signal Processing,2005,85(9):1805-1812.
[76]Asano F,et al.Combined approach of array processing and independent component analysis for blind separation of acoustic signals[J].IEEE Trans.on Speech and Audio Processing,2003,11(3):204-215.
[77]DiPietroPaolo D,et al..Comparison of BSS algorithms in SMCG data.International Congress Series[J].2007,1300:213-216.
[78]Besserve M,et al..Prediction of cognitive states using MEG and blind source separation[J].International Congress Series,2007,1300:205-208.
[79]Tang A C,et al..Validation of SOBI components from high-density EEG[J].NeuroImage,2005,25(2):539-553.
[80]Herault J,Jutten C.Space or time adaptive signal processing by neural network models[C].Neural Network for Computing.American institute for Physics.New York,1986.
[81]Tong L,et al..Indeterminacy and identifiability of blind identification[J].IEEE Trans.On Circuits and Systems,1991,38(5):499-509.
[82]Comon P,et al..Independent component analysis,A new concept?[J].Signal Processing,1994,36(3):287-314.
[83]Bell A J,Sejnowski T J.An information-maximization approach to blind separation and blind deconvolution[J].Neural Computation,1995,7(6):1129-1159.
[84]Cardoso J F.Irffomax and maximum likelihood for blind source separation[J].IEEE Signal Processing Letters,1997,4(4):112-114.
[85]Cichocki A,et al..A new online adaptive learning algorithm for blind separation of source signals[C]. In:ISANN94,Taiwan,1994:406-411.
[86]Cardoso J F,Laheld B H.Equivariant adaptive source separation[J].IEEE Trans.on Signal Processing,1996,44(12):3017-3029.
[87]Amari S I,Cichocki A,Yang H H.A new learning algorithm for blind signal separation[J].Advances in Neural Information Processing Systems,1996,8:757-764.
[88]Yang H H,Amari S I.Adaptive online learning algorithm for blind separation:maximum entropy and minimum mutual information[J].Neural Network,1997,9(7):1457-1481.
[89]Amari S I.Natural gradient works efficiently in learning[J].Neural Computation,1998,10(2):251-276.
[90]Hyvarinen A.Fast and robust fixed-point algorithms for independent component analysis[J].IEEE Trans.on Neural Networks,1999,10(3):626-634.
[91]Hyvarine A,Oja E.A fast fixed-point algorithm for independent component analysis[J].Neural Computation,1997,9(7):1483-1492.
[92]Girolarmi M,Fyfe C.An extended exploratory projection pursuit network with linear and nonlinear anti-Hebbian lateral eormections applied to the cocktail party problem[J].Neural Network,1997,10(9):1607-1618.
[93]Karhunen J,Oja E,et al.A class of neural networks for independent component analysis[J].IEEE Trans.on Neural Networks,1997,8(3):486-504.
[94]Yang B.Projection approximation subspace tracking[J].IEEE Trans.on Signal Processing,1995,43(1):95-107.
[95]Pajunen P,Karhunen J.Least-squares methods for blind source separation based on nonlinear PCA[J].Int J of Neural Systems,1998,8(12):601-612.
[96]Karhunen J,Pajunen P,Oja E.The nonlinear PCA criterion in blind source separation:Relations with other approaches[J].Neuroeomputing,1998,22(1):5-20.
[97]孙明轩.参数跟踪算法的收敛性质[J].西安工业学院学报,1994,14(4):16-22.
[98]Lozano L R.Independent Tracking and Regulation Adaptive Control with forgetting Factor[J].Automatica,1982,18(4):455-459.
[99]孙明轩,黄宝健.迭代学习控制[M].北京:国防工业出版社,1999.
[100]Sun M,He X.Iterative Learning Identification and Control of Discrete Time-Varying Systems[C].Proceedings of the 26th Chinese Control Conference(CCC07),Zhangjiajie,China,2007:520-524.
[101]Ljung L,Soderstrom T.Theory and Practice of Recursive Identification[M].MA:MIT Press,Cambridge,1983.
[102]Sankaran S G.Stereophonic acoustic echo cancellation using NLMS with orthogonal correction factor [A].Proceedings of International Workshop on Acoustic Echo and Noise Control,1999:40-43.
[103]Buchner H,et al.Robust extended multidelay filter and double-talk detector for acoustic echo cancellation[J].IEEE Transaction on Audio,Speech and Language Processing,2006,14(5):1633-1644.
[104]ITU-T Recommendation G.168,Echo Cancellers,2000.
[105]Asharif M R,Hayashi T,Yamashita K.Correlation LMS algorithm and its application to Double-Talk echo canceling[J].Electronics Letters,1999,41(3):194-195.
[106]Xu L.Least mean square error reconstruction principle for self-organizing neural nets[J].Neural Networks,1993,6:627-648.
[2]Goodwin G C,Sin K S.Adaptive filtering prediction and control[M].Prentice-hall,INC.,Englewood Cliffs,New Jersey,1984.
[3]Chen H F,Guo L.Identification and Stochastic Adaptive Control[M].Boston:Birkhauser,1991.
[4]Zadeh LA.From Circuit Theory to System Theory[J].Proc,IRE,1962,50(5):856-865.
[5]Ljung L.System Identification--Theory for the User,Second Edition[M].New Jersey:Prentice Hall,1999.
[6]方崇智,萧德云.过程辨识[M].北京:清华大学出版社,2001.
[7]Sorenson H W.Least-squares Estimation:from Gauss to Kalman[J].IEEE Spectrum,1970,7:63-68.
[8]Weiner N.Extrapolation,Interpolation and Smoothing of Stationary Time Series,with Engineering Applications[M].MA:MIT Press,1949.
[9]Kailath T.A view of three decades of linear filtering theory[J].IEEE Transactions of Information Theory,1974,20:146-181.
[10]Van Trees H L.Detection,Estimation,and Modulation Theory,Part Ⅰ[J].IEEE Transactions of Information Theory,1968,14:612-613.
[11]Van Trees H L.Detection,Estimation,and Modulation Theory,Part Ⅱ[J].IEEE Transactions of Information Theory,1972,18:650-651.
[12]Stiffler J J.Theory of Synchronous Communication[J].IEEE Transactions of Information Theory,1972,18:218-219.
[13]Lindsey W C.Synchronization Systems in Communication and Control[M].Prentice Hall:Englewood Cliffs,NJ,1972.
[14]Kalman R E.A New Approach to Linear Filtering and Prediction Problems[J].Journal of Basic Engineering,1960,82:34-45.
[15]Kalman R E.On the General Theory of Control System[J].Proceedings of First International Congress,IFAC,Moscow,USSR,1960,481-492.
[16]Haykin S.Adaptive Filter Theory,3rd ed[M].Prentice-Hall,Upper Saddle River,N.J.,1996.
[17]Willsky A.Digital Signal Processing and Control and Estimation Theory:Points of Tangency,Areas of Intersection,and Parallel Directions[M].MIT Press,Cambridge,Mass,1979.
[18]谢胜利,何昭水,高鹰.信号处理的自适应理论[M].北京:科学出版社,2006.
[19]Niedzwiecki M.Identification of time-varying processes[M].New York:Wiley,2000.
[20]Robbins H,Monro S.A stochastic approximation method[J].Annals of Mathematical Statistics,1951,22:400-407.
[21]Tsypkin Y Z.Adaption and Learning in Automatic Systems[M].Academic Press,New York,1971.
[22]Tsypkin Y Z.Foundation of the Theory of Learning Systems[M].Academic Press,New York,1973.
[23]Solo V.The limit behavior of LMS[J].IEEE Transactions on Acoustics,Speech,and Signal Processing, 1989,37(12):1909-1922.
[24]Widrow B,Hoff M E.Adaptive switching circuits[R].IRE WESCON Convention Record Part Ⅳ,1960,96-104.
[25]Ljung L,Gunnarsson S.Adaptation and tracking in system identification-a survey[J].Automatica,1990,26(1):7-21.
[26]Benveniste A.Design of adaptive algorithms for the tracking of time-varying systems[J].International Journal of Adaptive Control and Signal Processing,1987,1(1):3-29.
[27]Widrow B,Stearns S.Adaptive Signal Processing[M].Prentice-Hall,Englewood Cliffs,NJ,1985.
[28]Feuer A,Weinstein E.Convergence analysis of LMS filters with uneorrelated Gaussian data[J].IEEE Transactions on Acoustics,Speech,Signal Processing,1985,33:222-230.
[29]Weiss A,Mitra D.Digital adaptive filters:Conditions for convergence,rotes of convergence,effects of noise and errors arising from the implementation[J].IEEE Transaction on Information Theory,1979,25(6):637-652.
[30]Guo L,Meyn S P.Adaptive control for time-varying systems:A combination of Martingale and Markov chain techniques[J].International Journal of Adaptive Control and Signal Processing,1989,3(1):1-14.
[31]Guo L.On adaptive stabilization of time-varying stochastic systems[J].SIAM Journal on Control and Optimization,1990,28(6):1432-1451.
[32]Chen H F,Caines P E.On the adaptive control of a class of systems with random parameters and disturbances[J].Automatiea,1985,21(6):737-741.
[33]Ljung L.Analysis ofrecttrsive stochastic algorithms[J].IEEE Transactions on Automatic Control,1997,22(4):551-575.
[34]Macchi O,Eweda E.Second-order convergence analysis of stochastic adaptive linear filtering[J].IEEE Transactions on Automatic Control,1983,28(1):76-85.
[35]Eweda E,Macehi O.Tracking error bounds of adaptive nonstationary filtering[J].Automatiea,1985,21(3):293-302.
[36]Macchi O.Optimization of adaptive identification for time-varying filters[J].IEEE Transaction on Automatic Control,1986,31(3):283-287.
[37]Eleftheriou E,Falconer D D.Traeldng properties and steady-state performance of RLS adaptive filter algorithms[J].IEEE Transactions on Acoustics,Speech,and Signal Processing,1986,34(5):1097-1110.
[38]Fortesque T R,Kershenbaum L S,Ydstie B F.Implementation of self-tuning regulators with variable forgetting factors[J].Automatica,1981,17:831-835.
[39]Salgado M E,Goodwin G C,Middletton R H.Modified least squares algorithm incorporating exponential resetting and forgetting[J].International Journal of Control,1988,47(2):477-491.
[40]Widrow B,et al.Stationary and nonstationary learning characteristics of the LMS adaptive filter[J].IEEE Proceedings,1976,64(8):1151-1162.
[41]Eweda E,Macchi O.Convergence of the RLS and LMS adaptive filters[J].IEEE Transactions on Circuits and Systems,1987,34(7):799-803.
[42]Niedzwiecki M.On tracking characteristics of weighted least squares estimation applied to nonstationary system identification[J].IEEE Transaction on Automatic Control,1988,33(1):96-98.
[43]Guo L,Ljung L,Priouret P.Performance analysis of forgetting factor RLS algorithms[J].International Journal of Adaptive Control and Signal Processing,1993,7:525-537.
[44]Widrow B,Wallach E.On the statistical efficiency of the LMS algorithm with nonstationary inputs[J]. IEEE Transaction on Information Theory,1984,30(2):211-221.
[45]He Y C.On the stochastic approximation method and optimal filtering theory[J].Journal of Mathematical Analysis and Applications,1963,6(1):152-154.
[46]Astrom K J,Eykhoff P.System Identification-A Survey[J].Automatica,7(2):123-162,1971.
[47]Guo L.Stability of Recursive Stochastic Tracking Algorithms[J].SIAM Journal on Control and Optimization,1994,32(5):1195-1225.
[48]Ravikanth R,Meyn S P.Bounds on achievable performance in the identification and adaptive control of time-varying systems[J].IEEE Transaction on Automatic Control,1999,44(4):670-682.
[49]Niedzwiecki M.On the lower smoothing bound in identification of time-varying systems[J].Automatica,2008,44(2):459-464.
[50]Shen M F,Song R,Ting K H,Chart F H Y.A method for identifying non-Gaussian parametric model with time-varying coefficients[C].2003 IEEE International Conference on Acoustics,Speech,and Signal Processing,2003,6:Ⅵ629-Ⅵ632.
[51]Momen A,Masajedian S,Miar Y,Chime J D.A new method in blind estimation of fast time-varying channels based on subspace method[C].2004 Canadian Conference on Electrical and Computer Engineering,2004,4:1957-1961.
[52]Tugnait J K,Luo W.Blind identification of time-varying channels using multistep linear predictors[J].IEEE Transactions on Signal Processing,2004,52(6):1739-1749.
[53]Tugnait J K,Luo W.Blind space-time multiuser channel estimation in time-varying DS-CDMA systems [J].IEEE Transactions on Vehicular Technology,2006,55(1):207-218.
[54]Niedzwiecki M.On 'Cheap Smoothing' Opportunities in Identification of Time-varying Systems[R].Technical Report # ETI-2/2007.
[55]Niedzwiecki M,Kaczmarek P.Generalized adaptive notch and comb filters for identification of quasi-periodically varying systems[J].IEEE Transaction on Signal Processing,2005,53(12):4599-4609.
[56]Niedzwiecki M,Kaczmarek P.Estimation and tracking of complex-valued quasi-periodically varying systems[J].Auyomatiea,2005,41(9):1503-1516.
[57]Niedzwiecki M,Kaczmarek P.Tracking analysis of a generalized adaptive notch filter[J].IEEE Transaction on Signal Processing,2006,54(1):304-314.
[58]Niedzwiecki M,Kaczmarek P.Generalized adaptive notch filter with a self-optimization capability[J].IEEE Transaction on Signal Processing,2006,54(11):4185-4193.
[59]Niedzwiecki M,Sobocinski.A simple way of increasing estimation accuracy of generalized adaptive notch filters[J].IEEE Signal Processing Letters,2007,14(3):217-220.
[60]Niedzwiecki M,Soboeinski.On Tracking Properties of Real-Valued generalized Adaptive Notch Filters [J].IEEE Transaction on Signal Processing,2007,55(5):1688-1695.
[61]Sanyal S,Kukreja S L,Perreault E J,Westwiek D T.Identification of Linear Time Varying Systems using Basis Pursuit[C].Proceedings of the 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference,Shanghai,China,2005:22-25.
[62]Vipin B V,Parthasarathy H.Parameter estimation for time varying nonlinear circuit from state analysis and simulation[J].Communications m Nonlinear Science and Numerical Simulation,2008,13(4):839-846.
[63]Rajagopalan V,Chakraborty S,Ray A.Estimation of slowly varying parameters in nonlinear systems via symbolic dynamic filtering[J].Signal Processing,2008,88(2):339-348.
[64]Rajagopalan V,Ray A.Symbolic time series analysis via wavelet-based partitioning[J].Signal Processing,2006,86(11):3309-3320.
[65]Wang G Y.The Most General Time-Varying Filter Bank and Time-Varying Lapped Transforms[J].IEEE Transaction on Signal Processing,2006,54(10):3775-3789.
[66]Alamo T,Bravo J M,Camacho E F.Guaranteed state estimation by zonotopes[J].Automatica,2005,41(6):1035-1043.
[67]Bravo J M,Alamo T,Camaeho E F.Bounded error identification of systems with time-varying parameters[J].IEEE Transaction on Automatic Control,2006,51(7):1144-1150.
[68]程乾生.信号数字处理的数学原理[M].北京:石油工业出版社,1993.
[69]Huber P J.Robust Statistics[M].New York:Wiley,1981.
[70]Ozeki K,Umeda T.An adaptive filtering using an orthogonal projection to an affine subspace and its properties[J].IECE,1984,J67-A(2):126-132.
[71]Cioffi J M,Kailath T.Windowed fast transversal filters for recursive least squares adaptive filtering[J].IEEE Transactions on Acoustics,Speech,Signal Processing,1985,33(3):607-625.
[72]Sun W,Li H B.Blind channel estimation and detection for space-time coded CDMA in ISI channels[J].Digital Signal Processing,2007,17(1):280-296.
[73]Hoyer P O,Hyvarinen A.Independent component analysis applied to feature extraction from colour and stereo images[J].Network:Computation in Neural Systems,2000,11(3):191-210.
[74]Park H J,Lee T W.Capturing nonlinear dependencies in natural images using ICA and mixture of Laplacian distribution[J].Neurocomputing,2006,69(13):1513-1528.
[75]Barros A K,Ohnishi N.Single channel speech enhancement by efficient coding[J].Signal Processing,2005,85(9):1805-1812.
[76]Asano F,et al.Combined approach of array processing and independent component analysis for blind separation of acoustic signals[J].IEEE Trans.on Speech and Audio Processing,2003,11(3):204-215.
[77]DiPietroPaolo D,et al..Comparison of BSS algorithms in SMCG data.International Congress Series[J].2007,1300:213-216.
[78]Besserve M,et al..Prediction of cognitive states using MEG and blind source separation[J].International Congress Series,2007,1300:205-208.
[79]Tang A C,et al..Validation of SOBI components from high-density EEG[J].NeuroImage,2005,25(2):539-553.
[80]Herault J,Jutten C.Space or time adaptive signal processing by neural network models[C].Neural Network for Computing.American institute for Physics.New York,1986.
[81]Tong L,et al..Indeterminacy and identifiability of blind identification[J].IEEE Trans.On Circuits and Systems,1991,38(5):499-509.
[82]Comon P,et al..Independent component analysis,A new concept?[J].Signal Processing,1994,36(3):287-314.
[83]Bell A J,Sejnowski T J.An information-maximization approach to blind separation and blind deconvolution[J].Neural Computation,1995,7(6):1129-1159.
[84]Cardoso J F.Irffomax and maximum likelihood for blind source separation[J].IEEE Signal Processing Letters,1997,4(4):112-114.
[85]Cichocki A,et al..A new online adaptive learning algorithm for blind separation of source signals[C]. In:ISANN94,Taiwan,1994:406-411.
[86]Cardoso J F,Laheld B H.Equivariant adaptive source separation[J].IEEE Trans.on Signal Processing,1996,44(12):3017-3029.
[87]Amari S I,Cichocki A,Yang H H.A new learning algorithm for blind signal separation[J].Advances in Neural Information Processing Systems,1996,8:757-764.
[88]Yang H H,Amari S I.Adaptive online learning algorithm for blind separation:maximum entropy and minimum mutual information[J].Neural Network,1997,9(7):1457-1481.
[89]Amari S I.Natural gradient works efficiently in learning[J].Neural Computation,1998,10(2):251-276.
[90]Hyvarinen A.Fast and robust fixed-point algorithms for independent component analysis[J].IEEE Trans.on Neural Networks,1999,10(3):626-634.
[91]Hyvarine A,Oja E.A fast fixed-point algorithm for independent component analysis[J].Neural Computation,1997,9(7):1483-1492.
[92]Girolarmi M,Fyfe C.An extended exploratory projection pursuit network with linear and nonlinear anti-Hebbian lateral eormections applied to the cocktail party problem[J].Neural Network,1997,10(9):1607-1618.
[93]Karhunen J,Oja E,et al.A class of neural networks for independent component analysis[J].IEEE Trans.on Neural Networks,1997,8(3):486-504.
[94]Yang B.Projection approximation subspace tracking[J].IEEE Trans.on Signal Processing,1995,43(1):95-107.
[95]Pajunen P,Karhunen J.Least-squares methods for blind source separation based on nonlinear PCA[J].Int J of Neural Systems,1998,8(12):601-612.
[96]Karhunen J,Pajunen P,Oja E.The nonlinear PCA criterion in blind source separation:Relations with other approaches[J].Neuroeomputing,1998,22(1):5-20.
[97]孙明轩.参数跟踪算法的收敛性质[J].西安工业学院学报,1994,14(4):16-22.
[98]Lozano L R.Independent Tracking and Regulation Adaptive Control with forgetting Factor[J].Automatica,1982,18(4):455-459.
[99]孙明轩,黄宝健.迭代学习控制[M].北京:国防工业出版社,1999.
[100]Sun M,He X.Iterative Learning Identification and Control of Discrete Time-Varying Systems[C].Proceedings of the 26th Chinese Control Conference(CCC07),Zhangjiajie,China,2007:520-524.
[101]Ljung L,Soderstrom T.Theory and Practice of Recursive Identification[M].MA:MIT Press,Cambridge,1983.
[102]Sankaran S G.Stereophonic acoustic echo cancellation using NLMS with orthogonal correction factor [A].Proceedings of International Workshop on Acoustic Echo and Noise Control,1999:40-43.
[103]Buchner H,et al.Robust extended multidelay filter and double-talk detector for acoustic echo cancellation[J].IEEE Transaction on Audio,Speech and Language Processing,2006,14(5):1633-1644.
[104]ITU-T Recommendation G.168,Echo Cancellers,2000.
[105]Asharif M R,Hayashi T,Yamashita K.Correlation LMS algorithm and its application to Double-Talk echo canceling[J].Electronics Letters,1999,41(3):194-195.
[106]Xu L.Least mean square error reconstruction principle for self-organizing neural nets[J].Neural Networks,1993,6:627-648.