基于非稳定性的独立分量分析及四种新稳定性测度方法
|
摘要
独立分量分析(Independent Component Analysis, ICA)作为一种变换方法,在神经网络、盲源分离、统计分析等领域的有着广泛的应用,它在过去二十年尤其是近十年内得到了长足的发展。但是,现有的ICA方法不适用于重拖尾数据的处理问题。重拖尾信号常常被用Alpha稳定分布来描述,在计算机、物理、化学、经济、金融、地理等领域有广泛的应用,因此ICA方法在重拖尾数据中的不适用性大大局限了它的应用范围。
本文中我们提出了一种新的基于非稳定性的独立分量分析方法来解决这一问题,该方法通过极大化样本的非稳定性找出独立分量。同时我们还提出了极大化非稳定性准则来代替经典独立分量分析方法中的极大化非高斯性准则,并认为后者是前者的一种特殊情况。为了测量样本中的稳定性,我们还提出了一种稳定性测度方法和三种近似方法。最后我们设计了一种数值的梯度算法来实现这种新的ICA方法。
受广义中心极限定理的启发,在本文中我们提出了一种新的极大化非稳定性目标方程的独立主分量方法,这一方法将适用于重拖尾信号的盲源分离问题。这种新方法挑战了经典ICA方法中的极大化非高斯性的原则,指出极大化非高斯性原则应该被极大化非稳定性的原则所替代。在一定的约束下,我们还证明了经典的独立分量分析方法是基于非稳定性的独立分量分析方法的一种特例。为了测量数据的稳定性,我们引入了一种新的测度方法,称为Alpha负熵。在文章中,我们讨论了这种新方法作为稳定性测度理论上的优越性,并将它用做基于非稳定性的独立分量分析的目标方程。但是由于Alpha负熵计算复杂的缺点,我们又进一步提出了三种近似的稳定性测度方法：分数矩方法、对数矩方法、极值统计方法。在文章的最后,我们设计了一种数值梯度算法来实现极大化非稳定性的独立分量分析方法用以处理重拖尾信号的盲源分离问题。实验第一部分对分数矩、对数矩和极值统计方法和微分熵做了对比。实验第二部分分别利用Alpha负熵、分数矩方法、对数矩方法和极值统计方法做目标方程给出了重拖尾信号数据的盲源分离结果。结果显示了该方法在处理重拖尾信号上的成功,分离方向准确。
Independent Component Analysis (ICA), as a represent method, has wide applications in fields like neural network, blind source separation, statistical analysis etc., and it has been highly developed in the past twenty years especially recent ten years. But, ICA method is not suitable to solve problems concerning about heavy-tailed signals, which are always modeled byα-stable distribution and have wide applications in computer science, physics, chemistry, economics, finance, geography and so on.
We propose a new ICA method based on non-stability in this paper, which finds the independent components through the maximum of the non-stability of the sample. Also, we propose the maximum non-stability principle to take place of the maximum non-Gaussinity one in the classic ICA method, and point out that the later one is a special case of the former one. For the measure of stability, a brand new measure, called Alpha-negentropy, and three approximation stability measures are also proposed. In the end, we design a numerical algorithm based on gradient to realize our new ICA method.
Motivated by the Generalized Central Limit Theorem (GCLT), an important extension of classical CLT, we propose a new approach for ICA by maximizing the non-stability contrast function, which should suit heavy-tailed signal source separation problems. In this paper, we challenged the maximum non-Gaussinity principle and changed it into maximum non-stability principle. Also, we demonstrate that the classical ICA based on maximization of non-Gaussianity is a special case of the new approach of ICA which is based on maximization of non-stability with certain constraints. To be able to quantify non-stability, we propose a new measure of stability namely Alpha-negentropy, which is theoretical heretic and designed to be the contrast function. Three other approximation non-stability measures, FLOM method, logarithmic moment method and extreme value method are also introduced since Alpha-negentropy is computational complex. At the end of this paper, a numerical gradient algorithm for the maximization of the Alpha-negentropy with the objective of source separation of heavy-tailed signals is designed. The first part of experiment compares FLOM, logarithmic moment, extreme value method with the differential entropy. The other part evaluates this new ICA method using four non-stability methods as contrast functions, which show that ICA by maximum of non-stability performs successfully in heavy-tailed source separation problems.
本文中我们提出了一种新的基于非稳定性的独立分量分析方法来解决这一问题,该方法通过极大化样本的非稳定性找出独立分量。同时我们还提出了极大化非稳定性准则来代替经典独立分量分析方法中的极大化非高斯性准则,并认为后者是前者的一种特殊情况。为了测量样本中的稳定性,我们还提出了一种稳定性测度方法和三种近似方法。最后我们设计了一种数值的梯度算法来实现这种新的ICA方法。
受广义中心极限定理的启发,在本文中我们提出了一种新的极大化非稳定性目标方程的独立主分量方法,这一方法将适用于重拖尾信号的盲源分离问题。这种新方法挑战了经典ICA方法中的极大化非高斯性的原则,指出极大化非高斯性原则应该被极大化非稳定性的原则所替代。在一定的约束下,我们还证明了经典的独立分量分析方法是基于非稳定性的独立分量分析方法的一种特例。为了测量数据的稳定性,我们引入了一种新的测度方法,称为Alpha负熵。在文章中,我们讨论了这种新方法作为稳定性测度理论上的优越性,并将它用做基于非稳定性的独立分量分析的目标方程。但是由于Alpha负熵计算复杂的缺点,我们又进一步提出了三种近似的稳定性测度方法：分数矩方法、对数矩方法、极值统计方法。在文章的最后,我们设计了一种数值梯度算法来实现极大化非稳定性的独立分量分析方法用以处理重拖尾信号的盲源分离问题。实验第一部分对分数矩、对数矩和极值统计方法和微分熵做了对比。实验第二部分分别利用Alpha负熵、分数矩方法、对数矩方法和极值统计方法做目标方程给出了重拖尾信号数据的盲源分离结果。结果显示了该方法在处理重拖尾信号上的成功,分离方向准确。
Independent Component Analysis (ICA), as a represent method, has wide applications in fields like neural network, blind source separation, statistical analysis etc., and it has been highly developed in the past twenty years especially recent ten years. But, ICA method is not suitable to solve problems concerning about heavy-tailed signals, which are always modeled byα-stable distribution and have wide applications in computer science, physics, chemistry, economics, finance, geography and so on.
We propose a new ICA method based on non-stability in this paper, which finds the independent components through the maximum of the non-stability of the sample. Also, we propose the maximum non-stability principle to take place of the maximum non-Gaussinity one in the classic ICA method, and point out that the later one is a special case of the former one. For the measure of stability, a brand new measure, called Alpha-negentropy, and three approximation stability measures are also proposed. In the end, we design a numerical algorithm based on gradient to realize our new ICA method.
Motivated by the Generalized Central Limit Theorem (GCLT), an important extension of classical CLT, we propose a new approach for ICA by maximizing the non-stability contrast function, which should suit heavy-tailed signal source separation problems. In this paper, we challenged the maximum non-Gaussinity principle and changed it into maximum non-stability principle. Also, we demonstrate that the classical ICA based on maximization of non-Gaussianity is a special case of the new approach of ICA which is based on maximization of non-stability with certain constraints. To be able to quantify non-stability, we propose a new measure of stability namely Alpha-negentropy, which is theoretical heretic and designed to be the contrast function. Three other approximation non-stability measures, FLOM method, logarithmic moment method and extreme value method are also introduced since Alpha-negentropy is computational complex. At the end of this paper, a numerical gradient algorithm for the maximization of the Alpha-negentropy with the objective of source separation of heavy-tailed signals is designed. The first part of experiment compares FLOM, logarithmic moment, extreme value method with the differential entropy. The other part evaluates this new ICA method using four non-stability methods as contrast functions, which show that ICA by maximum of non-stability performs successfully in heavy-tailed source separation problems.
引文
[1]CJutten, J.Herault. Blind separation of sources, part Ⅰ:An adaptive algorithm based on neuromimetic architecture. Signal Processing,1991,24(1):1-10
[2]A.Hyvarinen, E.Oja. A fast fixed-point algorithm for independent component analysis. Neural Computation,1997,9(7):1483-1492.
[3]A.Hyvarinen. A family of fixed-point algorithms for independent component analysis. Proceeding of IEEE International C conference on Acoustics, Speech and Signal Processing (ICASSP97), Munich, Germany,1997,3917-3920.
[4]A.Hyvarinen. Fast and robust fixed-point algorithms for independent component analysis. IEEE Transactions on Neural Networks,1999,10 (3):626-633.
[5]P. Comon:Independent Component Analysis, A New Concept? Signal Processing 36,1994,287-314
[6]M. Sahmoudi, K. Abed-Meraim, M. Benidir. Blind Separation of Impulsive Alpha-Stable Sources Using Minimum Dispersion Criterion. IEEE Signal Processing Letters,2005, vol.12, no.4,281-284
[7]Nolan, J. P. Numerical calculation of stable densities and distribution functions. Commun Statist.-Stochastic Models 13,1997,159-774.
[8]Rachev, S. T. and S. Mittnik. Stable Paretian Models in Finance.2000, NewYork, NY:Wiley.
[9]McCulloch, J. H. Financial applications of stable distributions. In G S.1996, Handbook of Statistics, Maddala and C. R. Rao (Eds.)
[10]Frain, J. C. Maximum likelihood estimates of regression coecients with alpha stable residuals and day of week eects in total returns on equity indices.,2008, Department of Economics, Trinity College Dublin
[11]Asmussen, S., H. Schmidli, and V. Schmidt. Tail probabilities for non-standard risk and queueing processes with subexponential jumps. Advances in Applied Probability,1997
[12]Cormode, G. Stable distributions for stream computations:it's as easy as 0,1,2. In Proceedings of Workshop on Management and Processing of Data Streams (MPDS),2003
[13]Hetman, P., B. Szabat, K. Weron, and D. Wodzinski. On the Rajagopal relaxation-time distribution and its relationship to the Kohlrausch-Williams-Watts relaxation function. Journal of Non-Crystalline Solids,2003,330 (1-3),66-74.
[14]Bardou, F., J.-P. Bouchaud, A. Aspect, and C. Cohen-Tannoudji. Levy Statistics and Laser Cooling:HOw Rare Events Bring Atoms to Rest.2002, Cambridge: Cambridge Univ. Press.
[15]Rishmawi, S.. Fitting concentration data with stable distributions. Ph.D. thesis, 2005, American University, Washington, DC.
[16]Menabde, M. and M. Sivapalan. Modeling of rainfall time series and extremes, using bounded random cascades and Levy-stable distributions.2000, Water Resources Research 36,3293-3300.
[17]G Samorodnitsky and M. Taqqu. Stable non-Gaussian random processes. Chapman and Hall,1994, New York
[18]Hyvarinen, A. New approximations of differential entropy for independent component analysis and projection pursuit, Advances in Neural Information Processing Systems,1998, Vol.10.273-279
[19]John P. Nolan. Stable Distributions Models for Heavy Tailed Data. December 22, 2005, Report
[20]Lazo, A. and P. Rathie. On the entropy of continuous probability distributions, Theory, IEEE Transactions on,1978.24(1):120-122.
[21]A. Hyvarinen, E.Oja:Independent Component Analysis:Algorithms and Applications. Neural Networks 13,2004,411-430
[22]E. E. Kuruoglu:Density Parameter Estimation of Skewed alpha-Stable Distribution. IEEE transactions on signal processing,2001, vol.49, no.10,2192-2201,
[23]McCulloch, J. H.. Simple consistent estimators of stable distribution parameters. Communications inStatistics. Simulation and Computation 15,1986,1109-1136.
[24]Nikias, C. L. and M. Shao. Signal Processing with Alpha-Stable Distributions and Applications.1995, New York:Wiley.
[25]X. Y. Ma and C. L. Nikias, "Parameter estimation and blind channel identification in impulsive signal environments," IEEE Trans. Signal Processing,1995, vol.43, pp.2884-2897
[26]G Samorodnitsky and M. S Taqqu, Stable Non-Gaussian Random Processes, Socratic Models With Infinite Variance,1994, New York, Chapman and Hall
[27]G A. Tsihrintzis and C. L. Nikias, "Fast estimation of the parameters of alpha-stable impulsive interference," IEEE Trans. Signal Processing, vol.44,1996, 1492-1503
[2]A.Hyvarinen, E.Oja. A fast fixed-point algorithm for independent component analysis. Neural Computation,1997,9(7):1483-1492.
[3]A.Hyvarinen. A family of fixed-point algorithms for independent component analysis. Proceeding of IEEE International C conference on Acoustics, Speech and Signal Processing (ICASSP97), Munich, Germany,1997,3917-3920.
[4]A.Hyvarinen. Fast and robust fixed-point algorithms for independent component analysis. IEEE Transactions on Neural Networks,1999,10 (3):626-633.
[5]P. Comon:Independent Component Analysis, A New Concept? Signal Processing 36,1994,287-314
[6]M. Sahmoudi, K. Abed-Meraim, M. Benidir. Blind Separation of Impulsive Alpha-Stable Sources Using Minimum Dispersion Criterion. IEEE Signal Processing Letters,2005, vol.12, no.4,281-284
[7]Nolan, J. P. Numerical calculation of stable densities and distribution functions. Commun Statist.-Stochastic Models 13,1997,159-774.
[8]Rachev, S. T. and S. Mittnik. Stable Paretian Models in Finance.2000, NewYork, NY:Wiley.
[9]McCulloch, J. H. Financial applications of stable distributions. In G S.1996, Handbook of Statistics, Maddala and C. R. Rao (Eds.)
[10]Frain, J. C. Maximum likelihood estimates of regression coecients with alpha stable residuals and day of week eects in total returns on equity indices.,2008, Department of Economics, Trinity College Dublin
[11]Asmussen, S., H. Schmidli, and V. Schmidt. Tail probabilities for non-standard risk and queueing processes with subexponential jumps. Advances in Applied Probability,1997
[12]Cormode, G. Stable distributions for stream computations:it's as easy as 0,1,2. In Proceedings of Workshop on Management and Processing of Data Streams (MPDS),2003
[13]Hetman, P., B. Szabat, K. Weron, and D. Wodzinski. On the Rajagopal relaxation-time distribution and its relationship to the Kohlrausch-Williams-Watts relaxation function. Journal of Non-Crystalline Solids,2003,330 (1-3),66-74.
[14]Bardou, F., J.-P. Bouchaud, A. Aspect, and C. Cohen-Tannoudji. Levy Statistics and Laser Cooling:HOw Rare Events Bring Atoms to Rest.2002, Cambridge: Cambridge Univ. Press.
[15]Rishmawi, S.. Fitting concentration data with stable distributions. Ph.D. thesis, 2005, American University, Washington, DC.
[16]Menabde, M. and M. Sivapalan. Modeling of rainfall time series and extremes, using bounded random cascades and Levy-stable distributions.2000, Water Resources Research 36,3293-3300.
[17]G Samorodnitsky and M. Taqqu. Stable non-Gaussian random processes. Chapman and Hall,1994, New York
[18]Hyvarinen, A. New approximations of differential entropy for independent component analysis and projection pursuit, Advances in Neural Information Processing Systems,1998, Vol.10.273-279
[19]John P. Nolan. Stable Distributions Models for Heavy Tailed Data. December 22, 2005, Report
[20]Lazo, A. and P. Rathie. On the entropy of continuous probability distributions, Theory, IEEE Transactions on,1978.24(1):120-122.
[21]A. Hyvarinen, E.Oja:Independent Component Analysis:Algorithms and Applications. Neural Networks 13,2004,411-430
[22]E. E. Kuruoglu:Density Parameter Estimation of Skewed alpha-Stable Distribution. IEEE transactions on signal processing,2001, vol.49, no.10,2192-2201,
[23]McCulloch, J. H.. Simple consistent estimators of stable distribution parameters. Communications inStatistics. Simulation and Computation 15,1986,1109-1136.
[24]Nikias, C. L. and M. Shao. Signal Processing with Alpha-Stable Distributions and Applications.1995, New York:Wiley.
[25]X. Y. Ma and C. L. Nikias, "Parameter estimation and blind channel identification in impulsive signal environments," IEEE Trans. Signal Processing,1995, vol.43, pp.2884-2897
[26]G Samorodnitsky and M. S Taqqu, Stable Non-Gaussian Random Processes, Socratic Models With Infinite Variance,1994, New York, Chapman and Hall
[27]G A. Tsihrintzis and C. L. Nikias, "Fast estimation of the parameters of alpha-stable impulsive interference," IEEE Trans. Signal Processing, vol.44,1996, 1492-1503