欠定盲源信号分离的数学理论与算法
|
摘要
随着信息和计算机技术的发展,很多实际应用中通过传感器获得的是一些有用信号的混叠信号或带噪声的混叠信号,如何将这些隐藏在混叠信号中的原始信号分离出来,是一些应用中必须解决的问题,盲源分离技术正是在这种背景下应运而生的。
本文系统回顾了盲源分离技术的发展历史、研究现状,在总结盲源分离理论基础及经典算法的基础上,研究了欠定盲源分离技术相关理论。本文的重点是借助稀疏表示,采用“两步法”实现无噪声欠定盲源分离。这里我们在假定A已经估计出来的前提下,集中研究如何估计源信号s (t).此问题的核心就是:建立目标函数和设计优化算法。从不同的角度,在不同的情况下(如源信号是否独立同分布,源信号服从哪种分布等),建立不同的目标函数,从而设计相应的优化算法。根据欠定盲源分离的不同情况,本文研究了两种欠定盲源分离问题。一种是只有等式约束的非光滑凸优化问题;另一种是既有等式约束又有不等约束的非光滑非凸优化问题。
在解决这两类问题时本文采用以微分包含为模型的神经网络,得出解的存在性、唯一性,以及在参数足够大时,神经网络的轨道在有限步内达到可行域并且永驻其中。同时,本文还得出非光滑凸时轨道收敛到最优解集,而非光滑非凸时收敛到临界点集。
最后本文给出两个算例,进一步验证本文结论的可行性。
With the development of information and computer technique, in many applica- tions,the mixing signals or the mixing signals with noise of the source signals can be obtained by the sensors,how to separate the original source ignals from the mixing signals is the problem that must to be solved in some applications,the technique of blind source separation is being developed under this circumstance.
This dissertation reviews the development of BSS,the current research status,the related theory. In this article, we analyze and summarize the previous work of blind source separation including classical algorithm and theory. Emphasis of this paper achieve no noise blind source separation by“two-step”, with sparse representation. Here we have estimated on the assumption that A is known, focused on how to estimate the source signal s (t). The core of this problem is to establish the target function and design optimization algorithm. Under different circumstances, to establish different target function, thus design corresponding optimization algorithm. According to the different conditions of the underdetermined blind source separation, this paper studies two underdetermined blind source separation problems. One is only equality constraints of nonsmooth convex optimization problem, the other is a nonsmooth nonconvex optimization problem with a nonsmooth nonconvex objective function, a class of af?ne equality constraints, and a class of nonsmooth convex inequality constraints.
To solve these kinds of problems, we develops a neural network which is modeled by a differential inclusion. It is proved the existence, uniqueness of the solutions. Suf?ciently large penalty parameters, any trajectory of the neural network can reach the feasible region in ?nite time and stays there thereafter. Moreover, under the first one ,the trajectory of the neural network converges to the optimal solutions set of the neural network. Under the second one, the trajectory of the neural network converges to the set consisting of the critical points.
Finally, this paper contains two examples, which show the feasibility of the results in this paper.
本文系统回顾了盲源分离技术的发展历史、研究现状,在总结盲源分离理论基础及经典算法的基础上,研究了欠定盲源分离技术相关理论。本文的重点是借助稀疏表示,采用“两步法”实现无噪声欠定盲源分离。这里我们在假定A已经估计出来的前提下,集中研究如何估计源信号s (t).此问题的核心就是:建立目标函数和设计优化算法。从不同的角度,在不同的情况下(如源信号是否独立同分布,源信号服从哪种分布等),建立不同的目标函数,从而设计相应的优化算法。根据欠定盲源分离的不同情况,本文研究了两种欠定盲源分离问题。一种是只有等式约束的非光滑凸优化问题;另一种是既有等式约束又有不等约束的非光滑非凸优化问题。
在解决这两类问题时本文采用以微分包含为模型的神经网络,得出解的存在性、唯一性,以及在参数足够大时,神经网络的轨道在有限步内达到可行域并且永驻其中。同时,本文还得出非光滑凸时轨道收敛到最优解集,而非光滑非凸时收敛到临界点集。
最后本文给出两个算例,进一步验证本文结论的可行性。
With the development of information and computer technique, in many applica- tions,the mixing signals or the mixing signals with noise of the source signals can be obtained by the sensors,how to separate the original source ignals from the mixing signals is the problem that must to be solved in some applications,the technique of blind source separation is being developed under this circumstance.
This dissertation reviews the development of BSS,the current research status,the related theory. In this article, we analyze and summarize the previous work of blind source separation including classical algorithm and theory. Emphasis of this paper achieve no noise blind source separation by“two-step”, with sparse representation. Here we have estimated on the assumption that A is known, focused on how to estimate the source signal s (t). The core of this problem is to establish the target function and design optimization algorithm. Under different circumstances, to establish different target function, thus design corresponding optimization algorithm. According to the different conditions of the underdetermined blind source separation, this paper studies two underdetermined blind source separation problems. One is only equality constraints of nonsmooth convex optimization problem, the other is a nonsmooth nonconvex optimization problem with a nonsmooth nonconvex objective function, a class of af?ne equality constraints, and a class of nonsmooth convex inequality constraints.
To solve these kinds of problems, we develops a neural network which is modeled by a differential inclusion. It is proved the existence, uniqueness of the solutions. Suf?ciently large penalty parameters, any trajectory of the neural network can reach the feasible region in ?nite time and stays there thereafter. Moreover, under the first one ,the trajectory of the neural network converges to the optimal solutions set of the neural network. Under the second one, the trajectory of the neural network converges to the set consisting of the critical points.
Finally, this paper contains two examples, which show the feasibility of the results in this paper.
引文
1 F. H. Clarke. Optimization and Non-Smooth Analysis. Wiley, 1969:78~86
2 L. O. Chua, C. A. Desoer, E. S. Kuh. Linear and Nonlinear Circuits. McGraw-Hill Companies, 1987:112~131
3 M. Forti, P. Nistri, M. Quincampoix. Convergence of Neural Networks for Programming Problems Via a Nonsmooth ?ojasiewicz Inequality. IEEE Trans. Neural Netw.. 2006, 17(6):1471~1486
4 Y. Q. Li, A. Cichocki, S. Amari. Analysis of Sparse Representation and Blind Source Separation. Neural Computation. 2004, 16:1193~1234
5 P. Georgiev, F. Theis, A. Cichocki. Sparse Component Analysis and Blind Source Separation of Underdetermined Mixtures. IEEE Transactions on Neural Networks. 2005, 16(7):992~996
6 Y. Q. Li, A. Cichocki, S. Amari. Blind Estimation of Channel Parameters and Source Components for EEG Signals a Sparse Factorization Approach. IEEE Transactions on Neural Networks. 2006, 17(2):419~431
7 Z. S. He, S. L. Xie, S. X. Ding. Convolutive Blind Source Separation in the Frequency Domain Based on Sparse Representation. IEEE Transaction on Audio Speech and Language Processing. 2007, 15(5):1551~1563
8 Y. Q. Li, S. I. Amari, A. Cichocki, D. W. C. Ho, S. L. Xie. Underdetermined Blind Source Separation Based on Sparse Representation. IEEE Transactions on Signal Processing. 2006, 54(2):423~437
9 F. J. Theis, C. G. Puntonet, E. W. Lang. Median-Based Clustering for Underdetermined Blind Signal Processing. IEEE Transactions on Signal Processing Letters. 2006, 13(2):96~99
10 S. L. Xie, Z. S. He, Y. L. Fu. A Note on Stone’s Conjecture of Blind Signal Separation. Neural Computation. 2005, 17(2):321~330
11 S. Rayan, Y. Ozgur, J. M. Martin, A. Rafeef. Underdetermined Anechoic Blind Source Separation Via l qBasis-pursuit with q ? 1. IEEE Transactions on Signal Processing. 2007, 55(8):4004~4017
12 C. Jutten, J. Hérault. Blind Separation of Sources, Part I: An Adaptive Algorithm Based on Neuromimetic Architecture. Signal Processing. 1991, 24:1~10
13 P. Comon. Blind Separation of Sources, Part II: Problems Statement. SignalProcessing. 1991, 24:11~20
14 E. Sorouchyari. Blind Source Separation, Part III: Stability Analysis. Signal Processing. 1991, 24:21~29
15 P. Bofill, M. Zibulevsky. Underdetermined Blind Source Separation Using Sparse Representations. Signal Processing. 2001, 81(11):2353~2362
16 Y. Q. Li, A. Cichocki. Sparse Representation and its Applications in Blind Source Separation. Sergei Shishkin RIKEN Brain Science Institute. 2004:241~247
17 M. Zibulevsky, B. A. Pearmutter. Blind Source Separation by Sparse Decomposition in a Signal Dictionary. Neural Computation. 2001, 13:863~882
18 T. W. Lee, M. S. Lewicki, M. Girolami. Blind Source Separation of More Sources than Mixtures Using Overcomplete Representations. IEEE Transactions on Signal Processing Letter. 1999, 6(4):87~90
19 M. S. Lewiski, T. J. Sejnowski. Learning Overcomplete Representations. Neural Computation. 2000, 12:337~365
20 Y. Q. Li, J. Wang, J. M. Zhang. Blind Extraction of Singularly Mixed Source Signals. IEEE Transactions on Neural Networks. 2000, 11(6):1413~1422
21 Y. Q. Li, J. Wang, Sequential Blind Extraction of Instantaneously Mixed Sources. IEEE Transactions on Signal Processing. 2002, 50(5):997~1006
22 A. Cichocki, R. Unbehauer. Neural Networks for Optimization with Bounded Constraints. IEEE Transactions on Neural Networks. 1993, 4:293~304
23 J. D. Hopfield, D. W. Tank. Neural Computation of Decisions in Optimization Problems. Biol. Cybern. 1985, 52:141~151
24 D. W. Tank, J. J. Hopfield. Signal Decision Circuit and A Linear Programming Circuit. IEEE Transactions on Circuits and Systems. 1986, 33:533~541
25 X. P. Xue, W. Bian. A Project Neural Network for Solving Degenerate Convex Quadratic Proram. Neurocomputing. 2007, 70:2449~2459
26 P. Nistri, M. Quincampoix. On the Dynamics of A Differential Inclusion Built Upon A Nonconvex Constrained Minimization Problem. Journal of Optimization Theory and Applications. 2005, 123(3):659~672
27 L. V. Ferreira, E. Kaszkurewicz, A. Bhaya. Solving Systems of Linear Equations Via Gradient Systems with Discontinuous Right Hand Sides:Applications to LS-SVM. IEEE Transactions on Neural Networks. 2005,16(2):501~505
28 E. K. P. Chong, S. Hui, S. H. Zak. An Analysis of A Class of Neural Networks for Solving Linear Programming Problem. IEEE Transactions on Automatic Control. 1999, 44(11):1995~2006
29 M. Forti, P. Nistri, M. Quincampoix. Generalized Neural Network for Nonsmooth Nonlinear Programming Problems. IEEE Transactions on Circuits and Systems I. 2004, 51(9):1741~1754
30 T. Takagi, M. Sugeno. Fuzzy Identification of Systems and its Applications to Modeling and Control. IEEE Transactions on System. 1985, 15(1):116~ 123
31 B. Kosko. Fuzzy Systems as Universal Approximators. IEEE Transactions on Computers. 1994, 43(11):1329~1333
32 L. Wang, J. M. Mendel. Fuzzy Basis Functions, Universal Approximation and Orthogonal Least-squares Learning. IEEE Transactions on Neural Networks. 1992, 3(5):807~814
33 S. G. Kim, C. D. Yoo. Underdetermined Blind Source Separation Based on Generalized Gaussian Distribution. Machine Learning for Signal Processing. 2006:103~108
34 J. P. Aubin, H. Frankowska. Set-Valued Analysis. Boston MA: Birkhauser, 1990:37~43
35 E. K. P. Chong, S. Hui, S. H. Zak. An Analysis of A Cass of Neural Networks for Solving Linear Programming Problems. IEEE Trans. Autom. Control. 1999, 44(11):1995~2006
36 Y. Xia, J. Wang. A Recurrent Neural Network for Nonlinear Convex Programs Subject to Linear Constraint. IEEE Trans. Neural Netw.. 2005, 16(2):379~386
37 M. Forti, A. Tesi. New Conditions for Global Stability of Neural Networks with Application to Linear and Quadratic Programming Problems. IEEE IEEE Transactions on Circuits and Systems. 1995, 42(7):354~366
38 A. F. Filippov. Differential Equations With Discontinuous Right-Hand Side. Mathematics and its Applications. Kluwer Academic Publishers, 1988:93~ 107
39 R. A. DeCarlo, S. H. Zak, G. P.Matthews. Variable Structure Control ofNonlinear Multivariable Systems: A tutoral. Proc. IEEE. 1969, 76(2):212~232
40 S. C. Chan, K.M. Tsui. Wordlength Optimization of Linear Time-invariant Systems with Multiple. 2006, 54(4):845~854
41 E. Paschen, R. P. Mosby, Eds.. Poetry Speaks Expanded: Hear Poets Read Their Work From Tennyson to Plath. Sourcebooks. Naperville. 2001:237~249
42 G. D. Sandre, M. Forti, P. Nistri, A. Premoli. Dynamical Analysis of Full-range Cellular Neural Networks by Exploiting Differential Variational Inequalities. IEEE Transactions on Circuits and Systems. 2007, 54(8):1736~ 1749
43 Paul D. O’Grady1, Barak A. Pearlmutter. The LOST Algorithm: Finding Lines and Separating Speech Mixtures. Hindawi Publishing Corporation EURASIP Journal on Advances in Signal Processing. 2008:137~145
44 P. D. O’Grady, B. A. Pearlmutter. Soft-LOST: EM on a mixture of oriented lines in Proceedings of the 5th International Conference on Independent Component Analysis and Blind Signal Separation (ICA’04). Granada. Spain. 2004, 3195:430~436
45 P. D. O’Grady, B. A. Pearlmutter. Hard-LOST: Modi?ed k-Means for Oriented Lines in Proceedings of the Irish Signals and Systems Conference. Belfast. UK. 2004:247~252
46 J. J. Hop?eld, D. W. Tank. Neural Computation of Decision in Optimization Problems. Biological Cybernetics. 1985, 52:141~152
47 M. P. Kennedy, L. O. Chua. Neural Networks for Nonlinear Programming. IEEE Transactions on Circuits and Systems. 1988, 35(5):554~562
48 D. W. Tank, J. J. Hop?eld. Simple Neural Optimization Networks: An A/D Converter, Signal Decision Circuit, and A Linear Programming Circuit. IEEE Transactions on Circuits and Systems. 1986, 33(5):533~541
2 L. O. Chua, C. A. Desoer, E. S. Kuh. Linear and Nonlinear Circuits. McGraw-Hill Companies, 1987:112~131
3 M. Forti, P. Nistri, M. Quincampoix. Convergence of Neural Networks for Programming Problems Via a Nonsmooth ?ojasiewicz Inequality. IEEE Trans. Neural Netw.. 2006, 17(6):1471~1486
4 Y. Q. Li, A. Cichocki, S. Amari. Analysis of Sparse Representation and Blind Source Separation. Neural Computation. 2004, 16:1193~1234
5 P. Georgiev, F. Theis, A. Cichocki. Sparse Component Analysis and Blind Source Separation of Underdetermined Mixtures. IEEE Transactions on Neural Networks. 2005, 16(7):992~996
6 Y. Q. Li, A. Cichocki, S. Amari. Blind Estimation of Channel Parameters and Source Components for EEG Signals a Sparse Factorization Approach. IEEE Transactions on Neural Networks. 2006, 17(2):419~431
7 Z. S. He, S. L. Xie, S. X. Ding. Convolutive Blind Source Separation in the Frequency Domain Based on Sparse Representation. IEEE Transaction on Audio Speech and Language Processing. 2007, 15(5):1551~1563
8 Y. Q. Li, S. I. Amari, A. Cichocki, D. W. C. Ho, S. L. Xie. Underdetermined Blind Source Separation Based on Sparse Representation. IEEE Transactions on Signal Processing. 2006, 54(2):423~437
9 F. J. Theis, C. G. Puntonet, E. W. Lang. Median-Based Clustering for Underdetermined Blind Signal Processing. IEEE Transactions on Signal Processing Letters. 2006, 13(2):96~99
10 S. L. Xie, Z. S. He, Y. L. Fu. A Note on Stone’s Conjecture of Blind Signal Separation. Neural Computation. 2005, 17(2):321~330
11 S. Rayan, Y. Ozgur, J. M. Martin, A. Rafeef. Underdetermined Anechoic Blind Source Separation Via l qBasis-pursuit with q ? 1. IEEE Transactions on Signal Processing. 2007, 55(8):4004~4017
12 C. Jutten, J. Hérault. Blind Separation of Sources, Part I: An Adaptive Algorithm Based on Neuromimetic Architecture. Signal Processing. 1991, 24:1~10
13 P. Comon. Blind Separation of Sources, Part II: Problems Statement. SignalProcessing. 1991, 24:11~20
14 E. Sorouchyari. Blind Source Separation, Part III: Stability Analysis. Signal Processing. 1991, 24:21~29
15 P. Bofill, M. Zibulevsky. Underdetermined Blind Source Separation Using Sparse Representations. Signal Processing. 2001, 81(11):2353~2362
16 Y. Q. Li, A. Cichocki. Sparse Representation and its Applications in Blind Source Separation. Sergei Shishkin RIKEN Brain Science Institute. 2004:241~247
17 M. Zibulevsky, B. A. Pearmutter. Blind Source Separation by Sparse Decomposition in a Signal Dictionary. Neural Computation. 2001, 13:863~882
18 T. W. Lee, M. S. Lewicki, M. Girolami. Blind Source Separation of More Sources than Mixtures Using Overcomplete Representations. IEEE Transactions on Signal Processing Letter. 1999, 6(4):87~90
19 M. S. Lewiski, T. J. Sejnowski. Learning Overcomplete Representations. Neural Computation. 2000, 12:337~365
20 Y. Q. Li, J. Wang, J. M. Zhang. Blind Extraction of Singularly Mixed Source Signals. IEEE Transactions on Neural Networks. 2000, 11(6):1413~1422
21 Y. Q. Li, J. Wang, Sequential Blind Extraction of Instantaneously Mixed Sources. IEEE Transactions on Signal Processing. 2002, 50(5):997~1006
22 A. Cichocki, R. Unbehauer. Neural Networks for Optimization with Bounded Constraints. IEEE Transactions on Neural Networks. 1993, 4:293~304
23 J. D. Hopfield, D. W. Tank. Neural Computation of Decisions in Optimization Problems. Biol. Cybern. 1985, 52:141~151
24 D. W. Tank, J. J. Hopfield. Signal Decision Circuit and A Linear Programming Circuit. IEEE Transactions on Circuits and Systems. 1986, 33:533~541
25 X. P. Xue, W. Bian. A Project Neural Network for Solving Degenerate Convex Quadratic Proram. Neurocomputing. 2007, 70:2449~2459
26 P. Nistri, M. Quincampoix. On the Dynamics of A Differential Inclusion Built Upon A Nonconvex Constrained Minimization Problem. Journal of Optimization Theory and Applications. 2005, 123(3):659~672
27 L. V. Ferreira, E. Kaszkurewicz, A. Bhaya. Solving Systems of Linear Equations Via Gradient Systems with Discontinuous Right Hand Sides:Applications to LS-SVM. IEEE Transactions on Neural Networks. 2005,16(2):501~505
28 E. K. P. Chong, S. Hui, S. H. Zak. An Analysis of A Class of Neural Networks for Solving Linear Programming Problem. IEEE Transactions on Automatic Control. 1999, 44(11):1995~2006
29 M. Forti, P. Nistri, M. Quincampoix. Generalized Neural Network for Nonsmooth Nonlinear Programming Problems. IEEE Transactions on Circuits and Systems I. 2004, 51(9):1741~1754
30 T. Takagi, M. Sugeno. Fuzzy Identification of Systems and its Applications to Modeling and Control. IEEE Transactions on System. 1985, 15(1):116~ 123
31 B. Kosko. Fuzzy Systems as Universal Approximators. IEEE Transactions on Computers. 1994, 43(11):1329~1333
32 L. Wang, J. M. Mendel. Fuzzy Basis Functions, Universal Approximation and Orthogonal Least-squares Learning. IEEE Transactions on Neural Networks. 1992, 3(5):807~814
33 S. G. Kim, C. D. Yoo. Underdetermined Blind Source Separation Based on Generalized Gaussian Distribution. Machine Learning for Signal Processing. 2006:103~108
34 J. P. Aubin, H. Frankowska. Set-Valued Analysis. Boston MA: Birkhauser, 1990:37~43
35 E. K. P. Chong, S. Hui, S. H. Zak. An Analysis of A Cass of Neural Networks for Solving Linear Programming Problems. IEEE Trans. Autom. Control. 1999, 44(11):1995~2006
36 Y. Xia, J. Wang. A Recurrent Neural Network for Nonlinear Convex Programs Subject to Linear Constraint. IEEE Trans. Neural Netw.. 2005, 16(2):379~386
37 M. Forti, A. Tesi. New Conditions for Global Stability of Neural Networks with Application to Linear and Quadratic Programming Problems. IEEE IEEE Transactions on Circuits and Systems. 1995, 42(7):354~366
38 A. F. Filippov. Differential Equations With Discontinuous Right-Hand Side. Mathematics and its Applications. Kluwer Academic Publishers, 1988:93~ 107
39 R. A. DeCarlo, S. H. Zak, G. P.Matthews. Variable Structure Control ofNonlinear Multivariable Systems: A tutoral. Proc. IEEE. 1969, 76(2):212~232
40 S. C. Chan, K.M. Tsui. Wordlength Optimization of Linear Time-invariant Systems with Multiple. 2006, 54(4):845~854
41 E. Paschen, R. P. Mosby, Eds.. Poetry Speaks Expanded: Hear Poets Read Their Work From Tennyson to Plath. Sourcebooks. Naperville. 2001:237~249
42 G. D. Sandre, M. Forti, P. Nistri, A. Premoli. Dynamical Analysis of Full-range Cellular Neural Networks by Exploiting Differential Variational Inequalities. IEEE Transactions on Circuits and Systems. 2007, 54(8):1736~ 1749
43 Paul D. O’Grady1, Barak A. Pearlmutter. The LOST Algorithm: Finding Lines and Separating Speech Mixtures. Hindawi Publishing Corporation EURASIP Journal on Advances in Signal Processing. 2008:137~145
44 P. D. O’Grady, B. A. Pearlmutter. Soft-LOST: EM on a mixture of oriented lines in Proceedings of the 5th International Conference on Independent Component Analysis and Blind Signal Separation (ICA’04). Granada. Spain. 2004, 3195:430~436
45 P. D. O’Grady, B. A. Pearlmutter. Hard-LOST: Modi?ed k-Means for Oriented Lines in Proceedings of the Irish Signals and Systems Conference. Belfast. UK. 2004:247~252
46 J. J. Hop?eld, D. W. Tank. Neural Computation of Decision in Optimization Problems. Biological Cybernetics. 1985, 52:141~152
47 M. P. Kennedy, L. O. Chua. Neural Networks for Nonlinear Programming. IEEE Transactions on Circuits and Systems. 1988, 35(5):554~562
48 D. W. Tank, J. J. Hop?eld. Simple Neural Optimization Networks: An A/D Converter, Signal Decision Circuit, and A Linear Programming Circuit. IEEE Transactions on Circuits and Systems. 1986, 33(5):533~541