盲源分离的极大似然估计算法研究与应用
|
摘要
盲源分离(BSS)是一种多维信号处理方法,它是指在源信号以及混合模型都未知的情况下,仅从观测信号中恢复出源信号各个独立分量的过程。盲源分离己成为现代信号处理领域研究的热点问题,在通信、语音处理、图像处理、地震勘探、生物医学、雷达以及经济数据分析等领域具有非常重要的理论意义和广泛的应用价值。
本文提出了新的盲源分离算法,并给出了算法在图像分离中的应用。本文所做的主要工作:
(1)从盲源分离的理论出发,由盲源分离问题引出独立成分分析。阐述了独立成分分析的发展历史和研究现状,以及在独立成分分析研究中存在的问题。
(2)对独立成分分析理论基础进行了简单描述,讨论了独立成分分析研究中的主要问题。包括独立成分分析的数学定义、基本假设、相关的数学理论基础和实现途径等。
(3)介绍和推导了基于独立成分分析的盲源分离算法的几种典型的学习算法,重点研究了FastICA算法。FastICA算法基于非高斯性最大化原理,使用固定点迭代理论寻找非高斯性最大值,采用牛顿迭代算法对观测变量的大量采样点进行处理,每次从观测信号中分离出一个独立分量,是独立成分分析的一种快速算法。
(4)分别提出了基于简化牛顿法和修正牛顿法的算法,并给出了具体的实现步骤,将两种算法通过仿真试验进行了对比,实验结果表明,基于修正牛顿法的算法要优于前者。该算法减少了运算量,而且能够很好地分离出各个独立成分。最后计算机仿真验证了基于修正牛顿法的极大似然估计盲源分离算法在图像分离方面的有效性。
Blind Sources Separation (BSS) is one of the multiple signals processing method; it is the process of recovering unknown independent source signals from sensor measurements which are unknown combinations of the source signals. BSS has become a hot topic of modern signal processing, so it has very important theory significance and utility value in communication , speech processing , image processing, biomedicine and radar technology, and it even can be applied to financial data analysis.
This paper studies new algorithm and its applications in image separation. The major contribution of this paper is summarized as follow:
1. BSS theories are firstly presented, and then Independent Component Analysis (ICA) is derived from BSS. Introduce the development history and the current research status of ICA, and the problems existed in ICA study.
2. Simple mathematical preliminaries in ICA technique were given,main problems of the research of ICA are discussed, including the mathematical definition of ICA,the assumptions made about ICA problems and the mathematical theory and methods commonly used in ICA, etc.
3. Several classical algorithms of BSS based on ICA and their derivations are introduced. Further, the FastICA algorithm is studied. FastICA is a fast algorithm of ICA, which is based on Fixed-point iteration theory to fix the non-Gaussian maximum. FastICA algorithm processes a large amount of sample point of received signals via Newton iterative algorithm, and recovers one independent component from the receiving signals one time.
4. Introduce two algorithms based on Simplified Newton Method and Improved Newton Method, and compare these two algorithms through experiments. The experiment results prove the advantages of the algorithm which is based on Improved Newton Method, The algorithm reduces computation. It also separates independent components very well. Finally, computer simulations illustrate that the Maximum Likelihood Estimation algorithms based on Improved Newton Method can well separate mixed images.
本文提出了新的盲源分离算法,并给出了算法在图像分离中的应用。本文所做的主要工作:
(1)从盲源分离的理论出发,由盲源分离问题引出独立成分分析。阐述了独立成分分析的发展历史和研究现状,以及在独立成分分析研究中存在的问题。
(2)对独立成分分析理论基础进行了简单描述,讨论了独立成分分析研究中的主要问题。包括独立成分分析的数学定义、基本假设、相关的数学理论基础和实现途径等。
(3)介绍和推导了基于独立成分分析的盲源分离算法的几种典型的学习算法,重点研究了FastICA算法。FastICA算法基于非高斯性最大化原理,使用固定点迭代理论寻找非高斯性最大值,采用牛顿迭代算法对观测变量的大量采样点进行处理,每次从观测信号中分离出一个独立分量,是独立成分分析的一种快速算法。
(4)分别提出了基于简化牛顿法和修正牛顿法的算法,并给出了具体的实现步骤,将两种算法通过仿真试验进行了对比,实验结果表明,基于修正牛顿法的算法要优于前者。该算法减少了运算量,而且能够很好地分离出各个独立成分。最后计算机仿真验证了基于修正牛顿法的极大似然估计盲源分离算法在图像分离方面的有效性。
Blind Sources Separation (BSS) is one of the multiple signals processing method; it is the process of recovering unknown independent source signals from sensor measurements which are unknown combinations of the source signals. BSS has become a hot topic of modern signal processing, so it has very important theory significance and utility value in communication , speech processing , image processing, biomedicine and radar technology, and it even can be applied to financial data analysis.
This paper studies new algorithm and its applications in image separation. The major contribution of this paper is summarized as follow:
1. BSS theories are firstly presented, and then Independent Component Analysis (ICA) is derived from BSS. Introduce the development history and the current research status of ICA, and the problems existed in ICA study.
2. Simple mathematical preliminaries in ICA technique were given,main problems of the research of ICA are discussed, including the mathematical definition of ICA,the assumptions made about ICA problems and the mathematical theory and methods commonly used in ICA, etc.
3. Several classical algorithms of BSS based on ICA and their derivations are introduced. Further, the FastICA algorithm is studied. FastICA is a fast algorithm of ICA, which is based on Fixed-point iteration theory to fix the non-Gaussian maximum. FastICA algorithm processes a large amount of sample point of received signals via Newton iterative algorithm, and recovers one independent component from the receiving signals one time.
4. Introduce two algorithms based on Simplified Newton Method and Improved Newton Method, and compare these two algorithms through experiments. The experiment results prove the advantages of the algorithm which is based on Improved Newton Method, The algorithm reduces computation. It also separates independent components very well. Finally, computer simulations illustrate that the Maximum Likelihood Estimation algorithms based on Improved Newton Method can well separate mixed images.
引文
1 A. Cichochi, S. Amari. Adaptive Blind Signal and Image Processing: Learning Algorithms and Applications. New York: Wiley. 2004:10~68
2 W. Lu, J. C. Rajapakse. Approach and Applications of Constrained ICA. IEEE Transactions on Neural Networks. 2005,16(1):203~212
3 W. Lu, J. C. Rajapakse. ICA with Reference. Neuro Computing. 2006, 69(16-18):2244~2257
4 D. Sh. Huang, J. X. Mi. A New Constrained Independent Component Analysis Method. IEEE Transactions on Neural Networks. 2007,18(5):1532~1535
5余英林,谢胜利,蔡汉添等.信号处理新方法导论.清华大学出版社. 2004:12~56
6 C. Jutten, J. Herault. Blind Separation of Sources, PartⅠ: An Adaptive Algorithm Based on Neuromimetic Architecture. Signal Processing. 1991, 24:1~10
7 J. F. Cardoso, A. Souloumiac. Blind Beam forming for Non-Gaussian Signals. In IEEE Proceedings-F. 1993, 140(6):362~370
8 C. G. Puntonet, A. Prieto and J. Ortega. New Geometrical Approach for Blind Separation of Sources Mapped to a Neural Network. International Workshop on Neural Networks for Identification, Control, Robotics and signal/Image Processing. 1996:174~182
9 H. H. Yang. Adaptive Online learning Algorithms for Blind Separation- maximum Entropy and Minimum Mutual Information. Neural Computation. 1997, 9:1457~1482
10 P. Comon. Independent Component Analysis, a New Concept?. Signal Processing. 1994, 36:287~314
11 A. J. Bell, T. J. Sejnowski. An Information-maximization Approach to Blind Separation and Blind Deconvolution. Neural Computation. 1995, 7(6):1004~1034
12 J. F. Cardoso, B. H. Laheld. Equivariant Adaptive Source Separation. IEEE Transactions Signal Processing. 1996, 44(12):3017~3030
13 S. Amari. Natural Gradient Works Efficiently in Learning. Neural Computation. 1998, 10:251~276
14 B. A. Draper, K. Baek and M. S. Bartlett. Recognizing Faces with PCA and ICA. Computer Vision and Image Understanding. 2003, 91:115~137
15 K. C. Kwak, W. Pedrycz. Face Recognition Using an Enhanced IndependentComponent Analysis Approach. IEEE Transactions on Neural Networks. 2007, 18:530~541
16 T. Bakir, A. Peter and R. Riley. Non-negative Maximum Likelihood ICA for Blind Source Separation of Images and Signals with Application to Hyperspectral Image Subpixel Demixing. IEEE International Conference on Image Processing. 2006:3237~3240
17 C. Platt, F. Faggin. Networks for the Separation of Sources That are Super-imposed and Delayed. Advances in Neural Information Processing Systems. 1991:730~737
18 T. W. Lee, A. J. Bell and R. H. Lambert. Blind Separation of Delayed and Convolved Sources. Advances in Neural Information Processing System. Cambridge MA: MIT Press. 1997:758~764
19何振亚,刘琚,杨绿溪.盲均衡何信道参数估计的一种ICA和进化计算方.中国科学. 2000, 30(1): 1~7
20 Z. Y. He, J. Liu and L. X. Yang. Blind Separation of Images Using Edgeworth Expansion Based on ICA Algorithm. Chinese Journal of Electronics. 1999, 8(3):278~282
21凌燮亭.近场宽带信号源的盲分离.电子学报. 1996, 24(7):87~92
22朱孝龙.盲自适应信号分离的并行实现方法研究.西安电子科技大学. 2003:15~34
23张贤达,朱孝龙,保铮.基于分阶段学习的盲信号分离.中国科学. 2002, 32(5):693~703
24 J. Cheng, Y. W. Chen and H. Q. Lu. Color and Texture Based Image Segmentation Using Local Feature Analysis Approach. Third International Symposium on Multispectral Image Processing and Pattern Recognition. 2003,5286:600~604
25黄启宏,王帅,刘钊.改进的基于独立成分分析的图像特征提取算法.光电工程. 2007, 34(1):121~125
26黄启宏,段昶,刘钊.基于独立分量分析的图像特征提取及泊松噪声去除.光电工程. 2006, 33(11):128~132
27 Y. G. Liu, G. Chen and J. W. Lu. Face Recognition Based on Independent Component Analysis and Fuzzy Support Vector Machine. Proceedings of 6th WCICA. 2006:5~8
28安高云,阮秋琦.一种基于小波分解及鲁棒估计的ICA算法及其在人脸识别中的应用.电子学报. 2006, 34(10):1900~1905
29王毅,齐华,郝重阳.一种基于独立分量分析的模糊图像盲分离算法.计算机应用. 2006, 26(10):2366~2368
30杨俊安,解光军,庄镇泉.量子遗传算法及其在图像盲分离中的应用研究.计算机辅助设计与图形学学报. 2003, 15(7):847~851
31刘琚,张新刚,孙建德.一种基于ICA的图像水印方法.电路与系统学报. 2003, 8(3):55~59
32张力,钱恭斌,纪震.基于ICA的智能数字水印技术.电子与信息学报. 2005, 27(3):478~482
33陈蜜,伭??李德仁.独立分量分析的图像融合算法.光电工程. 2007, 34(6):82~87
34 A. Hyv?rinen, P. Pajunen . Nonlinear Independent Component Analysis: Existence and Uniqueness Results.Neural Networks. 1999, 12(3):429~439
35 A. Hyv?rinen, J. Karhunen and E. Oja.Independent Component Analysis.A Wiley-Interscience Publication. 2001:56~90
36 A. Hyv?rinen. Gaussian Moments for Noisy Independent Component Analysis. IEEE Signal Processing Letters. 1999, 6(6):145~147
37 P. Comon, P. Chevalier. Blind Source Separation: Models, Concepts, Algorithms and Performance, Unsupervised Adaptive Filtering. John Wiley&Sons. 2000:139~189
38 M. Lazaro et al. Stochastic Blind Equalization Based on PDF Fitting Using Parzen Estimator. IEEE Transactions on Signal Processing. 2005, 53(2):696~704
39 X. Li. Blind Channel Estimation and Equalization in Wireless Sensor Networks Based on Correlations Among Sensors. IEEE Transactions on Signal Processing. 2005, 53(4):1511~1519
40 Q. Y. Li, E. W. Bai and Z. Ding. Blind Source Separation of Signal with Known Alphabets Usingε-approximation Algorithms. IEEE Transactions Signal Processing. 2003, 51(1):1~10
41 Y. W. Yao, F. B. Giannakis. On Regularity and Identifiability of Blind Source Separation under Constant-modulus Constraints. IEEE Transactions Signal Processing. 2005, 53(4):1272~1281
42 S. Buzzi, A. D. Maio. Code-aided Blind Detection of the Transmission Rate for Multirate Direct-sequence CDMA Systems. IEEE Transactions on Signal Processing. 2005,53(6):2182~2192
43 X. G. Doukopoulos, G. V. Moustakides. Adaptive Power Techniques for Blind Channel Estimation in CDMA Systems. IEEE Transactions on Signal Processing. 2005,53(3):1110~1120
44何振亚.自适应信号处理.科学出版社. 2002:21~56
45 T. W. Lee. A Unifying Information-theoretic Framework for Independent Component Analysis. Computers and Mathematics with Applications. 2000,31(11):1~21
46 J. F. Cardoso. Blind Signal Separation: Statistical Principles. Proceedings of the IEEE. 1998,86(10):2009~2025
47 A. Hyv?rinen , E. Oja. Independent Component Analysis: Algorithms and Applications. Neural Networks. 2000,13:411~430
48 A. Hyv?rinen, P. O. Hoyer. Emergence of Phase and Shift Invariant Features by Decomposition of Natural Images into Independent Feature Subspaces. Neural Computation. 2000,12(7):1705~1720
49杨福生,洪波.独立分量分析的原理与应用.北京:清华大学出版社. 2006:1~46
50 T. W. Lee et al. Independent Component Analysis Using an Extended InfoMax Algorithm for Mixed Sub-Gaussian and Super-Gaussian Sources. Neural Computation. 1999, 11(2):409~433
2 W. Lu, J. C. Rajapakse. Approach and Applications of Constrained ICA. IEEE Transactions on Neural Networks. 2005,16(1):203~212
3 W. Lu, J. C. Rajapakse. ICA with Reference. Neuro Computing. 2006, 69(16-18):2244~2257
4 D. Sh. Huang, J. X. Mi. A New Constrained Independent Component Analysis Method. IEEE Transactions on Neural Networks. 2007,18(5):1532~1535
5余英林,谢胜利,蔡汉添等.信号处理新方法导论.清华大学出版社. 2004:12~56
6 C. Jutten, J. Herault. Blind Separation of Sources, PartⅠ: An Adaptive Algorithm Based on Neuromimetic Architecture. Signal Processing. 1991, 24:1~10
7 J. F. Cardoso, A. Souloumiac. Blind Beam forming for Non-Gaussian Signals. In IEEE Proceedings-F. 1993, 140(6):362~370
8 C. G. Puntonet, A. Prieto and J. Ortega. New Geometrical Approach for Blind Separation of Sources Mapped to a Neural Network. International Workshop on Neural Networks for Identification, Control, Robotics and signal/Image Processing. 1996:174~182
9 H. H. Yang. Adaptive Online learning Algorithms for Blind Separation- maximum Entropy and Minimum Mutual Information. Neural Computation. 1997, 9:1457~1482
10 P. Comon. Independent Component Analysis, a New Concept?. Signal Processing. 1994, 36:287~314
11 A. J. Bell, T. J. Sejnowski. An Information-maximization Approach to Blind Separation and Blind Deconvolution. Neural Computation. 1995, 7(6):1004~1034
12 J. F. Cardoso, B. H. Laheld. Equivariant Adaptive Source Separation. IEEE Transactions Signal Processing. 1996, 44(12):3017~3030
13 S. Amari. Natural Gradient Works Efficiently in Learning. Neural Computation. 1998, 10:251~276
14 B. A. Draper, K. Baek and M. S. Bartlett. Recognizing Faces with PCA and ICA. Computer Vision and Image Understanding. 2003, 91:115~137
15 K. C. Kwak, W. Pedrycz. Face Recognition Using an Enhanced IndependentComponent Analysis Approach. IEEE Transactions on Neural Networks. 2007, 18:530~541
16 T. Bakir, A. Peter and R. Riley. Non-negative Maximum Likelihood ICA for Blind Source Separation of Images and Signals with Application to Hyperspectral Image Subpixel Demixing. IEEE International Conference on Image Processing. 2006:3237~3240
17 C. Platt, F. Faggin. Networks for the Separation of Sources That are Super-imposed and Delayed. Advances in Neural Information Processing Systems. 1991:730~737
18 T. W. Lee, A. J. Bell and R. H. Lambert. Blind Separation of Delayed and Convolved Sources. Advances in Neural Information Processing System. Cambridge MA: MIT Press. 1997:758~764
19何振亚,刘琚,杨绿溪.盲均衡何信道参数估计的一种ICA和进化计算方.中国科学. 2000, 30(1): 1~7
20 Z. Y. He, J. Liu and L. X. Yang. Blind Separation of Images Using Edgeworth Expansion Based on ICA Algorithm. Chinese Journal of Electronics. 1999, 8(3):278~282
21凌燮亭.近场宽带信号源的盲分离.电子学报. 1996, 24(7):87~92
22朱孝龙.盲自适应信号分离的并行实现方法研究.西安电子科技大学. 2003:15~34
23张贤达,朱孝龙,保铮.基于分阶段学习的盲信号分离.中国科学. 2002, 32(5):693~703
24 J. Cheng, Y. W. Chen and H. Q. Lu. Color and Texture Based Image Segmentation Using Local Feature Analysis Approach. Third International Symposium on Multispectral Image Processing and Pattern Recognition. 2003,5286:600~604
25黄启宏,王帅,刘钊.改进的基于独立成分分析的图像特征提取算法.光电工程. 2007, 34(1):121~125
26黄启宏,段昶,刘钊.基于独立分量分析的图像特征提取及泊松噪声去除.光电工程. 2006, 33(11):128~132
27 Y. G. Liu, G. Chen and J. W. Lu. Face Recognition Based on Independent Component Analysis and Fuzzy Support Vector Machine. Proceedings of 6th WCICA. 2006:5~8
28安高云,阮秋琦.一种基于小波分解及鲁棒估计的ICA算法及其在人脸识别中的应用.电子学报. 2006, 34(10):1900~1905
29王毅,齐华,郝重阳.一种基于独立分量分析的模糊图像盲分离算法.计算机应用. 2006, 26(10):2366~2368
30杨俊安,解光军,庄镇泉.量子遗传算法及其在图像盲分离中的应用研究.计算机辅助设计与图形学学报. 2003, 15(7):847~851
31刘琚,张新刚,孙建德.一种基于ICA的图像水印方法.电路与系统学报. 2003, 8(3):55~59
32张力,钱恭斌,纪震.基于ICA的智能数字水印技术.电子与信息学报. 2005, 27(3):478~482
33陈蜜,伭??李德仁.独立分量分析的图像融合算法.光电工程. 2007, 34(6):82~87
34 A. Hyv?rinen, P. Pajunen . Nonlinear Independent Component Analysis: Existence and Uniqueness Results.Neural Networks. 1999, 12(3):429~439
35 A. Hyv?rinen, J. Karhunen and E. Oja.Independent Component Analysis.A Wiley-Interscience Publication. 2001:56~90
36 A. Hyv?rinen. Gaussian Moments for Noisy Independent Component Analysis. IEEE Signal Processing Letters. 1999, 6(6):145~147
37 P. Comon, P. Chevalier. Blind Source Separation: Models, Concepts, Algorithms and Performance, Unsupervised Adaptive Filtering. John Wiley&Sons. 2000:139~189
38 M. Lazaro et al. Stochastic Blind Equalization Based on PDF Fitting Using Parzen Estimator. IEEE Transactions on Signal Processing. 2005, 53(2):696~704
39 X. Li. Blind Channel Estimation and Equalization in Wireless Sensor Networks Based on Correlations Among Sensors. IEEE Transactions on Signal Processing. 2005, 53(4):1511~1519
40 Q. Y. Li, E. W. Bai and Z. Ding. Blind Source Separation of Signal with Known Alphabets Usingε-approximation Algorithms. IEEE Transactions Signal Processing. 2003, 51(1):1~10
41 Y. W. Yao, F. B. Giannakis. On Regularity and Identifiability of Blind Source Separation under Constant-modulus Constraints. IEEE Transactions Signal Processing. 2005, 53(4):1272~1281
42 S. Buzzi, A. D. Maio. Code-aided Blind Detection of the Transmission Rate for Multirate Direct-sequence CDMA Systems. IEEE Transactions on Signal Processing. 2005,53(6):2182~2192
43 X. G. Doukopoulos, G. V. Moustakides. Adaptive Power Techniques for Blind Channel Estimation in CDMA Systems. IEEE Transactions on Signal Processing. 2005,53(3):1110~1120
44何振亚.自适应信号处理.科学出版社. 2002:21~56
45 T. W. Lee. A Unifying Information-theoretic Framework for Independent Component Analysis. Computers and Mathematics with Applications. 2000,31(11):1~21
46 J. F. Cardoso. Blind Signal Separation: Statistical Principles. Proceedings of the IEEE. 1998,86(10):2009~2025
47 A. Hyv?rinen , E. Oja. Independent Component Analysis: Algorithms and Applications. Neural Networks. 2000,13:411~430
48 A. Hyv?rinen, P. O. Hoyer. Emergence of Phase and Shift Invariant Features by Decomposition of Natural Images into Independent Feature Subspaces. Neural Computation. 2000,12(7):1705~1720
49杨福生,洪波.独立分量分析的原理与应用.北京:清华大学出版社. 2006:1~46
50 T. W. Lee et al. Independent Component Analysis Using an Extended InfoMax Algorithm for Mixed Sub-Gaussian and Super-Gaussian Sources. Neural Computation. 1999, 11(2):409~433
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |