Zadeh的隶属函数对似然方法、语义通信和统计学习的意义
|
摘要
流行的似然方法不合适数据先验分布(即信源)可变场合。为此,我们把Zadeh的隶属函数看做预测模型,用隶属函数和可变信源产生似然函数,用平均对数标准(normalized)似然度定义语义信息测度。这样可以保证:(1)坚持使用最大似然准则;(2)预测模型适合信源可变场合;(3)得到的语义贝叶斯预测兼容贝叶斯定理;(4)预测模型能表达语义,便于理解。一组隶属函数构成一个语义信道,优化隶属函数就是使语义信道匹配Shannon信道,产生多标签模糊分类。文中介绍了通过两种信道相互匹配求解最大似然度的迭代算法。几个例子显示这种算法用于检验、估计和混合模型时,收敛快速且可靠。
The popular likelihood method cannot be properly used in cases where the prior distribution of data(or sources) are variable.Hence,we use Zadeh's membership function as the predictive model, use this function with a changeable source to produce a likelihood function,and define the semantic information measure with average log-normalized-likelihood. Then we can ensure that(1) the maximum likelihood criterion is always adopted;(2) the predictive model may be used in cases where sources are changeable;(3) the probability prediction is compatible with the Bayes' theorem;(4) a predictive model may indicate the semantic meaning of a hypothesis and may be more understandable. A group of membership functions form a semantic channel. To optimize a group of membership functions is to let a semantic channel match a Shannon's channel to make a multi-class and multi-label fuzzy classification. Through two channels' mutual matching,we can obtain an iterative algorithm for maximum mutual information and maximum likelihood. Several examples show that this algorithm for tests,estimations,and mixture models is fast and reliable.
The popular likelihood method cannot be properly used in cases where the prior distribution of data(or sources) are variable.Hence,we use Zadeh's membership function as the predictive model, use this function with a changeable source to produce a likelihood function,and define the semantic information measure with average log-normalized-likelihood. Then we can ensure that(1) the maximum likelihood criterion is always adopted;(2) the predictive model may be used in cases where sources are changeable;(3) the probability prediction is compatible with the Bayes' theorem;(4) a predictive model may indicate the semantic meaning of a hypothesis and may be more understandable. A group of membership functions form a semantic channel. To optimize a group of membership functions is to let a semantic channel match a Shannon's channel to make a multi-class and multi-label fuzzy classification. Through two channels' mutual matching,we can obtain an iterative algorithm for maximum mutual information and maximum likelihood. Several examples show that this algorithm for tests,estimations,and mixture models is fast and reliable.
引文
[1] Zadeh L A.Fuzzy sets[J].Information and Control,1965,8(3):338～353.
[2] Dubois D,Moral S,Prade H.A semantics for possibility theory based on likelihoods[J].Journal of Mathematical Analysis and Applications,1997,205(20):359～380.
[3] Cattaneo M E G V.The likelihood interpretation as the foundation of fuzzy set theory[J].International Journal of Approximate Reasoning,Available online 22 August 2017.
[4] Yang C C.Fuzzy Bayesian inference[C]//Computational Cybernetics and Simulation,IEEE International Conference on Systems,Man,and Cybernetics Conference Proceedings,Orlando,1997,Vol.3:2707～2712.
[5] Viertl R.Foundations of fuzzy bayesian inference[J].Journal of Uncertain Systems,2008,2(3):187～191.
[6] Thomas S F.Possibilistic uncertainty and statistical inference[C]//ORSA/TIMS Meeting,Houston,1981.
[7] 鲁晨光.B-模糊集合代数和广义交互熵公式[J].模糊系统和数学,1991,5(1):76～80.
[8] Zadeh L A.Probability measures of fuzzy events[J].Journal of Mathematical Analysis and Applications,1968,23(2):421～427.
[9] Davidson D.Truth and meaning[J].Synthese,1967,17:304～323.
[10] 汪培庄.模糊集和随机集落影[M].北京师范大学出版社,1985.
[11] 鲁晨光.广义信息论[M].中国科学技术大学出版社,1993.
[12] Shannon C E.A mathematical theory of communication[J].Bell System Technical Journal,1948,27(3):379～429;623～656.
[13] Lu C.A generalization of Shannon’s information theory[J].Int.J.of General Systems,1999,28(6):453～490.
[14] Fisher R A.On the mathematical foundations of theoretical statistics[J].Philo.Trans.Roy.Soc.,1922,A222:309～368.
[15] Cover T M,Thomas J A.Elements of information theory[M] 2nd Edition.New York:John Wiley & Sons,2006.
[16] Popper K.Conjectures and refutations[M].Repr.London and New York:Routledge,1963/2005.
[17] Carnap R,Bar-Hillel Y.An outline of a theory of semantic information[R].Tech.Rep.No.247,Research Lab.of Electronics,MIT,1952.
[18] Akaike H.A new look at the statistical model identification[J].IEEE Transactions on Automatic Control,1974,19:716～723.
[19] 汪培庄.因素空间和数据科学[J].辽宁工程技术大学学报,2015,34(2):273～280.
[20] Thornbury J R,Fryback D G,Edwards W.Likelihood ratios as a measure of the diagnostic usefulness of excretory urogram information[J].Radiology,1975,114(3):561～565.
[21] Kok M,Dahlin J,Schon B,Wills T B.A Newton-based maximum likelihood estimation in nonlinear state space models[OL].IFAC-PapersOnLine,2015,48:398～403.
[22] Zhang M L,Zhou Z H.A review on multilabel learning algorithms[J].IEEE Trans.Knowledge and Data Engineering,2014,26(8):1819～1837.
[23] Dempster A P,Laird N M,Rubin D B.Maximum likelihood from incomplete data via the EM algorithm[J].Journal of the Royal Statistical Society,Series B,1977,39(1):1～38.
[24] Barron A,Roos T,Watanabe K.Bayesian properties of normalized maximum likelihood and its fast computation[C]//IEEE IT Symposium on Information Theory,2014:1667～1671.
[25] Lu C.Semantic channel and Shannon channel mutually match and iterate for tests and estimations with maximum mutual information and maximum likelihood[C]//Proceedings of International Conference on Big Data and Smart Computing,Shanghai,2018.
[26] Lu C.Channels’ matching algorithm for mixture models[C]//Proceedings of International Conference on Intelligence Science,Shanghai,2017.
(1)https://en.wikipedia.org/wiki/Bayesian_inference.
[2] Dubois D,Moral S,Prade H.A semantics for possibility theory based on likelihoods[J].Journal of Mathematical Analysis and Applications,1997,205(20):359～380.
[3] Cattaneo M E G V.The likelihood interpretation as the foundation of fuzzy set theory[J].International Journal of Approximate Reasoning,Available online 22 August 2017.
[4] Yang C C.Fuzzy Bayesian inference[C]//Computational Cybernetics and Simulation,IEEE International Conference on Systems,Man,and Cybernetics Conference Proceedings,Orlando,1997,Vol.3:2707～2712.
[5] Viertl R.Foundations of fuzzy bayesian inference[J].Journal of Uncertain Systems,2008,2(3):187～191.
[6] Thomas S F.Possibilistic uncertainty and statistical inference[C]//ORSA/TIMS Meeting,Houston,1981.
[7] 鲁晨光.B-模糊集合代数和广义交互熵公式[J].模糊系统和数学,1991,5(1):76～80.
[8] Zadeh L A.Probability measures of fuzzy events[J].Journal of Mathematical Analysis and Applications,1968,23(2):421～427.
[9] Davidson D.Truth and meaning[J].Synthese,1967,17:304～323.
[10] 汪培庄.模糊集和随机集落影[M].北京师范大学出版社,1985.
[11] 鲁晨光.广义信息论[M].中国科学技术大学出版社,1993.
[12] Shannon C E.A mathematical theory of communication[J].Bell System Technical Journal,1948,27(3):379～429;623～656.
[13] Lu C.A generalization of Shannon’s information theory[J].Int.J.of General Systems,1999,28(6):453～490.
[14] Fisher R A.On the mathematical foundations of theoretical statistics[J].Philo.Trans.Roy.Soc.,1922,A222:309～368.
[15] Cover T M,Thomas J A.Elements of information theory[M] 2nd Edition.New York:John Wiley & Sons,2006.
[16] Popper K.Conjectures and refutations[M].Repr.London and New York:Routledge,1963/2005.
[17] Carnap R,Bar-Hillel Y.An outline of a theory of semantic information[R].Tech.Rep.No.247,Research Lab.of Electronics,MIT,1952.
[18] Akaike H.A new look at the statistical model identification[J].IEEE Transactions on Automatic Control,1974,19:716～723.
[19] 汪培庄.因素空间和数据科学[J].辽宁工程技术大学学报,2015,34(2):273～280.
[20] Thornbury J R,Fryback D G,Edwards W.Likelihood ratios as a measure of the diagnostic usefulness of excretory urogram information[J].Radiology,1975,114(3):561～565.
[21] Kok M,Dahlin J,Schon B,Wills T B.A Newton-based maximum likelihood estimation in nonlinear state space models[OL].IFAC-PapersOnLine,2015,48:398～403.
[22] Zhang M L,Zhou Z H.A review on multilabel learning algorithms[J].IEEE Trans.Knowledge and Data Engineering,2014,26(8):1819～1837.
[23] Dempster A P,Laird N M,Rubin D B.Maximum likelihood from incomplete data via the EM algorithm[J].Journal of the Royal Statistical Society,Series B,1977,39(1):1～38.
[24] Barron A,Roos T,Watanabe K.Bayesian properties of normalized maximum likelihood and its fast computation[C]//IEEE IT Symposium on Information Theory,2014:1667～1671.
[25] Lu C.Semantic channel and Shannon channel mutually match and iterate for tests and estimations with maximum mutual information and maximum likelihood[C]//Proceedings of International Conference on Big Data and Smart Computing,Shanghai,2018.
[26] Lu C.Channels’ matching algorithm for mixture models[C]//Proceedings of International Conference on Intelligence Science,Shanghai,2017.
(1)https://en.wikipedia.org/wiki/Bayesian_inference.