非参数先验分布的确定及其应用
|
摘要
统计问题中,贝叶斯方法在很多方面已经硕果累累,不过在处理非参数方面却仍存在很大的差距,这主要由于在参数空间上寻找有效先验分布是非常困难的,具体到非参数问题就是在给定样本空间取一个概率分布集。
基于Ferguson1973年的文章,在非参数问题中,对先验分布有两方面的要求:
(1)样本空间中,相对于概率分布空间上的某些适当的(弱)拓扑,先验分布必须有足够大的支撑。这就保证了先验选择的灵活性与广泛性,以便于找到最适合模型的分布函数。
(2)在给定先验分布类和观测样本时,后验分布必须易于计算,至少有可行的计算方法。从而保证在实际中的应用价值。
然而这两个要求是相悖的,一方的满足必须以牺牲另一方为条件。我们通常的处理方法是通过放宽第一个条件,而将第二个条件设置为共轭类来构造分布类。
参看最近几十年的文章,我们可以发现,在处理非参数贝叶斯问题中用到最多的先验分布,都是现已有的几种具体的先验,如Dirichlet过程,Talifree过程,中立过程,Polya树等。由于先验分布的限制,所以贝叶斯方法在处理非参数问题时,受到了阻力。因此,有必要研究在确定非参数问题中是否存在确定先验分布的一般方法或者是在一定限制条件下确定先验分布的一般方法这一基本问题。本文基于Ferguson对先验分布提出的两方面的要求和现已知的先验分布的构造方法,讨论了在可数样本空间和不可数样本空间上的先验分布的一些构造方法及相应先验分布的性质,并且给出了Dirichlet过程先验在估计后验均值方面的应用。
本文主要做了以下几方面的工作:
1.给出了先验分布在可数样本空间上的构造,通过规范化构造和Stick-breaking构造两种方法进行说明,并说明了构造方法的可行性。
2.给出了先验分布在不可数样本空间上的构造,通过Binning构造,增量过程构造,剖分树构造等六种不同的构造方法确定先验分布。
3.讨论了几种先验分布的一些性质和重要结论。
4.给出了Dirichlet过程先验在估计后验均值方面的应用。
The Bayesian approach to statistical problems,though fruitful in many ways,has been rather unsuccessful in treating nonparametric problems.This is due primarily to the difficulty in finding workable prior distributions on the parameter space,which in nonparametric problems is taken to be a set of probability distributions on a given sample space.
Based on the paper of Ferguson in 1973, there are two desirable properties of a prior distribution for nonparametric problems:
(1) The support of the prior distribution should be large-with respect to some suitable topology on the space of probability distributions on the sample space.This can assure the feasibility and universality of the prior,so we can find the best model for the distribution.
(2) Posterior distributions given a sample of observations from the true probability should be manageable analytically.It requires the Posterior distributions have the same forms as the priors,or they are conjugate classes,or they can easily be computed.
These properties are antagonistic in the sense that one may be obtained at the expense of the other.We usually broad a class of prior distributions in the sense of (1),for which (2) is realized by given in the sense of conjugate class.
Refer to the papers in the past few decades, the prior distributions we used most in treating nonparametric problems are those prior classes, eg: Dirichlet processes, Tailfree processes, neutral processes, Polya tree and so on. The Bayesian approach to statistical problems has been unsuccessful in treating nonparametic problems. This is due primarily to the limitations of prior distribution. It is necessary to consider whether there is a general method of construct prior distribution under some conditions. Based on two desirable properties of a prior distribution for nonparametric problems and some known prior distributions construction, some methods of construct prior distribution on countable sample spaces and uncountable sample spaces are introduced and given an algorithm to estimate the values of posterior means with a Dirichlet process prior.
This paper does the work as following:
1. Given methods of construction of prior distributions on countablesample spaces, i.e. construction via normalization and construction via stick-breaking
2. Given methods of construction of prior distributions on uncount?able sample spaces, i.e. construct prior via binning,via increasing processes,via partitioning tree and so on.
3. Discussed some properties and important facts of prior distribu?tion.
4. Given an algorithm to estimate the values of posterior means witha Dirichlet process prior.
基于Ferguson1973年的文章,在非参数问题中,对先验分布有两方面的要求:
(1)样本空间中,相对于概率分布空间上的某些适当的(弱)拓扑,先验分布必须有足够大的支撑。这就保证了先验选择的灵活性与广泛性,以便于找到最适合模型的分布函数。
(2)在给定先验分布类和观测样本时,后验分布必须易于计算,至少有可行的计算方法。从而保证在实际中的应用价值。
然而这两个要求是相悖的,一方的满足必须以牺牲另一方为条件。我们通常的处理方法是通过放宽第一个条件,而将第二个条件设置为共轭类来构造分布类。
参看最近几十年的文章,我们可以发现,在处理非参数贝叶斯问题中用到最多的先验分布,都是现已有的几种具体的先验,如Dirichlet过程,Talifree过程,中立过程,Polya树等。由于先验分布的限制,所以贝叶斯方法在处理非参数问题时,受到了阻力。因此,有必要研究在确定非参数问题中是否存在确定先验分布的一般方法或者是在一定限制条件下确定先验分布的一般方法这一基本问题。本文基于Ferguson对先验分布提出的两方面的要求和现已知的先验分布的构造方法,讨论了在可数样本空间和不可数样本空间上的先验分布的一些构造方法及相应先验分布的性质,并且给出了Dirichlet过程先验在估计后验均值方面的应用。
本文主要做了以下几方面的工作:
1.给出了先验分布在可数样本空间上的构造,通过规范化构造和Stick-breaking构造两种方法进行说明,并说明了构造方法的可行性。
2.给出了先验分布在不可数样本空间上的构造,通过Binning构造,增量过程构造,剖分树构造等六种不同的构造方法确定先验分布。
3.讨论了几种先验分布的一些性质和重要结论。
4.给出了Dirichlet过程先验在估计后验均值方面的应用。
The Bayesian approach to statistical problems,though fruitful in many ways,has been rather unsuccessful in treating nonparametric problems.This is due primarily to the difficulty in finding workable prior distributions on the parameter space,which in nonparametric problems is taken to be a set of probability distributions on a given sample space.
Based on the paper of Ferguson in 1973, there are two desirable properties of a prior distribution for nonparametric problems:
(1) The support of the prior distribution should be large-with respect to some suitable topology on the space of probability distributions on the sample space.This can assure the feasibility and universality of the prior,so we can find the best model for the distribution.
(2) Posterior distributions given a sample of observations from the true probability should be manageable analytically.It requires the Posterior distributions have the same forms as the priors,or they are conjugate classes,or they can easily be computed.
These properties are antagonistic in the sense that one may be obtained at the expense of the other.We usually broad a class of prior distributions in the sense of (1),for which (2) is realized by given in the sense of conjugate class.
Refer to the papers in the past few decades, the prior distributions we used most in treating nonparametric problems are those prior classes, eg: Dirichlet processes, Tailfree processes, neutral processes, Polya tree and so on. The Bayesian approach to statistical problems has been unsuccessful in treating nonparametic problems. This is due primarily to the limitations of prior distribution. It is necessary to consider whether there is a general method of construct prior distribution under some conditions. Based on two desirable properties of a prior distribution for nonparametric problems and some known prior distributions construction, some methods of construct prior distribution on countable sample spaces and uncountable sample spaces are introduced and given an algorithm to estimate the values of posterior means with a Dirichlet process prior.
This paper does the work as following:
1. Given methods of construction of prior distributions on countablesample spaces, i.e. construction via normalization and construction via stick-breaking
2. Given methods of construction of prior distributions on uncount?able sample spaces, i.e. construct prior via binning,via increasing processes,via partitioning tree and so on.
3. Discussed some properties and important facts of prior distribu?tion.
4. Given an algorithm to estimate the values of posterior means witha Dirichlet process prior.
引文
[1]Andrea Ongaro,Carla Cattaneo(2004).Discrete random probability measures:a general framework for nonparametric Bayesian inference.Statistics & Probability Letters67 33-45.
[2]Berger.J.o.统计决策理论及贝叶斯分析[M].贾乃光,译(1998).北京:中国统计出版社,1985.
[3]Ferguson.T.(1973).A Bayesian analysis of some nonparametric problems.Ann.Statist.1,209-230.
[4]Jayaram Sethuraman(1994).A constructive definition of dirichlet priors.Statistica Sinica4 639-650.
[5]Aaron A.D'Souza.Notes on Dirichlet Process.
[6]Peter Muller.Fernando A.Quintana.(?)(2004).Nonparametric Bayesian Data Analysis.Statist.Sci.19,no.1,95-110.
[7]KJELL DOKSUM(1974).Tailfree and neutral random probabilities and their posterior distributions.The Annals of Probability 1974,Vol.2,No.2,183-201.
[8]Ferguson.T.(1974).Prior distributions on spaces of probability measures.The Annals of Statistics1974,Vol.2,No.4,615-629.
[9]DAVID BLACKWELL AND JAMES B.MACQUEEN(1973).Ferguson distributions via polya urn schemes.Ann.Statist.1 353-355.
[10]Blackwell,D.(1973),Discreteness of Ferguson selections.Ann.Statist.1356-358.
[11]Jeremy Oakley and Anthony O'Hagan.October 8,2003.Uncertainty in prior elicitations:a nonparametric approach.Research Report No.521/02.
[12]R.M.Balan(2004).Q-markov random probability measures and their posterior distributions.Stochastic processes and their applications 109(2004)295-316.
[13]Jaynes,E.T.(1968),Prior probabilities IEEE Transactions on Systems,and Cybernetics,SSC-4.227.
[14]Kotz.s.吴喜之。现代贝叶斯统计学[M]北京:中国统计出版社,2000.
[15]Press,S.J.(1989)贝叶斯统计学,原理,模型及应用,廖文,陈安贵等译,中国统计出版社,1992。
[16]陈希孺,数理统计引论,科学出版社,1997。
[17]陈希孺,高等数理统计学,中国科学技术大学出版社,1999。
[18]成平,陈希孺,陈桂景,吴传义,参数估计,上海科学出版社,1985。
[19]峁诗松,贝叶斯统计,中国统计出版社,1999。
[20]张尧庭,陈汉峰,贝叶斯统计推断,科学出版社,1991。
[21]张金槐,唐雪梅,Bayes方法(修改版),国防科技大学出版社,1993。
[22]峁诗松,王静龙,濮晓龙,高等数理统计,高等教育出版社,1998。
[23]黎子良,统计推断与决策,南开大学出版社,1987。
[24]陈希孺,数理统计中的两个学派-频率学派和Bayes学派,数理统计和应用概率,1990,5(4)。
[25]陈珽,决策分析,科学出版社,1997。
[26]Cohn.D.L.1980.Measure Theory.Birkhauser,Boston.
[27]Hjort,N.L.2000.Bayesian analysis for a generalized Dirichlet process prior.Statistical Research Report.Department of Mathematics,University of Oslo.
[28]Hjort,N.L.,Ongaro,A.,2003.Bayesian inference using an extension of the Dirichlet process.Statistical Research Report.Department of Mathematics,University of Oslo.
[29]Jayaram Sethuraman and Tiwari,R.C.(1982).Convergence of Dirichlet measures and the interpretation of their parameter.Statistical Decision Theory and Related Topics Ⅲ2,305-315.
[30]Wilks,S.S.(1962).Mathematical Statistics.John Wiley,a)New York.
[31]Thomas Bayes.An essay towards solving a problem in the doctrine of chances,published in.Philosophical Transactions of the Royal Society.of London in 1763.
[32]Bayes' Theorem.Joseph Berkson.The Annals of Mathematical Statistics,Vol.1,No.1(Feb.,1930),pp.42-56.
[33]Wald,A,(1950),Statistic Decision Function,Wiley,New York.(中译本,王福保译,统计决策函数,上海科学出版社)。
[34]Savage,L.J(1954),Fundations of Statistics,Wiley,New York.
[35]Subhashis Ghosal(2005),Theory of nonparametric Bayesian inference.
[36]Dykstra,R.L.,and laud,P.(1981)A Bayesian nonparametric approach to reliability.Ann.Statist.9,356-367.
[37]Hjort,N.L.,(1990).Nonparametric Bayes estimators based on Bata process in models for life history data.Ann.Statist.18,1259-1294.
[38]Walker,S.,Muliere,P.,1997.Beta-Stacy processes and a generalization of the Polya-urn scheme.Ann.Statist.25.,1762-1780.
[39]Walker,S.,Muliere,P.,1999.A characterization of a neutral to the right prior via an extension of Johnson's sufficientness postulate.Ann.Statist.27,589-599.
[40]Martz,Walter,1982,Bayesian Reliability Analysis.
[41]Lavine,M.(1992),Some Aspects of Polya Tree Distribution for Statistical Modelling,Ann.Statist.,20,1222-1235.
[42]Lavine,M.(1994),More Aspects of Polya Tree Distribution for Statistical Modelling,Ann.Statist.,Vol.22,No.3,1161-1176.
[43]Michael D.Escobar(1994),Estimateing Normal means with a Dirichlet process prior,Journal of American Statistical Association,Vol.89,No.425.
[44]北京大学数学系几何与代数教研室代数小组,高等代数,北京:高等教育出版社,1988。
[45]刘玉琏;傅沛仁,数学分析讲义,北京:高等教育出版社,1985。
[46]盛骤,谢式干等,概率论与数理统计,高等教育出版社,2001。
[47]李裕奇等,概率论与数理统计,国防工业出版社,2004。
[48]李裕奇,随机过程,国防工业出版社,2003。
[49](英)艾奇逊,J.,成分数据的统计分析,武汉:中国地质大学出版社,1990。
[50]严加安,测度论讲义,北京:科学出版社,1998。
[51]Leonard,Thomas,Hsu,John S.J.,贝叶斯方法,北京:中国机械出版社,2005。
[52]朱慧明,韩玉启,贝叶斯多元统计推断理论,北京:科学出版社,2006。
[53]陈希孺等,非参数统计,上海:上海科学技术出版社,1989。
[54]李裕奇,刘海燕,赵联文,非参数统计方法,成都:西南交通大学出版社,1998。
[55]张尧庭,方开泰(1982,1997),多元统计分析引论,科学出版社。
[56]胡迪鹤,分析概率论,科学出版社,1984。
[2]Berger.J.o.统计决策理论及贝叶斯分析[M].贾乃光,译(1998).北京:中国统计出版社,1985.
[3]Ferguson.T.(1973).A Bayesian analysis of some nonparametric problems.Ann.Statist.1,209-230.
[4]Jayaram Sethuraman(1994).A constructive definition of dirichlet priors.Statistica Sinica4 639-650.
[5]Aaron A.D'Souza.Notes on Dirichlet Process.
[6]Peter Muller.Fernando A.Quintana.(?)(2004).Nonparametric Bayesian Data Analysis.Statist.Sci.19,no.1,95-110.
[7]KJELL DOKSUM(1974).Tailfree and neutral random probabilities and their posterior distributions.The Annals of Probability 1974,Vol.2,No.2,183-201.
[8]Ferguson.T.(1974).Prior distributions on spaces of probability measures.The Annals of Statistics1974,Vol.2,No.4,615-629.
[9]DAVID BLACKWELL AND JAMES B.MACQUEEN(1973).Ferguson distributions via polya urn schemes.Ann.Statist.1 353-355.
[10]Blackwell,D.(1973),Discreteness of Ferguson selections.Ann.Statist.1356-358.
[11]Jeremy Oakley and Anthony O'Hagan.October 8,2003.Uncertainty in prior elicitations:a nonparametric approach.Research Report No.521/02.
[12]R.M.Balan(2004).Q-markov random probability measures and their posterior distributions.Stochastic processes and their applications 109(2004)295-316.
[13]Jaynes,E.T.(1968),Prior probabilities IEEE Transactions on Systems,and Cybernetics,SSC-4.227.
[14]Kotz.s.吴喜之。现代贝叶斯统计学[M]北京:中国统计出版社,2000.
[15]Press,S.J.(1989)贝叶斯统计学,原理,模型及应用,廖文,陈安贵等译,中国统计出版社,1992。
[16]陈希孺,数理统计引论,科学出版社,1997。
[17]陈希孺,高等数理统计学,中国科学技术大学出版社,1999。
[18]成平,陈希孺,陈桂景,吴传义,参数估计,上海科学出版社,1985。
[19]峁诗松,贝叶斯统计,中国统计出版社,1999。
[20]张尧庭,陈汉峰,贝叶斯统计推断,科学出版社,1991。
[21]张金槐,唐雪梅,Bayes方法(修改版),国防科技大学出版社,1993。
[22]峁诗松,王静龙,濮晓龙,高等数理统计,高等教育出版社,1998。
[23]黎子良,统计推断与决策,南开大学出版社,1987。
[24]陈希孺,数理统计中的两个学派-频率学派和Bayes学派,数理统计和应用概率,1990,5(4)。
[25]陈珽,决策分析,科学出版社,1997。
[26]Cohn.D.L.1980.Measure Theory.Birkhauser,Boston.
[27]Hjort,N.L.2000.Bayesian analysis for a generalized Dirichlet process prior.Statistical Research Report.Department of Mathematics,University of Oslo.
[28]Hjort,N.L.,Ongaro,A.,2003.Bayesian inference using an extension of the Dirichlet process.Statistical Research Report.Department of Mathematics,University of Oslo.
[29]Jayaram Sethuraman and Tiwari,R.C.(1982).Convergence of Dirichlet measures and the interpretation of their parameter.Statistical Decision Theory and Related Topics Ⅲ2,305-315.
[30]Wilks,S.S.(1962).Mathematical Statistics.John Wiley,a)New York.
[31]Thomas Bayes.An essay towards solving a problem in the doctrine of chances,published in.Philosophical Transactions of the Royal Society.of London in 1763.
[32]Bayes' Theorem.Joseph Berkson.The Annals of Mathematical Statistics,Vol.1,No.1(Feb.,1930),pp.42-56.
[33]Wald,A,(1950),Statistic Decision Function,Wiley,New York.(中译本,王福保译,统计决策函数,上海科学出版社)。
[34]Savage,L.J(1954),Fundations of Statistics,Wiley,New York.
[35]Subhashis Ghosal(2005),Theory of nonparametric Bayesian inference.
[36]Dykstra,R.L.,and laud,P.(1981)A Bayesian nonparametric approach to reliability.Ann.Statist.9,356-367.
[37]Hjort,N.L.,(1990).Nonparametric Bayes estimators based on Bata process in models for life history data.Ann.Statist.18,1259-1294.
[38]Walker,S.,Muliere,P.,1997.Beta-Stacy processes and a generalization of the Polya-urn scheme.Ann.Statist.25.,1762-1780.
[39]Walker,S.,Muliere,P.,1999.A characterization of a neutral to the right prior via an extension of Johnson's sufficientness postulate.Ann.Statist.27,589-599.
[40]Martz,Walter,1982,Bayesian Reliability Analysis.
[41]Lavine,M.(1992),Some Aspects of Polya Tree Distribution for Statistical Modelling,Ann.Statist.,20,1222-1235.
[42]Lavine,M.(1994),More Aspects of Polya Tree Distribution for Statistical Modelling,Ann.Statist.,Vol.22,No.3,1161-1176.
[43]Michael D.Escobar(1994),Estimateing Normal means with a Dirichlet process prior,Journal of American Statistical Association,Vol.89,No.425.
[44]北京大学数学系几何与代数教研室代数小组,高等代数,北京:高等教育出版社,1988。
[45]刘玉琏;傅沛仁,数学分析讲义,北京:高等教育出版社,1985。
[46]盛骤,谢式干等,概率论与数理统计,高等教育出版社,2001。
[47]李裕奇等,概率论与数理统计,国防工业出版社,2004。
[48]李裕奇,随机过程,国防工业出版社,2003。
[49](英)艾奇逊,J.,成分数据的统计分析,武汉:中国地质大学出版社,1990。
[50]严加安,测度论讲义,北京:科学出版社,1998。
[51]Leonard,Thomas,Hsu,John S.J.,贝叶斯方法,北京:中国机械出版社,2005。
[52]朱慧明,韩玉启,贝叶斯多元统计推断理论,北京:科学出版社,2006。
[53]陈希孺等,非参数统计,上海:上海科学技术出版社,1989。
[54]李裕奇,刘海燕,赵联文,非参数统计方法,成都:西南交通大学出版社,1998。
[55]张尧庭,方开泰(1982,1997),多元统计分析引论,科学出版社。
[56]胡迪鹤,分析概率论,科学出版社,1984。