鞅方法与经验过程方法在统计中的应用
|
摘要
本文利用鞅方法和经验过程方法研究了相依样本的非参数统计与极限定理,建立了若干相关的结果.
其一,利用鞅方法和分组技术,我们得到了ρ-混合样本核密度估计fn,K(x)的中心极限定理以及分布函数的估计Fn,K(x)=f-∞xfn,K(t)dt的中心极限定理；建立了混合相依样本条件期望的矩不等式.作为应用,研究了‖fn,K(x)-Efn,K(x)‖p在一致范数和积分Lp范数下的收敛速度；最后给出了|fn,K(x)-Efn,K(x)|在紧集以及整个实空间R上的强一致收敛速度.在适当的条件下,我们证明了关于独立同分布样本的最优结果对混合样本同样是最优的.
其二,利用分块技巧,我们构造出一列m相依随机变量,由此将问题从相依转化为独立情形,然后借助于独立序列一些精细的概率不等式,给出了一类平稳过程核密度估计的渐近性质：首先建立了密度估计与其均值(真实密度)偏差的点态和一致最优弱收敛速度；其次得到了点态和一致最优强收敛速度.
其三,我们建立了小波回归估计的中心极限定理.其次利用经验过程方法,强期望上界的熵估计,Bernstein不等式以及Talagrand不等式,我们得到了一致范数下几乎必然收敛的最优速度.
其四,利用鞅方法和分块技巧,在较弱的相依速度条件下,我们建立了ρ-混合随机变量部分和在方差有限和无界情形下的强不变原理,改进了已有的一些结果.作为应用,得到了方差有限和无界情形下的一些重对数律型结果,并且解决了Shao(1993a)文中的一个猜想.
最后,借助于概率不等式和弱不变原理,我们研究了负相依随机变量部分和与最大部分和完全矩收敛的精确渐近性,给出了一些更一般的新的结果.
Using the martingale and empirical process methods, this dissertation in-vestigates the nonparametric statistics and limit theorems for dependent obser-vations, many results are constructed. The main contents include the following aspects.
Firstly, Using the martingale method and blocking technique, the central limit theorems (CLT) for the kernel density estimator fn,K(x) and the distribu-tion function estimator Fn,K(x)=f-∞x fn,K(t)dt are established. Further,we construct a moment inequality of the conditional expectation for dependent ran-dom variables. As applications, the convergence rates of‖fn,K(x)-Efnt,Kx(x)‖p in sup-norm loss and integral Lp-norm loss are proved. Moreover, the a.s. con-vergence rates of the supremum of|fn,K(x)-E fn,K(x)|over a compact set and the whole real line are obtained. It is showed, under suitable conditions, that the optimal rates for i.i.d. random variables are also optimal for dependent ones.
Secondly, using the blocking technique, we construct m dependent random variables, then we reduce the investigation for dependent samples to that for independent ones. Several limit theorems are established:First, the pointwise and the uniformly optimal weak convergence rates of the deviation of the kernel density estimator with respect to its mean (and the true density function) are derived. Moreover, the pointwise and the uniformly optimal strong convergence rates are obtained.
Thirdly, we establish the CLT for wavelet regression estimator. Moreover, using empirical process methods, the upper bound for the strong expectation with respect to the entropy, Bernstein inequality and Talagrand inequality, we derive the a.s. best possible convergence rate in sup-norm loss for wavelet regression estimators.
Fourthly, using the martingale method and blocking technique, we establish strong invariance principles for sums of stationary ρ-mixing random variables with finite and unbounded second moments under weaker mixing rates. Some earlier results are improved. As applications, some LIL results with finite and unbounded variance are obtained, also a conjecture raised by Shao (1993a) is solved.
Finally, using the probability inequalities and the weak invariance principles, the limit behavior of the complete moment convergence of partial sums and maximal partial sums for negatively associated random variables is investigated. Some more general and new results are given.
其一,利用鞅方法和分组技术,我们得到了ρ-混合样本核密度估计fn,K(x)的中心极限定理以及分布函数的估计Fn,K(x)=f-∞xfn,K(t)dt的中心极限定理；建立了混合相依样本条件期望的矩不等式.作为应用,研究了‖fn,K(x)-Efn,K(x)‖p在一致范数和积分Lp范数下的收敛速度；最后给出了|fn,K(x)-Efn,K(x)|在紧集以及整个实空间R上的强一致收敛速度.在适当的条件下,我们证明了关于独立同分布样本的最优结果对混合样本同样是最优的.
其二,利用分块技巧,我们构造出一列m相依随机变量,由此将问题从相依转化为独立情形,然后借助于独立序列一些精细的概率不等式,给出了一类平稳过程核密度估计的渐近性质：首先建立了密度估计与其均值(真实密度)偏差的点态和一致最优弱收敛速度；其次得到了点态和一致最优强收敛速度.
其三,我们建立了小波回归估计的中心极限定理.其次利用经验过程方法,强期望上界的熵估计,Bernstein不等式以及Talagrand不等式,我们得到了一致范数下几乎必然收敛的最优速度.
其四,利用鞅方法和分块技巧,在较弱的相依速度条件下,我们建立了ρ-混合随机变量部分和在方差有限和无界情形下的强不变原理,改进了已有的一些结果.作为应用,得到了方差有限和无界情形下的一些重对数律型结果,并且解决了Shao(1993a)文中的一个猜想.
最后,借助于概率不等式和弱不变原理,我们研究了负相依随机变量部分和与最大部分和完全矩收敛的精确渐近性,给出了一些更一般的新的结果.
Using the martingale and empirical process methods, this dissertation in-vestigates the nonparametric statistics and limit theorems for dependent obser-vations, many results are constructed. The main contents include the following aspects.
Firstly, Using the martingale method and blocking technique, the central limit theorems (CLT) for the kernel density estimator fn,K(x) and the distribu-tion function estimator Fn,K(x)=f-∞x fn,K(t)dt are established. Further,we construct a moment inequality of the conditional expectation for dependent ran-dom variables. As applications, the convergence rates of‖fn,K(x)-Efnt,Kx(x)‖p in sup-norm loss and integral Lp-norm loss are proved. Moreover, the a.s. con-vergence rates of the supremum of|fn,K(x)-E fn,K(x)|over a compact set and the whole real line are obtained. It is showed, under suitable conditions, that the optimal rates for i.i.d. random variables are also optimal for dependent ones.
Secondly, using the blocking technique, we construct m dependent random variables, then we reduce the investigation for dependent samples to that for independent ones. Several limit theorems are established:First, the pointwise and the uniformly optimal weak convergence rates of the deviation of the kernel density estimator with respect to its mean (and the true density function) are derived. Moreover, the pointwise and the uniformly optimal strong convergence rates are obtained.
Thirdly, we establish the CLT for wavelet regression estimator. Moreover, using empirical process methods, the upper bound for the strong expectation with respect to the entropy, Bernstein inequality and Talagrand inequality, we derive the a.s. best possible convergence rate in sup-norm loss for wavelet regression estimators.
Fourthly, using the martingale method and blocking technique, we establish strong invariance principles for sums of stationary ρ-mixing random variables with finite and unbounded second moments under weaker mixing rates. Some earlier results are improved. As applications, some LIL results with finite and unbounded variance are obtained, also a conjecture raised by Shao (1993a) is solved.
Finally, using the probability inequalities and the weak invariance principles, the limit behavior of the complete moment convergence of partial sums and maximal partial sums for negatively associated random variables is investigated. Some more general and new results are given.
引文
[1]Alam, K., Saxena, K. M. L. (1981). Positive dependence in multivariate distributions. Communications in Statistics 10,1183-1196.
[2]Antoniadis, A., Pham, D.T. (1998). Wavelet regression for random or irreg-ular design. Computational Statistics and Data Analysis 28,353-369.
[3]Balan, R. M., Kulik, R. (2005). Self-normalized weak invariance principle for mixing sequences. Lab Reas Probab Stat Univ Ottawa-Carleton Univ, Tech Rep Ser,417,1-15.
[4]Balan, R. M., Zamfirescu, I. M. (2006). Strong approximation for mixing sequences with infinite variance. Electronic Communications in Probability 11,11-23.
[5]Barlow, R. E., Proschan, F. (1981). Statistical theory of reliability and life testing:Probability Models. Silver Spring.
[6]Billingsley, P. (1968). Convergence of probability measure. Wiley, New York.
[7]Bickel, P. J., Rosenblatt, M. (1973). On some global measures of the devia-tions of density function estimates. Annals of Statistics 1,1071-1095.
[8]Bosq, D. (1996). Nonparametric Statistics for Stochastic Processes:Estima-tion and Prediction. Lecture Notes in Statistics 110, Springer, New York.
[9]Bradley, R. C., Bryc, W. (1985). Multilinear forms and measures of de-pendence between random variables. Journal of Multivariate Analysis 16, 335-367.
[10]Cai, T. (1999). Adaptive wavelet estimation:A block thresholding and oracle inequality approach. Annals of Statistics 27,898-924.
[11]Cai, T. T., Zhou, H. H. (2006). A data-driven block thresholding approach to wavelet estimation. Manuscript.
[12]Cai, Z. W. (1990). Strong consistency and rates for the probability den-sity estimators for weakly dependent random variables. Journal of Systems Science and Mathematical Sciences 10,360-370.
[13]Chow, Y. S. (1988). On the rate of moment convergence of sample sums and extremes. Bulletin of the Institute of Mathematics. Academia Sinica 16,177-201.
[14]Chung, K. L. (1948). On the maximum partial sums of sequences of indepen-dent random variables. Transactions of the American Mathematical Society 64,205-233.
[15]Cox, J. T., Grimmett, G. (1984). Central limit theorems for associated ran-dom variables and the percolation model. Annals of Probability 12,514-528.
[16]Csaki, E. (1977). The law of iterated logarithm for normalized empirical distributuion function. Zeitschrift fur Wahrshcheinlichkeitstheorie und Ver-wandte Gebiete 38.147-167.
[17]Csaki, E. (1978). On the lower limits of maxima of Wiener process and par-tial sums. Zeitschrift fur Wahrshcheinlichkeitstheorie und Verwandte Gebi-ete 43,205-222.
[18]Csorgo, M., Revesz, P. (1975a). A new method to prove Strassen type laws of invariance principle, I. Zeitschrift fur Wahrshcheinlichkeitstheorie und Verwandte Gebiete 31,255-260.
[19]Csorgo, M., Revesz, P. (1975b). A new method to prove Strassen type laws of invariance principle, Ⅱ. Zeitschrift fur Wahrshcheinlichkeitstheorie und Verwandte Gebiete 31,261-269.
[20]Csorgo, M., Revesz, P. (1981). Strong approximation in probability and statistics. Academic Press, New York.
[21]Csorgo, M., Szyszkowicz, B., Wang, Q. (2003). Donsker's theorem for self-normalized partial sums processes. Annals of Probability 31,1228-1240.
[22]Daubechies, I. (1992). Ten Lectures on Wavelets. CBMS-NSF Regional Conf. Ser. in Applied Mathematics. SIAM, Philadelphia.
[23]Deheuvels, P. (2000). Uniform limit laws for kernel density estimators on possibly unbounded intervals. In Recent Advances in Reliability Theory: Methodology, Practice and Inference Birkhauser, Boston.
[24]Pena V de la, Gine, E. (1999). Decoupling, from dependence to independence. Springer-Verlag, New York.
[25]Delyon, B., Juditsky, A. (1996). On minimax wavelat estimators. Appl. Com-put Harmon. Anal.3,215-228.
[26]Donoho, B., Johnstone, I. M., Keekyacharian, G., Picard, D. (1995). Wavelet shrinkage:Asymptopia? Journal of the Royal Statistical Society:Series B 57,301-369.
[27]Donoho, B., Johnstone, I. M., Keekyacharian, G., Picard, D. (1996). Density estimation by wavelet thresholding. Annals of Statistics 24,508-539.
[28]Donsker, M. (1951). An invariance principle for certain probability limit theorems. Memoirs of the American Mathematical Society 6,1-4.
[29]Dudley, R. M. (1984). A course on empirical processes. In Ecole d'Ete de Probabilites de Saint-Flour XII-1982. Lecture Notes in Mathematics 1097, 2-142.
[30]Dvoretzky, A., Kiefer, J., Wolfowitz, J. (1956). Asymptotic minimax char-acter of the sample distribution function and of the classical multinomial estimator. Ann. Math. Statist.27,642-669.
[31]Einmahl, U. (1987a). Strong invariance principle for partial sums of inde-pendent random vectors. Annals of Probability 15,1419-1440.
[32]Einmahl, U. (1987b). A useful estimate in the multi-dimensional invariance principle. Probability Theory and Related Fields 76,81-101.
[33]Einmahl, U. (1989). Extensions of results of Komlos Major, and Tusnady to the multivariate case. Journal of Multivariate Analysis 28,20-68.
[34]Einmahl, U. (2007). A generalization of Strassen's functional LIL. Journal of Theoretical Probability 20,901-915.
[35]Einmahl, U. (2009). A new strong invariance principle for sums of indepen-dent random vectors. Journal of Mathematical Sciences 163,311-327.
[36]Einmahl, U., Li, D. (2005). Some results on two-sided LIL behavior. Annals of Probability 33,1601-1624.
[37]Einmahl, U., Mason, D. M. (1996). Some universal results on the behavior of increments of partial sum. Annals of Probability 24,1388-1407.
[38]Einmahl,U., Mason, D. M. (2000). An empirical process approach to the uniform consistency of kernel-type function estimators. Journal of Theoret-ical Probability 13,1-37.
[39]Einmahl, U., Mason, D. M. (2005). Uniform in bandwidth consistency of kernel-type function estimators. Annals of Statistics 33,1380-1403.
[40]Erdos, P., Revesz, P. (1990). A new law of iterated logarithm. Acta Mathe-matica Hungarica 55,125-131.
[41]Fan, J. (1992). Design-adaptive nonparametric regression. Journal of the American Statistical Association 87,998-1004.
[42]Fan, J. (1993). Local linear regression smoothers and their minimax efficien-cies. Annals of Statistics 21,196-216.
[43]Fan, J., Yao, Q. (1998). Efficient estimation of conditional variance functions in stochastic regression. Biometrika 85,645-660.
[44]Fan, J. Q., Yao, Q. (2003). Nonlinear Time Series. Nonparametric and Para-metric Methods. Springer, New York.
[45]Freedman, D. A. (1975). On tail probabilities for martingales. Annals of Probability 3,100-118.
[46]Fodes, A. (1974). Density estimation for dependent sample. Studia Scien-tiarum Mathematicarum Hungarica 9,443-452.
[47]Gine, E., Guillou, A. (2002a). On consistency of kernel density estimators for randomly censored data:Rates holding uniformly over adaptive intervals. Annales de l'Institut Henri Poincare. Probabilites et Statistiques 37,503-522.
[48]Gine, E., Guillou, A. (2002b). Rates of strong consistency for multivariate kernel density estimators. Annales de l'Institut Henri Poincare. Probabilites et Statistiques 38,907-922.
[49]Gine, E., Koltchinskii, V., Zinn, J. (2004). Weighted uniform consistency of kernel density estimators. Annals of Probability 32 2570-2605.
[50]Gine, E., Nickl, R. (2009a). An exponential inequality for the distribution function of the kernel density estimator, with applications to adaptive esti-mation. Probability Theory and Related Fields 143,569-596.
[51]Gine, E., Nickl, R. (2009b). Uniform limit theorems for wavelet density estimators. Annals of Probability 37,1605-1646.
[52]Gine, E., Zinn, J. (1984). Some limit theorems for empirical processes. An-nals of Probability 12,929-989.
[53]Gordin, M. I. (1969). The central limit theorem for stationary Markov pro-cesses. Dokl. Akad. Nauk. SSR.188,739-741.
[54]Gordin, M. I, Lifsic, B. (1978). The central limit theorem for stationary Markov processes. Soviet. Math. Dokl.19,392-394.
[55]Hall, P., Heyde, C. C. (1986). Martingale Limit Theory and its Applications. Academic Press, New York.
Hall, P., Kerkyacharian, G., Picard, D. (1998). Block threshold rules for curve estimation using kernel and wavelet methods. Annals of Statistics 26, 922-942.
[57]Hanson, D. L., Russo, P. (1983). Some results on increments of the Wiener processes with applications to lag sums of i.i.d. random variables. Annals of Probability 11,609-623.
[58]Herrndorf, N. (1983). The Invariance Principle for p-mixing Sequences. Zeitschrift fur Wahrshcheinlichkeitstheorie und Verwandte Gebiete 63,97-108.
[59]Ho, H. C., Hsing, T. (1997). Limit theorems for functional of moving aver-ages. Annals of Probability 25,1636-1669.
[60]Huang, W., Zhang, L. X. (2005). Precise asymptotics in the laws of the logarithm in the Hilbert space. Journal of Mathematical Analysis and Ap-plications 304,734-758.
[61]Hsu, P. L., Robbins, H. (1947). Complete convergence and the law of large numbers. Proceedings of the National Academy of Sciences of the United States of America 33,25-31.
[62]Ibragimov, I. A. (1962). Some limit theorems for stationary processes. The-ory of Probability and its Applications 7,349-382.
[63]Joag-Dev K., Proschan, F. (1983). Negative association of random variables with applications. Annals of Statistics 11,286-295.
[64]Juditsky, A., Lambert-Lacroix, S. (2004). On minimax density estimation on K. Bernoulli 10,187-220.
[65]Kallianpur, G. (1981). Some ramifications of Wiener's ideas on nonlinear prediction, in:P. Masani (Ed.), Norbert Wiener, Collected Works,3,402-424. MIT Press, Cambridge.
[66]Kerkyacharian, G., Picard, D. (1992). Density estimation in Besov spaces. Statistics and Probability Letters 13,15-24.
[67]Klass, M. J. (1976a). Forward a universal law of the iterated logarithm I. Zeitschrift fur Wahrshcheinlichkeitstheorie und Verwandte Gebiete 36,165-178.
[68]Klass, M. J. (1976b). Forward a universal law of the iterated logarithm II. Zeitschrift fur Wahrshcheinlichkeitstheorie und Verwandte Gebiete 39,151-165.
[69]Kolmogorov, A. N., Rozanov, U. A. (1960). On the strong mixing conditions of a stationary Gaussian process. Theory of Probability and its Applications 2,222-227.
[70]Komlos, J., Major, P., Tusnady, G. (1975). An approximation of partial sums of independent R.V.'s and the sample D.F. I. Zeitschrift fur Wahrshchein-lichkeitstheorie und Verwandte Gebiete 32,111-131.
[71]Komlos, J., Major, P., Tusnady, G. (1976). An approximation of partial sums of independent R.V.'s and the sample D.F. II. Zeitschrift fur Wahrshchein-lichkeitstheorie und Verwandte Gebiete 34,35-58.
[72]Kuelbs, J. (1976). The law of the itrated logarithm and related strong con-vergence theorem for Banach space valued random variables. Lecture Notes in Mathematics 539, Springer, New York.
[73]Liang, H. Y. (2000). Complete convergence for weighted sums of NA random variabls. Statistics and Probability Letters 48,317-325.
[74]Liang, H. Y., Su, C. (1999). Complete convergence for weighted sums of NA sequences. Statistics and Probability Letters 45,85-95.
[75]Liebscher, E. (1996). Strong convergence of sums of α-mixing random vari-ables with applications todensity estimation. Stochastic Processes with their Applications 65 69-80.
[76]Lin, Z., Lu, C. (1992). Strong Limit Theorems. Science Press and Kluwer Academic Publishers.
[77]Lin, Z., Lu, C. (1997). Limit Theory on Mixing Dependent Random Vari-ables. Science Press and Kluwer Academic Publishers.
[78]Liu, L. X., Wu, R. (2002). Inequality of maximal partial sum and the law of iterated logarithm for sequences of NA random variables. Acta Mathematicae Sinica 45,969-978.
[79]Liu,W., Lin, Z. (2009). Strong approximation for a class of stationary pro-cesses. Stochastic Processes with their Applications 119,249-280.
[80]Machkouri, M. E. (2011). Kernel density estimation for stationary random fields. Manuscript.
[81]Major, P. (1976). The approximation of partial sums of independent rv's. Zeitschrift fur Wahrshcheinlichkeitstheorie und Verwandte Gebiete 35,213-220.
[82]Major, P. (1979). An improvement of Strassen's invariance priciple. Annals of Probability 7,55-61.
[83]Mallat, S. G. (1989). A theory for multiresolution signal decomposition:the wavelet representation. IEEE Trans. Pattern Anal. Mach. Intelligence 11, 674-693.
[84]Maxwell, M., Woodroofe, M. (2000). Central limit theorems for additive functional of Markov chains. Annals of Probability 28,713-724.
[85]Meyer, Y. (1992). Wavelets and Operators. Cambridge Studies in Advanced Mathematics Cambridge University Press, Cambridge.
[86]Montgomery-Smith, S. J. (1993). Comparison of sums of independent iden-tically distributed random vectors. Probability and Mathematical Statistics 14 281-285.
[87]Matula, P. (1992). A note on the almost sure convergence of sums of neg-atively dependent random variables. Statistics and Probability Letters 15, 209-213.
[88]Nagaev, S. V. (1979). Large deviations of sums of independent random vari-ables. Annals of Probability 7,745-789.
[89]Newman, C. M. (1984) Asymptotic independent and limit theorems for pos-itively and negatively dependent random variables. In:Tong, Y. L.(Ed.), Inequalities in Statistics and Probability. IMS. Hayward, CA,127-140.
[90]Neumann, M. H. (1998). Strong approximation of density estimators from weakly dependent observations by density estimators from independent ob-servations. Annals of Statistics 26,2014-2048.
[91]Nze, P. A., Rios, R. (1995). Density estimation in the L∞ norm for mixing processes (in French). C.R. Acad. Sci. Paris 320 1259-1262.
[92]Parzen, E. (1962). On the estimation of a probability density function and the mode. Ann. Math. Statist.33,1065-1076.
[93]Peligrad, M. (1982). Invariance principles for mixing sequences of random variables. Annals of Probability 10,968-981.
[94]Peligrad, M. (1992). Properties of uniform consistency of the kernel estima-tors of density and of regression functions under dependent assumptions. Stock. Reports 40,147-168.
[95]Peligrad, M., Shao, Q. M. (1995). Estimation of the variance of partial sums for ρ-mixing random variables. Journal of Mutivariate Analysis 52,140-157.
[96]Peligrad, M.,Utev, S. (2005). A new maximal inequality and invariance principle for stationary sequences. Annals of Probability 33,798-815.
[97]Petrov, V. V. (1995). Limit theorems of probability theory-sequences of in-dependent random variables. Oxford Science Publications, Oxford.
Priestley, M. B. (1988). Non-linear and Non-stationary Time Series Analy-sis. Academic Press, New York.
[99]Rosenblatt, M. (1956). Remarks on some nonparametric estimates of a den-sity function. Ann. Math. Statist.27,832-835.
[100]Roussas, G. G. (1988). Nonparametric estimation in mixing sequences of random variables. Journal of Statistical Planning and Inference 18,135-149.
[101]Ruschendorf, L. (1977). Consistency of estimators for multivariate density functions and for the mode. Sankhya 39,243-250.
[102]Samur, J. D. (1985). A note on the convergence to Gaussian laws of sums of stationary φ-mixing triangular arrays. Probability in Banach space, V (Medford, Mass.,1984), Lecture Notes in Mathematics Berlin:Springer. 1153,387-399.
[103]Shao, Q. M. (1989a). On the invariance principle for ρ-mixing sequences of random variables. Chinese Annals of Mathematics 10B,427-433.
[104]Shao, Q. M. (1989b). On the complete convergence for ρ-mixing sequence. Acta Mathematica Sinica 32,377-393.
[105]Shao, Q. M. (1993a). Almost sure invariance principles for mixing sequences of random variables. Stochastic Processes with their Applications 48,319-334.
[106]Shao, Q. M. (1993b). On the invariance principle for stationary ρ-mixing sequence with infinite variance. Chinese Annals of Mathematics 14B,27-42.
[107]Shao, Q. M. (1995a). Strong approximation theorems for independent ran-dom variables and their applications. Journal of Multivariate Analysis 52, 107-130.
[108]Shao, Q. M. (1995b). Maximal inequality for partial sums of ρ-mixing sequences. Annals of Probability 23,948-965.
[109]Shao, Q. M., Su, C. (1999). The law of the the iterated logarithm for NA random variables. Stochastic Processes with their Applications 83,139-148.
[110]Shao, Q. M. (2000). A comparison theorem on maximum inequalities be-tween negatively associated sequences and independent random variables. Journal of Theoretical Probability 13,343-356.
[111]Stout, W. (1974). Almost Sure Convergence. Academic Press, New York.
[112]Strassen, V. (1964). An invariance principle for the law of the iterated log-arithm. Zeitschrift fur Wahrshcheinlichkeitstheorie und Verwandte Gebiete 3,211-226.
[113]Su, C., Zhao, L. C., Wang, Y. B. (1996). The moment inequalities and weak convergence for negatively associated sequences. Science in China Series A 26,1091-1099.
[114]Talagrand, M. (1994). Sharper bounds for Gaussian and empirical pro-cesses. Annals of Probability 22,28-76.
[115]Talagrand, M. (1996). New concentration inequalities in product spaces. Inventiones Mathematicae 126,505-563.
[116]Tong, H. (1990). Non-linear Time Series:A Dynamical System Approach. Oxford University Press.
[117]Wiener, N. (1958). Nonlinear Problems in Random Theory. MIT Press, John Wiley.
[118]Withers, C. S. (1981). Conditions for linear process to be strongly mixing. Zeitschrift fur Wahrshcheinlichkeitstheorie und Verwandte Gebiete 57,477-480.
[119]Woodroofe, M. (1967). On the maximum deviation of the sample density. Ann. Math. Statist.38,475-481.
[120]Woodroofe, M. (1970). Discussion of "Density estimates and Markov se-quences" by M. Rosenblatt. In nonparametric Techniques in Statistical In-ference. Cambridge University Press.
[121]Wu, W. B. (2007). Strong invariance principles for dependent random vari-ables. Annals of Probability 35,2294-2320.
[122]Wu, W. B., Woodroofe,M. (2004). Martingale approximations for sums of stationary processes. Annals of Probability 32,1674-1690.
[123]Wu, W. B., Huang, Y., Huang, Y. (2010). Kernel estimation for time se-ries:An asymptotic theory. Stochastic Processes with their Applications 120, 2412-2431.
[124]Yakowitz, S. (1985). nonparametric density estimation, prediction and re-gression for Markov sequences. Journal of the American Statistical Associa-tion 80,215-221.
[125]Yu, B. (1993). Density estimation in the L∞ norm for dependent data with applications to the Gibbs sampler. Annals of Statistics 21,711-735.
[126]Zhang, L. X. (2004). Some limit results on the law of the iterated logarithm of NA sequences. Acta Mathematica Sinica 47,541-552.
[127]Zhang, L. X. (2001). The weak convergence for functions of negatively associated random variables. Journal of Multivariate Analysis 78,272-298.
[128]Zhang, L. X., Wen, J. W.(2001). A weak convergence for negatively asso-ciated fields. Statistics and Probability Letters 53,259-267.
[2]Antoniadis, A., Pham, D.T. (1998). Wavelet regression for random or irreg-ular design. Computational Statistics and Data Analysis 28,353-369.
[3]Balan, R. M., Kulik, R. (2005). Self-normalized weak invariance principle for mixing sequences. Lab Reas Probab Stat Univ Ottawa-Carleton Univ, Tech Rep Ser,417,1-15.
[4]Balan, R. M., Zamfirescu, I. M. (2006). Strong approximation for mixing sequences with infinite variance. Electronic Communications in Probability 11,11-23.
[5]Barlow, R. E., Proschan, F. (1981). Statistical theory of reliability and life testing:Probability Models. Silver Spring.
[6]Billingsley, P. (1968). Convergence of probability measure. Wiley, New York.
[7]Bickel, P. J., Rosenblatt, M. (1973). On some global measures of the devia-tions of density function estimates. Annals of Statistics 1,1071-1095.
[8]Bosq, D. (1996). Nonparametric Statistics for Stochastic Processes:Estima-tion and Prediction. Lecture Notes in Statistics 110, Springer, New York.
[9]Bradley, R. C., Bryc, W. (1985). Multilinear forms and measures of de-pendence between random variables. Journal of Multivariate Analysis 16, 335-367.
[10]Cai, T. (1999). Adaptive wavelet estimation:A block thresholding and oracle inequality approach. Annals of Statistics 27,898-924.
[11]Cai, T. T., Zhou, H. H. (2006). A data-driven block thresholding approach to wavelet estimation. Manuscript.
[12]Cai, Z. W. (1990). Strong consistency and rates for the probability den-sity estimators for weakly dependent random variables. Journal of Systems Science and Mathematical Sciences 10,360-370.
[13]Chow, Y. S. (1988). On the rate of moment convergence of sample sums and extremes. Bulletin of the Institute of Mathematics. Academia Sinica 16,177-201.
[14]Chung, K. L. (1948). On the maximum partial sums of sequences of indepen-dent random variables. Transactions of the American Mathematical Society 64,205-233.
[15]Cox, J. T., Grimmett, G. (1984). Central limit theorems for associated ran-dom variables and the percolation model. Annals of Probability 12,514-528.
[16]Csaki, E. (1977). The law of iterated logarithm for normalized empirical distributuion function. Zeitschrift fur Wahrshcheinlichkeitstheorie und Ver-wandte Gebiete 38.147-167.
[17]Csaki, E. (1978). On the lower limits of maxima of Wiener process and par-tial sums. Zeitschrift fur Wahrshcheinlichkeitstheorie und Verwandte Gebi-ete 43,205-222.
[18]Csorgo, M., Revesz, P. (1975a). A new method to prove Strassen type laws of invariance principle, I. Zeitschrift fur Wahrshcheinlichkeitstheorie und Verwandte Gebiete 31,255-260.
[19]Csorgo, M., Revesz, P. (1975b). A new method to prove Strassen type laws of invariance principle, Ⅱ. Zeitschrift fur Wahrshcheinlichkeitstheorie und Verwandte Gebiete 31,261-269.
[20]Csorgo, M., Revesz, P. (1981). Strong approximation in probability and statistics. Academic Press, New York.
[21]Csorgo, M., Szyszkowicz, B., Wang, Q. (2003). Donsker's theorem for self-normalized partial sums processes. Annals of Probability 31,1228-1240.
[22]Daubechies, I. (1992). Ten Lectures on Wavelets. CBMS-NSF Regional Conf. Ser. in Applied Mathematics. SIAM, Philadelphia.
[23]Deheuvels, P. (2000). Uniform limit laws for kernel density estimators on possibly unbounded intervals. In Recent Advances in Reliability Theory: Methodology, Practice and Inference Birkhauser, Boston.
[24]Pena V de la, Gine, E. (1999). Decoupling, from dependence to independence. Springer-Verlag, New York.
[25]Delyon, B., Juditsky, A. (1996). On minimax wavelat estimators. Appl. Com-put Harmon. Anal.3,215-228.
[26]Donoho, B., Johnstone, I. M., Keekyacharian, G., Picard, D. (1995). Wavelet shrinkage:Asymptopia? Journal of the Royal Statistical Society:Series B 57,301-369.
[27]Donoho, B., Johnstone, I. M., Keekyacharian, G., Picard, D. (1996). Density estimation by wavelet thresholding. Annals of Statistics 24,508-539.
[28]Donsker, M. (1951). An invariance principle for certain probability limit theorems. Memoirs of the American Mathematical Society 6,1-4.
[29]Dudley, R. M. (1984). A course on empirical processes. In Ecole d'Ete de Probabilites de Saint-Flour XII-1982. Lecture Notes in Mathematics 1097, 2-142.
[30]Dvoretzky, A., Kiefer, J., Wolfowitz, J. (1956). Asymptotic minimax char-acter of the sample distribution function and of the classical multinomial estimator. Ann. Math. Statist.27,642-669.
[31]Einmahl, U. (1987a). Strong invariance principle for partial sums of inde-pendent random vectors. Annals of Probability 15,1419-1440.
[32]Einmahl, U. (1987b). A useful estimate in the multi-dimensional invariance principle. Probability Theory and Related Fields 76,81-101.
[33]Einmahl, U. (1989). Extensions of results of Komlos Major, and Tusnady to the multivariate case. Journal of Multivariate Analysis 28,20-68.
[34]Einmahl, U. (2007). A generalization of Strassen's functional LIL. Journal of Theoretical Probability 20,901-915.
[35]Einmahl, U. (2009). A new strong invariance principle for sums of indepen-dent random vectors. Journal of Mathematical Sciences 163,311-327.
[36]Einmahl, U., Li, D. (2005). Some results on two-sided LIL behavior. Annals of Probability 33,1601-1624.
[37]Einmahl, U., Mason, D. M. (1996). Some universal results on the behavior of increments of partial sum. Annals of Probability 24,1388-1407.
[38]Einmahl,U., Mason, D. M. (2000). An empirical process approach to the uniform consistency of kernel-type function estimators. Journal of Theoret-ical Probability 13,1-37.
[39]Einmahl, U., Mason, D. M. (2005). Uniform in bandwidth consistency of kernel-type function estimators. Annals of Statistics 33,1380-1403.
[40]Erdos, P., Revesz, P. (1990). A new law of iterated logarithm. Acta Mathe-matica Hungarica 55,125-131.
[41]Fan, J. (1992). Design-adaptive nonparametric regression. Journal of the American Statistical Association 87,998-1004.
[42]Fan, J. (1993). Local linear regression smoothers and their minimax efficien-cies. Annals of Statistics 21,196-216.
[43]Fan, J., Yao, Q. (1998). Efficient estimation of conditional variance functions in stochastic regression. Biometrika 85,645-660.
[44]Fan, J. Q., Yao, Q. (2003). Nonlinear Time Series. Nonparametric and Para-metric Methods. Springer, New York.
[45]Freedman, D. A. (1975). On tail probabilities for martingales. Annals of Probability 3,100-118.
[46]Fodes, A. (1974). Density estimation for dependent sample. Studia Scien-tiarum Mathematicarum Hungarica 9,443-452.
[47]Gine, E., Guillou, A. (2002a). On consistency of kernel density estimators for randomly censored data:Rates holding uniformly over adaptive intervals. Annales de l'Institut Henri Poincare. Probabilites et Statistiques 37,503-522.
[48]Gine, E., Guillou, A. (2002b). Rates of strong consistency for multivariate kernel density estimators. Annales de l'Institut Henri Poincare. Probabilites et Statistiques 38,907-922.
[49]Gine, E., Koltchinskii, V., Zinn, J. (2004). Weighted uniform consistency of kernel density estimators. Annals of Probability 32 2570-2605.
[50]Gine, E., Nickl, R. (2009a). An exponential inequality for the distribution function of the kernel density estimator, with applications to adaptive esti-mation. Probability Theory and Related Fields 143,569-596.
[51]Gine, E., Nickl, R. (2009b). Uniform limit theorems for wavelet density estimators. Annals of Probability 37,1605-1646.
[52]Gine, E., Zinn, J. (1984). Some limit theorems for empirical processes. An-nals of Probability 12,929-989.
[53]Gordin, M. I. (1969). The central limit theorem for stationary Markov pro-cesses. Dokl. Akad. Nauk. SSR.188,739-741.
[54]Gordin, M. I, Lifsic, B. (1978). The central limit theorem for stationary Markov processes. Soviet. Math. Dokl.19,392-394.
[55]Hall, P., Heyde, C. C. (1986). Martingale Limit Theory and its Applications. Academic Press, New York.
Hall, P., Kerkyacharian, G., Picard, D. (1998). Block threshold rules for curve estimation using kernel and wavelet methods. Annals of Statistics 26, 922-942.
[57]Hanson, D. L., Russo, P. (1983). Some results on increments of the Wiener processes with applications to lag sums of i.i.d. random variables. Annals of Probability 11,609-623.
[58]Herrndorf, N. (1983). The Invariance Principle for p-mixing Sequences. Zeitschrift fur Wahrshcheinlichkeitstheorie und Verwandte Gebiete 63,97-108.
[59]Ho, H. C., Hsing, T. (1997). Limit theorems for functional of moving aver-ages. Annals of Probability 25,1636-1669.
[60]Huang, W., Zhang, L. X. (2005). Precise asymptotics in the laws of the logarithm in the Hilbert space. Journal of Mathematical Analysis and Ap-plications 304,734-758.
[61]Hsu, P. L., Robbins, H. (1947). Complete convergence and the law of large numbers. Proceedings of the National Academy of Sciences of the United States of America 33,25-31.
[62]Ibragimov, I. A. (1962). Some limit theorems for stationary processes. The-ory of Probability and its Applications 7,349-382.
[63]Joag-Dev K., Proschan, F. (1983). Negative association of random variables with applications. Annals of Statistics 11,286-295.
[64]Juditsky, A., Lambert-Lacroix, S. (2004). On minimax density estimation on K. Bernoulli 10,187-220.
[65]Kallianpur, G. (1981). Some ramifications of Wiener's ideas on nonlinear prediction, in:P. Masani (Ed.), Norbert Wiener, Collected Works,3,402-424. MIT Press, Cambridge.
[66]Kerkyacharian, G., Picard, D. (1992). Density estimation in Besov spaces. Statistics and Probability Letters 13,15-24.
[67]Klass, M. J. (1976a). Forward a universal law of the iterated logarithm I. Zeitschrift fur Wahrshcheinlichkeitstheorie und Verwandte Gebiete 36,165-178.
[68]Klass, M. J. (1976b). Forward a universal law of the iterated logarithm II. Zeitschrift fur Wahrshcheinlichkeitstheorie und Verwandte Gebiete 39,151-165.
[69]Kolmogorov, A. N., Rozanov, U. A. (1960). On the strong mixing conditions of a stationary Gaussian process. Theory of Probability and its Applications 2,222-227.
[70]Komlos, J., Major, P., Tusnady, G. (1975). An approximation of partial sums of independent R.V.'s and the sample D.F. I. Zeitschrift fur Wahrshchein-lichkeitstheorie und Verwandte Gebiete 32,111-131.
[71]Komlos, J., Major, P., Tusnady, G. (1976). An approximation of partial sums of independent R.V.'s and the sample D.F. II. Zeitschrift fur Wahrshchein-lichkeitstheorie und Verwandte Gebiete 34,35-58.
[72]Kuelbs, J. (1976). The law of the itrated logarithm and related strong con-vergence theorem for Banach space valued random variables. Lecture Notes in Mathematics 539, Springer, New York.
[73]Liang, H. Y. (2000). Complete convergence for weighted sums of NA random variabls. Statistics and Probability Letters 48,317-325.
[74]Liang, H. Y., Su, C. (1999). Complete convergence for weighted sums of NA sequences. Statistics and Probability Letters 45,85-95.
[75]Liebscher, E. (1996). Strong convergence of sums of α-mixing random vari-ables with applications todensity estimation. Stochastic Processes with their Applications 65 69-80.
[76]Lin, Z., Lu, C. (1992). Strong Limit Theorems. Science Press and Kluwer Academic Publishers.
[77]Lin, Z., Lu, C. (1997). Limit Theory on Mixing Dependent Random Vari-ables. Science Press and Kluwer Academic Publishers.
[78]Liu, L. X., Wu, R. (2002). Inequality of maximal partial sum and the law of iterated logarithm for sequences of NA random variables. Acta Mathematicae Sinica 45,969-978.
[79]Liu,W., Lin, Z. (2009). Strong approximation for a class of stationary pro-cesses. Stochastic Processes with their Applications 119,249-280.
[80]Machkouri, M. E. (2011). Kernel density estimation for stationary random fields. Manuscript.
[81]Major, P. (1976). The approximation of partial sums of independent rv's. Zeitschrift fur Wahrshcheinlichkeitstheorie und Verwandte Gebiete 35,213-220.
[82]Major, P. (1979). An improvement of Strassen's invariance priciple. Annals of Probability 7,55-61.
[83]Mallat, S. G. (1989). A theory for multiresolution signal decomposition:the wavelet representation. IEEE Trans. Pattern Anal. Mach. Intelligence 11, 674-693.
[84]Maxwell, M., Woodroofe, M. (2000). Central limit theorems for additive functional of Markov chains. Annals of Probability 28,713-724.
[85]Meyer, Y. (1992). Wavelets and Operators. Cambridge Studies in Advanced Mathematics Cambridge University Press, Cambridge.
[86]Montgomery-Smith, S. J. (1993). Comparison of sums of independent iden-tically distributed random vectors. Probability and Mathematical Statistics 14 281-285.
[87]Matula, P. (1992). A note on the almost sure convergence of sums of neg-atively dependent random variables. Statistics and Probability Letters 15, 209-213.
[88]Nagaev, S. V. (1979). Large deviations of sums of independent random vari-ables. Annals of Probability 7,745-789.
[89]Newman, C. M. (1984) Asymptotic independent and limit theorems for pos-itively and negatively dependent random variables. In:Tong, Y. L.(Ed.), Inequalities in Statistics and Probability. IMS. Hayward, CA,127-140.
[90]Neumann, M. H. (1998). Strong approximation of density estimators from weakly dependent observations by density estimators from independent ob-servations. Annals of Statistics 26,2014-2048.
[91]Nze, P. A., Rios, R. (1995). Density estimation in the L∞ norm for mixing processes (in French). C.R. Acad. Sci. Paris 320 1259-1262.
[92]Parzen, E. (1962). On the estimation of a probability density function and the mode. Ann. Math. Statist.33,1065-1076.
[93]Peligrad, M. (1982). Invariance principles for mixing sequences of random variables. Annals of Probability 10,968-981.
[94]Peligrad, M. (1992). Properties of uniform consistency of the kernel estima-tors of density and of regression functions under dependent assumptions. Stock. Reports 40,147-168.
[95]Peligrad, M., Shao, Q. M. (1995). Estimation of the variance of partial sums for ρ-mixing random variables. Journal of Mutivariate Analysis 52,140-157.
[96]Peligrad, M.,Utev, S. (2005). A new maximal inequality and invariance principle for stationary sequences. Annals of Probability 33,798-815.
[97]Petrov, V. V. (1995). Limit theorems of probability theory-sequences of in-dependent random variables. Oxford Science Publications, Oxford.
Priestley, M. B. (1988). Non-linear and Non-stationary Time Series Analy-sis. Academic Press, New York.
[99]Rosenblatt, M. (1956). Remarks on some nonparametric estimates of a den-sity function. Ann. Math. Statist.27,832-835.
[100]Roussas, G. G. (1988). Nonparametric estimation in mixing sequences of random variables. Journal of Statistical Planning and Inference 18,135-149.
[101]Ruschendorf, L. (1977). Consistency of estimators for multivariate density functions and for the mode. Sankhya 39,243-250.
[102]Samur, J. D. (1985). A note on the convergence to Gaussian laws of sums of stationary φ-mixing triangular arrays. Probability in Banach space, V (Medford, Mass.,1984), Lecture Notes in Mathematics Berlin:Springer. 1153,387-399.
[103]Shao, Q. M. (1989a). On the invariance principle for ρ-mixing sequences of random variables. Chinese Annals of Mathematics 10B,427-433.
[104]Shao, Q. M. (1989b). On the complete convergence for ρ-mixing sequence. Acta Mathematica Sinica 32,377-393.
[105]Shao, Q. M. (1993a). Almost sure invariance principles for mixing sequences of random variables. Stochastic Processes with their Applications 48,319-334.
[106]Shao, Q. M. (1993b). On the invariance principle for stationary ρ-mixing sequence with infinite variance. Chinese Annals of Mathematics 14B,27-42.
[107]Shao, Q. M. (1995a). Strong approximation theorems for independent ran-dom variables and their applications. Journal of Multivariate Analysis 52, 107-130.
[108]Shao, Q. M. (1995b). Maximal inequality for partial sums of ρ-mixing sequences. Annals of Probability 23,948-965.
[109]Shao, Q. M., Su, C. (1999). The law of the the iterated logarithm for NA random variables. Stochastic Processes with their Applications 83,139-148.
[110]Shao, Q. M. (2000). A comparison theorem on maximum inequalities be-tween negatively associated sequences and independent random variables. Journal of Theoretical Probability 13,343-356.
[111]Stout, W. (1974). Almost Sure Convergence. Academic Press, New York.
[112]Strassen, V. (1964). An invariance principle for the law of the iterated log-arithm. Zeitschrift fur Wahrshcheinlichkeitstheorie und Verwandte Gebiete 3,211-226.
[113]Su, C., Zhao, L. C., Wang, Y. B. (1996). The moment inequalities and weak convergence for negatively associated sequences. Science in China Series A 26,1091-1099.
[114]Talagrand, M. (1994). Sharper bounds for Gaussian and empirical pro-cesses. Annals of Probability 22,28-76.
[115]Talagrand, M. (1996). New concentration inequalities in product spaces. Inventiones Mathematicae 126,505-563.
[116]Tong, H. (1990). Non-linear Time Series:A Dynamical System Approach. Oxford University Press.
[117]Wiener, N. (1958). Nonlinear Problems in Random Theory. MIT Press, John Wiley.
[118]Withers, C. S. (1981). Conditions for linear process to be strongly mixing. Zeitschrift fur Wahrshcheinlichkeitstheorie und Verwandte Gebiete 57,477-480.
[119]Woodroofe, M. (1967). On the maximum deviation of the sample density. Ann. Math. Statist.38,475-481.
[120]Woodroofe, M. (1970). Discussion of "Density estimates and Markov se-quences" by M. Rosenblatt. In nonparametric Techniques in Statistical In-ference. Cambridge University Press.
[121]Wu, W. B. (2007). Strong invariance principles for dependent random vari-ables. Annals of Probability 35,2294-2320.
[122]Wu, W. B., Woodroofe,M. (2004). Martingale approximations for sums of stationary processes. Annals of Probability 32,1674-1690.
[123]Wu, W. B., Huang, Y., Huang, Y. (2010). Kernel estimation for time se-ries:An asymptotic theory. Stochastic Processes with their Applications 120, 2412-2431.
[124]Yakowitz, S. (1985). nonparametric density estimation, prediction and re-gression for Markov sequences. Journal of the American Statistical Associa-tion 80,215-221.
[125]Yu, B. (1993). Density estimation in the L∞ norm for dependent data with applications to the Gibbs sampler. Annals of Statistics 21,711-735.
[126]Zhang, L. X. (2004). Some limit results on the law of the iterated logarithm of NA sequences. Acta Mathematica Sinica 47,541-552.
[127]Zhang, L. X. (2001). The weak convergence for functions of negatively associated random variables. Journal of Multivariate Analysis 78,272-298.
[128]Zhang, L. X., Wen, J. W.(2001). A weak convergence for negatively asso-ciated fields. Statistics and Probability Letters 53,259-267.