Stein’s method for the bootstrap
Stein’s method for the bootstrap
复制标题
斯坦因引导法
DOI：
10.1214/lnms/1196283802
复制
发表时间：
2004
影响因子：
--
通讯作者：
G. Reinert
中科院分区：
文献类型：
--
作者：
Susan P. Holmes;G. Reinert
本文对许多已知的关于自举分布收敛于光滑统计量真分布规律的结果给出了新的证明，无论所研究的样本是来自随机变量的独立实现还是弱依赖的依赖实现。提出了一种新的自举方法，并证明了该方法在一致局部依赖条件下是有效的。所采用的技术基于Reinert[19]开发的Stein的经验过程方法。最后一节提供了一些仿真和应用，其中相关的matlab函数可以从第一作者那里获得。6.1概述Stein的方法通过对期望的近似证明了弱收敛结果，而不直接使用特征函数，使其可以用于具有依赖性的复杂问题。在这项工作中，我们展示了如何使用Stein的方法证明任何具有可交换权值的bootstrap的一致性，甚至提供了一些误差项。我们说，如果自举经验测度和以真实经验测度或真实均值测度为中心的高斯测度之间的距离随着样本量趋于无穷大而趋于零，则自举有效。我们的结果也提供了一个显式的误差界的差异，对任何有限的样本量的自举高斯性。许多结果本身是已知的，例如，参见Bickel和Freedman(1981), Singh（1981）在多项情况下的一致性结果或Praestgaard和Wellner（1993）在可交换权重情况下的一致性结果。然而，本文提出了一种新的自举边界误差项的方法，该方法不依赖于Edgeworth展开，而其他关于收敛率的理论工作则依赖于Edgeworth展开。在独立的情况下见[10,12,11,14,13]。使用Edgeworth展开式证明因变量的例子可以在Lahiri[15]和Politis等人的书[15]中找到。在这种方法中，我们不是直接比较两个分布，而是比较它们在某些测试函数上的Stein算子，并使用它们的差的期望来限定分布之间的实际距离。在定义了算子和符号之后，我们从紧跟Stein中心极限定理证明的均值的自举分布的一致性的简单例子开始。然后，我们将在第6.3节中使用经验过程来证明可交换权的一致性的一般情况。该内容下载自40.77.167.1在星期三，2016年8月3日05:28:45 UTC所有使用http://about.jstor.org/terms 96，我们在6.4节中表明，弱邻域依赖结构不会使引导过程无效，只要使用类似于Carlstein等人的块型引导。第6.5节给出了一些例子；我们给出了各种依赖结构和邻域引导在这些情况下提供的结果。6.2符号假设有一个概率空间（0,#，P）。我们叫E: X ？与R相关的期望？在X上，定义在0上具有有限期望的实值随机变量的空间。我们的第一个定理是Stein的正态近似定理的一个简单应用，它在Stein（1986）中得到了发展。我们可以证明平滑均值函数具有近似正态的自举分布。稍后将通过经验过程方法对这一结果进行概括。6.2.1可交换变量Stein在随机化方案中引入了可交换变量，该方案能够表征期望算子E的零空间ker E。根据定义，（X, X1）是一对可交换变量，当且仅当（X， X‘）的联合分布与（Xf， X）的分布相同，有时写为（X， X’） ？(X \)。在后面（X， X'）总是用来表示一个可交换的对。6.2.2反对称函数的操作符调用？在fl2上定义的有界可测反对称函数集。我们用？接线员？:哦?? ?X对应于每一个反对称的F in ？函数：TF，使得TF(x) = E(F(x, x ')\x)，其中E（A\x）是给定x = x的条件期望。6.2.3均值的引导为了说明该方法并获得第一个结果，设x = （?? ?）X2,…， xn)从一个分布p中得到的独立同分布观测值的样本，用Pn = ？年代?? ?% * ^ ?经验分布。引导取代了未知的分布？根据经验分布Pn计算p的统计泛函，一个自举样本的特征为？-向量k = （kx,k2，…）， Cn = {k = (*!,?？* ?) * ?+ ? ?+ ? ? = ?> *i>0， fe n}原始的多项引导使用？？= ?年代?W * ?>，其中k以多项式形式分布，特别是Eki = 1。稍后，我们还将考虑更普遍的加权自举，其中fcj可以简单地交换，例如Praestgaard和Wellner（1993）。首先，这里我们研究多项式权重k = (kx, ？吗?吗?kn)。作为第一步，Stein的方法提供了一种简单的方法来证明自举求和W = S *?-^？是渐近正态。为此，我们采用Stein（1986，第2章）的经典结果。假设W均值为0，方差为1，并且（W， W）是一个可交换对，使得有一个0 < ？< 1 with (6.1) E(W'\W) = (1 X)W。此内容下载自40.77.167.1在Wed， 03 Aug 2016 05:28:45 UTC所有使用服从http://about.jstor.org/terms 97 Stein（1986）证明了以下定理，并改进了Baldi， Rinott和Stein（1989）的边界。设（W， W）是满足（6.1）的可交换对，并设EW = 0, Var(W) = 1。对于任何连续有界函数h: IR ?？有界的，分段连续导数的IR，我们有\Eh(W) F/？| /'|3 < (sup/i inf h) Je (l ~E((W W')2\W) J + ^r\\h'\\E\W W根据经典Stein过程，我们使用独立于X的辅助随机变量（J， J）构建可交换对（k, k'），如下：？根据权值kj选择J，即P(I = I) = kj/n，如果/ = I，则k{减1，k[= ki 1]。吗?在1和n之间均匀地选择J，P(J = J) = ？，若J = J，则分量kj增加1，且kj = kj + 1。考虑k代表计数向量什么时候？球是扔进去的吗？骨灰盒，这对可互换的对应于随机选择一个球，把它从落地的地方拿出来，再扔一次；K '给出了新的计数向量。X的重新输入的引导和，以及它的可交换对应w = E? \/E?X?-X?)2为了应用Stein的方法，需要各种条件矩。在整个过程中，所有的计算都以样本X为条件，因此我们将E(W)写为本节中的E^(W)，并将Xi替换为观测值。为了简化计算，我们可以调整X的大小，使S？0，我们用s\ \？年代x % ?^ 54:=z S% ?？观察第一周（W′-W） = ？E(w w \ v) = ？(* fci年代kiXi ^ = J2E (kfi-kt \ k) ^ ^ 52 = ^ 2 te (n-kt-kt (n-l) \ k) ^ ? ?* ?“是吗？”*吗?”S2 S2 fci(n-l)lk
This paper gives new proofs for many known results about the convergence in law of the bootstrap distribution to the true distribution of smooth statistics, whether the samples studied come from independent realizations of a random variable or dependent realizations with weak dependence. Moreover it suggests a novel bootstrap procedure and provides a proof that this new bootstrap works under uniform local dependence. The techniques employed are based on Stein's method for empirical processes as developed by Reinert [19]. The last section provides some simulations and applications for which the relevant matlab functions are available from the first author. 6.1 Overview Stein's method proves weak-convergence results through approximations to expectations without direct use of characteristic functions, allowing it to be used in complex problems with dependence. In this work we show how consistency can be proved and even some error terms provided for any bootstrap with exchangeable weights using Stein's method. We say that the bootstrap works if the distance between the bootstrap empirical measure and a Gaussian measure centred around the true empirical measure, or the true mean measure, tends to zero as sample size tends to infinity. Our results also provide an explicit error bound for the difference to Gaussianity of the bootstrap for any finite sample size. Many of the results themselves are known, see for instance Bickel and Freedman(1981), Singh(1981) for the consistency results in the multinomial case or Praestgaard and Wellner(1993) for the case of exchangeable weights. However this paper proposes a new way of bounding error terms for the bootstrap that does not rely on Edgeworth expansions as does the other theoretical work on convergence rates to date. In the independent case see [10, 12, 11, 14, 13]. Examples of proofs for dependent variables using Edgeworth expansions can be found in Lahiri [15] and the book by Politis et al. [17]. Instead of comparing two distributions directly, in this approach we compare their Stein operators on certain test functions and the expectation of their difference is used to bound the actual distance between distributions. After defining the operators and notations, we start with the simple case of the consistency of the bootstrap distribution for die mean following Stein's proof of the central limit theorem closely. We then pass to the use of empirical processes to prove the general case of consistency for exchangeable weights in section 6.3. This approach does not depend strongly on the hypothesis of independence, 95 This content downloaded from 40.77.167.1 on Wed, 03 Aug 2016 05:28:45 UTC All use subject to http://about.jstor.org/terms 96 and we show in section 6.4 that a weak neighborhood dependency structure does not invalidate the bootstrap procedure as long as a block-type bootstrap similar to Carlstein et al. [3] is used. Section 6.5 presents some examples; we give various dependency structures and the results that the neighborhood bootstrap provides in these cases. 6.2 Notation Suppose we have a probability space (O, #, P). We will call E : X ?> R the expectation associated to ? on X, the space of real-valued random variables defined on O that have finite expectation. Our first theorem is a simple application of Stein's normal approximation theorems as developed in Stein (1986). We can show that smooth functions of means have bootstrap distributions that are approximately normal. Later a generalization of this result will be provided through the empirical process approach. 6.2.1 Exchangeable Variables Stein has introduced exchangeable variables in a randomization scheme that enables a characterization of the null space ker E of the expectation operator E. By definition, (X, X1) is a pair of exchangeable variables if and only if the joint distribution of the pair (X, X') is identical to the distribution of (Xf, X), written sometimes (X, X') ? (X\ X). In what follows (X, X') is always used to denote an exchangeable pair. 6.2.2 Operators of Antisymmetric Functions Call ? the set of bounded measurable antisymmetric functions defined on fl2. We will denote by ? the operator ? : ? ?? X which associates to every antisymmetric F in ? the function: TF such that TF(x) = E(F(X, X')\x) where E(A\x) is the conditional expectation given X = x. 6.2.3 Bootstrap of the mean To illustrate the method and for first results, let X = (??, X2,..., xn) he a sample of independent identically distributed observations from a distribution P. Denote by Pn = ? S? ??% *^? empirical distribution. The bootstrap replaces the unknown distribution ? by the empirical distribution Pn in the computation of statistical functionals of P. A bootstrap sample is characterized by a ?-vector k = (kx,k2,..., kn) of the simplex Cn = {k = (*!,??? ,*?), *? + ??? + ??=?> *i>0, fe eN} The original multinomial bootstrap uses ?? = ? S? W*?> where the k is distributed as a multinomial, in particular Eki = 1. Later we will also consider more generally weighted bootstraps where the fcj's are simply exchangeable as in Praestgaard and Wellner (1993) for instance. Firstly, here we study the original bootstrap with multinomial weights k = (kx, ? ? ? , kn). As a first step, Stein's method provides a straightforward way of showing that the bootstrap sum W = S *?-^? is asymptotically normal. To this purpose, we employ the classical result from Stein (1986, Chapter 2). Assume that W is mean zero, variance 1, and that (W, W) is an exchangeable pair such that there is a 0 < ? < 1 with (6.1) E(W'\W) = (1 X)W. This content downloaded from 40.77.167.1 on Wed, 03 Aug 2016 05:28:45 UTC All use subject to http://about.jstor.org/terms 97 Stein (1986) proved the following theorem, with an improvement on the bounds by Baldi, Rinott and Stein (1989). Theorem 6.1 Let (W, W) be an exchangeable pair satisfying (6.1) and assume EW = 0, Var(W) = 1. For any continuous, bounded function h : IR ?? IR with bounded, piecewise continuous derivatives, we have \Eh(W) F/?| /'|3 < (sup/i inf h) Je (l ~E((W W')2\W) J + ^r\\h'\\E\W W Following the classical Stein procedure we build the exchangeable pair (k, k'), using the auxiliary random variables (J, J), independent of X, as follows: ? Choose J according to the weight kj, that is : P(I = i) = kj/n, if / = i, decrease k{ by 1, k[ = ki 1. ? Choose J uniformly between 1 and n,P(J = j) = ?, if J = j, increase the component kj by 1, and kj = kj + 1. Thinking of k representing the count vector when ? balls are thrown into ? urns, this exchangeable pair corresponds to choosing one of the balls at random, taking it out of the urn where it landed, and throwing it again; k' gives the new count vector. the recentered bootstrap sum of X, and its exchangeable counterpart w = E ?\/E?X?-X?)2 To apply Stein's method, various conditional moments are needed. Throughout all calculations are conditional on the sample X, so we will write E(W) to mean E^(W) throughout this section, and replace the Xi's by the observed values ??. To simplify the calculations we can rescale X so that S? Xi ? 0 and we use s\ \? S% x? an(^ 54 :=z S% ??? Observe first Ek(W' -W) = ?E(W W\V) = ?(S *fci S kiXi^ = J2E(kfi-kt\k)^ ^ 52 = ^2TE(n-kt-kt(n-l)\k)^ ?? *?' si ? *?' S2 S2 fci(n-l)lk) 52