Sufficiency, Completeness & the Rao–Blackwell and Lehmann–Scheffé Theorems

Let $(\mathcal{X}, \mathcal{A}, \mathcal{P})$ be a statistical experiment where $\mathcal{P} = \{P_\theta : \theta \in \Theta\}$ is a dominated family of probability measures. A statistic $T: \mathcal{X} \to \mathcal{T}$ is any Borel-measurable mapping from the sample space $\mathcal{X}$ into a metric space $\mathcal{T}$ (typically $\mathbb{R}^k$ with $k < n$). A statistic partitions the sample space $\mathcal{X}$ into level sets or equivalence classes:

Observing $T(\mathbf{X}) = t$ informs the investigator that the realization $\mathbf{X}$ lies somewhere within the sub-manifold $A_t$, while the specific location within $A_t$ is discarded.

Formal Definition of Sufficiency

A statistic $T(\mathbf{X})$ is said to be sufficient for the parameter $\theta$ (or for the family $\mathcal{P}$) if the conditional distribution of the sample $\mathbf{X}$ given $T(\mathbf{X}) = t$ is completely independent of the parameter $\theta$ for all $t \in \mathcal{T}$.

Formally, for every observable event $A \in \mathcal{A}$ and for all $\theta \in \Theta$:

where $P(A \mid t)$ does not depend on $\theta$.

The operational interpretation of sufficiency is profound: once the numerical value of $T(\mathbf{X})$ is calculated and made known, the original raw data $\mathbf{X}$ contain zero additional information regarding $\theta$. An independent researcher armed only with $T(\mathbf{x})$ could perform a computer simulation from the conditional distribution $P(\cdot \mid t)$ to generate synthetic data $\mathbf{X}^*$ whose joint distribution under $P_\theta$ is mathematically indistinguishable from the original experimental data $\mathbf{X}$. Thus, no inferential precision is sacrificed by discarding the raw data and retaining only $T(\mathbf{X})$.

Verifying sufficiency directly from the definition requires calculating the conditional distribution of $\mathbf{X}$ given $T$, which is often technically demanding. The Neyman–Fisher Factorization Theorem provides an elegant, necessary, and sufficient algebraic criterion based directly on the structure of the joint density function.

Theorem: Neyman-Fisher Factorization Criterion

Let $\mathcal{P} = \{f(\mathbf{x}; \theta) : \theta \in \Theta\}$ be a family of probability densities dominated by a $\sigma$-finite measure $\mu$ on $\mathcal{X}$. A statistic $T(\mathbf{X})$ is sufficient for $\theta$ if and only if there exist non-negative, measurable functions $g(t; \theta)$ and $h(\mathbf{x})$ such that for all $\mathbf{x} \in \mathcal{X}$ and all $\theta \in \Theta$:

where:

1. $g(T(\mathbf{x}); \theta)$ depends on the data $\mathbf{x}$ exclusively through the summary statistic $T(\mathbf{x})$, and depends on the unknown parameter $\theta$.
2. $h(\mathbf{x})$ is a non-negative function of the data that is strictly independent of the parameter $\theta$.

Proof for the Discrete Case:

Suppose $X$ takes values in a countable sample space $\mathcal{X}$. Necessity ($\implies$): Assume $T(\mathbf{X})$ is sufficient. Then the conditional probability $\mathbb{P}(\mathbf{X} = \mathbf{x} \mid T(\mathbf{X}) = T(\mathbf{x}))$ is independent of $\theta$. Denote this conditional probability by $h_0(\mathbf{x})$. By the definition of conditional probability:

Define $g(t; \theta) = \mathbb{P}_\theta(T(\mathbf{X}) = t)$ and $h(\mathbf{x}) = h_0(\mathbf{x})$. Then:

which matches the factorization form exactly.

Sufficiency ($\impliedby$): Assume $f(\mathbf{x}; \theta) = g(T(\mathbf{x}); \theta) h(\mathbf{x})$. Compute the marginal probability mass function of $T(\mathbf{X})$:

Now evaluate the conditional probability for any point $\mathbf{x}$ such that $T(\mathbf{x}) = t$:

The term $g(t; \theta)$ cancels out completely! The resulting conditional probability depends solely on the known function $h(\cdot)$ and is completely free of $\theta$. Thus $T(\mathbf{X})$ is sufficient.

In the general measure-theoretic setting (Halmos and Savage, 1949), the proof utilizes the Radon–Nikodym derivative with respect to a dominating finite mixture measure $\lambda = \sum c_i P_{\theta_i}$, verifying that the conditional expectation operator $\mathbb{E}_\theta[\cdot \mid T]$ is $\theta$-invariant.

Sufficiency alone does not guarantee maximal data compression. The raw data vector itself, $T_{\text{raw}}(\mathbf{X}) = (X_1, \dots, X_n)$, is trivially sufficient because $h(\mathbf{x}) = 1$ and $g(\mathbf{x}; \theta) = f(\mathbf{x}; \theta)$. Similarly, any one-to-one transformation or augmented vector $(T(\mathbf{X}), X_1)$ is sufficient if $T(\mathbf{X})$ is sufficient. To achieve the greatest possible simplification, we must identify the statistic that achieves the coarsest sufficient partition of the sample space.

Definition of Minimal Sufficiency

A sufficient statistic $T(\mathbf{X})$ is said to be minimal sufficient if it is a function of any other sufficient statistic. That is, if $S(\mathbf{X})$ is any arbitrary sufficient statistic for $\theta$, then there exists a measurable function $\psi$ such that:

Geometrically, if $\mathcal{A}_T$ and $\mathcal{A}_S$ represent the partitions generated by $T$ and $S$, then every level set of $S$ is contained within a level set of $T$. Minimal sufficiency means that $T$ provides the maximum possible data reduction without sacrificing sufficiency.

The Lehmann–Scheffé Theorem for Minimal Sufficiency

E. L. Lehmann and H. Scheffé (1950) formulated a direct and universal criterion for identifying minimal sufficient statistics through likelihood ratios.

Theorem: Likelihood Ratio Criterion

Let $f(\mathbf{x}; \theta)$ be the density function of the sample. Suppose there exists a statistic $T(\mathbf{X})$ such that for any two sample realizations $\mathbf{x}, \mathbf{y} \in \mathcal{X}$, the likelihood ratio:

is completely independent of $\theta$ if and only if $T(\mathbf{x}) = T(\mathbf{y})$. Then $T(\mathbf{X})$ is a minimal sufficient statistic for $\theta$.

Proof:

1. Sufficiency of $T$: Select a fixed representative point $\mathbf{x}_t$ from each level set $A_t = \{ \mathbf{x} : T(\mathbf{x}) = t \}$. For any $\mathbf{x} \in A_t$, by assumption the ratio $\frac{f(\mathbf{x}; \theta)}{f(\mathbf{x}_t; \theta)}$ is independent of $\theta$. Define:

and define $g(t; \theta) = f(\mathbf{x}_t; \theta)$. Then:

By the Neyman–Fisher Factorization Theorem, $T(\mathbf{X})$ is sufficient.

2. Minimality of $T$: Let $S(\mathbf{X})$ be any other sufficient statistic. By factorization, $f(\mathbf{x}; \theta) = g_S(S(\mathbf{x}); \theta) h_S(\mathbf{x})$. If $S(\mathbf{x}) = S(\mathbf{y})$, then:

which is independent of $\theta$. But by the hypothesis of the theorem, the likelihood ratio is independent of $\theta$ only if $T(\mathbf{x}) = T(\mathbf{y})$. Thus, $S(\mathbf{x}) = S(\mathbf{y}) \implies T(\mathbf{x}) = T(\mathbf{y})$, which guarantees that $T$ is a functional mapping of $S$. Hence $T(\mathbf{X})$ is minimal sufficient.

In contrast to sufficient statistics, which concentrate all information regarding $\theta$, some statistics contain absolutely zero information about $\theta$.

Definition of an Ancillary Statistic

A statistic $V(\mathbf{X})$ is called an ancillary statistic (or ancillary) if its marginal probability distribution is completely independent of the unknown parameter $\theta$.

For example, in a location family $f(x; \mu) = f_0(x - \mu)$, the sample range $R = X_{(n)} - X_{(1)}$ or any sample spacing $X_i - X_j$ is ancillary because shifting the data by a constant $\mu$ leaves all differences unchanged. Similarly, in a scale family $f(x; \sigma) = \frac{1}{\sigma} f_0(x/\sigma)$, scale-free ratios $X_i / X_j$ or Studentized residuals are ancillary.

While ancillary statistics contain zero direct information about $\theta$, they frequently serve as conditioning variables that indicate the precision or reliability of an experiment.

Basu's Theorem

In 1955, Debabrata Basu discovered a remarkable theorem connecting sufficiency, completeness, and ancillarity that allows statisticians to establish the statistical independence of two random variables without ever evaluating a joint integral or convolution.

Theorem: Basu's Theorem

If $T(\mathbf{X})$ is a complete sufficient statistic for a parameter $\theta$, and $V(\mathbf{X})$ is any ancillary statistic, then $T(\mathbf{X})$ and $V(\mathbf{X})$ are statistically independent under every $P_\theta \in \mathcal{P}$.

Proof:

Let $B$ be any Borel set in the range of $V$. Because $V$ is ancillary, the marginal probability $\mathbb{P}(V(\mathbf{X}) \in B) = p_B$ is a constant independent of $\theta$. Now, consider the conditional probability of the event $\{V \in B\}$ given $T$:

Because $T$ is a sufficient statistic, this conditional probability is independent of $\theta$, making $\eta(T)$ a legitimate statistic. By the law of total expectation:

Rearranging this expectation:

Now, consider the function $g(T) = \eta(T) - p_B$. Since $T$ is a complete statistic, the only function whose expectation is identically zero across the entire parameter space is the zero function almost everywhere:

Because the conditional distribution of $V$ given $T$ equals the marginal distribution of $V$ for all measurable sets $B$, $T(\mathbf{X})$ and $V(\mathbf{X})$ are strictly statistically independent.

Applications of Basu's Theorem:

Basu's Theorem provides an instant proof that for an i.i.d. Gaussian sample $X_1, \dots, X_n \sim \mathcal{N}(\mu, \sigma^2)$:

• The sample mean $\bar{X}$ is complete sufficient for $\mu$ when $\sigma^2$ is fixed.
• The sample variance $S^2 = \frac{1}{n-1} \sum (X_i - \bar{X})^2$ is translation-invariant, hence ancillary for $\mu$.
• By Basu's Theorem, $\bar{X}$ and $S^2$ are strictly independent, without any tedious orthogonal rotations or Cochran's theorem proofs!

The Conditionality Principle and Ancillarity Paradoxes

Allan Birnbaum (1962) proved a celebrated foundational theorem uniting the Sufficiency Principle and the Conditionality Principle:

• Sufficiency Principle (S): If $T(\mathbf{X})$ is a sufficient statistic, then any two experimental outcomes $\mathbf{x}_1, \mathbf{x}_2$ with $T(\mathbf{x}_1) = T(\mathbf{x}_2)$ must yield identical statistical inferences regarding $\theta$.
• Conditionality Principle (C): If an experiment is chosen from two candidate experiments $E_1$ and $E_2$ by a fair coin flip $J \in \{1, 2\}$ whose outcome is observed, then inference regarding $\theta$ should depend only on the chosen experiment $E_J$, conditioning on the realized ancillary indicator $J$.

Birnbaum proved that the Sufficiency Principle and the Conditionality Principle logically imply the Likelihood Principle (L):

The Likelihood Principle asserts that all experimental evidence regarding $\theta$ is contained exclusively in the realized likelihood function $L(\theta; \mathbf{x})$, and that unobserved sample paths, stopping rules, and sample space counterfactuals are irrelevant to valid inference. However, frequentist statistical inference depends intrinsically on error rates integrated over the entire sample space $\mathcal{X}$. Conditioning on ancillary statistics serves as an essential bridge: by conditioning on ancillary variables (such as sample size in random sample designs or the configuration statistic in location-scale families), frequentist tests and confidence intervals recover conditional validity without violating the spirit of conditionality.

Completeness is a strong uniqueness property of a family of distributions that prevents distinct functions of a sufficient statistic from having identical expectations.

Formal Definition of Completeness

Let $\mathcal{P} = \{P_\theta : \theta \in \Theta\}$ be a family of probability measures on $(\mathcal{X}, \mathcal{A})$, and let $T(\mathbf{X})$ be a statistic with induced family of distributions $\mathcal{P}_T = \{P_{\theta}^T : \theta \in \Theta\}$. The statistic $T$ (or the family $\mathcal{P}_T$) is said to be complete if for any measurable real-valued function $g$:

If this condition holds only for all bounded measurable functions $g$ (i.e., $\sup_t |g(t)| < \infty$), the statistic $T$ is said to be boundedly complete. Completeness implies bounded completeness, but the converse is not true.

Operational Meaning: Uniqueness of Estimators

Suppose $T$ is complete, and suppose we have two estimators $g_1(T)$ and $g_2(T)$ that are both unbiased estimators of the same parametric function $\tau(\theta)$:

Subtracting the two equations:

By completeness, $g_1(T) - g_2(T) = 0$ almost everywhere. Therefore:

Completeness guarantees that there can exist at most one function of $T$ that is unbiased for any given parametric function $\tau(\theta)$!

In parametric modeling, proving completeness directly from integrals can be formidable. Fortunately, for exponential families, completeness is guaranteed by the deep analytic properties of the Laplace transform.

Theorem: Completeness of Exponential Families

Consider a $k$-parameter exponential family with probability density function:

Let $\mathbf{T}(\mathbf{X}) = (T_1(\mathbf{X}), \dots, T_k(\mathbf{X}))^T$. If the natural parameter space:

contains a $k$-dimensional open rectangle or open ball in $\mathbb{R}^k$, then the vector statistic $\mathbf{T}(\mathbf{X})$ is a complete minimal sufficient statistic for $\boldsymbol{\theta}$.

Proof Sketch (Laplace Transform Uniqueness):

The condition $\mathbb{E}_{\boldsymbol{\eta}}[g(\mathbf{T})] = 0$ translates into the integral equation:

where $\nu$ is the base measure induced by $h(\mathbf{x})$. Decompose $g(\mathbf{t})$ into its positive and negative parts, $g(\mathbf{t}) = g^+(\mathbf{t}) - g^-(\mathbf{t})$:

The left and right sides are the bilateral Laplace transforms of the non-negative measures $d\mu_1(\mathbf{t}) = g^+(\mathbf{t}) d\nu(\mathbf{t})$ and $d\mu_2(\mathbf{t}) = g^-(\mathbf{t}) d\nu(\mathbf{t})$. By the Lerch–Laplace Inversion Theorem, if two Laplace transforms of Radon measures coincide on an open set in $\mathbb{R}^k$, their underlying measures must be identical: $\mu_1 \equiv \mu_2$. Thus $g^+(\mathbf{t}) = g^-(\mathbf{t})$ almost everywhere, implying $g(\mathbf{t}) = 0$ almost everywhere.

If the natural parameter space $\mathcal{H}$ is curved or restricted to a lower-dimensional manifold (such as $\mathcal{N}(\theta, \theta^2)$ where $\eta_1 = 1/\theta$ and $\eta_2 = -1/(2\theta^2)$), $\mathcal{H}$ contains no open ball in $\mathbb{R}^2$, and completeness fails! In curved exponential families, the minimal sufficient statistic is typically not complete.

The Pitman–Koopman–Darmois Theorem

Can an analyst achieve finite-dimensional data reduction in every distribution family? In 1935–1936, E. J. G. Pitman, B. O. Koopman, and G. Darmois independently proved a celebrated structural theorem that establishes the unique mathematical supremacy of the exponential family:

Theorem (Pitman–Koopman–Darmois):

Let $X_1, X_2, \dots, X_n$ be an independent, identically distributed sample from a family of continuous distributions $\mathcal{P} = \{f(x; \theta) : \theta \in \Theta\}$ whose support is independent of $\theta$, and whose density is continuously differentiable in $x$. Then there exists a sufficient statistic $T(\mathbf{X}) = (T_1(\mathbf{X}), \dots, T_k(\mathbf{X}))^T$ whose dimension $k$ remains strictly fixed and bounded as the sample size $n \to \infty$ if and only if the distribution belongs to an exponential family:

If a distribution family does not belong to the exponential family (such as the Cauchy distribution, the Student's $t$-distribution, or the Weibull distribution with unknown shape), then as $n$ grows, the minimal sufficient statistic is the entire order statistic vector $(X_{(1)}, X_{(2)}, \dots, X_{(n)})$ of dimension $n$! In non-exponential families, no data compression is mathematically possible without discarding essential information regarding $\theta$.

The Pitman–Koopman–Darmois Theorem

Can an analyst achieve finite-dimensional data reduction in every distribution family? In 1935–1936, E. J. G. Pitman, B. O. Koopman, and G. Darmois independently proved a celebrated structural theorem that establishes the unique mathematical supremacy of the exponential family:

Theorem (Pitman–Koopman–Darmois):

Let $X_1, X_2, \dots, X_n$ be an independent, identically distributed sample from a family of continuous distributions $\mathcal{P} = \{f(x; \theta) : \theta \in \Theta\}$ whose support is independent of $\theta$, and whose density is continuously differentiable in $x$. Then there exists a sufficient statistic $T(\mathbf{X}) = (T_1(\mathbf{X}), \dots, T_k(\mathbf{X}))^T$ whose dimension $k$ remains strictly fixed and bounded as the sample size $n \to \infty$ if and only if the distribution belongs to an exponential family:

If a distribution family does not belong to the exponential family (such as the Cauchy distribution, the Student's $t$-distribution, or the Weibull distribution with unknown shape), then as $n$ grows, the minimal sufficient statistic is the entire order statistic vector $(X_{(1)}, X_{(2)}, \dots, X_{(n)})$ of dimension $n$! In non-exponential families, no data compression is mathematically possible without discarding essential information regarding $\theta$.

We now turn to the constructive methods of finding optimal estimators. In 1945, C. R. Rao, and independently David Blackwell in 1947, discovered that conditioning any preliminary unbiased estimator on a sufficient statistic strictly improves—or at least never degrades—its performance under any convex loss function.

Theorem: The Rao–Blackwell Theorem

Let $\mathbf{X} \sim P_\theta$, and let $T(\mathbf{X})$ be a sufficient statistic for $\theta$. Let $W(\mathbf{X})$ be an unbiased estimator of $\tau(\theta)$ with $\text{Var}_\theta(W) < \infty$. Define the conditioned statistic:

Then:

1. $\phi(T)$ is a legitimate estimator that does not depend on $\theta$.
2. $\phi(T)$ is an unbiased estimator of $\tau(\theta)$:

1. The variance of $\phi(T)$ is strictly bounded above by the variance of $W$:

Equality holds if and only if $W$ is already a deterministic function of $T$ almost everywhere ($W = \phi(T)$ a.e.).

Proof:

1. Observable Statistic: Because $T$ is sufficient for $\theta$, the conditional distribution of $\mathbf{X}$ given $T = t$ is independent of $\theta$. Therefore, the conditional expectation $\phi(t) = \mathbb{E}[W(\mathbf{X}) \mid T(\mathbf{X}) = t]$ does not involve $\theta$, making $\phi(T)$ an observable estimator.

2. Unbiasedness: By the Law of Total Expectation (tower property):

3. Variance Reduction: By the Law of Total Variance (variance decomposition formula):

Because variance is non-negative, $\text{Var}(W \mid T) \ge 0$, and thus $\mathbb{E}_\theta[\text{Var}(W \mid T)] \ge 0$. Therefore:

Equality holds if and only if $\mathbb{E}_\theta[\text{Var}(W \mid T)] = 0$, which implies $\text{Var}(W \mid T) = 0$ almost everywhere, meaning $W$ is a deterministic function of $T$.

Extension to Convex Loss (Blackwell's Formulation)

If estimation error is evaluated by any strictly convex loss function $L(\theta, d)$ (such as absolute error $|d - \theta|$ or asymmetric loss), Jensen's Inequality for conditional expectations yields:

Conditioning on a sufficient statistic reduces risk under every convex loss function.

While the Rao–Blackwell theorem proves that conditioning on a sufficient statistic reduces variance, it does not guarantee that the resulting estimator $\phi(T)$ has the absolute minimum variance among all possible unbiased estimators. What if we had started with a different crude estimator $W^*$? Would conditioning $W^*$ on $T$ produce a different improved estimator $\phi^*(T)$?

The celebrated Lehmann–Scheffé Theorem (1950) provides the definitive answer: if the sufficient statistic is also complete, then the Rao–Blackwell estimator is unique and is guaranteed to be the Uniformly Minimum Variance Unbiased Estimator (UMVUE).

Theorem: The Lehmann–Scheffé Theorem

Let $T(\mathbf{X})$ be a complete sufficient statistic for $\theta$.

1. If $g(T)$ is any unbiased estimator of $\tau(\theta)$ with finite variance, then $g(T)$ is the unique Uniformly Minimum Variance Unbiased Estimator (UMVUE) of $\tau(\theta)$.
2. If $W(\mathbf{X})$ is any unbiased estimator of $\tau(\theta)$, then the conditioned estimator:

is the unique UMVUE of $\tau(\theta)$.

Proof:

Let $U(\mathbf{X})$ be any arbitrary unbiased estimator of $\tau(\theta)$ with $\text{Var}_\theta(U) < \infty$. Define $g^*(T) = \mathbb{E}[U \mid T]$. By the Rao–Blackwell theorem:

1. $\mathbb{E}_\theta[g^*(T)] = \tau(\theta)$.
2. $\text{Var}_\theta(g^*(T)) \le \text{Var}_\theta(U)$.

Now, suppose $g(T)$ is any function of $T$ that is unbiased for $\tau(\theta)$. Then:

Because $T$ is complete, this implies that:

Therefore, $g(T)$ is identical to $g^*(T)$ almost everywhere! Consequently:

Because $U$ was completely arbitrary, $g(T)$ has a variance less than or equal to that of every competing unbiased estimator across the entire parameter space $\Theta$. Furthermore, if another estimator $U^*$ had equal variance, equality in Cauchy–Schwarz would force $U^* = g(T)$ a.e. Thus $g(T)$ is the unique UMVUE.

Two Universal Recipes for Constructing UMVUEs

The Lehmann–Scheffé Theorem provides two distinct operational recipes for finding UMVUEs:

1. Method 1 (Direct Unbiasing of the Complete Sufficient Statistic): Find a complete sufficient statistic $T$. Propose a candidate function $g(T)$. Compute its expectation $\mathbb{E}_\theta[g(T)]$. If $\mathbb{E}_\theta[g(T)] = \tau(\theta)$, then $g(T)$ is immediately the UMVUE.

2. Method 2 (Rao-Blackwell Conditioning): Construct a trivial, naive unbiased estimator $W(\mathbf{X})$ (even one depending only on the single first observation $X_1$). Then evaluate the conditional expectation $\phi(T) = \mathbb{E}[W \mid T]$. By Lehmann–Scheffé, $\phi(T)$ is automatically the UMVUE.

To appreciate the practical power of Method 2, consider estimating the probability of zero occurrences in a Poisson process: $\tau(\lambda) = \mathbb{P}(X = 0) = e^{-\lambda}$, based on an i.i.d. sample $X_1, \dots, X_n \sim \text{Poisson}(\lambda)$. The sum $T = \sum_{i=1}^n X_i \sim \text{Poisson}(n\lambda)$ is a complete sufficient statistic. Propose the simplest conceivable unbiased estimator of $\tau(\lambda)$ using only the first observation:

Notice that $\mathbb{E}[W] = \mathbb{P}(X_1 = 0) = e^{-\lambda}$, so $W$ is unbiased, but terribly crude: its variance is $e^{-\lambda}(1 - e^{-\lambda})$. Now, condition $W$ on the complete sufficient statistic $T = t$:

Using the fact that $\sum_{i=2}^n X_i \sim \text{Poisson}((n-1)\lambda)$:

By the Lehmann–Scheffé Theorem, the estimator:

is the unique UMVUE of $e^{-\lambda}$! Notice how conditioning effortlessly transformed a binary indicator $W \in \{0, 1\}$ into a smooth, highly efficient exponential estimator.

A fundamental consequence of the Rao–Blackwell Theorem is that any estimator $W(\mathbf{X})$ that is not almost surely a function of a sufficient statistic is strictly inadmissible under squared error loss, provided that $\mathbb{E}[\text{Var}(W \mid T)] > 0$. Conditioning on $T$ eliminates unnecessary Monte Carlo noise without changing the mean.

Furthermore, in distributions with parameter-dependent boundaries (such as $\text{Uniform}(\theta_1, \theta_2)$ or Pareto families), the complete sufficient statistic is composed of extreme order statistics $(X_{(1)}, X_{(n)})$. Rao-Blackwell conditioning on order statistics concentrates probability mass on the boundary simplex, providing optimal non-parametric and parametric estimators that achieve $\mathcal{O}(1/n^2)$ convergence rates.