Empirical distribution function

In statistics, an empirical distribution function is a cumulative probability distribution function that concentrates probability 1/"n" at each of the "n" numbers in a sample.

Let $X\_1,ldots,X\_n$ be iid random variables in $mathbb\{R\}$ with the cdf "F"("x").

The empirical distribution function $F\_n(x)$ based on sample $X\_1,ldots,X\_n$ is a step function defined by

:$F\_n(x)\; =\; frac\{\; mbox\{number\; of\; elements\; in\; the\; sample\}\; leq\; x\}n\; =\; frac\{1\}\{n\}\; sum\_\{i=1\}^n\; I(X\_i\; le\; x),$

where "I"("A") is the indicator of event "A".

For fixed "x", $I(X\_ileq\; x)$ is a Bernoulli random variable with parameter "p" = "F"("x"), hence $nF\_n(x)$ is a binomial random variable with mean "nF"("x") and variance "nF"("x")(1 − "F"("x")).

Asymptotical properties

* By the strong law of large numbers,

:: $F\_n(x)\; o\; F(x)$ almost surely for fixed "x".

:In other words, $F\_n(x)$ is a consistent unbiased estimator of the cumulative distribution function "F(x)".

* By the central limit theorem,

:: $sqrt\{n\}(F\_n(x)-F(x))$

converges in distribution to a normal distribution "N"(0, "F"("x")(1 − "F"("x"))) for fixed "x".:The Berry–Esséen theorem provides the rate of this convergence.
* By the Glivenko-Cantelli theorem $F\_n(x)\; o\; F(x)$ uniformly over "x", that is :: $|F\_n(x)-F(x)|\_infty\; o\; 0$ with probability 1. :The Dvoretzky-Kiefer-Wolfowitz inequality provides the rate of this convergence.
* Kolmogorov showed that :: $sqrt\{n\}|F\_n(x)-F(x)|\_infty$ converges in distribution to the Kolmogorov distribution, provided that "F"("x") is continuous.:The Kolmogorov-Smirnov test for "goodness-of-fit" is based on this fact.
* By Donsker's theorem,:: $sqrt\{n\}(F\_n-F)$, as a process indexed by "x", converges weakly in $ell^infty(mathbb\{R\})$ to a Brownian bridge "B"("F"("x")).

See also

* Càdlàg functions
* Empirical probability
* Empirical process