Neyman–Pearson lemma


Neyman–Pearson lemma

In statistics, the Neyman-Pearson lemma, named after Jerzy Neyman and Egon Pearson, states that when performing a hypothesis test between two point hypotheses H0θ = θ0 and H1θ = θ1, then the likelihood-ratio test which rejects H0 in favour of H1 when

\Lambda(x)=\frac{ L( \theta _{0} \mid x)}{ L (\theta _{1} \mid x)} \leq \eta

where

P(\Lambda(X)\leq \eta|H_0)=\alpha

is the most powerful test of size α for a threshold η. If the test is most powerful for all \theta_1 \in \Theta_1, it is said to be uniformly most powerful (UMP) for alternatives in the set \Theta_1 \, .

In practice, the likelihood ratio is often used directly to construct tests — see Likelihood-ratio test. However it can also be used to suggest particular test-statistics that might be of interest or to suggest simplified tests — for this one considers algebraic manipulation of the ratio to see if there are key statistics in it is related to the size of the ratio (i.e. whether a large statistic corresponds to a small ratio or to a large one).

Contents

Proof

Define the rejection region of the null hypothesis for the NP test as

R_{NP}=\left\{ X: \frac{L(\theta_{0},X)}{L(\theta_{1},X)} \leq \eta\right\} .

Any other test will have a different rejection region that we define as RA. Furthermore define the function of region, and parameter

P(R,\theta)=\int_R L(\theta|x)\, dx,

where this is the probability of the data falling in region R, given parameter θ.

For both tests to have significance level α, it must be true that

\alpha= P(R_{NP}, \theta_0)=P(R_A, \theta_0) \,.

However it is useful to break these down into integrals over distinct regions, given by

P(R_{NP} \cap R_A, \theta) + P(R_{NP} \cap R_A^c, \theta) = 
P(R_{NP},\theta) ,

and

 P(R_{NP} \cap R_A, \theta) + P(R_{NP}^c \cap R_A, \theta) =  P(R_A,\theta).

Setting θ = θ0 and equating the above two expression, yields that

P(R_{NP} \cap R_A^c, \theta_0) =  P(R_{NP}^c \cap R_A, \theta_0).

Comparing the powers of the two tests, which are P(RNP1) and P(RA1), one can see that

P(R_{NP},\theta_1) \geq P(R_A,\theta_1) \text{ if, and only if, }
P(R_{NP} \cap R_A^c, \theta_1) \geq P(R_{NP}^c \cap R_A, \theta_1).

Now by the definition of RNP ,

 P(R_{NP} \cap R_A^c, \theta_1)= \int_{R_{NP}\cap R_A^c} L(\theta_{1}|x)\,dx \geq \frac{1}{\eta} \int_{R_{NP}\cap R_A^c} L(\theta_0|x)\,dx = \frac{1}{\eta}P(R_{NP} \cap R_A^c, \theta_0)
 = \frac{1}{\eta}P(R_{NP}^c \cap R_A, \theta_0) = \frac{1}{\eta}\int_{R_{NP}^c \cap R_A} L(\theta_{0}|x)\,dx \geq \int_{R_{NP}^c\cap R_A} L(\theta_{1}|x)dx  = P(R_{NP}^c \cap R_A, \theta_1).

Hence the inequality holds.

Example

Let X_1,\dots,X_n be a random sample from the \mathcal{N}(\mu,\sigma^2) distribution where the mean μ is known, and suppose that we wish to test for H_0:\sigma^2=\sigma_0^2 against H_1:\sigma^2=\sigma_1^2. The likelihood for this set of normally distributed data is

L\left(\sigma^2;\mathbf{x}\right)\propto \left(\sigma^2\right)^{-n/2} \exp\left\{-\frac{\sum_{i=1}^n \left(x_i-\mu\right)^2}{2\sigma^2}\right\}.

We can compute the likelihood ratio to find the key statistic in this test and its effect on the test's outcome:

\Lambda(\mathbf{x}) = \frac{L\left(\sigma_1^2;\mathbf{x}\right)}{L\left(\sigma_0^2;\mathbf{x}\right)} = 
\left(\frac{\sigma_1^2}{\sigma_0^2}\right)^{-n/2}\exp\left\{-\frac{1}{2}(\sigma_1^{-2}-\sigma_0^{-2})\sum_{i=1}^n \left(x_i-\mu\right)^2\right\}.

This ratio only depends on the data through \sum_{i=1}^n \left(x_i-\mu\right)^2. Therefore, by the Neyman-Pearson lemma, the most powerful test of this type of hypothesis for this data will depend only on \sum_{i=1}^n \left(x_i-\mu\right)^2. Also, by inspection, we can see that if \sigma_1^2>\sigma_0^2, then \Lambda(\mathbf{x}) is an increasing function of \sum_{i=1}^n \left(x_i-\mu\right)^2. So we should reject H0 if \sum_{i=1}^n \left(x_i-\mu\right)^2 is sufficiently large. The rejection threshold depends on the size of the test. In this example, the test statistic can be shown to be a scaled Chi-square distributed random variable and an exact critical value can be obtained.

See also

References

External links

  • Cosma Shalizi, a professor of statistics at Carnegie Mellon University, gives an intuitive derivation of the Neyman-Pearson Lemma using ideas from economics

Wikimedia Foundation. 2010.

Look at other dictionaries:

  • Neyman-Pearson-Lemma — Das Neyman Pearson Lemma ist ein Satz der mathematischen Statistik, der eine Optimalitätsaussage über die Konstruktion eines Hypothesentests macht. Gegenstand des Neyman Pearson Lemmas ist das denkbar einfachste Szenario eines Hypothesentests:… …   Deutsch Wikipedia

  • Neyman-Pearson lemma — In statistics, the Neyman Pearson lemma states that when performing a hypothesis test between two point hypotheses H 0: θ = θ 0 and H 1: θ = θ 1, then the likelihood ratio test which rejects H 0 in favour of H 1 when:Lambda(x)=frac{ L( heta {0}… …   Wikipedia

  • Egon Sharpe Pearson — (* 11. August 1895 in Hampstead; † 12. Juni 1980 London) war ein britischer Statistiker. Er ist der Sohn von Karl Pearson. Pearson folgte seinem Vater als Professor für Statistik am University College London. Er war Herausgeber der Zeitschrift… …   Deutsch Wikipedia

  • Jerzy Neyman — (* 16. April 1894 in Bendery, Moldawien; † 5. August 1981 in Oakland, Kalifornien) war ein polnischer Mathematiker und Autor wichtiger statistischer Bücher. Das Neyman Pearson Lemma ist nach ihm benannt. Neyman in Warschau 1973 …   Deutsch Wikipedia

  • Jerzy Neyman — Born April 16, 1894(1894 04 16) Bendery, Bessarabia, Imperial Russia Died August 5, 1981(1981 …   Wikipedia

  • Egon Pearson — Egon Sharpe Pearson (* 11. August 1895 in Hampstead; † 12. Juni 1980 Midhurst) war ein britischer Statistiker. Er ist der Sohn von Karl Pearson. Pearson folgte seinem Vater als Professor für Statistik am University College London. Er war… …   Deutsch Wikipedia

  • Karl Pearson — Infobox Scientist name = Karl Pearson |300px caption = Karl Pearson (né Carl Pearson) birth date = birth date|1857|3|27|mf=y birth place = Islington, London, England death date = death date and age|1936|4|27|1857|3|27|mf=y death place =… …   Wikipedia

  • Egon Pearson — Egon Sharpe Pearson (Hampstead, 11 August 1895 – London, 12 June 1980) was the only son of Karl Pearson, and like his father, a leading British statistician. He went to Winchester School and Trinity College, Cambridge, and succeeded his father as …   Wikipedia

  • Type I and type II errors — In statistics, the terms Type I error (also, α error, or false positive) and type II error (β error, or a false negative) are used to describe possible errors made in a statistical decision process. In 1928, Jerzy Neyman (1894 1981) and Egon… …   Wikipedia

  • Founders of statistics — Statistics is the theory and application of mathematics to the scientific method including hypothesis generation, experimental design, sampling, data collection, data summarization, estimation, prediction and inference from those results to the… …   Wikipedia