Post-hoc analysis


Post-hoc analysis

Post-hoc analysis design and analysis of experiments, refers to looking at the data—after the experiment has concluded—for patterns that were not specified "a priori". It is also known as "data dredging" to evoke the sense that the more one looks the more likely something will be found. More subtly, each time a pattern in the data is considered, a statistical test is effectively performed. This greatly inflates the total number of statistical tests and necessitates the use of multiple testing procedures to compensate. However, this is difficult to do precisely and in fact most results of post-hoc analyses are reported as they are with unadjusted "p"-values. These "p"-values must be interpreted in light of the fact that they are a small and selected subset of a potentially large group of "p"-values. Results of post-hoc analysis should be explicitly labeled as such in reports and publications to avoid misleading readers.

In practice, post-hoc analysis is usually concerned with finding patterns in subgroups of the sample.

In Latin post-hoc means "after this".


=Student Newman-Keuls Post-Hoc ANOVA Analysis=

An example of a Post-hoc analysis would be a Student Newman-Keuls Test: "A different approach to evaluating a posteriori pairwise comparisons stems from the work of Student (1927), Newman (1939), and Keuls (1952). The Newman-Keuls procedure is based on a stepwise or layer approach to significance testing. Sample means are ordered from the smallest to the largest. The largest difference, which involves means that are r = p steps apart, is tested first at α level of significance; if significant, means that are r = p - 1 steps apart are tested at α level of significance and so on. The Newman-Keuls procedure provides an r-mean significance level equal to α for each group of r ordered means; that is, the probability of falsely rejecting the hypothesis that all means in an ordered group are equal to α. It follows that the concept of error rate applies neither on an experimentwise nor on a per comparison basis--the actual error rate falls somewhere between the two. The Newman-Keuls procedure, like Tukey's procedure, requires equal sample n's.

The critical difference y-hat(Wr), that two means separated by r steps must exceed to be declared significant is, according to the Newman-Keuls procedure,

: psi - widehat{W_{r = q_{alpha;p,v} sqrt{frac{MSE}{n ,

It should be noted that the Newman-Keuls and Tukey procedures require the same critical difference for the first comparison that is tested. The Tukey procedure uses this critical difference for all of the remaining tests while the Newman-Keuls procedure reduces the size of the critical difference, depending on the number of steps separating the ordered means. As a result, Newman-Keuls test is more powerful than Tukey's test. Remember, however, that Newman-Keuls procedure does not control the experimentwise error rate at α.

Frequently a test of the overall null hypothesis m1 =m2 …= mp is performed with an F statistic in ANOVA rather than with a range statistic. If the F statistic is significant, Shaffer (1979) recommends using the critical difference y-hat(Wr -1) instead of y-hat(Wr) to evaluate the largest pairwise comparison at the first step of the testing procedure. The testing procedure for all subsequent steps is unchanged. She has shown that the modified procedure leads to greater power at the first step without affecting control of the type I error rate. This makes dissonances, in which the overall null hypothesis is rejected by an F test without rejecting any one of the proper subsets of comparison, less likely." [http://www.ocair.org/files/KnowledgeBase/Statistics/Anova.asp]

See also

* The significance level α (alpha) in statistical hypothesis testing
*Subgroup analysis
*Post hoc ergo propter hoc

Bibliography

* http://www.ocair.org/files/KnowledgeBase/Statistics/Anova.asp
* http://www.eric.ed.gov/ERICWebPortal/custom/portlets/recordDetails/detailmini.jsp?_nfpb=true&_&ERICExtSearch_SearchValue_0=ED109185&ERICExtSearch_SearchType_0=no&accno=ED109185


Wikimedia Foundation. 2010.

Look at other dictionaries:

  • Post hoc ergo propter hoc — Post hoc redirects here. For the analytical technique, see Post hoc analysis. For the West Wing episode, see Post Hoc, Ergo Propter Hoc (The West Wing). Post hoc ergo propter hoc, Latin for after this, therefore because of this, is a logical… …   Wikipedia

  • Post hoc — is a Latin phrase, meaning after this or after the event . It may refer to: *Post hoc ergo propter hoc, a logical fallacy *Post hoc analysis, a form of statistical analysis …   Wikipedia

  • Analysis of variance — In statistics, analysis of variance (ANOVA) is a collection of statistical models, and their associated procedures, in which the observed variance in a particular variable is partitioned into components attributable to different sources of… …   Wikipedia

  • Post–Kyoto Protocol negotiations on greenhouse gas emissions — Post Kyoto negotiations refers to high level talks attempting to address global warming by limiting greenhouse gas emissions. Generally part of the United Nations Framework Convention on Climate Change (UNFCCC), these talks concern the period… …   Wikipedia

  • Post-Kyoto Protocol negotiations on greenhouse gas emissions — The Kyoto Protocol is an extension of the 1992 UN Framework Convention on Climate Change (UNFCCC), the world s first treaty to attempt to address global warming by limiting greenhouse gas emissions. The Protocol deals in detail with its first… …   Wikipedia

  • Subgroup analysis — Subgroup analysis, in the context of design and analysis of experiments, refers to looking for pattern in a subset of the subjectscite journal url=http://content.nejm.org/cgi/content/extract/354/16/1667 author=Lagakos SW journal=NEJM title=The… …   Wikipedia

  • Intelligence analysis management — This article deals with the roles of processing/analysis in the real world intelligence cycle as a part of intelligence cycle management. See Intelligence analysis for a discussion of the techniques of analysis. For a hierarchical list of… …   Wikipedia

  • Content analysis — or textual analysis is a methodology in the social sciences for studying the content of communication. Earl Babbie defines it as the study of recorded human communications, such as books, websites, paintings and laws. According to Dr. Farooq… …   Wikipedia

  • Multiple comparisons — In statistics, the multiple comparisons or multiple testing problem occurs when one considers a set of statistical inferences simultaneously.[1] Errors in inference, including confidence intervals that fail to include their corresponding… …   Wikipedia

  • Newman–Keuls method — In statistics, the Newman–Keuls method (named after D. Newman (1939),[1] and M. Keuls (1952)[2]) is a post hoc test used for comparisons after the performed F test (analysis of variance) is found to be significant. The Newman–Keuls method is very …   Wikipedia


Share the article and excerpts

Direct link
Do a right-click on the link above
and select “Copy Link”

We are using cookies for the best presentation of our site. Continuing to use this site, you agree with this.