ANOVA is an acronym for ANalysis Of VAriance. An ANOVA segregates different sources of variation seen in experimental results. Some of the sources are "explained" (usually due to the treatments the experimenter applied), while the remainder are lumped together as "unexplained" variation (also called the "Error term"). An ANOVA then tests if the variation associated with an explained source is large relative to the unexplained variation. If that ratio (the F statistic) is so large that the probability that it occurred by chance is low (for example, P<=0.05), we can conclude (at that level of probability) that that source of variation did have a significant effect.
For example, consider an experiment where three varieties of wheat were grown at four locations. At each of the locations, there were four blocks, within each of which were small plots for each of the varieties. The yield of each plot was measured. We wish to know if there is a significant difference in yield associated with the different varieties (one source of variation). We also wish to know if one location was superior to another. Finally, we wish to know if some varieties are superior at one location but inferior at another (that is, if there is an interaction of variety and location). The ANOVA procedure will answer these questions.
The layout of the various test plots and the method of assigning treatments to those plots constitutes the "experimental design." The wheat experiment, for example, is a "randomized complete blocks" experiment; all of the treatments occur once, randomly arranged in each block. Experimental designs can vary greatly. Each design requires a slightly different mathematical model and a slightly different procedure for analysis. Extensive discussions of different experimental designs and different ANOVA procedures can be found in statistics texts such as Gomez and Gomez (1984), Little and Hills (1978), Snedecor and Cochran (1980), and Sokal and Rohlf (1995).
CoStat can handle virtually any type of experimental design. It has a large number of pre-defined models that you can pick from a list (including: 1, 2, 3 and 4 way completely randomized, 1 and 2 way randomized blocks, latin square, nested, split plot, split-split plot, split block, some covariance designs, etc). Or, you can use a special language to describe different models.
Because the ANOVA procedure uses a Generalized Linear Models (GLM) approach, it can analyze unbalanced designs and experiments with missing values. It can calculate the Type I, II, or III Sums of Squares.
Before performing the ANOVA, CoStat performs Bartlett's test for homogeneity of variances, one of the assumptions of ANOVA.
After performing the ANOVA, the procedure can automatically run a means comparisons test (also called multiple comparisons of means) (for example, Duncan's, Student-Newman-Keuls (SNK), Tukey-Kramer, Tukey's HSD, or Least Significant Difference (LSD)).
Contrasts are related to multiple comparisons of means, but the tests are done during the ANOVA procedure. Contrasts are comparisons of different subsets of means and are planned before the experiment is conducted. For example, you might test the control against all other treatments. Contrasts are also called a priori comparisons, planned comparisons, and orthogonal contrasts (which indicates there is no overlap between the statistical questions asked by several contrasts).