t-test, F-test and chi-square test
Tests of significance: hypotheses, errors and p-values, one-sample, unpaired and paired t, the F-test for variances, and chi-square with examples.
On this page
A new wheat variety yields 0.5 t per ha more than the check in a five-plot trial. Is it really better, or could the difference be just plot-to-plot chance? Ninety F2 plants appear in a ratio that looks close to 9 : 3 : 3 : 1. Close enough? Questions like these are answered by tests of significance. They compare an observed difference with the variation expected by chance alone.
This article covers the three tests met most often in agricultural statistics: Student's t-test for means, the F-test for variances, and the chi-square (χ2) test for frequencies.
The logic of testing
- State the null hypothesis (H0), usually "no difference" or "the data fit the expected ratio", and the alternative (H1).
- Choose the level of significance α, usually 5% (0.05), sometimes 1%.
- Calculate a test statistic from the data.
- Compare it with the table (critical) value for the right degrees of freedom, or compute the p-value.
- If the statistic exceeds the critical value (p < α), reject H0: the difference is significant. Otherwise, there is not enough evidence to reject it.
There are two possible errors:
| H0 true | H0 false | |
|---|---|---|
| Reject H0 | Type I error (probability α) | Correct |
| Do not reject H0 | Correct | Type II error (probability β) |
The power of a test, 1 − β, is its chance of detecting a real difference. Power increases with more replications, larger true differences and lower variability.
A test is two-tailed when H1 is "different", and one-tailed when H1 specifies a direction, such as "higher".
Student's t-test
W. S. Gosset developed the t-test in 1908 while working at the Guinness brewery in Dublin, and published it under the pen name "Student". It is used for small samples when the population standard deviation is unknown, assuming the data are roughly normal.
One-sample t
This version tests whether a sample mean differs from a standard value μ0:
Two independent samples (unpaired t)
This version compares the means of two separate groups, such as two varieties on different plots:
If the two variances differ significantly, a modified test (Welch's t) is used instead.
Comparing two wheat varieties
Variety A yields 4.2, 4.6, 4.4, 4.8 and 4.5 t per ha (mean 4.5, s2 = 0.05). Variety B yields 4.8, 5.2, 4.6, 5.4 and 5.0 t per ha (mean 5.0, s2 = 0.10).
- F-test for equal variances: F = 0.10 ⁄ 0.05 = 2.0. The table value for df (4, 4) at 5% is 6.39, so the variances can be pooled.
- sp2 = (4 × 0.05 + 4 × 0.10) ⁄ 8 = 0.075.
- t = 0.5 ⁄ √[0.075 × (1⁄5 + 1⁄5)] = 0.5 ⁄ 0.173 = 2.89.
- With df = 8, the table value of t at 5% (two-tailed) is 2.306. Since 2.89 > 2.306, the difference is significant: B outyields A.
Paired t
The paired t-test is used when observations come in pairs on the same unit: before and after treatment on the same animals, two methods applied to the same samples, or two treatments on halves of the same field. It works on the differences d:
Pairing removes variation between units, so it is more powerful when units differ a lot.
The F-test
The F-test, named in honour of R. A. Fisher, compares two variances:
F is used to check equal variances before an unpaired t-test, to compare the precision of two methods, and above all in ANOVA. There, the treatment mean square is tested against the error mean square to compare many means at once.
The chi-square test
The chi-square test, introduced by Karl Pearson in 1900, compares observed frequencies (O) with expected frequencies (E):
Goodness of fit
This version tests whether data fit a theoretical ratio. It is the standard test in genetics.
Mendel's dihybrid ratio
Mendel's F2 of round–yellow × wrinkled–green peas gave 315 round yellow, 108 round green, 101 wrinkled yellow and 32 wrinkled green (total 556). Expected numbers for 9 : 3 : 3 : 1 are 312.75, 104.25, 104.25 and 34.75.
χ2 = (2.252 ⁄ 312.75) + (3.752 ⁄ 104.25) + (3.252 ⁄ 104.25) + (2.752 ⁄ 34.75) = 0.016 + 0.135 + 0.101 + 0.218 = 0.47.
With df = 3, the table value at 5% is 7.815. Because 0.47 is far below it, the data fit the 9 : 3 : 3 : 1 ratio well.
Test of independence
This version tests whether two classifications are associated, for example seed treatment (treated or untreated) and disease (present or absent), in a contingency table.
Some rules apply to all chi-square tests:
- Use counts, never percentages.
- Expected frequencies should be at least 5. Merge classes if they are not, or for 2 × 2 tables apply Yates' correction, |O − E| − 0.5.
- Chi-square tells you whether an association exists, not how strong it is.
Key points
- Testing compares an observed statistic with its chance distribution under H0. Significance means p < α, usually 0.05.
- A Type I error rejects a true H0 (probability α); a Type II error accepts a false one (β); power = 1 − β.
- t-tests compare means: one-sample (df n − 1), unpaired (df n1 + n2 − 2, pooled variance) and paired (on differences).
- F = larger variance ⁄ smaller variance. It checks equal variances and is the basis of ANOVA.
- χ2 = Σ(O − E)2 ⁄ E. Goodness of fit has df = classes − 1; independence has df = (r − 1)(c − 1). Expected counts should be at least 5.
What the exams ask
- Define null hypothesis, level of significance, Type I and Type II errors, and power.
- Carry out an unpaired or paired t-test on given data.
- Test a genetic ratio with chi-square.
- State the degrees of freedom for each test.
- Explain when to use paired rather than unpaired t, and when to apply Yates' correction.
Described in general terms: the kind of examination, not a past paper. Laisha never reproduces exam questions.
Questions people ask
Does "not significant" mean there is no difference?
No. It means the data do not provide enough evidence of a difference at the chosen level. A real difference may exist but be too small to detect with the replications used. Reporting the size of the difference with a confidence interval is more informative than "significant" or "not significant" alone.
Why is 5% the usual level?
It is a convention popularised by Fisher, balancing the risk of false claims against the risk of missing real effects. Stricter levels such as 1% are used when a false positive would be costly, for example when releasing a variety or approving a drug.
Can I use a t-test to compare five varieties two at a time?
It is better not to. Many pairwise tests raise the chance of at least one false significant result well above 5%. Use ANOVA first, then a planned comparison or a multiple-comparison method such as LSD or Tukey's test.
Sources
- Student. "The probable error of a mean." Biometrika 6, 1–25 (1908).
- Gomez, K. A. and Gomez, A. A. Statistical Procedures for Agricultural Research, 2nd edition. Wiley, 1984.
- Snedecor, G. W. and Cochran, W. G. Statistical Methods, 8th edition. Iowa State University Press, 1989.
- Panse, V. G. and Sukhatme, P. V. Statistical Methods for Agricultural Workers, 4th edition. ICAR, New Delhi, 1985.
More in statistics and research methods
Discussion
Questions, corrections and notes from readers on this article. Every comment is checked before it appears: keep it on the topic, and do not share phone numbers, links or contact details.
Last reviewed