Agricultural Statistics & Data Science · Statistical theory core · also Testing of Hypotheses, Statistical Inference (MSc core, part)
The logic and optimality of tests: errors and power, the Neyman-Pearson lemma, uniformly most powerful and unbiased tests, likelihood-ratio tests and their large-sample forms, and Wald's sequential probability ratio test.
In the exams: AICE-JRF Ag Stat Units 1-2 incl. SPRT (heavy); GATE ST 'Testing of Hypotheses' (heavy); IIT JAM MS (heavy); UPSC ISS Statistics I-II (heavy)
A statistical test is a rule for deciding, from data, between two statements about a population: a null hypothesis, usually 'no effect' or 'the standard holds', and an alternative. Agricultural research runs on such decisions: does a new variety out-yield the check, is the germination of a seed lot below the certification standard, are pest counts over-dispersed, does fertiliser response differ between sites? The theory of testing says how to make these decisions with controlled error rates and with the greatest possible power to detect real effects.
This subject builds on Probability Theory (limit theorems, Slutsky's lemma), Probability Distributions and Sampling Distributions (chi-square, t, F, the non-central distributions and the exponential family) and Statistical Inference I: Estimation (sufficiency, likelihood, Fisher information, pivots). It refers to those manuals rather than repeating them.
The chapters follow the classical order: hypotheses, errors, size and power; the Neyman-Pearson lemma; monotone likelihood ratio and uniformly most powerful tests; unbiased tests; likelihood-ratio tests and Wilks' theorem; Wald and score tests; the standard Z, t, chi-square and F tests used daily in agricultural research; the duality between tests and confidence intervals; multiple testing with family-wise error and false discovery rates; Wald's sequential probability ratio test with its operating-characteristic and average-sample-number functions; and power and sample-size calculation.
Examinations test all of this. The ICAR AICE-JRF/SRF Agricultural Statistics paper names Neyman-Pearson, UMP and likelihood-ratio tests in Unit 1 and the SPRT in Unit 2. GATE Statistics, IIT JAM Mathematical Statistics, the ISI tests and the Indian Statistical Service ask for most powerful critical regions, power functions, likelihood-ratio statistics and SPRT boundaries as numerical and multiple-select items. The entrance-level tests (AIEEA-PG, CUET-PG, SSC JSO, state services) concentrate on the standard tests and the two types of error.
A statistical hypothesis is a statement about the distribution of the data, usually about a parameter θ. A simple hypothesis specifies the distribution completely (θ = θ₀); a composite hypothesis specifies a set of values (θ ≤ θ₀, or μ = μ₀ with σ unknown). The null hypothesis H₀ is the one whose wrongful rejection is treated as the more serious error, or the one representing no change; the alternative H₁ is what the investigator hopes to show or wants to detect.
A test divides the sample space into a critical (rejection) region C and an acceptance region. If the data fall in C, H₀ is rejected. In practice the region is described by a test statistic T and a critical value: reject if T > c (one-sided) or |T| > c (two-sided). A randomised test is described by a test function φ(x) ∈ [0, 1], the probability of rejecting H₀ when x is observed; randomisation is needed in theory to achieve an exact size with discrete data, though it is rarely used in practice.
The choice between one-sided and two-sided alternatives must be made before seeing the data, from the scientific question. A seed-certification test asks whether germination is below the standard, which is one-sided; a varietal comparison asks whether two varieties differ, which is normally two-sided unless only an improvement would ever be acted upon.
Two errors are possible. A Type I error rejects a true H₀; a Type II error fails to reject a false H₀. Their probabilities are α = P(reject H₀ | H₀ true) and β = P(accept H₀ | H₁ true). The power of the test at θ is 1 − β(θ) = P(reject H₀ | θ). The power function π(θ) = P_θ(reject H₀) summarises the test: under H₀ it gives the Type I error probabilities, under H₁ the power.
The size of a test is sup over θ in H₀ of π(θ), the largest Type I error probability. A test is of level α if its size is at most α. For a fixed sample size, α and β cannot both be made arbitrarily small: shrinking the critical region lowers α and raises β. The Neyman-Pearson approach fixes α (conventionally 0.05 or 0.01) and seeks the test with the greatest power. Only increasing the sample size, or reducing the error variance by better design, lowers both.
A worked example: plot yields are N(μ, 16) and n = 16 plots are sampled; H₀: μ = 50 against H₁: μ = 52, rejecting H₀ when x̄ > 51.645. The standard error is 1, so α = P(Z > 1.645) = 0.05, and the power at μ = 52 is P(Z > −0.355) ≈ 0.639. With 64 plots the standard error halves, the critical value becomes 50.82, and the power rises to about 0.991.
The p-value is the probability, computed under H₀, of a test statistic at least as extreme as the one observed. Rejecting H₀ when p ≤ α is the same as rejecting when the statistic falls in the size-α critical region. The p-value is not the probability that H₀ is true, nor the probability that the result is due to chance, and 1 − p is not the probability that H₁ is true. Under a continuous null distribution the p-value is uniformly distributed on (0, 1), which is the basis of the multiple-testing methods of Chapter 9.
Statistical significance is not practical importance. A large trial can make a 0.5% yield difference 'significant'; a small one can miss a 10% difference. Reporting the estimated effect with a confidence interval, not only a p-value, lets readers judge both. The American Statistical Association's 2016 statement on p-values makes these points, and they matter especially when trial results are turned into recommendations to farmers.
Failure to reject H₀ does not establish H₀. To show that two varieties are equivalent, or that a residue is below a limit, the hypotheses must be reversed (equivalence or non-inferiority testing), so that the burden of proof lies on the claim of no important difference.
For testing a simple H₀: θ = θ₀ against a simple H₁: θ = θ₁, a test of size α is most powerful (MP) if no other test of size at most α has greater power at θ₁. The Neyman-Pearson lemma (1933) identifies it: reject H₀ when the likelihood ratio Λ(x) = L(θ₁; x)/L(θ₀; x) exceeds a constant k, accept when it is below k, and randomise when it equals k, with k (and the randomisation probability) chosen so that the size is exactly α. Any MP test has this form, apart from sets of probability zero.
The intuition: the lemma ranks sample points by how much more likely they are under H₁ than under H₀ and puts the most 'H₁-like' points into the critical region first, until the budget α is spent. The proof compares the power of any other size-α test φ with that of the likelihood-ratio test φ* through ∫(φ* − φ)(f₁ − kf₀) ≥ 0.
In practice the inequality Λ(x) > k is simplified to an equivalent inequality in a sufficient statistic. For a normal mean with known σ and μ₁ > μ₀, log Λ is increasing in x̄, so the MP test rejects when x̄ > μ₀ + z_α σ/√n. For Poisson counts with λ₁ > λ₀, Λ is increasing in ΣXᵢ, so the MP test rejects for large totals. For exponential lifetimes with rate λ₁ > λ₀ (shorter lives under H₁), Λ = (λ₁/λ₀)ⁿ exp[−(λ₁ − λ₀)ΣXᵢ] is decreasing in ΣXᵢ, so the MP test rejects for small totals.
With discrete data the size of a non-randomised test jumps. Testing H₀: p = 0.5 against H₁: p = 0.8 with n = 10 Bernoulli trials, the MP test rejects for large X. P(X ≥ 9 | p = 0.5) = 11/1024 ≈ 0.0107 and P(X ≥ 8 | p = 0.5) = 56/1024 ≈ 0.0547. A non-randomised test can have size 0.0107 or 0.0547, but not exactly 0.05. The randomised MP test of size 0.05 rejects when X ≥ 9 and, when X = 8, rejects with probability γ = (0.05 − 0.0107)/P(X = 8) = (0.05 − 0.0107)/(45/1024) ≈ 0.894.
Practitioners prefer to use the conservative non-randomised test and report the attained size or the p-value; mid-p-values are a compromise. The lemma also explains why the test's form does not depend on the specific alternative value when the likelihood ratio is monotone in one statistic, which is the bridge to uniformly most powerful tests in the next chapter.
Two further facts are worth knowing. The MP test is unbiased: its power at θ₁ is at least α. And the lemma extends to finding tests that maximise power subject to several side conditions (the generalised Neyman-Pearson lemma), which is the tool used to derive unbiased tests in Chapter 4.
A Bayesian reading of the lemma is also useful. Rejecting when L(θ₁)/L(θ₀) > k is the same as rejecting when the posterior odds of H₁ exceed k times the prior odds; with losses attached to the two errors, the Bayes rule has exactly the likelihood-ratio form, with k fixed by the prior and the losses rather than by α. The Neyman-Pearson choice of k by a fixed size is therefore one particular way of setting the threshold of a likelihood-ratio rule.
A seed lot is accepted if its germination rate is p₀ = 0.90 and should be rejected if it is p₁ = 0.80. Testing n = 50 seeds, the MP test rejects for small numbers germinated. Under H₀ the expected count is 45; P(X ≤ 40 | p = 0.90) ≈ 0.0245 and P(X ≤ 41 | p = 0.90) ≈ 0.0579, so the non-randomised test rejecting when X ≤ 40 has size 0.0245, and its power at p = 0.80, P(X ≤ 40 | p = 0.80) ≈ 0.556, is modest. Larger samples or a sequential plan (Chapter 10) improve the trade-off. These are the calculations behind the sampling rules in seed-testing and acceptance-sampling standards, although the actual rules and tolerances are set by national and international seed-testing bodies and vary between them. Raising the sample to 200 seeds, with the critical count chosen for a size near 0.05, lifts the power at p = 0.80 close to 1, which is why certification tests use several hundred seeds.
For a composite alternative, a level-α test is uniformly most powerful (UMP) if it is most powerful against every θ in H₁ simultaneously. UMP tests do not exist in general, but they do for one-sided problems in families with a monotone likelihood ratio.
A family {f(x; θ)} has a monotone likelihood ratio (MLR) in a statistic T(x) if, for every θ₁ > θ₀, the ratio f(x; θ₁)/f(x; θ₀) is a non-decreasing function of T(x). One-parameter exponential families f = h(x) exp[η(θ)T(x) − A(θ)] with η increasing have MLR in T; so do the uniform (0, θ) in X₍ₙ₎, the hypergeometric in the number of defectives, and the non-central t in its non-centrality.
Karlin-Rubin theorem: if the family has MLR in T, then for H₀: θ ≤ θ₀ against H₁: θ > θ₀ the test that rejects when T > c (randomising at T = c if needed), with P_θ₀(T > c) = α, is UMP of level α. Its power function is non-decreasing in θ, so its size is attained at θ₀. For H₀: θ ≥ θ₀ against θ < θ₀, reject for small T.
Examples. Normal mean with known σ: the test rejecting for x̄ > μ₀ + z_α σ/√n is UMP for μ > μ₀. Normal variance with known mean: reject for Σ(Xᵢ − μ)² > σ₀²χ²_(n, α) for σ² > σ₀². Bernoulli trials: reject for large ΣXᵢ. Poisson counts: reject for large ΣXᵢ. Exponential mean θ: reject for large ΣXᵢ when testing θ > θ₀. Uniform (0, θ): for H₀: θ ≤ θ₀ against θ > θ₀, reject when X₍ₙ₎ > θ₀(1 − α)^(1/n), which is UMP; for H₀: θ = θ₀ against the two-sided alternative θ ≠ θ₀, the uniform family is unusual in that a UMP test does exist: reject when X₍ₙ₎ > θ₀ or X₍ₙ₎ ≤ θ₀α^(1/n).
For two-sided alternatives in regular families, UMP tests generally do not exist. For a normal mean with known σ, the MP test against μ₁ > μ₀ rejects in the upper tail and against μ₁ < μ₀ in the lower tail; no single test is best on both sides. The two-sided test |Z| > z_(α/2) is a compromise, which is best among unbiased tests (Chapter 4). Similarly, when nuisance parameters are present (normal mean with σ unknown) UMP tests generally fail to exist, and the theory turns to unbiased, similar or invariant tests.
A practical reading: when the scientific question is truly one-sided, as in testing whether a germination rate falls below a standard or whether a residue exceeds a limit, the one-sided test is optimal and more powerful than a two-sided one at the same α. Choosing the side after seeing the data, however, doubles the real Type I error rate.
A test of level α is unbiased if its power is at least α at every θ in H₁: the test never rejects a false hypothesis less often than a true one. A biased test, such as a one-sided test applied to a two-sided problem, has power below α on one side, which is unreasonable. Among unbiased level-α tests, one that is most powerful at every alternative is uniformly most powerful unbiased (UMPU).
If the power function is continuous, an unbiased test must have power exactly α on the boundary between H₀ and H₁ (it is similar on the boundary). For a two-sided problem H₀: θ = θ₀ in a one-parameter exponential family, the UMPU test rejects when T < c₁ or T > c₂, with c₁ and c₂ chosen so that the size is α and the derivative of the power function at θ₀ is zero; equivalently E_θ₀[Tφ(T)] = αE_θ₀(T). For symmetric null distributions (normal, t) this gives equal tails; for skewed ones (chi-square for a variance) the UMPU cut-offs are not the equal-tail points, although equal tails are what is used in practice.
When the model has nuisance parameters, the theory looks for tests whose size does not depend on them. A test is similar on the boundary if its rejection probability equals α for every value of the nuisance parameter there. In a multiparameter exponential family f = h(x)exp[θU(x) + Σηⱼ Tⱼ(x) − A], the statistic T = (T₁, ..., T_k) is sufficient and complete for the nuisance parameters when θ is fixed. Conditioning on T removes them: tests with Neyman structure (conditional size α given T) are similar, and the UMPU test for θ is the conditional UMP or UMPU test based on U given T.
This machinery produces the standard tests as optimal. The one-sample and paired t-tests are UMPU for a normal mean with σ unknown (one-sided and two-sided). The two-sample pooled t-test is UMPU for equal means under equal variances. The chi-square test of a normal variance with μ unknown is UMPU (with the unequal-tail cut-offs). Fisher's exact test for a 2 × 2 table, which conditions on the margins, is the (randomised) UMPU test for the odds ratio. The F-test in analysis of variance is not UMPU but is UMP among invariant tests.
The point for an applied statistician is reassurance: the familiar tests are not arbitrary recipes; under their assumptions no other unbiased test does better. When those assumptions fail (non-normal errors, unequal variances, dependence) the optimality goes too, which is why robust and non-parametric alternatives exist.
When no UMP or UMPU test exists, the generalised likelihood ratio test (LRT) is the standard general method. For H₀: θ ∈ Θ₀ against H₁: θ ∈ Θ − Θ₀, the statistic is Λ = sup over Θ₀ of L(θ)/sup over Θ of L(θ), which lies between 0 and 1; H₀ is rejected when Λ is small, i.e. when −2 log Λ is large. Λ compares the best fit under the restriction with the best fit without it, using the maximum-likelihood estimates under each.
Many standard tests are LRTs. For a normal mean with σ unknown, Λ is a decreasing function of |t|, so the LRT is the two-sided t-test. For comparing k normal means with common variance, the LRT is the one-way ANOVA F-test. For a normal variance it is a chi-square test (with LRT cut-offs). For k independent samples, Bartlett's test of equal variances is a modified LRT. For contingency tables the LRT statistic is G² = 2ΣO log(O/E).
Wilks' theorem (1938): under regularity conditions, if H₀ is true and imposes r independent restrictions on the parameters (r = dimension of Θ minus dimension of Θ₀), then −2 log Λ converges in distribution to χ²_r. The large-sample LRT therefore rejects at level α when −2 log Λ > χ²_(r, α). The conditions are those for asymptotic normality of the MLE plus the requirement that the true value under H₀ lies in the interior of the parameter space. Under a local alternative, −2 log Λ is asymptotically non-central chi-square, which gives its power.
Examples. For a multinomial goodness-of-fit test with k classes and s estimated parameters, G² ~ χ²_(k−1−s) approximately. For an r × c contingency table testing independence, the degrees of freedom are (r − 1)(c − 1). For comparing nested generalised linear models, the deviance difference is approximately chi-square with degrees of freedom equal to the number of extra parameters. For testing a single parameter, −2 log Λ is approximately χ²₁, equivalent to a two-sided z-test.
Boundary problems need care. Testing whether a variance component is zero (H₀: σ²_g = 0 against σ²_g > 0, common in breeding trials) puts the null value on the edge of the parameter space; the limiting distribution is then a 50:50 mixture of a point mass at zero and χ²₁, so the naive χ²₁ p-value is about twice the correct one. Testing the number of components in a mixture violates the conditions more severely, and Wilks' theorem does not apply.
The LRT is invariant to reparameterisation, uses the full likelihood, and generalises easily, which is why it underlies model comparison in mixed models (with REML, only models with the same fixed effects can be compared by likelihood ratio), generalised linear models and genetic linkage analysis, where the LOD score is log₁₀ of a likelihood ratio and a LOD of 3 corresponds to a likelihood ratio of 1000. In small samples the chi-square approximation can be poor; exact tests, Bartlett-type corrections or simulation of the null distribution (parametric bootstrap) are the remedies.
A worked example: ten Poisson counts of lesions per leaf total 30, and H₀: λ = 2 is tested. The MLE is λ̂ = 3, and −2 log Λ = 2[30 log(3/2) − 10(3 − 2)] = 4.33, which exceeds χ²₁,₀.₀₅ = 3.841, so H₀ is rejected at 5%. For the same data the score statistic is 5.00 and the Wald statistic 3.33: the three asymptotically equivalent tests give noticeably different values at n = 10, a reminder that 'asymptotically equivalent' says nothing about small samples.
Three large-sample tests of H₀: θ = θ₀ are based on the likelihood. The likelihood-ratio test compares log-likelihoods at θ̂ and θ₀. The Wald test measures the distance of the MLE from θ₀ in standard-error units: W = (θ̂ − θ₀)² nI(θ̂), or (θ̂ − θ₀)²/SE(θ̂)², with I evaluated at the MLE. The score (Rao, Lagrange multiplier) test measures the slope of the log-likelihood at θ₀: S = U(θ₀)²/[nI(θ₀)], where U is the score; it needs only the fit under H₀. Under H₀ all three are asymptotically χ²₁ (χ²_r for r restrictions in the multiparameter case, using the information matrix) and asymptotically equivalent; under local alternatives they have the same non-central chi-square limit.
Geometrically, on a plot of the log-likelihood the LRT looks at the vertical drop from the maximum to θ₀, the Wald test at the horizontal distance between θ̂ and θ₀ scaled by the curvature at θ̂, and the score test at the slope at θ₀ scaled by the curvature at θ₀. For a quadratic log-likelihood (normal mean with known σ) all three coincide exactly.
Binomial proportion, H₀: p = p₀. Wald: z = (p̂ − p₀)/√(p̂(1 − p̂)/n). Score: z = (p̂ − p₀)/√(p₀(1 − p₀)/n), the familiar large-sample test that uses the null standard error. With 60 successes in 100 trials and p₀ = 0.5, the score z is 2.0 and the Wald z is 2.04. Pearson's chi-square statistic for goodness of fit and for contingency tables is a score statistic; the log-rank test in survival analysis is a score test.
The tests differ in finite samples and in convenience. The score test needs only the null fit, which makes it attractive for testing whether to add terms to a model (for example testing for over-dispersion in Poisson counts or for a genotype-by-environment term) without fitting the bigger model. The Wald test needs only the unrestricted fit, so it is what software prints next to each coefficient; but it is not invariant to reparameterisation (testing θ = 1 and log θ = 0 can give different answers), and it behaves badly when the estimate is near a boundary or when the log-likelihood is far from quadratic, as in logistic regression with large effects (the Hauck-Donner effect, where the Wald statistic decreases as the effect grows). The LRT is invariant and usually the most reliable of the three in moderate samples, at the cost of fitting both models.
For confidence intervals the same three ideas give Wald intervals, score (Wilson-type) intervals and likelihood-ratio intervals; Chapter 8 makes the link exact.
The standard tests follow from the sampling distributions in the distributions manual. One-sample z-test for a mean with known σ: Z = (X̄ − μ₀)/(σ/√n) ~ N(0, 1) under H₀; with large samples, s replaces σ. One-sample t-test with σ unknown and normal data: t = (X̄ − μ₀)/(s/√n) ~ t_(n−1). Paired t-test: the same on differences, with n − 1 df for n pairs, used when the same plots, animals or farms are measured twice; pairing removes between-unit variation and is often far more powerful than an unpaired comparison. Two-sample pooled t-test: t = (X̄₁ − X̄₂)/[s_p√(1/n₁ + 1/n₂)] with n₁ + n₂ − 2 df, where s_p² = [(n₁ − 1)s₁² + (n₂ − 1)s₂²]/(n₁ + n₂ − 2), assuming equal variances. Welch's test drops that assumption, using separate variances and approximate (Satterthwaite) degrees of freedom; it is the safer default when variances or sample sizes differ.
Large-sample tests for proportions: one sample, z = (p̂ − p₀)/√(p₀q₀/n); two samples, z = (p̂₁ − p̂₂)/√(p̄q̄(1/n₁ + 1/n₂)) with the pooled proportion p̄. Correlation: t = r√(n − 2)/√(1 − r²) with n − 2 df tests ρ = 0; Fisher's z tests ρ = ρ₀ ≠ 0. Regression slope: t = b/SE(b) with n − 2 df.
Chi-square test for a normal variance: χ² = (n − 1)s²/σ₀² ~ χ²_(n−1). It is sensitive to non-normality, unlike t-tests for means. Pearson's goodness-of-fit test: Σ(O − E)²/E, approximately χ² with (classes − 1 − parameters estimated) df, with small expected frequencies pooled. Test of independence in an r × c contingency table: Σ(O − E)²/E with E = row total × column total/grand total and (r − 1)(c − 1) df. For 2 × 2 tables, χ² = n(ad − bc)²/[(a + b)(c + d)(a + c)(b + d)] with 1 df; Yates' continuity correction subtracts n/2 from |ad − bc| and is conservative; Fisher's exact test is used for small expected counts. Tests of homogeneity of proportions across groups have the same arithmetic.
F-test for equality of two normal variances: F = s₁²/s₂² ~ F(n₁ − 1, n₂ − 1); conventionally the larger variance is put in the numerator and the upper α/2 point used for a two-sided test. Like the variance chi-square test, it is sensitive to non-normality, so Levene's test is preferred in practice. Analysis-of-variance F-tests (treatment mean square over error mean square) rest on Cochran's theorem and are robust to moderate non-normality with balanced data.
Typical agricultural applications: a paired t-test for yield before and after a management change on the same farms; a two-sample t-test for two varieties in a completely randomised trial; a chi-square test of a 3:1 or 9:3:3:1 segregation ratio; a contingency test of adoption against farm-size class; an F-test for heterogeneity of error variances before pooling experiments.
Six farms record yields (t/ha) of a new and an old practice on adjacent strips; the differences (new − old) are 0.4, 0.6, 0.2, 0.5, 0.3 and 0.4. Their mean is 0.40 and their standard deviation 0.141, so t = 0.40/(0.141/√6) = 6.93 with 5 df, far beyond t₀.₀₂₅,₅ = 2.571: a clear improvement. An unpaired analysis of the same strips would include the large farm-to-farm differences in the error and could easily miss the effect. The design, not the arithmetic, decides which test is valid: paired data must be analysed as pairs.
The assumptions behind the paired t-test are that the differences are independent and roughly normal; with six pairs normality cannot be checked well, and a sign test or Wilcoxon signed-rank test (treated in Nonparametric and Rank-Based Methods) is a useful companion analysis. Here all six differences are positive, so even the sign test gives a one-sided p-value of (1/2)⁶ = 0.016. When the conclusions of a parametric and a distribution-free test agree, the result is robust to the assumptions; when they disagree, the data deserve a closer look.
There is a one-to-one correspondence between families of tests and confidence sets. If, for each θ₀, A(θ₀) is the acceptance region of a level-α test of H₀: θ = θ₀, then the set C(x) = {θ₀ : x ∈ A(θ₀)} is a confidence set with coverage at least 1 − α. Conversely, given a 1 − α confidence set C(x), the test that rejects H₀: θ = θ₀ when θ₀ ∉ C(x) has level α. So a two-sided level-α test rejects exactly when the hypothesised value lies outside the 1 − α two-sided confidence interval, and a one-sided test corresponds to a one-sided confidence bound.
Optimality transfers too: inverting UMP one-sided tests gives uniformly most accurate confidence bounds (smallest probability of covering false values), and inverting UMPU tests gives uniformly most accurate unbiased intervals. The t-interval is the inversion of the t-test; the chi-square interval for σ² inverts the chi-square test (and the UMPU cut-offs give the unbiased interval); Wilson's interval inverts the score test; Clopper-Pearson inverts exact binomial tests; likelihood-ratio intervals invert the LRT.
The duality is practical. A confidence interval reports everything a test does and more: whether the null value is excluded, how large the effect is, and how precisely it is estimated. If the 95% interval for the yield advantage of a new variety is 0.1 to 0.5 t/ha, the two-sided test of zero advantage rejects at 5%, the test of an advantage of 0.6 t/ha also rejects, and a test of 0.3 does not. If the interval is −0.2 to 0.8 t/ha, the data are compatible with a small loss and with a large gain, and 'not significant' should not be read as 'no difference'.
The duality underlies equivalence testing: two one-sided tests (TOST) at level α each declare equivalence within ±Δ exactly when the 1 − 2α confidence interval lies inside (−Δ, Δ). This is how bioequivalence of generic veterinary drugs and equivalence of a new variety to a reference for regulatory purposes are assessed, with margins set by regulators and varying by country.
One caution: a test and an interval correspond only if they come from the same procedure. A Wald interval and a score test can disagree near the boundary, as can an exact test and an approximate interval, because they are inversions of different tests.
When many hypotheses are tested, some will be rejected by chance. With m independent true null hypotheses each tested at α = 0.05, the probability of at least one false rejection is 1 − 0.95ᵐ: 0.40 for m = 10, 0.64 for m = 20, and the expected number of false rejections is mα. Field trials with many varieties, multi-trait analyses, genome-wide marker scans and residue screening across many compounds all face this.
The family-wise error rate (FWER) is the probability of one or more false rejections among the family. The Bonferroni procedure tests each hypothesis at α/m (or multiplies each p-value by m); by Boole's inequality it controls the FWER at α under any dependence, but it is conservative when m is large or the tests are positively correlated. Šidák's procedure uses 1 − (1 − α)^(1/m), slightly less conservative, exact under independence. Holm's step-down procedure orders the p-values p₍₁₎ ≤ ... ≤ p₍ₘ₎ and rejects H₍₁₎, H₍₂₎, ... as long as p₍ⱼ₎ ≤ α/(m − j + 1), stopping at the first failure; it controls the FWER under any dependence and is uniformly more powerful than Bonferroni, so there is no reason to use plain Bonferroni instead. Hochberg's step-up procedure is more powerful still but needs independence or positive dependence.
For comparing treatment means in ANOVA there are tailored procedures: Tukey's honestly significant difference for all pairwise comparisons, Dunnett's test for comparisons with a control (check variety), and Scheffé's method for all contrasts. The least significant difference (LSD) without protection controls only the per-comparison rate; Fisher's protected LSD (only after a significant F) is a common compromise in agronomic reports.
When thousands of hypotheses are tested, as with marker-trait associations or gene expression, controlling the FWER leaves little power. The false discovery rate (FDR), introduced by Benjamini and Hochberg in 1995, is the expected proportion of false rejections among all rejections (taken as 0 when there are none). Controlling FDR at q accepts that a small fraction of discoveries are false in exchange for many more true discoveries.
The Benjamini-Hochberg (BH) step-up procedure: order the p-values p₍₁₎ ≤ ... ≤ p₍ₘ₎, find the largest k with p₍ₖ₎ ≤ kq/m, and reject H₍₁₎, ..., H₍ₖ₎. It controls the FDR at q (more precisely at m₀q/m, where m₀ is the number of true nulls) when the tests are independent or positively dependent; the Benjamini-Yekutieli variant divides q by Σ1/i for arbitrary dependence. Example: with m = 10 p-values 0.001, 0.004, 0.012, 0.020, 0.035, 0.060, 0.11, 0.25, 0.48 and 0.71, and q = 0.05, the thresholds kq/m are 0.005, 0.010, 0.015, 0.020, 0.025, ...; the largest k satisfying the condition is k = 4 (0.020 ≤ 0.020), so four hypotheses are rejected. Bonferroni at 0.005 would reject two and Holm also two.
Which rate to control depends on the cost of errors. Confirmatory claims (registering a variety, approving a product) call for FWER control; exploratory screening (which markers to follow up) is well served by FDR. Whatever is done, the number of tests performed should be reported, so that readers can judge the risk of chance findings.
In a sequential test the sample size is not fixed in advance: observations are taken one at a time and after each one the statistician decides to accept H₀, reject H₀ or continue. Abraham Wald developed the sequential probability ratio test (SPRT) during the Second World War for acceptance inspection; it is used in agriculture for sequential sampling plans in pest management (decide to spray, not to spray, or inspect more plants) and in quality control of seed and produce.
For simple H₀: θ = θ₀ against simple H₁: θ = θ₁, let Λₙ = Πf(xᵢ; θ₁)/Πf(xᵢ; θ₀) after n observations. Choose constants B < 1 < A. Continue sampling while B < Λₙ < A; accept H₀ when Λₙ ≤ B; reject H₀ when Λₙ ≥ A. Wald's approximations, which ignore the overshoot of the boundaries, are A ≈ (1 − β)/α and B ≈ β/(1 − α), where α and β are the desired error probabilities. With α = 0.05 and β = 0.10, A = 18 and B ≈ 0.105; on the log scale, log A = 2.890 and log B = −2.251. The actual error rates using these boundaries satisfy α′ + β′ ≤ α + β, and in practice they are close to the nominal values.
Working with Zᵢ = log[f(xᵢ; θ₁)/f(xᵢ; θ₀)], the SPRT is a random walk Σzᵢ between two parallel barriers log B and log A. For Bernoulli observations (infested or not), Σzᵢ = d log(p₁/p₀) + (n − d) log[(1 − p₁)/(1 − p₀)], with d the number of infested units, and the continuation region in the (n, d) plane is a band between two parallel straight lines d = h₀ + sn and d = h₁ + sn. For a normal mean with known σ, the boundaries are also parallel lines in the cumulative sum Σxᵢ. These lines are exactly what is printed on field sequential-sampling charts.
The SPRT terminates with probability one. Its operating characteristic (OC) function L(θ) is the probability of accepting H₀ when θ is the true value: L(θ₀) ≈ 1 − α and L(θ₁) ≈ β. Wald's approximation for general θ uses the non-zero value h(θ) solving E_θ[(f(X; θ₁)/f(X; θ₀))^h] = 1: L(θ) ≈ (A^h − 1)/(A^h − B^h). The average sample number (ASN) function E_θ(N) is the expected number of observations; by Wald's identity E(ΣZᵢ) = E(N)E(Z), giving E_θ(N) ≈ [L(θ) log B + (1 − L(θ)) log A]/E_θ(Z), valid when E_θ(Z) ≠ 0. The ASN is largest for θ between θ₀ and θ₁, where the evidence accumulates slowly.
Wald and Wolfowitz proved that among all tests (sequential or not) with error probabilities at most α and β, the SPRT minimises the expected sample size both under H₀ and under H₁. Typical savings are about half the fixed-sample size at the two hypothesised values. That is why sequential plans are attractive when observations are costly or slow, as with destructive sampling or plant-by-plant inspection.
Disadvantages: the sample size is random and occasionally large (truncated SPRTs cap it), the efficiency gain holds only near θ₀ and θ₁, and composite hypotheses need modifications. In pest management, sequential plans built on negative binomial counts with an estimated k are standard; their boundaries depend on the economic threshold, which varies by crop, pest and region.
An inspector wants to accept a field (no action) if the proportion of infested plants is p₀ = 0.10 and to recommend spraying if it is p₁ = 0.30, with α = 0.05 and β = 0.10. Per infested plant, log(p₁/p₀) = log 3 = 1.0986; per clean plant, log[(1 − p₁)/(1 − p₀)] = log(0.7/0.9) = −0.2513. After n plants with d infested, continue while −2.251 < 1.0986d − 0.2513n < 2.890. The slope of both lines is s = 0.2513/1.3499 = 0.186 infested plants per plant inspected; the intercepts are h₀ = −2.251/1.3499 = −1.668 and h₁ = 2.890/1.3499 = 2.141. So the inspector sprays as soon as d ≥ 2.141 + 0.186n and stops without action as soon as d ≤ −1.668 + 0.186n: for example, after 9 clean plants in a row (0.186 × 9 − 1.668 = 0.006 ≥ 0) the field is accepted. The plan's OC and ASN curves, computed from the formulas above, tell the extension service how often fields at each infestation level will be sprayed and how many plants a scout will inspect on average, which is what decides whether the plan is practical.
Power calculations answer two questions before an experiment: how likely is it to detect an effect of a given size, and how many replicates are needed for a stated power? For a one-sided z-test of H₀: μ = μ₀ at level α with known σ, the power at μ₁ = μ₀ + δ is Φ(δ√n/σ − z_α). Setting this equal to 1 − β gives n = (z_α + z_β)²σ²/δ². For a two-sided test replace z_α by z_(α/2) (ignoring the negligible far tail). For two independent groups with n per group, the variance of the difference doubles: n = 2(z_(α/2) + z_β)²σ²/δ² per group.
Example: to detect a difference of δ = 0.5 t/ha between two varieties with σ = 0.6 t/ha, two-sided α = 0.05 (z = 1.96) and power 0.80 (z_β = 0.8416), n = 2(2.8016)²(0.36)/0.25 = 22.6, so 23 plots per variety. With paired or blocked designs σ is the standard deviation of differences, often much smaller, and n falls accordingly.
With σ unknown and estimated, the t-test's power uses the non-central t distribution with non-centrality δ√n/σ (one sample) and requires iteration because the degrees of freedom depend on n; the normal formula then gives a slight underestimate, usually corrected by adding one or two observations. For ANOVA F-tests the non-central F with λ = rΣτᵢ²/σ² gives the power (see the distributions manual).
For a one-sample test of a proportion, n = [z_α√(p₀q₀) + z_β√(p₁q₁)]²/(p₁ − p₀)². For two proportions with equal groups, n per group ≈ [z_(α/2)√(2p̄q̄) + z_β√(p₁q₁ + p₂q₂)]²/(p₁ − p₂)², where p̄ is the average of p₁ and p₂. Detecting a rise in germination from 80% to 90% with α = 0.05 (two-sided) and power 0.8 needs about 199 seeds per group.
Agricultural scientists often think in coefficients of variation and percentage differences. If the CV of plot yields is c% and the smallest difference worth detecting is d% of the mean, the number of replicates per treatment for a two-treatment comparison is r ≈ 2(z_(α/2) + z_β)²(c/d)². With CV 12%, d = 10%, α = 0.05 and power 0.80, r ≈ 2 × 7.849 × 1.44 = 22.6. In a multi-treatment RCBD the same formula applies to each pairwise comparison, with the error degrees of freedom adding a small correction. This explains why trials that aim to detect 5% yield differences need many replicates or very uniform sites, and why reported trial CVs are scrutinised.
Three cautions. Power must be computed for an effect size chosen on scientific or economic grounds before the trial, not from the observed effect afterwards: 'observed power' is a monotone function of the p-value and adds nothing. Low power inflates reported effects among significant results, because only overestimates cross the threshold (the winner's curse), a known problem in small variety trials and in marker-trait association studies. And power depends on design as much as on n: blocking, covariates, paired comparisons and better field technique reduce σ and may be cheaper than more replicates. Reporting the planned effect size, α, power and resulting sample size in a trial protocol is good practice and increasingly required by funders and journals.
A final point concerns multi-location and multi-season trials. The relevant error for comparing varieties across a region is not the within-trial plot error but the variety-by-environment interaction, so power for a regional recommendation depends mainly on the number of locations and seasons, not on replicates per site. Adding replicates at one station cannot substitute for testing in more environments; power calculations for regional variety testing should use the interaction variance estimated from earlier trial series.
A student meets testing as the second half of the inference core and as a heavy block of AICE-JRF, GATE ST and IIT JAM. The concrete skills are: writing the most powerful critical region from the Neyman-Pearson lemma and computing its size and power; recognising MLR and stating the UMP test; deriving a likelihood-ratio test and its asymptotic degrees of freedom; choosing the right standard test for a given design; computing SPRT boundaries; and computing the sample size for a stated power. A thesis needs the same skills to justify every significance statement and to plan replication.
A statistician five years into work at a research station uses testing every day: choosing paired, pooled or Welch t-tests correctly, checking segregation ratios and contingency tables, deciding between Tukey, Dunnett and protected LSD for variety trials, controlling false discoveries in marker screens, using likelihood-ratio tests for variance components with the boundary correction, designing sequential sampling plans for pest scouting, and computing replication from CV before a trial is approved. Explaining to scientists that 'not significant' is not 'no effect' is part of the job.
A senior statistician with twenty-five years of experience sets testing policy for an institute or a regulatory programme: the significance levels and multiple-comparison rules in variety-release and registration testing, equivalence margins with regulators, standards for reporting effect sizes and intervals, power requirements in trial protocols, and sequential plans for surveillance. Such a statistician reviews claims of new technologies, spots selective reporting and underpowered trials, and explains to policy makers what a test result does and does not show.
A PhD with long experience develops tests for new problems: optimal or invariant tests for complex designs, likelihood-ratio and score tests for mixed and genetic models with boundary parameters, sequential and group-sequential designs for long-term trials, and multiple-testing methods for genomic and spatial data with dependence. Such a researcher proves size and power properties, checks them by simulation at realistic sample sizes, and teaches the theory to doctoral students.
Proofs are sketched rather than given in full. Not covered in depth: invariant tests beyond a mention, locally most powerful tests, Bayesian testing and Bayes factors (in Bayesian Inference), group-sequential designs and alpha-spending, non-parametric tests (in Nonparametric and Rank-Based Methods), multivariate tests such as Hotelling's T² (in Multivariate Analysis), and permutation and bootstrap tests. Seed-testing tolerances, economic thresholds for sequential pest plans and regulatory equivalence margins vary by country, crop and agency and are not quoted. Tables of z, t, chi-square and F points are not reproduced; the values used are stated where needed.
Original practice questions written to the syllabus and pattern of the exams named in each question's examTags. They are not past papers and do not reproduce any board's questions. Every numerical answer was computed with a script; the values needed are stated in the question.
These notes are free to read here, with no account.
What the Laisha app adds:
Free for ten days, then a subscription. We would rather write that here than have you find it later.