Agricultural Engineering · Foundations · also Agricultural Statistics and Data Analysis, Statistical Methods
Describing data, probability distributions, regression and designed experiments, with the frequency analysis that hydrology and quality control rely on.
In the exams: GATE AG Section 1 (probability and statistics asked every year 2019–26); ICAR AIEEA-PG Unit I; NCEES FE 6–9 questions; NCEES PE statistics
Probability, statistics and data analysis is the part of engineering mathematics that deals with variation. Every quantity an agricultural engineer measures varies: the moisture content of grain samples drawn from one bin, the draft of a plough across a field, the discharge of emitters along a drip lateral, the annual maximum rainfall at a station, the weight of filled bags leaving a packing line. Statistics gives the tools to summarise that variation, to say how far a sample tells us about the whole, to decide whether a difference between two machines or two treatments is real or chance, and to design experiments and quality checks that give a trustworthy answer for the least effort.
This manual follows the ten units of the course in order: measures of central tendency and dispersion; probability and random variables; the binomial, Poisson and normal distributions; correlation and linear regression; sampling and sampling distributions; tests of hypothesis with the z, t, chi-square and F statistics; the completely randomised and randomised block designs; frequency analysis of rainfall and floods; statistical quality control charts; and data analysis with spreadsheets and scripts. Each chapter defines its terms, gives the working formulas, shows how they are used on agricultural-engineering data, and ends with the points a student must carry into an exam hall.
The subject is examined everywhere an agricultural engineer is examined. GATE Agricultural Engineering places probability and statistics inside its Engineering Mathematics section, and Laisha's count of the 2019–2026 papers found at least one item on it in every year. ICAR AIEEA-PG puts statistics in Unit I alongside mathematics. The NCEES FE Other Disciplines exam gives Probability and Statistics its own block of 6–9 questions, and the NCEES PE Agricultural and Biological Engineering specification lists statistics among its common system applications. Hydrology papers then use the same ideas again as frequency analysis, and processing and machinery papers use them as quality control and field-test analysis.
Critical values of z, t, chi-square and F are taken from tables or software. In an exam the value is either given or must be read from a supplied table; this manual quotes only the handful of values every student should know (for example 1.645, 1.96 and 2.576 for the standard normal) and otherwise tells you which table and which degrees of freedom to use.
Data are either qualitative (categories such as soil texture class or tractor make) or quantitative. Quantitative data are discrete (counts: weeds per quadrat, defective seeds per hundred) or continuous (measurements: yield, moisture, draft). A population is the whole set of values of interest; a sample is the part actually observed. A number computed from a population, such as the population mean μ, is a parameter; the same quantity computed from a sample, the sample mean x̄, is a statistic and is used to estimate the parameter.
The arithmetic mean is the sum of the observations divided by their number, x̄ = Σx/n. For a frequency distribution with class marks x and frequencies f it is Σfx/Σf. The mean uses every observation, which makes it efficient but sensitive to outliers: one blocked nozzle reading of 0.2 L/min among twenty readings near 1.0 L/min pulls the mean noticeably. The sum of deviations from the mean is always zero, and the sum of squared deviations is smallest when taken about the mean; both facts are used later in regression. A weighted mean Σwx/Σw is used when observations carry different importance, for example the average moisture of grain lots of different masses, where each lot's moisture must be weighted by its dry-matter or wet mass rather than averaged plainly.
The geometric mean, the n-th root of the product of n positive values, is the correct average for rates that multiply, such as annual growth rates or dilution factors. The harmonic mean, n divided by the sum of reciprocals, is the correct average of rates expressed per unit of a fixed quantity, for example the average speed of a tractor that covers equal distances at different speeds. For any set of positive values that are not all equal, AM > GM > HM.
The median is the middle value when data are sorted: the ((n+1)/2)-th value for odd n and the mean of the two middle values for even n. For grouped data, median = L + ((N/2 − F)/f)·h, where L is the lower boundary of the median class, N the total frequency, F the cumulative frequency before the median class, f the frequency of the median class and h the class width. The median is unaffected by extreme values, so it is preferred for skewed data such as rainfall depths, farm sizes or incomes.
The mode is the most frequent value. For grouped data, mode = L + (f1 − f0)/(2f1 − f0 − f2)·h, where f1 is the modal-class frequency and f0, f2 the frequencies of the classes before and after it. For moderately skewed unimodal distributions Karl Pearson's empirical relation holds approximately: mode ≈ 3·median − 2·mean. In a symmetric distribution the three coincide. In a positively (right) skewed distribution, which is typical of daily rainfall and of particle sizes after grinding, mean > median > mode; in a negatively skewed one the order is reversed.
Quartiles Q1 and Q3 cut off the lowest and highest quarters of the data; deciles and percentiles generalise the idea. A box plot draws the median, the quartiles and whiskers, and flags points beyond 1.5 times the interquartile range from the box as possible outliers. Different textbooks and programs use slightly different interpolation rules for quartiles of small samples, so hand values can differ from software output in the second decimal; state the rule you used.
The range is the largest minus the smallest value; it is quick but depends on only two observations. The interquartile range Q3 − Q1 and the quartile deviation (Q3 − Q1)/2 are robust to outliers. The mean absolute deviation is Σ|x − x̄|/n.
The variance is the mean squared deviation. For a population, σ² = Σ(x − μ)²/N. For a sample used to estimate the population variance, s² = Σ(x − x̄)²/(n − 1). The divisor n − 1, the degrees of freedom, is used because deviations are measured from x̄ rather than μ and one degree of freedom is spent estimating x̄; dividing by n would underestimate σ² on average. A convenient computing form is Σ(x − x̄)² = Σx² − (Σx)²/n. The standard deviation, the square root of the variance, is in the units of the data. Adding a constant to every observation leaves the variance unchanged; multiplying every observation by c multiplies the standard deviation by |c| and the variance by c².
The coefficient of variation, CV = (s/x̄)·100 per cent, is dimensionless and compares variability of quantities with different means or units. It is the standard measure of uniformity in machinery testing: the CV of seed spacing from a planter, of discharge between sprayer nozzles, or of metering rate between the outlets of a seed drill. In field experiments the CV of the error term is quoted to judge the precision of the trial. Acceptable CV limits for a given test are set by the relevant national or international test code and differ between crops and machines, so always read the limit from the code being applied rather than from memory.
Skewness measures asymmetry. Pearson's coefficient (mean − mode)/s, or 3(mean − median)/s, and the moment coefficient based on the third central moment are both used. Kurtosis, based on the fourth moment, describes how heavy the tails are relative to the normal distribution, whose moment kurtosis is 3.
A random experiment is one whose outcome cannot be predicted exactly, such as testing a seed for germination or recording whether a day is rainy. The set of all possible outcomes is the sample space S, and an event is a subset of S. Probability P(A) satisfies three axioms: P(A) ≥ 0, P(S) = 1, and for mutually exclusive events P(A ∪ B) = P(A) + P(B). From these follow P(A′) = 1 − P(A) for the complement and the addition rule P(A ∪ B) = P(A) + P(B) − P(A ∩ B) for any two events.
When all outcomes are equally likely, P(A) is the number of favourable outcomes divided by the total, and counting rules are needed. The number of ordered arrangements of r items chosen from n is nPr = n!/(n − r)!; the number of unordered selections is nCr = n!/(r!(n − r)!). Drawing 3 soil samples to analyse from 10 bags is a combination problem; assigning 3 different treatments to 3 of 10 plots in order is a permutation problem.
The conditional probability of A given that B has occurred is P(A | B) = P(A ∩ B)/P(B), provided P(B) > 0. Rearranged, this gives the multiplication rule P(A ∩ B) = P(B)·P(A | B). Events are independent when P(A ∩ B) = P(A)·P(B), which is the same as P(A | B) = P(A). Do not confuse independent with mutually exclusive: two mutually exclusive events with non-zero probabilities cannot be independent, because if one happens the other cannot.
If B1, B2, …, Bk are mutually exclusive and together make up S (a partition), then for any event A the theorem of total probability gives P(A) = Σ P(Bi)·P(A | Bi). Bayes' theorem reverses the conditioning: P(Bj | A) = P(Bj)·P(A | Bj) / Σ P(Bi)·P(A | Bi).
A typical engineering use is tracing a defect to its source. Suppose three seed-processing lines produce 50, 30 and 20 per cent of the output of a plant and their defective rates are 1, 2 and 3 per cent. The overall defective rate is 0.5·0.01 + 0.3·0.02 + 0.2·0.03 = 0.017. If a defective packet is found, the probability that it came from the third line is 0.2·0.03/0.017 ≈ 0.353, although that line makes only a fifth of the output. The same reasoning is used to interpret screening tests: when a condition is rare, even an accurate test produces many false positives, because the prior probability P(Bj) is small.
A random variable X assigns a number to each outcome. It is discrete if it takes countable values (number of breakdowns in a season) and continuous if it takes any value in an interval (time to failure, rainfall depth). A discrete variable has a probability mass function p(x) with Σp(x) = 1; a continuous variable has a probability density function f(x) ≥ 0 with ∫f(x)dx = 1 over its range, and probabilities are areas: P(a ≤ X ≤ b) = ∫ from a to b of f(x)dx. For a continuous variable P(X = a) = 0. The cumulative distribution function F(x) = P(X ≤ x) rises from 0 to 1 and, for a continuous variable, f(x) = dF/dx.
The expected value or mean is E(X) = Σx·p(x) or ∫x·f(x)dx. The variance is Var(X) = E[(X − μ)²] = E(X²) − [E(X)]². Linear rules follow: E(aX + b) = aE(X) + b and Var(aX + b) = a²Var(X). For any two variables E(X + Y) = E(X) + E(Y); for independent variables Var(X ± Y) = Var(X) + Var(Y) — note the plus sign for a difference as well. This is why tolerances stack: the variance of the total length of three independently made parts is the sum of their variances.
Two continuous distributions are worth knowing besides the normal. The uniform distribution on [a, b] has f(x) = 1/(b − a), mean (a + b)/2 and variance (b − a)²/12. The exponential distribution with rate λ has f(x) = λe^(−λx) for x ≥ 0, mean 1/λ, variance 1/λ², and P(X > t) = e^(−λt). It describes the time between random events, such as the time between breakdowns of a machine with a constant failure rate, and it is memoryless: the probability of surviving another t hours does not depend on how long the machine has already run.
A binomial experiment has a fixed number n of independent trials, each with only two outcomes (success or failure) and the same success probability p. The number of successes X then has P(X = x) = nCx·p^x·(1 − p)^(n − x), x = 0, 1, …, n. Its mean is np and its variance np(1 − p), so the variance is always less than the mean. Germination of seeds in a test, the number of defective items in a sample drawn from a large lot, and the number of years out of n in which rainfall exceeds a threshold are all modelled as binomial when the independence and constant-p conditions are reasonable.
The distribution is symmetric when p = 0.5, right-skewed when p < 0.5 and left-skewed when p > 0.5. Cumulative probabilities such as P(X ≤ 2) are sums of individual terms, and 'at least one' questions are quickest through the complement: P(X ≥ 1) = 1 − (1 − p)^n. When sampling without replacement from a small lot the trials are not independent and the hypergeometric distribution is exact; the binomial is an adequate approximation when the sample is a small fraction (commonly taken as under about 5–10 per cent) of the lot.
The Poisson distribution describes the number of events occurring at random in a fixed interval of time, length, area or volume when events occur independently at a constant average rate λ: P(X = x) = e^(−λ)·λ^x/x!, x = 0, 1, 2, …. Its mean and variance are both λ, a property used to check whether count data are Poisson: if the variance of counts is much larger than the mean, the events are clustered (over-dispersed). Examples are the number of weeds per quadrat in a uniformly infested field, the number of breakdowns of a pump in a month, or the number of blemishes per metre of a polyethylene film.
The rate scales with the interval: if a combine averages 0.5 breakdowns per 100 hours, the number in 300 hours is Poisson with λ = 1.5. The Poisson distribution is also the limit of the binomial when n is large and p small with np held fixed; a common rule of thumb uses it when n ≥ 20 and p ≤ 0.05, with λ = np. The sum of independent Poisson variables is Poisson with the rates added.
The normal distribution has density f(x) = (1/(σ√(2π)))·exp(−(x − μ)²/(2σ²)). It is symmetric about μ, bell-shaped, and its mean, median and mode coincide. Its points of inflection are at μ ± σ. Any normal variable is converted to the standard normal Z (mean 0, SD 1) by z = (x − μ)/σ, and probabilities are read from the standard normal table of Φ(z) = P(Z ≤ z).
Values worth memorising: about 68.27 per cent of the area lies within μ ± σ, 95.45 per cent within μ ± 2σ and 99.73 per cent within μ ± 3σ. The central 90, 95 and 99 per cent lie within ±1.645, ±1.960 and ±2.576 standard deviations. One-tailed upper 5 and 1 per cent points are 1.645 and 2.326. Φ(1) ≈ 0.8413, Φ(2) ≈ 0.9772, Φ(1.5) ≈ 0.9332.
The normal distribution matters because many measured quantities are approximately normal and, more importantly, because averages and sums of many independent variables are approximately normal whatever their own distribution (the central limit theorem, Chapter 5). The binomial may be approximated by a normal with μ = np and σ = √(np(1 − p)) when np and n(1 − p) are both at least about 5 (some texts use 10). Because a discrete count is replaced by a continuous curve, a continuity correction of 0.5 is applied: P(X ≤ 40) is approximated by P(Y < 40.5). The Poisson with large λ is similarly approximated by a normal with mean and variance λ.
If ln X is normal, X is log-normal. Log-normal models suit positive, right-skewed quantities such as particle sizes, hydraulic conductivity and flood peaks, and are used again in Chapter 8.
When two quantities are measured on the same units — tractor forward speed and wheel slip, soil moisture and cone index, grain moisture and drying time — the first step is a scatter diagram. It shows whether the relation is roughly linear, whether it is increasing or decreasing, and whether a few points dominate.
The covariance is sxy = Σ(x − x̄)(y − ȳ)/(n − 1). Its sign shows the direction of association, but its size depends on the units. Pearson's product-moment correlation coefficient removes the units: r = Sxy/√(Sxx·Syy), where Sxy = Σ(x − x̄)(y − ȳ) = Σxy − ΣxΣy/n, Sxx = Σ(x − x̄)² and Syy = Σ(y − ȳ)². The value lies between −1 and +1. It is unchanged by adding constants to x or y or by multiplying either by a positive constant (change of origin and scale); multiplying one variable by a negative constant changes only its sign.
Correlation measures only linear association. A perfect parabola symmetric about x̄ gives r = 0 although y is completely determined by x. A high r does not show that x causes y: both may respond to a third variable, as crop yield and irrigation depth both do to season. r is sensitive to outliers and to the range of x sampled.
To test whether a sample correlation shows a real linear association, H0: ρ = 0 is tested with t = r√(n − 2)/√(1 − r²) on n − 2 degrees of freedom. For ranked data or monotonic but non-linear relations Spearman's rank correlation is used: rs = 1 − 6Σd²/(n(n² − 1)), where d is the difference between the ranks of each pair (with no ties).
Regression gives an equation for predicting y from x. The simple linear model is y = β0 + β1x + ε, where ε is random error with mean zero and constant variance. The method of least squares chooses the estimates b0 and b1 that minimise the sum of squared vertical deviations Σ(y − b0 − b1x)². Setting the derivatives to zero gives the normal equations, whose solution is b1 = Sxy/Sxx and b0 = ȳ − b1x̄. The fitted line therefore always passes through (x̄, ȳ), and the residuals e = y − ŷ sum to zero.
There are two regression lines. The line of y on x has slope byx = Sxy/Sxx = r·sy/sx; the line of x on y, used to predict x from y, has slope bxy = Sxy/Syy = r·sx/sy. They intersect at (x̄, ȳ), and byx·bxy = r², so r is the geometric mean of the two regression coefficients and has their common sign. If the two lines coincide, r = ±1; if r = 0 they are perpendicular (parallel to the axes).
The coefficient of determination R² = SSR/SST = 1 − SSE/SST is the fraction of the total variation in y explained by the regression; in simple linear regression R² = r². Here SST = Syy, SSR = b1·Sxy and SSE = SST − SSR. The standard error of estimate se = √(SSE/(n − 2)) is the typical size of a residual in the units of y. The slope is tested with t = b1/(se/√Sxx) on n − 2 degrees of freedom; for simple regression this t test, the correlation t test and the F test of the regression ANOVA (F = t²) are the same test.
Residual plots — residuals against fitted values and against x — are the main check on the model. A curved pattern shows that a straight line is the wrong form; a funnel shape shows that the variance is not constant; one very large residual may be a recording error or a real but unusual event.
Many engineering relations are linear after a transformation. A power law y = a·x^b becomes ln y = ln a + b·ln x; an exponential decay y = a·e^(−kx), as in thin-layer drying of grain, becomes ln y = ln a − kx. The regression is then done on the transformed variables and the constants recovered by taking antilogs. The least-squares criterion then applies to the transformed scale, so the back-transformed curve is not the least-squares fit on the original scale.
Multiple linear regression extends the model to several predictors, y = β0 + β1x1 + … + βkxk + ε, fitted by least squares in matrix form b = (XᵀX)⁻¹Xᵀy. Adjusted R² penalises extra predictors, because ordinary R² never decreases when a variable is added. Strongly correlated predictors (multicollinearity) make individual coefficients unstable even when the fit is good. Prediction outside the range of the observed x values (extrapolation) is unreliable, because nothing in the data shows that the relation continues. Calibration curves for sensors, draft equations for implements, and yield-response curves are all regression models and carry these same cautions.
It is rarely possible to measure a whole population: every kernel in a silo, every emitter in an orchard, every bag from a season's production. A sample is taken instead, and its statistics are used to estimate the population parameters. A sample is only useful if it is representative, which in practice means it was chosen by a probability method so that sampling error can be calculated.
In simple random sampling every possible sample of size n has the same chance of selection, achieved with random numbers. Systematic sampling takes every k-th unit after a random start; it is easy in the field (every 10th plant along a row, every 20th bag on a conveyor) but can be badly biased if the population has a periodic pattern matching k. Stratified sampling divides the population into internally similar groups (strata) such as soil types or fields, and samples each; it gives more precise estimates than simple random sampling of the same size when the strata differ. Cluster sampling selects whole groups (villages, fields) and measures all or some units within them; it is cheaper but usually less precise per unit measured. Multistage sampling combines these and is the basis of most national crop-cutting surveys. Grain sampling from bins, trucks and flowing streams follows official probe and diverter-sampler procedures, which differ by country and grain standard.
Sampling error is the chance difference between a statistic and the parameter; it shrinks as n grows. Non-sampling errors — bias in selection, faulty instruments, recording mistakes, non-response — do not shrink with n, and a large biased sample is still biased.
A statistic varies from sample to sample, and its probability distribution is its sampling distribution. For the sample mean of n independent observations from a population with mean μ and SD σ, E(x̄) = μ and the standard deviation of x̄, called the standard error, is σ/√n. Quadrupling the sample size halves the standard error. When sampling without replacement from a finite population of N units, the standard error is multiplied by the finite population correction √((N − n)/(N − 1)), which matters only when n is a noticeable fraction of N (a common rule is more than 5 per cent).
If the population is normal, x̄ is exactly normal for any n. The central limit theorem states that, whatever the shape of the population (provided its variance is finite), the distribution of x̄ approaches the normal as n increases. In practice n of about 30 is often enough for moderately skewed populations; strongly skewed data need more. This is why normal-based methods work for averages of rainfall, of yields or of machine outputs even when individual values are skewed.
A sample proportion p̂ = x/n has mean p and standard error √(p(1 − p)/n), and it is approximately normal when np and n(1 − p) are both reasonably large. The difference between two independent sample means has standard error √(σ1²/n1 + σ2²/n2).
When σ is unknown and estimated by s, the quantity (x̄ − μ)/(s/√n) follows Student's t distribution with n − 1 degrees of freedom if the population is normal. The t distribution is symmetric about zero like Z but has heavier tails; its critical values are larger than z values for small samples and approach them as the degrees of freedom increase (for example, the two-tailed 5 per cent point is 2.262 for 9 df, 2.042 for 30 df and tends to 1.96).
If s² is the variance of a sample of n from a normal population, (n − 1)s²/σ² follows a chi-square distribution with n − 1 degrees of freedom. Chi-square is non-negative and right-skewed, with mean equal to its degrees of freedom. The ratio of two independent sample variances, each divided by its population variance, follows the F distribution with (n1 − 1, n2 − 1) degrees of freedom; F is also non-negative and right-skewed. These three distributions supply the critical values for the tests of Chapter 6 and the analysis of variance of Chapter 7.
A point estimate is a single value (x̄ for μ). An estimator is unbiased if its expected value equals the parameter (x̄ and s² with divisor n − 1 are unbiased), and more efficient than another if its variance is smaller. An interval estimate gives a range that contains the parameter with stated confidence. The 100(1 − α) per cent confidence interval for μ is x̄ ± z(α/2)·σ/√n when σ is known, and x̄ ± t(α/2, n − 1)·s/√n when it is estimated from a small normal sample. A 95 per cent confidence interval means that intervals built this way contain μ in 95 per cent of repeated samples; any one interval either contains μ or does not.
The sample size needed to estimate a mean within a margin of error E with confidence 1 − α is n = (z(α/2)·σ/E)², rounded up. For a proportion, n = z²·p(1 − p)/E², and using p = 0.5 gives the largest (safest) n when nothing is known about p. Halving the margin of error requires four times the sample.
A test of hypothesis decides between a null hypothesis H0, usually a statement of no effect or no difference (the new metering mechanism gives the same mean seed rate as the old), and an alternative H1. The alternative is two-sided (μ ≠ μ0) when a difference in either direction matters, and one-sided (μ > μ0 or μ < μ0) when only one direction is of interest and this is decided before looking at the data.
A test statistic is computed from the sample and compared with the critical value that cuts off a tail area α, the level of significance, under H0. If the statistic falls in the rejection region, H0 is rejected. Rejecting a true H0 is a Type I error with probability α. Failing to reject a false H0 is a Type II error with probability β, and the power of the test is 1 − β. For a fixed sample size, lowering α raises β; the only way to reduce both is to increase n or reduce the error variance.
The p-value is the probability, computed under H0, of a test statistic at least as extreme as the one observed. H0 is rejected at level α when p ≤ α. A small p-value says the data are unlikely under H0; it is not the probability that H0 is true, and statistical significance is not the same as practical importance: with a very large sample a trivial difference can be highly significant. Report the estimated effect and its confidence interval alongside the test.
A two-sided test at level α and a 100(1 − α) per cent confidence interval agree: H0: μ = μ0 is rejected exactly when μ0 lies outside the interval.
For a single mean with σ known (or n large), z = (x̄ − μ0)/(σ/√n). With σ unknown and a small sample from a normal population, t = (x̄ − μ0)/(s/√n) on n − 1 degrees of freedom.
For two independent means with equal population variances, the pooled variance is sp² = ((n1 − 1)s1² + (n2 − 1)s2²)/(n1 + n2 − 2) and t = (x̄1 − x̄2)/(sp·√(1/n1 + 1/n2)) on n1 + n2 − 2 degrees of freedom. When the variances clearly differ, Welch's test uses √(s1²/n1 + s2²/n2) with approximate degrees of freedom; most software reports it by default. With large samples the same statistic is referred to the normal table.
When the same units are measured twice — the same ten plots with and without a subsoiler, the same operators on two seat designs, the same tractor with two tyre inflation pressures — the observations are paired. The paired t test works on the differences d: t = d̄/(sd/√n) with n − 1 degrees of freedom. Pairing removes the unit-to-unit variation and usually gives a far more sensitive test than treating the two sets as independent, which is a frequent examination trap.
For proportions with large samples, z = (p̂ − p0)/√(p0(1 − p0)/n) for one proportion, and for two, z = (p̂1 − p̂2)/√(p̂(1 − p̂)(1/n1 + 1/n2)) using the pooled proportion p̂.
The chi-square goodness-of-fit test compares observed frequencies O with frequencies E expected under a hypothesised distribution: χ² = Σ(O − E)²/E, with degrees of freedom equal to the number of classes minus one minus the number of parameters estimated from the data. It is used to check, for example, whether seed germination counts follow a stated ratio, or whether annual rainfall fits a normal distribution. Expected frequencies should not be too small; a common rule is at least 5 in each class, merging neighbouring classes where necessary.
The chi-square test of independence uses an r × c contingency table: E for each cell = (row total × column total)/grand total, and df = (r − 1)(c − 1). It tests whether two classifications — say, tractor age group and whether a hydraulic fault occurred — are associated. For a 2 × 2 table with small expected frequencies, Yates' continuity correction or an exact test is used. The chi-square test of a single variance uses χ² = (n − 1)s²/σ0² on n − 1 df.
The F test compares two variances from independent normal samples: F = s1²/s2², conventionally with the larger variance in the numerator so that only the upper tail of the F table is needed, with (n1 − 1, n2 − 1) degrees of freedom. Placing the larger variance on top makes a two-sided test at level α use the upper α/2 point. It is used to compare the precision of two instruments or the uniformity of two machines, and to check the equal-variance assumption of the pooled t test. The F test is sensitive to non-normality. Its main use, the analysis of variance, follows in Chapter 7.
An experiment applies treatments (the conditions being compared — tillage methods, drying temperatures, nozzle types, fertiliser doses) to experimental units (plots, batches, animals, machine runs) and measures a response. The variation among units treated alike is experimental error. R.A. Fisher set out three basic principles. Replication — applying each treatment to more than one unit — provides an estimate of error and reduces the standard error of treatment means, which is σ/√r for r replications. Randomisation — allotting treatments to units by a random process — protects against systematic bias and makes the error estimate valid. Local control — grouping similar units into blocks and comparing treatments within blocks — removes known sources of variation from the error.
The analysis of variance (ANOVA) partitions the total sum of squares of the observations into parts attributable to each source of variation. Each sum of squares divided by its degrees of freedom is a mean square; the ratio of the treatment mean square to the error mean square is an F statistic, compared with the F table at (treatment df, error df). ANOVA assumes that effects are additive, that errors are independent, normally distributed and of equal variance across treatments. Data such as counts and percentages often need a transformation (square root, logarithm, or angular) before analysis to meet these assumptions.
In a completely randomised design (CRD) the t treatments are allotted to the experimental units entirely at random, with r replications each (unequal replication is allowed). It is suited to homogeneous material: laboratory and greenhouse trials, drying and storage experiments, and machine tests on uniform samples. It is the simplest and most flexible design and leaves the maximum degrees of freedom for error, but if the units are not uniform all their variation goes into the error and the design is inefficient.
With N = rt observations, grand total G and treatment totals Ti, the correction factor is CF = G²/N. Total SS = Σy² − CF; treatment SS = ΣTi²/r − CF; error SS = total SS − treatment SS. Degrees of freedom are N − 1 in total, t − 1 for treatments and N − t = t(r − 1) for error. F = treatment MS/error MS. If F exceeds the tabled value, at least one treatment mean differs from another, and the means are then compared, for example by the least significant difference LSD = t(α/2, error df)·√(2·MSE/r). The coefficient of variation of the experiment is √MSE divided by the grand mean, times 100.
When the experimental area or material has a known gradient — a slope, a fertility trend, batches of raw material processed on different days, different operators — the units are grouped into r blocks of t units each, as uniform as possible within a block, and every treatment appears once in every block, randomised afresh in each. This is the randomised block design (RBD), also called the randomised complete block design, and it is the most widely used field design. Blocks should be laid out with their long side across the fertility gradient, so that plots within a block are alike.
The ANOVA adds a block line. Block SS = ΣBj²/t − CF, where Bj are the block totals; error SS = total SS − treatment SS − block SS. Degrees of freedom: blocks r − 1, treatments t − 1, error (r − 1)(t − 1), total rt − 1. Blocking removes the between-block variation from the error, so if the blocks really differ the error mean square falls and the test becomes more sensitive, at the cost of r − 1 error degrees of freedom. If the blocks turn out to be alike, little is lost. A missing plot in an RBD can be estimated by the formula y = (rB + tT − G)/((r − 1)(t − 1)), where B, T and G are the totals of the block, treatment and whole experiment excluding the missing value; one error degree of freedom is then subtracted.
The Latin square design controls two sources of variation (rows and columns) at once with t × t units and error df (t − 1)(t − 2). Factorial experiments test two or more factors together and estimate their interaction; they can be laid out in a CRD or RBD, and split-plot designs are used when one factor needs large plots (such as a tillage implement) and another can be applied to small ones.
Hydrological design — the height of a farm pond spillway, the capacity of a drainage channel, the size of a culvert under a field road, the depth of a contour bund — needs the magnitude of an event with a stated chance of being exceeded. Frequency analysis estimates it from a record of past events, usually the annual maximum series (the largest value in each year). A partial duration series, which takes all events above a threshold, is used for short records and frequent events.
The return period or recurrence interval T of an event of magnitude x is the average interval in years between exceedances; its annual exceedance probability is P = 1/T. A 50-year flood has a 2 per cent chance of being equalled or exceeded in any one year, not a guarantee of occurring once in 50 years. The probability that it is not exceeded in a year is q = 1 − P. Over a design life of n years, assuming independent years, the probability of at least one exceedance — the risk — is R = 1 − (1 − 1/T)^n, and the probability of exactly k exceedances is binomial: nCk·P^k·q^(n − k). A structure designed for the 100-year flood with a 50-year life has a risk of about 39.5 per cent. To limit risk to R over n years, the required return period is T = 1/(1 − (1 − R)^(1/n)).
To see how a record fits a distribution, the annual maxima are ranked in descending order (m = 1 for the largest) and each is given an empirical exceedance probability by a plotting-position formula. The Weibull formula P = m/(n + 1), so T = (n + 1)/m, is the most widely used in hydrology texts; others (California m/n, Hazen (2m − 1)/(2n), Gringorten) exist and give slightly different values, especially at the extremes. The values are plotted on probability paper constructed so that the chosen distribution plots as a straight line; the fitted line is then read or extrapolated to the design return period. Extrapolating far beyond the record length is uncertain, and many agencies limit it.
Chow's general equation expresses the T-year magnitude as xT = x̄ + K·s, where x̄ and s are the mean and standard deviation of the annual series and K is a frequency factor that depends on the distribution and T.
For the normal distribution K is the standard normal deviate for exceedance probability 1/T (K = 0 for T = 2 years; 0.842 for 5; 1.282 for 10; 1.645 for 20; 2.054 for 50; 2.326 for 100). Annual rainfall totals at many stations are close to normal. Annual maximum rainfall and floods are usually right-skewed, so skewed distributions are used.
The Gumbel (extreme value type I) distribution is widely used for annual maxima of rainfall and floods. For a long record its frequency factor is K = −(√6/π)·[0.5772 + ln(ln(T/(T − 1)))]. This gives K ≈ −0.164 for T = 2, 1.305 for T = 10, 2.044 for T = 25, 2.592 for T = 50 and 3.137 for T = 100. For short records the reduced variate is standardised using the mean ȳn and standard deviation Sn of the reduced variate for sample size n, taken from Gumbel's table: K = (yT − ȳn)/Sn, with yT = −ln(ln(T/(T − 1))). The Gumbel distribution has a fixed skewness of about 1.14.
In the log-normal method the logarithms of the data are analysed with the normal frequency factor, yT = ȳ + z·sy, and xT is the antilog. The log-Pearson type III method does the same with logarithms but takes K from a table that depends on the skew coefficient of the logarithms as well as T; with zero skew it reduces to the log-normal. Log-Pearson III is the method prescribed for flood-flow frequency by United States federal guidelines; other countries adopt other distributions, such as the generalised extreme value or Gumbel, so use the method your national guideline names.
Intensity–duration–frequency relations, which give the rainfall intensity for a duration and return period used in the rational method, are built by repeating the frequency analysis for annual maxima of each duration. The resulting empirical formulas and their constants are specific to a station or region.
Every process varies. Common-cause (chance) variation is the steady background produced by many small factors; assignable (special) causes are identifiable events — a worn cutter, a change of raw material batch, a new operator — that shift or widen the output. A process operating with only common-cause variation is in statistical control. Walter Shewhart's control chart plots a statistic from small samples (subgroups) taken at regular intervals against a centre line and upper and lower control limits placed three standard errors of the statistic from the centre line. With normal data a point falls outside three-sigma limits by chance only about 0.27 per cent of the time, so a point outside the limits is taken as a signal to look for an assignable cause.
Besides points outside the limits, non-random patterns signal trouble: a run of several consecutive points on one side of the centre line, a steady trend up or down, or cycles. Different texts and standards use slightly different run rules (for example seven, eight or nine points in a row on one side), so apply the rule set your organisation or standard specifies.
Control limits describe what the process does; specification limits describe what the customer requires. They are different things and must never be drawn as if interchangeable. A process can be in control and still produce out-of-specification items if its natural spread is wider than the tolerance.
For a measured characteristic — fill weight of bags, moisture after drying, bearing bore diameter — subgroups of n items (commonly 4 or 5) are taken, and each subgroup's mean x̄ and range R are plotted. The centre lines are the grand mean x̿ and the mean range R̄. The limits use constants tabulated by subgroup size: for the x̄ chart, x̿ ± A2·R̄; for the R chart, LCL = D3·R̄ and UCL = D4·R̄. For n = 5 the constants are A2 = 0.577, D3 = 0 and D4 = 2.114; for n = 4 they are A2 = 0.729, D3 = 0 and D4 = 2.282. The process standard deviation is estimated as σ̂ = R̄/d2, with d2 = 2.326 for n = 5 and 2.059 for n = 4. The R chart should be judged first: if the spread is not in control, the x̄ chart limits, which depend on R̄, are not meaningful. For larger subgroups the x̄ and s chart replaces the range with the standard deviation.
Process capability compares the natural spread of an in-control process with the specification width. Cp = (USL − LSL)/(6σ) measures potential capability assuming the process is centred; Cpk = min(USL − μ, μ − LSL)/(3σ) also accounts for centring. Cp = 1 means the ±3σ spread just fills the tolerance; higher is better, and customers and industries set their own minimum values.
When items are simply classed as conforming or non-conforming — cracked kernels in a sample, leaking bottles, bags under weight — the p chart plots the fraction defective in each subgroup of size n. Its centre line is p̄, the overall fraction defective, and its limits are p̄ ± 3√(p̄(1 − p̄)/n), with a negative lower limit set to zero. The np chart plots the number defective, with limits np̄ ± 3√(np̄(1 − p̄)); it is easier to read when n is constant. When subgroup sizes vary, p-chart limits are computed for each subgroup.
The c chart plots the number of defects (non-conformities) per unit of constant size — blemishes per square metre of film, weld defects per frame, weeds per quadrat in a quality check of a weeder — using the Poisson model, with centre line c̄ and limits c̄ ± 3√c̄. The u chart is its counterpart for units of varying size, plotting defects per unit.
Acceptance sampling is related but different: a sample is drawn from a lot and the lot is accepted if the number of defectives is no more than an acceptance number. The probability of acceptance for each true fraction defective, plotted as the operating characteristic (OC) curve, is calculated from the binomial or Poisson distribution. Acceptance sampling judges lots; control charts manage the process that makes them.
Most errors in engineering data analysis happen before any statistics is done. Keep raw data in a tidy layout: one row per observation, one column per variable, one header row with names and units, and no merged cells or colour-coded meaning. Record the date, location, instrument and operator as columns rather than in notes. Keep the raw file unchanged and do all cleaning in a separate, documented step so that every result can be traced back. Missing values should be left blank or coded consistently, never entered as zero, which is a real value.
Check the data before analysing them: sort each column to see the extremes, plot histograms and scatter diagrams, and compare summary statistics with physical limits (moisture content cannot exceed 100 per cent on the wet basis; wheel slip is not negative in a drawbar test). Unit conversions and date formats are common sources of silent errors when files move between programs and countries (decimal comma versus decimal point, day-month versus month-day order).
Spreadsheets are the everyday tool for small data sets and field calculations. Most spreadsheet programs offer the same statistical functions under similar names: the mean (AVERAGE), median (MEDIAN), sample and population standard deviation (typically STDEV.S or STDEV, and STDEV.P), variance, correlation (CORREL), regression slope and intercept (SLOPE, INTERCEPT, and LINEST for full output), and the normal, t, chi-square and F distributions with their inverses for p-values and critical values. Always check whether a function returns the sample or the population version and whether a distribution function returns a one-tailed or two-tailed probability, as names and conventions differ between programs and versions.
Absolute references ($A$1) fix a cell when a formula is copied; relative references move with it; getting this wrong is a classic source of error in calibration sheets. Trendlines on charts give a quick regression, but the equation shown is rounded and should not be copied for design use; compute coefficients in cells instead. Spreadsheets are hard to audit when formulas are hidden in many cells, and very large or repeated analyses are better done with a script.
A script — in a language such as Python with its scientific libraries (NumPy for arrays, SciPy.stats for distributions and tests, pandas for tables, statsmodels for regression and ANOVA) or R, which was built for statistics — records every step of an analysis as text. Anyone can rerun it on new data and get the same result, which is the meaning of reproducible analysis. A script can read hundreds of data-logger files, clean them by the same rules, compute statistics and draw plots in one run, which a spreadsheet cannot do reliably.
Good practice is to keep data and code separate, to comment the purpose of each step, to fix the random seed when simulating so results can be repeated, and to check the script against a small example worked by hand. When reporting, state the method (for example 'two-sample t test with pooled variance'), the test statistic, degrees of freedom and p-value, and the estimated effect with its confidence interval.
Simulation (the Monte Carlo method) uses a script to draw many random samples from assumed distributions and so to estimate probabilities that are hard to derive: the chance that a reservoir runs dry in a season, the distribution of total harvesting time given random breakdowns, or the reliability of a system of pumps. Bootstrap resampling, which repeatedly resamples the observed data with replacement, gives standard errors and confidence intervals without assuming a distribution.
Modern agricultural machines, sensors and weather stations produce large, continuous data streams: yield-monitor data, soil-sensor logs, telemetry. The same principles apply at scale — clean first, plot before modelling, separate data from code, and never trust a result that cannot be reproduced.
A student uses this subject every week of the degree. Laboratory reports need the mean, standard deviation and CV of repeated readings, and an honest statement of uncertainty. Machinery test exercises ask whether a new metering plate is more uniform than the old one, which is a CV comparison and an F or t test. Hydrology assignments ask for a design rainfall from a station record using Weibull plotting and a Gumbel fit. Final-year projects are usually small designed experiments, and choosing CRD or RBD, the number of replications and the right ANOVA decides whether the project can show anything at all. For GATE and ICAR PG, fast and accurate use of the probability rules and the three distributions is the core skill.
Five years into practice, an engineer in a testing station, processing plant or irrigation agency uses statistics to make and defend decisions. Examples: setting up x̄ and R charts on a packing line and deciding when a shift in fill weight needs the machine stopped; checking emitter discharge uniformity in a drip system against the design figure; fitting a calibration regression for a moisture meter and stating its standard error; choosing the design rainfall for a farm pond spillway at an acceptable risk over its life; reporting a field trial to a client with the effect size and confidence interval rather than a bare 'significant'. The common failure at this stage is treating paired data as independent, or using a trendline equation outside the range of the data.
With twenty-five years of experience, the engineer is setting the sampling plans, test protocols and design standards others follow, and reviewing their analyses. The statistical questions become ones of judgement: whether a short rainfall record can support a 100-year estimate, how much extrapolation a regulator will accept, whether a machine that passed a test on ten runs will perform the same across a region, what acceptance sampling plan balances the producer's and consumer's risk for a procurement contract, and how to design a multi-location trial so that the conclusions travel. The senior engineer also has to explain uncertainty to managers and farmers in plain words and to resist decisions built on a single significant p-value.
A researcher with long field experience uses this subject as the base for advanced methods: mixed models for multi-location and repeated-measures trials, response-surface designs for process optimisation, non-stationary frequency analysis where climate change breaks the assumption that past annual maxima represent the future, spatial statistics for precision agriculture, and machine-learning models on sensor and yield-monitor data. The core ideas remain those of this course — sampling error, replication and randomisation, the difference between correlation and cause, the danger of extrapolation — and the researcher's job is to make sure a sophisticated model does not hide a basic failure of design. Reproducible scripts and open data are now expected by journals and funders.
This manual covers the ten units of the course at undergraduate engineering depth. It does not cover: exact small-sample tables for Gumbel's reduced mean and standard deviation (yn, Sn), the log-Pearson III frequency-factor table, or tables of control-chart constants beyond subgroup sizes 4 and 5 — read these from the table supplied in the exam or guideline; non-parametric tests beyond Spearman's rank correlation (Mann–Whitney, Wilcoxon, Kruskal–Wallis); multiple-comparison procedures other than the LSD (Duncan, Tukey, Scheffé); the full analysis of factorial, split-plot and Latin square designs; mixed models, time series, geostatistics and machine learning; and regional flood-frequency methods and non-stationary analysis. Acceptable CV limits for machinery tests and national design-flood rules are not quoted because they differ by code and country.
Original practice questions written to the syllabus and pattern of the exams named in each question's examTags. They are not past papers and do not reproduce any board's questions. Every numerical answer and every numerical distractor was computed with a script (Python with NumPy and SciPy) before it was written; where a decision needs a critical value, the value is given in the question.
These notes are free to read here, with no account.
What the Laisha app adds:
Free for ten days, then a subscription. We would rather write that here than have you find it later.