Agricultural Statistics & Data Science · Statistical theory core · also Probability Theory (MSc core), Measure-theoretic probability (PhD)
Probability from its axioms to the limit theorems: random variables, expectation, generating functions, inequalities and modes of convergence, ending with the laws of large numbers and the central limit theorem that justify most of applied statistics.
In the exams: AICE-JRF Ag Stat Unit 1 (heavy); GATE ST 'Probability' and limit theorems (heavy); IIT JAM MS (heavy); UPSC ISS Statistics I (heavy); ISI admission (heavy); CSIR-NET Mathematical Sciences Unit 4 (present)
Probability theory is the mathematics of uncertainty. It starts from three axioms laid down by A. N. Kolmogorov in 1933 and builds, step by step, the objects every statistician uses daily: events and their probabilities, random variables and their distributions, expectations and moments, generating functions, and finally the limit theorems that explain why averages of many noisy measurements behave so predictably. Without this subject, estimation, testing, design of experiments and sample surveys are recipes without reasons.
Agriculture is full of chance. Whether a seed germinates, how many aphids sit on a leaf, what a plot yields, whether a diagnostic test on a seed lot comes back positive, how long a pump runs before it fails: each is a random outcome. The agricultural statistician turns these into random variables, asks what their distributions and moments are, and uses the law of large numbers and the central limit theorem to justify the standard errors and confidence intervals printed in every trial report and crop-estimation bulletin.
This manual follows the order of a first-year master's course in agricultural statistics or mathematical statistics: axioms and sigma-fields, conditional probability and Bayes' theorem, random variables, expectation and generating functions, joint distributions and conditional expectation, transformations, inequalities, modes of convergence, laws of large numbers, central limit theorems with the delta method, and a closing chapter on the measure theory that underlies everything. Standard distributions themselves are developed in the companion subject, Probability Distributions and Sampling Distributions.
Worldwide the subject is examined in the same way: short derivations and multi-step numericals in admission tests such as GATE Statistics, IIT JAM Mathematical Statistics, the ISI tests, CUET-PG and the Chinese postgraduate statistics paper; theorem recognition and application in the ICAR AICE-JRF/SRF Agricultural Statistics paper (Unit 1) and CSIR-NET; and both objective and derivation-style questions in the Indian Statistical Service and state statistical services. Master's and doctoral programmes from Ludhiana and New Delhi to Davis, Wageningen, Piracicaba and Los Baños teach the same core.
A random experiment is any procedure whose outcome cannot be predicted with certainty but whose possible outcomes can be listed or described: sowing a seed and recording whether it germinates, counting the aphids on a leaf, weighing the grain harvested from a plot. The set of all possible outcomes is the sample space, written Ω. It may be finite (germinated or not), countably infinite (number of aphids: 0, 1, 2, ...) or uncountable (plot yield, any non-negative real number).
An event is a subset of Ω to which we wish to assign a probability. Events are combined with set operations: the union A ∪ B (A or B occurs), the intersection A ∩ B (both occur), the complement Aᶜ (A does not occur) and the difference A − B. Two events are mutually exclusive, or disjoint, when A ∩ B is empty. De Morgan's laws, (A ∪ B)ᶜ = Aᶜ ∩ Bᶜ and (A ∩ B)ᶜ = Aᶜ ∪ Bᶜ, hold for any finite or countable collection and are used constantly when a probability is easier to compute for the complement.
For a finite Ω with equally likely outcomes the classical definition applies: P(A) is the number of outcomes in A divided by the number in Ω. Counting then rests on permutations, n!/(n − r)! ordered selections of r from n, and combinations, C(n, r) = n!/(r!(n − r)!) unordered selections. The classical definition fails when outcomes are not equally likely or Ω is infinite; the relative-frequency view (P(A) as the long-run proportion of repetitions in which A occurs) and the subjective view (P(A) as a coherent degree of belief, the basis of Bayesian statistics) are interpretations, but neither is a mathematical definition. The axiomatic approach supplies the definition and leaves interpretation to the user.
It might seem natural to assign a probability to every subset of Ω. For finite or countable Ω this works, and the power set (the collection of all subsets) is used. For Ω = ℝ it does not: there is no countably additive measure, defined on all subsets of the real line, that agrees with length on intervals and is unchanged by translation (the Vitali construction shows this). So probability is defined on a suitable collection of subsets called a sigma-field (σ-field, or σ-algebra).
A collection 𝓕 of subsets of Ω is a σ-field if (i) Ω belongs to 𝓕; (ii) whenever A belongs to 𝓕, so does Aᶜ; and (iii) whenever A₁, A₂, ... belong to 𝓕, so does their countable union. It follows that the empty set and countable intersections also belong to 𝓕. A collection closed only under complements and finite unions is a field (an algebra); every σ-field is a field but not conversely.
Examples: the trivial σ-field {∅, Ω}; the power set; and, for a single event A, the σ-field generated by A, which is {∅, A, Aᶜ, Ω}, the smallest σ-field containing A. In general the σ-field generated by a collection 𝓒 is the intersection of all σ-fields containing 𝓒. The Borel σ-field on ℝ is the σ-field generated by the open intervals; it equally is generated by the half-lines (−∞, x], which is why a distribution function determines a probability on all Borel sets. The pair (Ω, 𝓕) is a measurable space, and the triple (Ω, 𝓕, P) is a probability space. In exam questions on σ-fields, check the three conditions directly and remember that a union of two σ-fields need not be a σ-field, while an intersection always is.
A probability P is a function on 𝓕 satisfying three axioms: (1) P(A) ≥ 0 for every event A; (2) P(Ω) = 1; (3) countable additivity: if A₁, A₂, ... are pairwise disjoint, P(∪Aᵢ) = ΣP(Aᵢ).
Everything else is derived. P(∅) = 0; P(Aᶜ) = 1 − P(A); if A ⊂ B then P(A) ≤ P(B); P(A) ≤ 1. The addition rule P(A ∪ B) = P(A) + P(B) − P(A ∩ B) is a theorem, not an axiom. Its extension, the inclusion-exclusion formula, for three events reads P(A ∪ B ∪ C) = ΣP(A) − ΣP(A ∩ B) + P(A ∩ B ∩ C). Boole's inequality P(∪Aᵢ) ≤ ΣP(Aᵢ) gives an upper bound without knowing intersections, and Bonferroni's inequality P(A ∩ B) ≥ P(A) + P(B) − 1 gives a lower bound for a joint probability; the latter is the basis of the Bonferroni correction for multiple comparisons in field trials.
Countable additivity, rather than mere finite additivity, is what makes probability behave well under limits. It is equivalent (given finite additivity) to continuity: if A₁ ⊂ A₂ ⊂ ... increase to A, then P(Aₙ) → P(A); if A₁ ⊃ A₂ ⊃ ... decrease to A, then P(Aₙ) → P(A). These continuity properties are used to prove that a distribution function is right-continuous with limits 0 and 1, and they are behind the Borel-Cantelli lemmas of Chapter 8.
Two cautions. An event of probability zero need not be impossible: for a continuous yield variable, the event that yield equals exactly 3.000 t/ha has probability zero yet can occur. And probability one does not mean certainty; it means 'almost surely'.
When we learn that an event B has occurred, the relevant sample space shrinks to B. The conditional probability of A given B is P(A | B) = P(A ∩ B)/P(B), defined when P(B) > 0. For fixed B, P(· | B) is itself a probability satisfying all three axioms, so every rule of Chapter 1 applies to it.
Rearranged, the definition gives the multiplication rule P(A ∩ B) = P(B) P(A | B) = P(A) P(B | A), and for several events P(A₁ ∩ A₂ ∩ ... ∩ Aₙ) = P(A₁) P(A₂ | A₁) P(A₃ | A₁ ∩ A₂) ... This is how sequential sampling without replacement is handled. If a bag holds 4 healthy and 2 diseased tubers and two are drawn without replacement, the probability that both are diseased is (2/6)(1/5) = 1/15; drawing with replacement would give (2/6)² = 1/9.
The law of total probability follows. If B₁, ..., Bₖ partition Ω (disjoint, union Ω, each with positive probability), then P(A) = Σ P(Bᵢ) P(A | Bᵢ). A seed company buying from three growers who supply 50%, 30% and 20% of its stock, with germination-failure rates of 2%, 3% and 5%, has an overall failure rate of 0.5(0.02) + 0.3(0.03) + 0.2(0.05) = 0.029. The partition is chosen so that the conditional probabilities are easy; that choice is the main skill in multi-stage problems.
Bayes' theorem reverses the conditioning: P(Bⱼ | A) = P(Bⱼ) P(A | Bⱼ) / Σ P(Bᵢ) P(A | Bᵢ). The P(Bⱼ) are prior probabilities, the P(A | Bⱼ) likelihoods and the P(Bⱼ | A) posterior probabilities. In the seed example, a failed seed came from the third grower with probability 0.010/0.029 ≈ 0.345, although that grower supplies only 20% of the stock.
Diagnostic testing is the classic application. The sensitivity of a test is P(positive | infected) and its specificity P(negative | not infected). The quantity a plant-health inspector actually needs is the positive predictive value P(infected | positive). With prevalence 2%, sensitivity 0.95 and specificity 0.90, PPV = 0.02 × 0.95 / (0.02 × 0.95 + 0.98 × 0.10) = 0.019/0.117 ≈ 0.162. Most positives are false positives because the condition is rare; confusing PPV with sensitivity is the commonest error in this topic, in examinations and in practice.
Bayes' theorem can be applied sequentially: today's posterior is tomorrow's prior. If several pieces of evidence are conditionally independent given the hypothesis, their likelihoods multiply. This is the logic of Bayesian inference, which treats unknown parameters as random variables with prior distributions, and it underlies modern disease-surveillance and crop-forecasting models.
Events A and B are independent if P(A ∩ B) = P(A) P(B); when P(B) > 0 this is the same as P(A | B) = P(A). Independence is a property of the probability measure, not of the sets: it is not the same as being disjoint. Two disjoint events, each with positive probability, are never independent, because P(A ∩ B) = 0 while P(A) P(B) > 0. If A and B are independent, so are A and Bᶜ, Aᶜ and B, and Aᶜ and Bᶜ.
For more than two events there are two notions. A₁, ..., Aₙ are pairwise independent if every pair satisfies the product rule, and mutually independent if the product rule holds for every sub-collection: P(Aᵢ ∩ Aⱼ ∩ Aₖ) = P(Aᵢ)P(Aⱼ)P(Aₖ) and so on, 2ⁿ − n − 1 conditions in all. Pairwise independence does not imply mutual independence. Bernstein's example: toss two fair coins; let A = first is a head, B = second is a head, C = the two tosses agree. Each has probability 1/2 and each pair intersects with probability 1/4, so they are pairwise independent; but A ∩ B ∩ C is 'both heads', with probability 1/4, not 1/8.
Independence makes 'at least one' problems simple: the probability that at least one of n independent events occurs is 1 − Π(1 − P(Aᵢ)). If each of four independent pheromone traps detects a pest that is present with probability 0.6, the probability that at least one detects it is 1 − 0.4⁴ = 0.9744. Conditional independence (independence under P(· | C)) neither implies nor is implied by unconditional independence, a point that matters when observations share a common cause such as a field or a season.
A random variable is a function X from Ω to the real line that is measurable: for every Borel set B, the set {ω : X(ω) ∈ B} belongs to the σ-field 𝓕. Measurability ensures that statements such as 'yield exceeds 5 t/ha' are events with probabilities. It suffices to check that {X ≤ x} belongs to 𝓕 for every real x, because the half-lines generate the Borel σ-field. A random vector (X₁, ..., Xₖ) is a measurable map into ℝᵏ.
The distribution, or law, of X is the probability P_X(B) = P(X ∈ B) on the Borel sets. Different random variables on different spaces can share a distribution; most of statistics depends on X only through its distribution. Sums, products, continuous functions, maxima and pointwise limits of random variables are again random variables.
An indicator variable 1_A, equal to 1 when A occurs and 0 otherwise, is the simplest random variable, and E(1_A) = P(A). Counting the germinated seeds in a tray is a sum of indicators, one per seed, which is why expectations of counts are so easy to find: the expected count is the sum of the individual probabilities, whether or not the seeds behave independently. Measurability is never an issue for variables built from ordinary formulas; it becomes important only in proofs and in the construction of exotic counterexamples.
The cumulative distribution function (CDF) of X is F(x) = P(X ≤ x). Every CDF has four properties: it is non-decreasing; it is right-continuous, F(x) = lim F(x + h) as h decreases to 0; F(x) → 0 as x → −∞; and F(x) → 1 as x → +∞. Conversely, every function with these four properties is the CDF of some random variable. Right-continuity follows from the convention P(X ≤ x); some older texts define F(x) = P(X < x), which makes F left-continuous, so read each book's convention.
From F one computes every probability: P(a < X ≤ b) = F(b) − F(a); P(X > x) = 1 − F(x); P(X = x) = F(x) − F(x−), the size of the jump at x; P(X < x) = F(x−). For example, if F(x) = 1 − e^(−2x) for x ≥ 0, then P(1 < X ≤ 2) = e^(−2) − e^(−4) ≈ 0.1170. A CDF has at most countably many jumps. The p-th quantile is ξ_p = inf{x : F(x) ≥ p}; the median is ξ_0.5. For the exponential distribution with rate λ, the median is (ln 2)/λ, smaller than the mean 1/λ, a sign of right skew.
X is discrete if it takes at most countably many values x₁, x₂, ... with probabilities p(xᵢ) = P(X = xᵢ) summing to 1; p is the probability mass function (pmf) and F is a step function. X is (absolutely) continuous if there is a non-negative function f, the probability density function (pdf), with F(x) = ∫ f(t) dt from −∞ to x. Then F is continuous, F′(x) = f(x) wherever f is continuous, P(X = x) = 0 for every x, and P(a ≤ X ≤ b) = ∫ f from a to b, whichever end-points are included.
A density is not a probability. It must be non-negative and integrate to 1, but it can exceed 1: the uniform density on (0, 0.5) equals 2. To find a normalising constant, integrate and set the integral equal to 1: for f(x) = kx² on (0, 3), k × 27/3 = 1 gives k = 1/9.
Many agricultural variables are mixed. Daily rainfall has a positive probability of being exactly zero and is continuous above zero; insurance claims and pesticide residue below a detection limit behave the same way. The CDF of such a variable has a jump at zero equal to P(X = 0) and is continuous elsewhere. The Lebesgue decomposition theorem states that any CDF is a mixture of a discrete part, an absolutely continuous part and a singular continuous part (continuous, but with derivative zero almost everywhere, like the Cantor function). Singular distributions rarely arise in applications but appear in theory examinations.
A final, useful result is the probability integral transform: if X has a continuous CDF F, then F(X) is uniform on (0, 1). Conversely, if U is uniform on (0, 1), F⁻¹(U) has CDF F, with F⁻¹ the quantile function. This is how random numbers from any distribution are generated in simulation studies.
The expectation, or mean, of X is E(X) = Σ x p(x) for a discrete variable and ∫ x f(x) dx for a continuous one, provided the sum or integral converges absolutely, that is E|X| < ∞. If it does not, the expectation does not exist: the standard Cauchy distribution, with density 1/(π(1 + x²)), has no mean because ∫|x| f(x) dx diverges. For any function g, E g(X) = Σ g(x)p(x) or ∫ g(x)f(x)dx, without first finding the distribution of g(X). For a non-negative variable, E(X) = ∫ from 0 to ∞ of (1 − F(x)) dx, and for a non-negative integer variable E(X) = Σ from k = 0 of P(X > k).
Expectation is linear: E(aX + bY + c) = aE(X) + bE(Y) + c, whether or not X and Y are independent. The r-th raw moment is μ′ᵣ = E(Xʳ) and the r-th central moment μᵣ = E(X − μ)ʳ. The variance is σ² = μ₂ = E(X²) − (E X)²; Var(aX + b) = a² Var(X). If the r-th moment exists, all lower moments exist.
Shape is summarised by Pearson's coefficients β₁ = μ₃²/μ₂³ and β₂ = μ₄/μ₂², or by γ₁ = μ₃/σ³ (skewness, which keeps the sign) and γ₂ = β₂ − 3 (excess kurtosis). For a normal distribution β₁ = 0 and β₂ = 3. A distribution with β₂ < 3 is platykurtic and with β₂ > 3 leptokurtic. Central moments are obtained from raw moments by binomial expansion; for example μ₃ = μ′₃ − 3μ′₂μ + 2μ³.
The moment-generating function (MGF) of X is M(t) = E(e^(tX)), when this is finite for all t in an open interval around 0. Its r-th derivative at 0 is the r-th raw moment. Three properties make it the workhorse of distribution theory: M_(aX+b)(t) = e^(bt) M_X(at); for independent X and Y, M_(X+Y)(t) = M_X(t) M_Y(t); and uniqueness: if two MGFs agree on an interval around zero, the distributions are identical. So a binomial MGF (q + pe^t)ⁿ identifies the binomial distribution, and the product of Poisson MGFs exp(λ₁(e^t − 1)) exp(λ₂(e^t − 1)) shows that independent Poisson counts add to a Poisson count with mean λ₁ + λ₂. The cumulant-generating function K(t) = log M(t) has derivatives at zero called cumulants: κ₁ is the mean, κ₂ the variance and κ₃ the third central moment; for the Poisson every cumulant equals λ. Cumulants of independent sums add.
The MGF does not always exist. The Cauchy, the lognormal (all moments finite, yet the MGF is infinite for every t > 0) and Student's t lack it.
For variables on 0, 1, 2, ... the probability-generating function (PGF) is G(s) = E(s^X) = Σ p(k)sᵏ, defined at least for |s| ≤ 1. Then p(k) = G⁽ᵏ⁾(0)/k!, E(X) = G′(1), E[X(X − 1)] = G″(1) (the second factorial moment), and Var(X) = G″(1) + G′(1) − [G′(1)]². PGFs are natural for counts of insects, eggs or offspring, and they handle random sums: if S = X₁ + ... + X_N with N independent of the iid Xᵢ, then G_S(s) = G_N(G_X(s)), the basis of branching-process models of pest outbreaks.
The characteristic function (CF) φ(t) = E(e^(itX)) = E cos(tX) + iE sin(tX) exists for every random variable and every real t, because |e^(itX)| = 1. Its properties: φ(0) = 1; |φ(t)| ≤ 1; φ is uniformly continuous; φ(−t) is the complex conjugate of φ(t); φ_(aX+b)(t) = e^(ibt)φ(at); and the CF of a sum of independent variables is the product of their CFs. φ is real-valued if and only if the distribution is symmetric about zero. If E|X|ʳ < ∞ then φ has r continuous derivatives and φ⁽ʳ⁾(0) = iʳ E(Xʳ).
The CF determines the distribution uniquely, and the inversion theorem recovers it: if ∫|φ(t)| dt < ∞, X has a continuous density f(x) = (1/2π) ∫ e^(−itx) φ(t) dt. Examples worth memorising: the standard normal has φ(t) = e^(−t²/2); the standard Cauchy has φ(t) = e^(−|t|), which is not differentiable at zero, matching the absence of a mean; the Poisson has exp(λ(e^(it) − 1)).
Lévy's continuity theorem links CFs to convergence: Xₙ converges in distribution to X if and only if φₙ(t) → φ(t) for every t; and if φₙ(t) converges pointwise to a function continuous at zero, that limit is a CF. This theorem is the standard route to the central limit theorem in Chapter 10. Bochner's theorem characterises CFs as the continuous, positive-definite functions with φ(0) = 1, which is mainly of theoretical interest.
For two random variables X and Y the joint CDF is F(x, y) = P(X ≤ x, Y ≤ y). In the discrete case the joint pmf p(x, y) gives the probability of each pair; in the continuous case a joint density f(x, y) satisfies P((X, Y) ∈ A) = ∬_A f(x, y) dx dy. Marginal distributions are obtained by summing or integrating out the other variable: f_X(x) = ∫ f(x, y) dy. For f(x, y) = x + y on the unit square, f_X(x) = x + 1/2.
X and Y are independent if F(x, y) = F_X(x)F_Y(y) for all x, y; equivalently the joint pmf or density factorises into a function of x times a function of y over a support that is a product set (a rectangle). The support condition is essential. The density f(x, y) = 8xy on 0 < y < x < 1 looks like a product, but the triangular support links the variables, and X and Y are dependent.
The covariance is Cov(X, Y) = E(XY) − E(X)E(Y) and the correlation ρ = Cov(X, Y)/(σ_X σ_Y), which lies in [−1, 1] by the Cauchy-Schwarz inequality, with |ρ| = 1 only if Y is a linear function of X with probability one. Var(aX + bY) = a²Var(X) + b²Var(Y) + 2ab Cov(X, Y). Independence implies zero covariance (when moments exist), but zero covariance does not imply independence: if X is standard normal and Y = X², Cov(X, Y) = E(X³) = 0 although Y is a function of X. Joint normality is the important special case where zero correlation does imply independence.
The conditional pmf of Y given X = x is p(y | x) = p(x, y)/p_X(x) and the conditional density f(y | x) = f(x, y)/f_X(x), for f_X(x) > 0. If (X, Y) is uniform on the triangle 0 < y < x < 1, then given X = x, Y is uniform on (0, x). The conditional expectation E(Y | X = x) = ∫ y f(y | x) dy is a function of x; written E(Y | X) it is a random variable.
Two identities carry much of applied statistics. The tower property (law of iterated expectations): E[E(Y | X)] = E(Y). The variance decomposition (law of total variance): Var(Y) = E[Var(Y | X)] + Var[E(Y | X)]: total variance is the average within-group variance plus the variance of group means. This is the population version of the analysis-of-variance split, and of the between- and within-cluster components in survey sampling.
E(Y | X) is the best predictor of Y from X in mean squared error: among all functions g, E[Y − g(X)]² is smallest when g(X) = E(Y | X). The best linear predictor is the regression line μ_Y + ρ(σ_Y/σ_X)(X − μ_X); the two coincide when (X, Y) is bivariate normal.
In measure-theoretic terms, E(Y | 𝓖) for a sub-σ-field 𝓖 is the 𝓖-measurable random variable whose integral over every set in 𝓖 matches that of Y; it exists by the Radon-Nikodym theorem (Chapter 11) and is unique up to sets of probability zero.
Suppose the number of egg masses N on a plant is Poisson with mean 4, and the number of eggs in each mass is independent of N and of the other masses, with mean 20 and variance 25. Let S be the total eggs on a plant. Conditioning on N: E(S | N) = 20N and Var(S | N) = 25N. Hence E(S) = 20 × 4 = 80 (Wald's identity, E(S) = E(N)E(X)), and Var(S) = E(25N) + Var(20N) = 25 × 4 + 400 × 4 = 1700. A student who uses only the within part, 100, under-states the variance seventeen-fold; the variation in the number of masses dominates. The same structure describes total claims in crop insurance (number of claims times claim size) and total yield from a random number of fruiting branches.
The calculation shows the general pattern: Var(S) = E(N)Var(X) + Var(N)[E(X)]². With PGFs the whole distribution follows from G_S(s) = G_N(G_X(s)).
Given X with a known distribution and Y = g(X), there are three standard routes to the distribution of Y: the distribution-function (CDF) method, the change-of-variable (Jacobian) method and the MGF or CF method.
In the CDF method, write F_Y(y) = P(g(X) ≤ y), express the event in terms of X, compute it, and differentiate. If X is uniform on (0, 1) and Y = −2 ln X, then F_Y(y) = P(X ≥ e^(−y/2)) = 1 − e^(−y/2) for y > 0: Y is exponential with mean 2, which is also chi-square with 2 degrees of freedom. If Z is standard normal and Y = Z², then F_Y(y) = P(−√y ≤ Z ≤ √y) = 2Φ(√y) − 1, and differentiating gives f_Y(y) = (2πy)^(−1/2) e^(−y/2), the chi-square density with one degree of freedom.
For a one-to-one, differentiable g with inverse x = g⁻¹(y), the change-of-variable formula is f_Y(y) = f_X(g⁻¹(y)) |dx/dy|. The absolute value matters for decreasing g. For Y = aX + b, f_Y(y) = f_X((y − b)/a)/|a|. When g is not one-to-one, split the domain into pieces on which it is, and add the contributions from each piece: for Y = Z², the two branches ±√y each contribute, which is why the density above has no factor 1/2. Forgetting a branch is the commonest mistake.
For a one-to-one transformation (U, V) = (u(X, Y), v(X, Y)) with inverse x = x(u, v), y = y(u, v), the joint density of (U, V) is f_(U,V)(u, v) = f_(X,Y)(x(u, v), y(u, v)) |J|, where J is the determinant of the matrix of partial derivatives ∂(x, y)/∂(u, v) of the inverse map. Equivalently |J| = 1/|∂(u, v)/∂(x, y)|. The new support must be worked out carefully; most errors are in the limits, not the algebra.
Standard results obtained this way: if X and Y are independent exponential with the same rate, U = X + Y is gamma with shape 2 and V = X/(X + Y) is uniform on (0, 1), independent of U; more generally if X ~ Gamma(α) and Y ~ Gamma(β) are independent with a common scale, X/(X + Y) is Beta(α, β) independent of X + Y. The ratio of two independent standard normals has the standard Cauchy distribution. If X and Y are independent standard normals, in polar coordinates R² = X² + Y² is exponential with mean 2 and the angle is uniform on (0, 2π), independent of R; reversing this gives the Box-Muller method of generating normal variates.
When only one function U is wanted, introduce an auxiliary V (often V = X), find the joint density and integrate V out. For U = X + Y this gives the convolution formula f_U(u) = ∫ f_X(x) f_Y(u − x) dx for independent X and Y. The sum of two independent uniforms on (0, 1) has the triangular density f(s) = s on (0, 1) and 2 − s on (1, 2), so P(S ≤ 0.5) = 0.125.
When Y is a sum of independent variables, the MGF method is fastest: multiply the MGFs and recognise the product. Sums of independent normals are normal; sums of independent gammas with a common scale are gamma; sums of independent Poissons are Poisson; a sum of n independent exponentials with rate λ is gamma with shape n.
Minima and maxima are handled with the CDF method. If X₁, ..., Xₙ are independent with CDFs Fᵢ, then P(max ≤ y) = Π Fᵢ(y) and P(min > y) = Π (1 − Fᵢ(y)). For two independent exponential failure times with rates λ₁ and λ₂, the minimum is exponential with rate λ₁ + λ₂: a pump with components failing at rates 0.02 and 0.03 per hour first fails after a mean of 1/0.05 = 20 hours. The general theory of ordered values belongs to order statistics, treated in the distributions subject.
Simulation uses these results in reverse: the inverse-CDF method X = F⁻¹(U), the Box-Muller transformation, and sums or ratios of simpler variables. A statistician checking a new estimator by Monte Carlo is applying exactly the transformations of this chapter.
Inequalities give guaranteed bounds when the distribution is unknown and only a moment or two is available. They are crude, but they hold for every distribution, and they are the tools used to prove the laws of large numbers.
Markov's inequality: if X ≥ 0 and a > 0, then P(X ≥ a) ≤ E(X)/a. If the mean yield loss to a pest is 5 kg per plot, at most a quarter of plots can lose 20 kg or more. The proof is one line: a·1{X ≥ a} ≤ X, then take expectations. More generally P(|X| ≥ a) ≤ E|X|ʳ/aʳ.
Chebyshev's inequality applies Markov to (X − μ)²: P(|X − μ| ≥ kσ) ≤ 1/k², or equivalently P(|X − μ| ≥ c) ≤ σ²/c². At least 1 − 1/k² of any distribution lies within k standard deviations of its mean: 75% within two, about 88.9% within three. For a normal distribution the true figures are about 95.4% and 99.7%, which shows how conservative Chebyshev is. With mean 50 and standard deviation 4, P(|X − 50| ≥ 10) ≤ 16/100 = 0.16, while the normal value is about 0.012. The one-sided version, Cantelli's inequality, gives P(X − μ ≥ kσ) ≤ 1/(1 + k²).
Chebyshev also gives a distribution-free sample size. To estimate a proportion within ±ε with probability at least 1 − δ, it is enough that p(1 − p)/(nε²) ≤ δ; using p(1 − p) ≤ 1/4, n ≥ 1/(4ε²δ). For ε = 0.05 and δ = 0.05 this is n = 2000, against about 385 from the normal approximation: the price of making no distributional assumption.
Jensen's inequality: if g is convex and E(X) exists, then E g(X) ≥ g(E X); for concave g the inequality reverses. Equality needs X to be constant or g linear on the support. Consequences: E(X²) ≥ (E X)², so variance is non-negative; for positive X, E(1/X) ≥ 1/E(X); and E(log X) ≤ log E(X), so the geometric mean never exceeds the arithmetic mean. In agriculture this explains why the yield predicted at average weather differs from the average yield over weather when response is curved, and why a back-transformed mean of log-transformed data under-estimates the arithmetic mean.
The Cauchy-Schwarz inequality: [E(XY)]² ≤ E(X²)E(Y²), with equality if and only if one variable is a multiple of the other with probability one. Applied to X − μ_X and Y − μ_Y it gives Cov(X, Y)² ≤ Var(X)Var(Y), that is |ρ| ≤ 1. Hölder's inequality generalises it: E|XY| ≤ (E|X|ᵖ)^(1/p)(E|Y|^q)^(1/q) for 1/p + 1/q = 1. Minkowski's inequality is the triangle inequality for the r-th mean norm: (E|X + Y|ʳ)^(1/r) ≤ (E|X|ʳ)^(1/r) + (E|Y|ʳ)^(1/r) for r ≥ 1.
Lyapunov's inequality states that (E|X|ʳ)^(1/r) is non-decreasing in r > 0. In particular E|X| ≤ (E X²)^(1/2), and if a higher absolute moment is finite every lower one is too. This is why convergence in r-th mean implies convergence in s-th mean for s < r (Chapter 8).
Practical caution: these bounds are for reasoning, not for routine reporting. When a reasonable model exists, its exact probabilities are far sharper; the inequalities matter when no model can be trusted or when a theorem must hold for every distribution.
A sequence of random variables X₁, X₂, ... can converge to X in several senses.
Almost surely (with probability one): P(Xₙ → X) = 1; the sequence converges for every ω outside a set of probability zero. In probability: for every ε > 0, P(|Xₙ − X| > ε) → 0. In r-th mean (r > 0): E|Xₙ − X|ʳ → 0; r = 2 is mean-square convergence. In distribution (in law, weakly): Fₙ(x) → F(x) at every point x where F is continuous.
The relations: almost-sure convergence implies convergence in probability; convergence in r-th mean implies convergence in probability (by Markov's inequality) and in s-th mean for s < r (by Lyapunov); convergence in probability implies convergence in distribution. None of the reverse implications holds in general. Convergence in distribution to a constant c does imply convergence in probability to c. Convergence in probability implies almost-sure convergence along some subsequence.
Counterexamples are a favourite of examiners. If Xₙ = n with probability 1/n and 0 otherwise, Xₙ → 0 in probability, but E(Xₙ) = 1 for every n, so there is no convergence in mean. The 'typewriter' sequence of indicators of shrinking intervals that sweep repeatedly across (0, 1) converges to 0 in probability and in mean, yet converges at no point, so not almost surely. And if X is standard normal and Xₙ = −X for all n, then Xₙ → X in distribution (both are standard normal) but not in probability.
Why 'continuity points' in the definition? Take Xₙ = 1/n, a constant. It obviously should converge to 0, but Fₙ(0) = 0 for every n while the limit CDF has F(0) = 1. Requiring convergence only at continuity points of F removes this artefact.
The continuous mapping theorem: if Xₙ converges to X almost surely, in probability or in distribution, and g is continuous (on a set with probability one under X), then g(Xₙ) converges to g(X) in the same mode. So if the sample mean converges in probability to μ, its square converges to μ² and its logarithm to log μ (for μ > 0).
Slutsky's lemma: if Xₙ → X in distribution and Yₙ → c in probability, with c a constant, then Xₙ + Yₙ → X + c, XₙYₙ → cX, and Xₙ/Yₙ → X/c (for c ≠ 0), all in distribution. The constant is essential; if Yₙ converges to a non-degenerate random variable, nothing can be said without the joint distribution. Slutsky is what justifies replacing σ by the sample standard deviation s in the statistic √n(X̄ − μ)/s: the numerator √n(X̄ − μ)/σ tends to N(0, 1) by the central limit theorem, σ/s → 1 in probability, and their product therefore tends to N(0, 1). The same argument underpins every large-sample confidence interval in a field-trial report. As an exercise: if Xₙ → N(0, 1) in distribution and Yₙ → 2 in probability, then XₙYₙ + 1 → N(1, 4).
For events A₁, A₂, ..., the event '{Aₙ infinitely often}' is lim sup Aₙ = ∩ₙ ∪_(k ≥ n) A_k: infinitely many of the Aₙ occur.
First Borel-Cantelli lemma: if Σ P(Aₙ) < ∞, then P(Aₙ infinitely often) = 0. No independence is needed. Second lemma: if the Aₙ are independent and Σ P(Aₙ) = ∞, then P(Aₙ infinitely often) = 1. With P(Aₙ) = 1/n² only finitely many occur almost surely; with independent Aₙ and P(Aₙ) = 1/n, infinitely many occur almost surely.
The lemmas are the standard way to prove almost-sure convergence: to show Xₙ → X almost surely, show that for each ε > 0, Σ P(|Xₙ − X| > ε) < ∞ (complete convergence). They also prove a zero-one law: for independent events, P(lim sup Aₙ) is either 0 or 1. Kolmogorov's zero-one law extends this to every tail event of an independent sequence. A practical reading: if a rare pest incursion has a summable sequence of yearly probabilities, only finitely many incursions will ever occur; if the yearly risk stays constant and years are independent, incursions will recur indefinitely, however small the risk.
A law of large numbers says that the average of many random variables settles down to a constant. Let X̄ₙ = (X₁ + ... + Xₙ)/n. A weak law (WLLN) gives convergence in probability; a strong law (SLLN) gives almost-sure convergence.
Chebyshev's WLLN: if the Xᵢ are uncorrelated with means μᵢ and variances bounded by a common constant, then X̄ₙ − (Σμᵢ)/n → 0 in probability. The proof is Chebyshev's inequality: Var(X̄ₙ) ≤ C/n → 0. For iid variables with variance σ², P(|X̄ₙ − μ| ≥ ε) ≤ σ²/(nε²); with σ² = 25, n = 100 and ε = 1 the bound is 0.25. More generally, Markov's WLLN needs only Var(ΣXᵢ)/n² → 0, which allows correlated variables.
Bernoulli's law, the oldest, is the special case of iid Bernoulli trials: the proportion of germinated seeds in a sample converges in probability to the germination probability p. It is the mathematical form of the relative-frequency interpretation of probability.
Khintchine's WLLN: if the Xᵢ are iid with finite mean μ, then X̄ₙ → μ in probability. No finite variance is required; the proof uses characteristic functions or truncation. The finite mean is needed, though: for iid standard Cauchy variables, X̄ₙ is itself standard Cauchy for every n, so it never concentrates around any value. A field instrument whose errors have Cauchy-like heavy tails does not improve by averaging, which is one argument for medians and robust estimators.
Kolmogorov's SLLN: if X₁, X₂, ... are iid, then X̄ₙ → μ almost surely if and only if E|X₁| < ∞, with μ = E(X₁). For independent but not identically distributed variables, Kolmogorov's condition Σ Var(Xₙ)/n² < ∞ is sufficient for X̄ₙ − E(X̄ₙ) → 0 almost surely. Borel's SLLN for Bernoulli trials is the special case where the proportion of successes converges almost surely to p.
The difference between weak and strong laws is the difference between the two modes. The weak law says that for each large n, a large deviation of X̄ₙ is unlikely; the strong law says that with probability one the whole sequence of running averages eventually stays within any ε of μ.
Applications are everywhere. The Glivenko-Cantelli theorem, sometimes called the fundamental theorem of statistics, states that the empirical CDF Fₙ converges to F uniformly in x, almost surely: sup |Fₙ(x) − F(x)| → 0. Monte Carlo integration rests on the SLLN: to evaluate ∫₀¹ g(x) dx, average g(Uᵢ) over many uniform random numbers. Consistency of the method-of-moments estimators, of sample variances and of sample quantiles follows from the laws of large numbers combined with the continuous mapping theorem.
A practitioner's reading: the laws of large numbers say that averages over many independent plots, farms or seasons are reliable. They do not say how fast; the rate of convergence, of order 1/√n, is the business of the central limit theorem. They also need independence or weak dependence: averaging plots that share one uncontrolled source of error, such as a single site or year, does not average that error away.
A central limit theorem (CLT) says that a suitably standardised sum of many small independent effects is approximately normal, whatever their own distribution.
Lindeberg-Lévy CLT: if X₁, X₂, ... are iid with mean μ and finite variance σ² > 0, then √n(X̄ₙ − μ)/σ converges in distribution to N(0, 1). The proof expands the characteristic function: the CF of the standardised sum is [1 − t²/(2n) + o(1/n)]ⁿ → e^(−t²/2), and Lévy's continuity theorem completes it. If plot yields have mean 40 kg and standard deviation 8 kg, the mean of 64 plots has standard error 1 kg, and P(X̄ > 42) ≈ P(Z > 2) ≈ 0.0228.
For independent but non-identical variables, with means μᵢ, variances σᵢ² and sₙ² = Σσᵢ², Liapounov's CLT requires that for some δ > 0, Σ E|Xᵢ − μᵢ|^(2+δ) / sₙ^(2+δ) → 0; then Σ(Xᵢ − μᵢ)/sₙ → N(0, 1). The Lindeberg condition, that for every ε > 0, (1/sₙ²) Σ E[(Xᵢ − μᵢ)² 1{|Xᵢ − μᵢ| > εsₙ}] → 0, is weaker and sufficient; Feller showed it is also necessary when no single term dominates (maxᵢ σᵢ²/sₙ² → 0). Liapounov's condition implies Lindeberg's. The point of these conditions is that no individual term may contribute a non-negligible share of the total variance.
The De Moivre-Laplace theorem is the CLT for binomial counts: (X − np)/√(np(1 − p)) → N(0, 1). For discrete counts a continuity correction improves the approximation: P(X ≤ k) ≈ Φ((k + 0.5 − np)/√(npq)). With n = 100 seeds and germination probability 0.8, P(X ≤ 75) ≈ Φ(−1.125) ≈ 0.130 with the correction, against Φ(−1.25) ≈ 0.106 without it. The Berry-Esseen theorem bounds the error of the normal approximation by a constant times E|X − μ|³/(σ³√n), so skewed variables need larger n.
The CLT turns statements about averages into normal-table calculations. To have P(|X̄ − μ| < d) ≥ 0.95 with known σ, take n ≥ (1.96σ/d)². With σ = 3 and d = 0.5 this is 138.3, so 139 observations; Chebyshev would demand 720. The CLT also explains why the sum of twelve uniform (0, 1) variables minus 6, which has mean 0 and variance 1, was once used as a quick normal generator, and why sample proportions, totals and regression coefficients are reported with normal-based standard errors.
The same logic applies to proportions. For a sample proportion p̂ from n = 400 Bernoulli trials with p = 0.5, the standard error is √(0.25/400) = 0.025, so P(p̂ > 0.55) ≈ P(Z > 2) ≈ 0.0228 without continuity correction. Two warnings belong with every such calculation. First, the CLT is a limit statement; it does not say how large n must be, and for strongly skewed variables such as pest counts with many zeros or insurance losses, n of 30, the rule of thumb in elementary texts, can be far too small. Second, the CLT needs finite variance; for heavy-tailed variables without it, sums are attracted to non-normal stable laws instead.
If √n(Tₙ − θ) → N(0, σ²) in distribution and g is differentiable at θ with g′(θ) ≠ 0, then √n(g(Tₙ) − g(θ)) → N(0, [g′(θ)]²σ²). The proof is a first-order Taylor expansion plus Slutsky's lemma. If g′(θ) = 0, a second-order delta method gives a scaled chi-square limit instead.
Examples: for Poisson counts with mean λ, √n(log X̄ − log λ) → N(0, λ/λ²) = N(0, 1/λ); for a binomial proportion p̂, the log-odds log(p̂/(1 − p̂)) has asymptotic variance 1/(np(1 − p)); the asymptotic variance of the sample coefficient of variation and of ratio estimators in surveys is derived the same way.
A variance-stabilising transformation chooses g so that [g′(θ)]²σ²(θ) is constant, by solving g′(θ) ∝ 1/σ(θ). For Poisson-type counts (σ² = λ) this gives g = √x, with asymptotic variance 1/4 for √X̄ after scaling by n; for binomial proportions (σ² = p(1 − p)) it gives the angular transformation arcsin √p; for variables whose standard deviation is proportional to the mean it gives log x. These are the square-root, arcsine and logarithmic transformations recommended before analysis of variance of insect counts, percentage infestation and multiplicative growth data in agricultural experiments.
The delta method extends to vectors: √n(g(Tₙ) − g(θ)) → N(0, ∇g′ Σ ∇g). Applied to the ratio of two means it gives the familiar first-order variance of the ratio estimator.
Modern probability is a branch of measure theory, and the PhD-level parts of the syllabus (AICE-JRF, CSIR-NET, ISI) expect the vocabulary. A measure μ on a measurable space (Ω, 𝓕) is a function from 𝓕 to [0, ∞] with μ(∅) = 0 that is countably additive. A probability is a measure with total mass one. Counting measure assigns to each set its number of points; Lebesgue measure on the Borel sets of ℝ assigns to each interval its length. Carathéodory's extension theorem guarantees that a countably additive set function on a field extends uniquely (in the σ-finite case) to the σ-field it generates; this is how Lebesgue measure, product measures and probability distributions specified by a CDF are built.
Sets of measure zero matter. Every countable set, including the rationals, has Lebesgue measure zero; the Cantor set is uncountable yet also has measure zero. A property holds almost everywhere (a.e.), or almost surely in probability, if it fails only on a set of measure zero.
A function f from (Ω, 𝓕) to ℝ is measurable if {f ≤ x} ∈ 𝓕 for every x. Sums, products, continuous functions, suprema, infima and pointwise limits of sequences of measurable functions are measurable. A simple function takes finitely many values, f = Σ aᵢ 1_(Aᵢ) with measurable Aᵢ; every non-negative measurable function is the pointwise limit of an increasing sequence of non-negative simple functions.
The Lebesgue integral is built in three steps. For a non-negative simple function, ∫ f dμ = Σ aᵢ μ(Aᵢ). For a non-negative measurable f, ∫ f dμ is the supremum of the integrals of simple functions lying below f. For a general f = f⁺ − f⁻, ∫ f dμ = ∫ f⁺ dμ − ∫ f⁻ dμ when at least one is finite; f is integrable when ∫ |f| dμ < ∞. Expectation is exactly this integral: E(X) = ∫ X dP. The single definition covers discrete, continuous and mixed variables: with counting measure it becomes a sum, with Lebesgue measure an ordinary integral.
The Lebesgue integral integrates more functions than the Riemann integral. The Dirichlet function, 1 on the rationals and 0 on the irrationals in [0, 1], is not Riemann integrable, but its Lebesgue integral is 0 because the rationals have measure zero. Whenever a bounded function on a closed interval is Riemann integrable, the two integrals agree.
Three convergence theorems justify exchanging limits and integrals. Monotone convergence: if 0 ≤ f₁ ≤ f₂ ≤ ... and fₙ → f pointwise, then ∫ fₙ → ∫ f. Fatou's lemma: for non-negative fₙ, ∫ lim inf fₙ ≤ lim inf ∫ fₙ. Dominated convergence: if fₙ → f almost everywhere and |fₙ| ≤ g with g integrable, then ∫ fₙ → ∫ f. In probability language: if Xₙ → X almost surely and |Xₙ| ≤ Y with E(Y) < ∞, then E(Xₙ) → E(X). The example Xₙ = n·1{U < 1/n}, with U uniform on (0, 1), converges to 0 almost surely while E(Xₙ) = 1, showing that some domination is needed.
A measure ν is absolutely continuous with respect to μ if μ(A) = 0 implies ν(A) = 0. The Radon-Nikodym theorem (for σ-finite measures) says that then there is a non-negative measurable f with ν(A) = ∫_A f dμ for all A; f = dν/dμ is the Radon-Nikodym derivative. A probability density is precisely the Radon-Nikodym derivative of the distribution of X with respect to Lebesgue measure, and a pmf is the derivative with respect to counting measure. The same theorem defines conditional expectation and likelihood ratios.
Fubini's and Tonelli's theorems allow iterated integration over product spaces: for a non-negative or integrable function, the double integral equals either iterated integral. This is the justification for computing joint probabilities as repeated integrals and for interchanging expectation and summation.
For the applied agricultural statistician the measure-theoretic layer is mostly invisible, but it settles questions that otherwise cause confusion: why some sets cannot be given probabilities, why expectation is defined the same way for every type of variable, when a derivative may be taken inside an expectation (dominated convergence again), and what 'almost surely' means in the strong law. Examinations at PhD entrance test definitions and the three convergence theorems more than proofs.
A student meets probability as the first serious theory course of the master's programme and as the heaviest block of GATE ST, IIT JAM MS and AICE-JRF Unit 1. The concrete tasks are: computing conditional and Bayes probabilities without confusing P(A | B) with P(B | A); deriving a distribution by the CDF or Jacobian method with correct supports; using MGFs and PGFs to get moments and identify sums; stating the exact conditions of each law of large numbers and CLT; and building a counterexample when asked whether one mode of convergence implies another. A thesis needs the same tools to justify the standard errors and large-sample tests it reports.
A statistician five years into work at a research institute or a crop-estimation agency uses probability to make decisions: computing the positive predictive value of a seed-health or disease test at the prevalence actually expected, deciding whether a normal approximation is adequate for a count or proportion at the sample size available, choosing a square-root or arcsine transformation for counts and percentages before analysis of variance, deriving delta-method standard errors for ratios and indices, and running Monte Carlo checks of an unfamiliar estimator. Explaining to agronomists why averaging many plots helps, and why it does not remove a common site effect, is daily work.
A senior statistician with twenty-five years of experience leads the methods behind national programmes: crop-insurance loss models built from random sums of claims, risk statements for pest surveillance from detection probabilities of trap networks, and approximation rules written into survey manuals. The senior's job is to know where the textbook assumptions fail: heavy-tailed losses for which means and variances barely exist, dependence across plots that defeats the law of large numbers, and rare events for which the normal approximation is useless. Such a statistician sets the standards that juniors apply and defends them before review committees.
A PhD with long experience works at the level of the theory itself: proving consistency and asymptotic normality of new estimators for designs, surveys and genetic models through laws of large numbers, Lindeberg-type CLTs for non-identical and weakly dependent variables, and the delta method; using measure-theoretic conditional expectation in Bayesian and mixed models; and supervising students through Borel-Cantelli and dominated-convergence arguments. Such a researcher also judges when asymptotic results are trustworthy at the sample sizes of real field experiments, and when exact, simulation-based or robust alternatives are needed.
This manual states theorems with their conditions and gives short proof sketches only; full proofs of the strong law, the Lindeberg-Feller theorem, the inversion and continuity theorems, Carathéodory extension and Radon-Nikodym are left to the cited texts. Martingales, stochastic processes, infinitely divisible and stable laws, large deviations and the law of the iterated logarithm are not covered (stochastic processes have their own subject). Standard distributions are developed in Probability Distributions and Sampling Distributions. The Berry-Esseen constant is not quoted because published values differ by source and are revised over time. Statistical tables are not reproduced; questions state the values they need.
Original practice questions written to the syllabus and pattern of the exams named in each question's examTags. They are not past papers and do not reproduce any board's questions. Every numerical answer was computed with a script; the values needed are stated in the question.
These notes are free to read here, with no account.
What the Laisha app adds:
Free for ten days, then a subscription. We would rather write that here than have you find it later.