Agricultural Statistics & Data Science · Statistical theory core · also Applied Stochastic Processes, Markov Chains and Processes
Random processes in time: Markov chains and their classification and limits, Poisson and birth-death processes, branching processes and Brownian motion, applied to population growth, epidemics, queues and weather sequences such as wet- and dry-day chains.
In the exams: GATE ST 'Stochastic Processes' (heavy); UPSC ISS Statistics I (present); ISI admission (present); CSIR-NET Unit 4 (present); light in AICE-JRF Ag Stat
A stochastic process is a family of random variables indexed by time (or space): the weather on successive days, the number of insects on a plant as the season progresses, the size of a pest population over generations, the queue of trucks at a grain market, the price of a commodity through the trading day. Probability theory describes single random variables and their limits; stochastic processes describe how randomness unfolds and accumulates through time, and how the past constrains the future.
Agriculture is full of such processes. Daily rainfall occurrence is well described by a Markov chain in which tomorrow's state depends on today's; pest and pathogen populations grow and die out as birth-death and branching processes; arrivals of farmers, trucks and calls form Poisson processes; machinery failures and replacements form renewal processes; and the random-walk and Brownian-motion models underlie price risk and the diffusion of organisms. The same mathematics gives the probability that an introduced pest dies out, the expected length of a dry spell, the waiting time at a milk-collection centre and the long-run cost of a replacement policy.
This manual covers the definition and classification of processes; discrete-time Markov chains with transition matrices and the Chapman-Kolmogorov equations; the classification of states; limiting and stationary distributions; the Poisson process; birth, death and birth-death processes; branching processes and extinction; M/M/1 and M/M/c queues; Brownian motion; Markov-chain models of rainfall and crop sequences; and an introduction to renewal processes. It builds on Probability Theory (conditional probability, generating functions, limit theorems) and Probability Distributions and Sampling Distributions (the exponential, gamma, geometric and Poisson distributions), and links to Categorical Data Analysis and GLMs for estimating transition probabilities from count data.
GATE Statistics has a full section on stochastic processes (Markov chains, Poisson processes, birth-death processes); the Indian Statistical Service, ISI and CSIR-NET include them; and the ICAR AICE-JRF paper touches them lightly through weather and population models. Examination questions are mostly computational: n-step transition probabilities, stationary distributions, absorption probabilities, Poisson-process probabilities, extinction probabilities and queue characteristics.
A stochastic process is a collection {X(t), t ∈ T} of random variables defined on a common probability space, where T is the index set (usually time) and the set of possible values of X(t) is the state space. Processes are classified by these two sets: discrete time and discrete state (a Markov chain of daily wet or dry days); discrete time and continuous state (weekly soil-moisture readings); continuous time and discrete state (the number of insects on a plant, a queue length); continuous time and continuous state (Brownian motion, a continuously recorded temperature). A realisation, the sequence of values actually observed, is called a sample path or trajectory.
The joint behaviour of a process is specified by its finite-dimensional distributions, the joint distributions of (X(t₁), ..., X(tₖ)) for every finite set of times. Kolmogorov's extension theorem guarantees that a consistent family of such distributions defines a process. In practice, processes are specified through simpler structural properties.
A process is strictly stationary if its finite-dimensional distributions are unchanged by a shift in time, and weakly (second-order, covariance) stationary if its mean is constant and its covariance Cov(X(t), X(t + h)) depends only on the lag h. Weak stationarity is the basis of time-series analysis (see the time-series subject); strict stationarity implies weak stationarity when second moments exist, but not conversely, except for Gaussian processes.
A process has independent increments if X(t₂) − X(t₁), X(t₃) − X(t₂), ... are independent for t₁ < t₂ < t₃ < ...; it has stationary increments if the distribution of X(t + h) − X(t) depends only on h. The Poisson process and Brownian motion have both. A process has the Markov property if, given the present, the future is independent of the past: P(X(tₙ₊₁) ∈ A | X(tₙ), ..., X(t₁)) = P(X(tₙ₊₁) ∈ A | X(tₙ)). Every process with independent increments is Markov, but most Markov processes (rainfall chains, birth-death processes) do not have independent increments. A martingale is a process whose expected next value, given the past, equals its present value; a fair game's fortune is the classic example, and martingale methods give elegant proofs of extinction and ruin probabilities, though they are beyond this introduction.
A discrete-time Markov chain {Xₙ, n = 0, 1, 2, ...} on a countable state space has the Markov property with time-homogeneous one-step transition probabilities pᵢⱼ = P(Xₙ₊₁ = j | Xₙ = i). They form the transition matrix P, whose entries are non-negative and whose rows sum to 1 (a stochastic matrix). Together with the initial distribution π⁽⁰⁾, P determines the whole process: P(X₀ = i₀, X₁ = i₁, ..., Xₙ = iₙ) = π⁽⁰⁾ᵢ₀ pᵢ₀ᵢ₁ ⋯ pᵢₙ₋₁ᵢₙ.
Example: a two-state weather chain with states D (dry) and W (wet) has P(D → W) = 0.3 and P(W → W) = 0.6, so P = [[0.7, 0.3], [0.4, 0.6]]. If today is dry, the probability of the sequence dry, wet, wet over the next three days (starting from today) is 0.3 × 0.6 = 0.18 for the two transitions.
Two modelling decisions come before any calculation. First, the state space: rainfall can be coded as dry/wet, or as dry/light/heavy, and a finer coding needs more data to estimate its larger matrix (k(k − 1) free transition probabilities for k states). Second, homogeneity in time: a single matrix for the whole year mixes monsoon and dry-season behaviour, so agrometeorological work fits separate matrices by month or lets the probabilities vary smoothly with the day of the year. A chain can also be made Markov by enlarging the state: if tomorrow depends on the last two days, the pair (yesterday, today) is a four-state first-order chain.
The n-step transition probabilities pᵢⱼ⁽ⁿ⁾ = P(X_(m+n) = j | X_m = i) satisfy the Chapman-Kolmogorov equations pᵢⱼ⁽ᵐ⁺ⁿ⁾ = Σₖ pᵢₖ⁽ᵐ⁾pₖⱼ⁽ⁿ⁾: to go from i to j in m + n steps, the chain must be somewhere (k) after m steps. In matrix form P⁽ᵐ⁺ⁿ⁾ = P⁽ᵐ⁾P⁽ⁿ⁾, so the n-step matrix is the n-th power Pⁿ. The distribution of the chain at time n is the row vector π⁽ⁿ⁾ = π⁽⁰⁾Pⁿ.
For the weather chain, P² = [[0.7 × 0.7 + 0.3 × 0.4, 0.7 × 0.3 + 0.3 × 0.6], [0.4 × 0.7 + 0.6 × 0.4, 0.4 × 0.3 + 0.6 × 0.6]] = [[0.61, 0.39], [0.52, 0.48]]. So if today is dry, the day after tomorrow is wet with probability 0.39. For a two-state chain with p₀₁ = a and p₁₀ = b, the powers have the closed form Pⁿ = [1/(a + b)][[b, a], [b, a]] + [(1 − a − b)ⁿ/(a + b)][[a, −a], [−b, b]], which shows the geometric approach to the limit at rate |1 − a − b|ⁿ.
Transition probabilities are estimated from an observed sequence by p̂ᵢⱼ = nᵢⱼ/nᵢ., the proportion of transitions out of i that go to j; these are the maximum-likelihood estimates, and the counts nᵢⱼ can be analysed with the contingency-table and log-linear methods of the categorical-data subject, for example to test whether the transition probabilities change between seasons or decades.
State j is accessible from i if pᵢⱼ⁽ⁿ⁾ > 0 for some n; i and j communicate if each is accessible from the other. Communication is an equivalence relation that splits the state space into classes; a chain with a single class is irreducible. A class is closed if no state outside it is accessible from it; an absorbing state (pᵢᵢ = 1) is a closed class by itself.
Let fᵢᵢ be the probability that the chain, starting at i, ever returns to i. State i is recurrent if fᵢᵢ = 1 and transient if fᵢᵢ < 1. Equivalently, i is recurrent if and only if Σₙ pᵢᵢ⁽ⁿ⁾ = ∞ (the expected number of returns is infinite). Recurrence and transience are class properties. A recurrent state is positive recurrent if its mean recurrence time μᵢ is finite and null recurrent if μᵢ = ∞. In a finite chain, at least one state is recurrent, there are no null-recurrent states, and every state of a finite irreducible chain is positive recurrent.
The simple random walk on the integers (step +1 with probability p, −1 with probability q = 1 − p) is the standard infinite example: it is recurrent if p = 1/2 (in fact null recurrent) and transient if p ≠ 1/2. Pólya's theorem: the symmetric random walk is recurrent in one and two dimensions but transient in three or more.
The period of state i is d(i) = gcd{n ≥ 1 : pᵢᵢ⁽ⁿ⁾ > 0}. If d(i) = 1 the state is aperiodic. Period is a class property. A chain that alternates deterministically between two states has period 2; any state with pᵢᵢ > 0 is aperiodic. An irreducible, aperiodic, positive-recurrent chain is called ergodic.
When a chain has absorbing states, the questions are which absorbing state is reached and how long it takes. With transient states T and absorbing states A, write the transition matrix in canonical form with Q (transitions among transient states) and R (transient to absorbing). The fundamental matrix N = (I − Q)⁻¹ gives the expected number of visits to each transient state; the expected time to absorption from each transient state is N1, and the absorption probabilities are NR.
The gambler's-ruin problem is the classic case: a random walk on 0, 1, ..., N absorbed at 0 and N, starting at i. With p ≠ q, the probability of reaching N before 0 is [1 − (q/p)ⁱ]/[1 − (q/p)ᴺ]; with p = q = 1/2 it is i/N and the expected duration is i(N − i). The same mathematics describes a small population that either dies out or reaches a carrying capacity, or a farmer's cash reserve that may be exhausted by a run of poor seasons.
A probability vector π is a stationary (invariant) distribution of a chain if πP = π: if the chain starts in π, it stays in π for ever. It is found by solving πP = π together with Σπⱼ = 1. For the two-state chain with p₀₁ = a and p₁₀ = b, π = (b/(a + b), a/(a + b)). For the weather chain with P(D → W) = 0.3 and P(W → D) = 0.4, π(W) = 0.3/(0.3 + 0.4) = 0.429: in the long run about 43% of days are wet.
An irreducible chain has a stationary distribution if and only if it is positive recurrent, in which case π is unique and πⱼ = 1/μⱼ, the reciprocal of the mean recurrence time: with π(W) = 0.429, the mean time between wet days (from one wet day to the next) is 1/0.429 = 2.33 days. A chain satisfying detailed balance, πᵢpᵢⱼ = πⱼpⱼᵢ for all i, j, is reversible, and the balance equations then give π directly; birth-death chains are always reversible, which is why their stationary distributions have product form (Chapter 6).
For an ergodic chain (irreducible, aperiodic, positive recurrent), pᵢⱼ⁽ⁿ⁾ → πⱼ as n → ∞ for every starting state i: the chain forgets where it started, and every row of Pⁿ converges to π. For the weather chain, the rows of Pⁿ approach (0.571, 0.429) at the geometric rate 0.3ⁿ, since 1 − a − b = 0.3; after a week the starting state hardly matters. Periodic chains still have stationary distributions but pᵢⱼ⁽ⁿ⁾ oscillates and does not converge; time averages converge nevertheless.
The ergodic theorem for Markov chains is a law of large numbers: for an irreducible positive-recurrent chain, the long-run proportion of time spent in state j converges almost surely to πⱼ, and (1/n)Σf(Xₖ) → Σπⱼf(j) for any bounded f. This is why the long-run proportion of wet days, the average yield over a cropping sequence or the average cost of a stocking policy can be computed from π, and it is the theoretical foundation of Markov chain Monte Carlo (see Bayesian Inference): a chain designed to have the posterior as its stationary distribution produces averages that converge to posterior expectations.
Absorbing chains and transient states behave differently: the limit of pᵢⱼ⁽ⁿ⁾ is zero for transient j, and the limiting distribution depends on the starting state through the absorption probabilities.
A counting process {N(t), t ≥ 0} is a Poisson process with rate λ > 0 if N(0) = 0, it has independent increments, and the number of events in any interval of length t is Poisson with mean λt: P(N(s + t) − N(s) = k) = e^(−λt)(λt)ᵏ/k!. Equivalently, events occur singly, independently, at a constant rate: P(one event in (t, t + h)) = λh + o(h) and P(two or more) = o(h). It models arrivals of trucks at a market, calls to an advisory helpline, emergence of insects in a constant environment, and lightning-caused fires in a season.
Example: farmers arrive at a soil-testing laboratory at λ = 3 per hour. The number arriving in 2 hours is Poisson with mean 6; P(no arrivals in 30 minutes) = e^(−1.5) = 0.223; P(exactly 2 in an hour) = e^(−3)3²/2! = 0.224.
Whether a real arrival stream is Poisson is an empirical question. Three checks are standard: the counts in equal intervals should have variance close to their mean (index of dispersion near 1); the inter-arrival times should look exponential (an exponential probability plot or a KS test); and counts in disjoint intervals should be uncorrelated. Arrivals scheduled by appointment are more regular than Poisson, and arrivals of groups (several trucks from one village) are more clustered; in both cases queueing and risk calculations based on the Poisson assumption will be wrong in a predictable direction.
The times between successive events, T₁, T₂, ..., are independent exponential variables with mean 1/λ, and the time Sₙ of the n-th event is gamma (Erlang) with shape n and rate λ. The memoryless property of the exponential means that the waiting time from any fixed moment to the next event is again exponential with mean 1/λ, however long one has already waited; the apparent paradox that a randomly arriving observer waits as long, on average, as a full inter-arrival time is the inspection paradox of renewal theory (Chapter 11).
Superposition: independent Poisson processes with rates λ₁ and λ₂ combine into a Poisson process with rate λ₁ + λ₂, and the next event comes from the first process with probability λ₁/(λ₁ + λ₂). Thinning: if each event is independently kept with probability p, the kept events form a Poisson process with rate λp, independent of the discarded ones with rate λ(1 − p): if insects land at rate 10 per hour and each independently is a vector species with probability 0.2, vector landings form a Poisson process with rate 2 per hour. Conditional uniformity: given N(t) = n, the n event times are distributed as the order statistics of n independent uniforms on (0, t).
Extensions: the non-homogeneous Poisson process has a time-varying rate λ(t), with N(t) Poisson with mean ∫₀ᵗλ(u)du, suitable for seasonal insect emergence or diurnal arrivals; the compound Poisson process attaches a random size Yᵢ to each event, S(t) = ΣYᵢ, with E(S(t)) = λtE(Y) and Var(S(t)) = λtE(Y²), the standard model for aggregate rainfall over a period or total insurance claims (compare the random sums of the probability manual). Spatial Poisson processes describe random scatter of plants or weeds over a field; departures from it (clustering, regularity) are the subject of spatial point-process statistics.
A continuous-time birth-death process on the states 0, 1, 2, ... moves from n to n + 1 at rate λₙ (a birth) and from n to n − 1 at rate μₙ (a death); the time spent in state n before moving is exponential with rate λₙ + μₙ. The transition probabilities satisfy the Kolmogorov forward equations, for example p′ₙ(t) = λₙ₋₁pₙ₋₁(t) − (λₙ + μₙ)pₙ(t) + μₙ₊₁pₙ₊₁(t) for the probability of being in state n at time t.
The Poisson process is the pure birth process with constant rate λₙ = λ. The Yule (linear pure birth) process has λₙ = nλ: each individual independently splits at rate λ. Starting from one individual, X(t) is geometric on 1, 2, ... with P(X(t) = n) = e^(−λt)(1 − e^(−λt))ⁿ⁻¹, and E(X(t)) = e^(λt): exponential growth, as in an unchecked aphid colony or bacterial culture. The linear pure death process has μₙ = nμ: each of N₀ individuals dies independently at rate μ, so X(t) is binomial(N₀, e^(−μt)) with mean N₀e^(−μt); seed survival in a seed bank and decay of a pesticide-exposed cohort follow this pattern.
In the linear birth-death process each individual gives birth at rate λ and dies at rate μ, so λₙ = nλ and μₙ = nμ. The mean population size is E(X(t)) = X(0)e^((λ−μ)t): growth if λ > μ and decline otherwise. State 0 is absorbing, and the probability of eventual extinction starting from i individuals is 1 if λ ≤ μ and (μ/λ)ⁱ if λ > μ. Even a population that grows on average can die out by chance when it starts small: an introduced biocontrol agent with λ = 0.3 and μ = 0.2 per day released as 5 individuals goes extinct with probability (2/3)⁵ = 0.132, and with 20 individuals the risk falls to (2/3)²⁰ ≈ 0.0003, which is why releases use many individuals.
When births and deaths do not lead to absorption (for example λₙ = λ constant, as immigration, and μₙ = nμ), the process can have a stationary distribution, found from detailed balance: πₙ = π₀ Π from k = 1 to n of (λₖ₋₁/μₖ), normalised to sum to 1, provided the sum converges. For immigration at rate λ and independent deaths at rate μ per individual, πₙ is Poisson with mean λ/μ: the equilibrium number of insects on a plant that receives immigrants at rate 6 per day and loses each insect at rate 0.5 per day is Poisson with mean 12. The M/M/1 and M/M/c queues of Chapter 8 are birth-death processes with arrivals as births and service completions as deaths.
A Galton-Watson branching process models a population in discrete generations in which each individual independently produces a random number of offspring with the same distribution pₖ = P(k offspring), with probability-generating function G(s) = Σpₖsᵏ, mean m = G′(1) and variance σ². Let Zₙ be the generation size with Z₀ = 1. The PGF of Zₙ is the n-fold composition Gₙ(s) = G(G(⋯G(s))), by the random-sum result of the probability manual. The mean grows geometrically: E(Zₙ) = mⁿ. The variance is σ²mⁿ⁻¹(mⁿ − 1)/(m − 1) for m ≠ 1 and nσ² for m = 1.
Processes are called subcritical (m < 1), critical (m = 1) and supercritical (m > 1). Applications include the spread of a pest or pathogen from a single introduction in its early phase (each infected plant infecting a random number of others), the survival of a new mutation or a rare allele, the persistence of a family name (Galton's original problem) and the early stage of an epidemic, where m plays the role of the basic reproduction number R₀.
The probability q that the population eventually dies out is the smallest non-negative root of the equation s = G(s). If m ≤ 1 then q = 1 (extinction is certain, except in the trivial case p₁ = 1); if m > 1 then q < 1. Starting from Z₀ = k individuals, the extinction probability is qᵏ.
Example: an introduced insect produces 0, 1 or 2 surviving offspring with probabilities 0.25, 0.25 and 0.5. Then m = 0.25 + 1.0 = 1.25 > 1, G(s) = 0.25 + 0.25s + 0.5s², and s = G(s) gives 0.5s² − 0.75s + 0.25 = 0, i.e. s² − 1.5s + 0.5 = 0, with roots 0.5 and 1. So q = 0.5: even though the population grows on average, half of single introductions die out. With 10 independent founders the extinction probability is 0.5¹⁰ ≈ 0.001. For Poisson offspring with mean m, q solves q = e^(m(q−1)); for m = 2, q ≈ 0.203.
This result has direct practical meaning in biosecurity and biocontrol: the establishment probability of an invader depends strongly on propagule pressure (the number of founders), and early eradication can succeed by chance-driven extinction even when the reproduction number exceeds 1. Conversely, for a released biocontrol agent, multiple releases of moderate size are more reliable than one release, if the founders act independently. Real populations add environmental variation, density dependence and spatial spread, which are handled by extensions of the basic model.
Queueing theory studies waiting lines: trucks at a grain procurement centre, farmers at a milk-collection point, tractors waiting for a repair workshop, samples awaiting analysis in a laboratory. In Kendall's notation A/B/c, A is the arrival process, B the service-time distribution and c the number of servers; M (Markovian) denotes Poisson arrivals or exponential service times.
In the M/M/1 queue, customers arrive as a Poisson process with rate λ and are served one at a time with exponential service times of rate μ. The queue length is a birth-death process with λₙ = λ and μₙ = μ. It is stable if the traffic intensity ρ = λ/μ < 1, and then the stationary distribution is geometric: Pₙ = (1 − ρ)ρⁿ. Standard results: probability the server is idle, P₀ = 1 − ρ; mean number in the system L = ρ/(1 − ρ); mean number waiting Lq = ρ²/(1 − ρ); mean time in the system W = 1/(μ − λ); mean waiting time before service Wq = ρ/(μ − λ). Little's law L = λW (and Lq = λWq) holds for very general queues, not only M/M/1.
Example: milk-supply farmers arrive at a single testing counter at 4 per hour and testing takes on average 10 minutes (μ = 6 per hour). Then ρ = 2/3, L = 2, Lq = 1.33, W = 0.5 hour and Wq = 1/3 hour (20 minutes). As ρ approaches 1, waiting grows without bound: at λ = 5.5 per hour, W = 2 hours.
In the M/M/c queue there are c identical servers, each with rate μ, and the system is stable if ρ = λ/(cμ) < 1. The stationary probabilities follow from birth-death balance with death rates μₙ = min(n, c)μ. The probability that an arriving customer must wait is given by the Erlang C formula, and Wq = P(wait)/(cμ − λ). Adding servers reduces waiting sharply when utilisation is high: with λ = 10 per hour and μ = 6 per hour, one server is unstable, two servers (ρ = 0.83) give long queues, and three servers (ρ = 0.56) give short ones. Pooling two separate single-server queues into one two-server queue always reduces mean waiting, a principle used when designing procurement-centre layouts.
Other models relax the assumptions: M/G/1 (general service times; the Pollaczek-Khinchine formula Lq = (λ²σ² + ρ²)/[2(1 − ρ)] shows that service-time variability, not only its mean, causes waiting), finite-capacity queues M/M/1/K in which arrivals are turned away when the yard is full, and priority queues. Real arrival patterns at markets are non-homogeneous (peaks after harvest and in the morning), so steady-state formulas are used with care, often alongside simulation. Queueing models guide decisions about the number of weighbridges, testing counters or harvesters to provide, balancing the cost of servers against the cost of waiting (spoilage of milk, perishable produce or farmers' time).
Standard Brownian motion (the Wiener process) {W(t), t ≥ 0} has W(0) = 0, independent increments, normally distributed increments W(t) − W(s) ~ N(0, t − s) for s < t, and continuous sample paths. It follows that W(t) ~ N(0, t), Cov(W(s), W(t)) = min(s, t), and the process is a martingale. Brownian motion with drift μ and diffusion coefficient σ is X(t) = μt + σW(t), with X(t) ~ N(μt, σ²t).
Brownian motion arises as the limit of a scaled random walk (Donsker's invariance principle, a functional central limit theorem): if steps of ±1 are taken every 1/n units of time and scaled by 1/√n, the path converges in distribution to Brownian motion. Its paths are continuous everywhere but differentiable nowhere, and they have infinite length over any interval, reflecting the √t scaling of displacement: over time t the typical displacement is of order √t, not t. Scaling property: for c > 0, W(ct)/√c is again a standard Brownian motion.
A dispersal illustration: if a released insect's position along a field edge behaves like Brownian motion with σ² = 4 m² per hour, then after 9 hours its displacement is N(0, 36), with standard deviation 6 m, and about 95% of insects lie within ±11.8 m of the release point. Quadrupling the time only doubles the spread, which is why dispersal experiments report the mean squared displacement per unit time (the diffusion coefficient) rather than a speed. Adding a drift term models wind-assisted movement, and the probability of crossing a boundary, such as a buffer strip around a trial, follows from the reflection principle.
The reflection principle gives the distribution of the running maximum: P(max over 0 ≤ s ≤ t of W(s) ≥ a) = 2P(W(t) ≥ a) = 2[1 − Φ(a/√t)] for a > 0. For example, the probability that a standard Brownian motion reaches level 2 by time 4 is 2[1 − Φ(1)] = 0.317. The time to first reach level a has the inverse-Gaussian (Lévy) distribution in the driftless case, and the probability that Brownian motion with negative drift ever exceeds a level is exp(−2|μ|a/σ²). The Brownian bridge, Brownian motion conditioned to return to 0 at time 1, is the limit of the empirical-process statistic behind the Kolmogorov-Smirnov test (see the nonparametric manual).
In agriculture and related fields, Brownian motion and its relatives model the random dispersal of organisms (the mean squared displacement grows linearly in time, the basis of diffusion models of insect and seed dispersal), fluctuations of commodity prices (geometric Brownian motion, log S(t) = log S(0) + (μ − σ²/2)t + σW(t), used in option and storage-value models), measurement drift in sensors, and genetic drift in the diffusion approximation of population genetics. These applications use the mathematics in approximate form; real dispersal kernels often have heavier tails than the Gaussian, and real price series show volatility clustering that Brownian motion does not capture.
Gabriel and Neumann (1962) showed that the occurrence of wet and dry days at Tel Aviv was well described by a first-order two-state Markov chain, and the model has since been fitted to rainfall records worldwide. With states D (dry) and W (wet), the parameters are p₀₁ = P(wet tomorrow | dry today) and p₁₁ = P(wet tomorrow | wet today); usually p₁₁ > p₀₁, reflecting the persistence of weather systems. They are estimated from daily records by transition counts, p̂₀₁ = n₀₁/n₀. and p̂₁₁ = n₁₁/n₁., typically separately for each month or season because rainfall regimes change through the year.
The model gives practically useful quantities directly. The long-run probability of a wet day is π(W) = p₀₁/(1 − p₁₁ + p₀₁). The lengths of wet spells are geometric with mean 1/(1 − p₁₁), and of dry spells with mean 1/p₀₁. The probability of a dry spell of at least k days, given that a dry spell has started, is (1 − p₀₁)ᵏ⁻¹. Example: with p₀₁ = 0.2 and p₁₁ = 0.6 during the monsoon, π(W) = 0.2/0.6 = 0.333, mean wet spell 2.5 days, mean dry spell 5 days, and the probability that a dry spell lasts at least 10 days is 0.8⁹ = 0.134. Such probabilities underlie advice on sowing dates (sow when the risk of a long dry spell after sowing is acceptable), the timing of irrigation and spraying, and the design of crop-insurance triggers based on dry spells.
The first-order assumption can be tested against a second-order chain, in which tomorrow depends on the last two days, by a likelihood-ratio (G²) or chi-square test on the transition counts, or by comparing models with AIC or BIC; in some climates second-order chains fit dry-spell lengths better. Logistic regression of the wet/dry state on the previous days' states and on seasonal harmonics (see the categorical-data subject) is a flexible way of fitting non-homogeneous chains whose transition probabilities vary smoothly through the year. Rainfall amounts on wet days are then modelled separately, usually by a gamma or mixed-exponential distribution, giving the occurrence-amount 'weather generators' (such as the Richardson type) used to drive crop-simulation models.
Markov chains also describe land use and crop sequences: the state is the crop grown on a field (cereal, pulse, oilseed, fallow) and the transition matrix, estimated from repeated surveys or remote sensing, describes rotations. The stationary distribution projects the long-run land-use mix if transition behaviour stays constant; absorbing states (land lost to urban use) give expected time to conversion. Markov models of livestock herd dynamics (age or parity classes) and of disease states in plants and animals (healthy, latent, infectious, removed) use the same machinery. The main caution is time-homogeneity: policy, prices and climate change transition probabilities, so projections are scenarios, not forecasts.
A worked crop-sequence example: fields are classified each year as rice (R), wheat (W) or fallow (F), and survey data give the transition matrix with rows R: (0.6, 0.3, 0.1), W: (0.5, 0.4, 0.1) and F: (0.4, 0.2, 0.4). Solving πP = π gives a long-run mix of about 54% rice, 32% wheat and 14% fallow if behaviour stays constant; comparing this with the current mix shows the direction in which land use is drifting, and re-estimating the matrix after a policy change shows how the projected mix responds.
A renewal process generalises the Poisson process by allowing the times between events to have any distribution F on (0, ∞), not only the exponential. Inter-renewal times X₁, X₂, ... are independent and identically distributed with mean μ; the n-th renewal occurs at Sₙ = X₁ + ... + Xₙ; and N(t) counts renewals by time t. Typical renewals: replacement of a failed pump or tractor tyre, successive outbreaks of a pest after control, successive harvests of a perennial crop, refilling of a storage tank.
The renewal function m(t) = E[N(t)] = Σₙ Fₙ(t), where Fₙ is the distribution of Sₙ, satisfies the renewal equation m(t) = F(t) + ∫₀ᵗ m(t − x) dF(x). For the Poisson process m(t) = λt. Limit theorems: N(t)/t → 1/μ almost surely (a strong law), the elementary renewal theorem m(t)/t → 1/μ, and a central limit theorem: N(t) is approximately normal with mean t/μ and variance tσ²/μ³ for large t. If pump lives have mean 2 years and SD 0.5 years, over 40 years about 20 pumps are needed, with variance 40 × 0.25/8 = 1.25.
If one inspects a renewal process at a fixed time t and measures the length of the interval that contains t, its mean is not μ but approaches E(X²)/E(X) = μ + σ²/μ for large t, which exceeds μ: longer intervals are more likely to be 'caught'. For the Poisson process, the interval containing t has mean 2/λ (twice the mean inter-arrival time) for large t. The same bias affects field surveys that sample plants, fields or lesions at a moment in time (length-biased sampling), and must be corrected when estimating durations from snapshots, such as the length of infectious periods estimated from currently infected animals.
The renewal-reward theorem: if each cycle earns (or costs) a random reward Rᵢ, with cycles iid, then the long-run reward per unit time is E(R)/E(X) almost surely. This underlies replacement policies. Example: replacing a pump at failure costs 30,000 (including lost irrigation), replacing it preventively at age T costs 10,000; with the cycle length min(life, T) and cost depending on which comes first, the policy minimising E(cost per cycle)/E(cycle length) is the optimal age-replacement policy, preferable to replacement on failure when lifetimes have an increasing hazard (wear-out). For exponential lifetimes (constant hazard), preventive replacement never helps, because a working pump is always as good as new: the memoryless property again.
Alternating renewal processes, in which periods of operation alternate with periods of repair, give the long-run availability of a machine as E(up time)/[E(up time) + E(down time)]: a harvester that runs on average 90 days between breakdowns and takes 10 days to repair is available 90% of the time, whatever the shapes of the two distributions. This simple result is often all that is needed to compare maintenance contracts or to decide how many standby machines a custom-hiring centre should keep.
A student needs fluency with transition matrices and their powers, solving πP = π, classifying states and computing periods, absorption probabilities and expected times, Poisson-process probabilities including thinning and superposition, birth-death equilibria, branching-process extinction from the generating function, M/M/1 characteristics and Little's law, and Brownian-motion probabilities via the reflection principle. GATE ST tests these as numerical items; agrometeorology theses use the rainfall chain directly.
A statistician five years into work at an agrometeorology unit, plant-protection service or agribusiness firm fits wet/dry Markov chains to station records to produce dry-spell probabilities for sowing advisories, builds weather generators for crop models, estimates establishment and extinction probabilities for introduced pests and biocontrol releases, sizes testing counters and weighbridges with queueing models, and evaluates replacement policies for machinery with renewal-reward arguments.
A senior statistician with twenty-five years of experience decides which stochastic models are fit for institutional products: the weather-generator assumptions behind crop-insurance and advisory systems, the population models used in biosecurity risk assessments, and the queueing and logistics models in procurement planning. Such a statistician insists on checking time-homogeneity and model order against data and on communicating model-based probabilities with their limitations.
A PhD with long experience develops stochastic models for agricultural systems: non-homogeneous and hidden Markov models for rainfall under climate change, spatial branching and diffusion models of pest invasion, stochastic epidemic models for plant and animal diseases, and continuous-time Markov models of herd and land-use dynamics, together with their estimation from incomplete data. Such a researcher proves model properties, fits them by likelihood or Bayesian methods, and supervises doctoral students.
Proofs are sketched rather than given. Not covered: martingale theory beyond a mention, stochastic calculus and stochastic differential equations, hidden Markov models, continuous-time Markov chains in general form beyond birth-death processes, spatial point processes and stochastic epidemic models in detail, and time-series models (in their own subject). Rainfall-chain parameters vary by station, season and period and are not quoted except as illustrations. Statistical tables are not reproduced; questions state the values they need.
Original practice questions written to the syllabus and pattern of the exams named in each question's examTags. They are not past papers and do not reproduce any board's questions. Every numerical answer was computed with a script; the values needed are stated in the question.
These notes are free to read here, with no account.
What the Laisha app adds:
Free for ten days, then a subscription. We would rather write that here than have you find it later.