Agricultural Statistics & Data Science · Foundations · also Mathematical Analysis, Advanced Calculus, Mathematics for Statistics (part)
The rigorous calculus behind limits theorems and estimation: sequences, series, continuity, differentiability and Riemann integration, at the level GATE Statistics and JAM Mathematical Statistics examine and that measure-theoretic probability later builds on.
In the exams: GATE ST 'Calculus' section (heavy); IIT JAM MS mathematics component (heavy); AIEEA-PG Group 07 Unit II-III mathematics (present); ISI admission tests (heavy); UPSC ISS Statistics I (light)
Real analysis is the rigorous study of the real numbers and of the functions, limits, series and integrals built on them. For a statistician it is not an optional refinement of calculus. The laws of large numbers and the central limit theorem are statements about limits of sequences; consistency of an estimator is convergence; moment generating functions are power series; probability densities are integrals that must converge; the gamma and beta functions sit inside half the distributions in a textbook; and maximum likelihood is calculus in several variables with constraints. A student who can differentiate and integrate mechanically but cannot say when a series converges, when an improper integral exists, or why a limit may be taken under an integral sign will meet difficulty in every later course on probability, inference and asymptotic theory.
This manual covers the real number system and countability; sequences and their convergence, including monotone and Cauchy sequences; series and tests of convergence; power series and radius of convergence; limits, continuity and uniform continuity; differentiability, the mean value theorems and Taylor's theorem; L'Hospital's rule and indeterminate forms; Riemann integration and improper integrals; the gamma and beta functions; functions of several variables, with partial derivatives, Jacobians, extrema and Lagrange multipliers; and double and triple integrals with change of variables. Each chapter states the definitions and theorems precisely, explains why they matter in statistics, and works the kinds of short problems that examinations set.
Graduate entrance examinations in statistics and mathematical statistics, statistical institute admission tests and national eligibility tests in the mathematical sciences test this material heavily, in multiple-choice and numerical-answer form: convergence of sequences and series, radius of convergence, limits by L'Hospital's rule, applications of the mean value theorem, evaluation of improper integrals with gamma and beta functions, extrema of functions of two variables and Jacobians of transformations. Master's admission examinations in agricultural statistics include a mathematics block covering much of the same ground at an introductory level.
A set is a collection of objects; A ∪ B, A ∩ B and A \ B denote union, intersection and difference, and De Morgan's laws state that the complement of a union is the intersection of the complements, (A ∪ B)ᶜ = Aᶜ ∩ Bᶜ, and the complement of an intersection is the union of the complements. These rules are used constantly in probability, where events are sets. A function f from A to B assigns to each element of A exactly one element of B. It is injective (one-to-one) if different inputs give different outputs, surjective (onto) if every element of B is an output, and bijective if it is both, in which case it has an inverse. Two sets have the same cardinality if a bijection exists between them.
A set is countable if it is finite or can be put in one-to-one correspondence with the natural numbers N = {1, 2, 3, …}. The integers are countable (list them as 0, 1, −1, 2, −2, …), and so, perhaps surprisingly, are the rationals Q: arrange fractions p/q in a grid by numerator and denominator and list them along diagonals, skipping repeats. A countable union of countable sets is countable, and any subset of a countable set is countable. Cantor's diagonal argument shows that the real numbers in (0, 1) are not countable: given any list of decimal expansions, construct a number whose n-th digit differs from the n-th digit of the n-th number in the list; it is not in the list. Hence R is uncountable, and so is the set of irrationals, since R = Q ∪ (irrationals) and Q is countable. The power set of N, the set of all its subsets, is also uncountable. In probability this distinction separates discrete distributions, whose support is countable and whose probabilities can be summed, from continuous ones, for which every single point has probability zero.
The real numbers form an ordered field: addition and multiplication obey the usual algebraic rules, and the order is compatible with them. What distinguishes R from Q is completeness, stated as the least upper bound property: every non-empty set of real numbers that is bounded above has a least upper bound, or supremum, in R. The rationals lack this: the set of rationals whose square is less than 2 has no rational supremum. The supremum of a set S is the number M such that every element of S is at most M and, for every ε > 0, some element exceeds M − ε; it need not belong to S (the supremum of (0, 1) is 1). The infimum is defined similarly. From completeness follow the Archimedean property (for any real x there is a natural number n > x) and the density of the rationals (between any two reals lies a rational, and also an irrational). The absolute value satisfies the triangle inequality |a + b| ≤ |a| + |b|.
A set is open if every point has a neighbourhood (a small interval around it) inside the set, and closed if it contains all its limit points; a limit point of S is a point every neighbourhood of which contains points of S other than itself. The Bolzano–Weierstrass theorem states that every bounded infinite subset of R has a limit point, and the Heine–Borel theorem states that a subset of R is compact (every open cover has a finite subcover) if and only if it is closed and bounded. Compactness of closed bounded intervals is what makes the extreme value and uniform continuity theorems work.
A sequence (aₙ) converges to a limit L if for every ε > 0 there is an N such that |aₙ − L| < ε for all n ≥ N. The limit, when it exists, is unique, and every convergent sequence is bounded, though not every bounded sequence converges: (−1)ⁿ is bounded and oscillates. Limits respect algebra: if aₙ → a and bₙ → b, then aₙ + bₙ → a + b, aₙbₙ → ab, and aₙ/bₙ → a/b if b ≠ 0. The sandwich (squeeze) theorem states that if aₙ ≤ bₙ ≤ cₙ and aₙ and cₙ both tend to L, so does bₙ; it gives, for example, sin(n)/n → 0. Standard limits used repeatedly in statistics include rⁿ → 0 for |r| < 1; n^(1/n) → 1; a^(1/n) → 1 for a > 0; (1 + x/n)ⁿ → eˣ; and nᵏ/aⁿ → 0 for a > 1, so that exponentials dominate powers. The limit (1 − λ/n)ⁿ → e^(−λ) underlies the Poisson approximation to the binomial.
The monotone convergence theorem states that a monotone sequence converges if and only if it is bounded; an increasing sequence bounded above converges to its supremum. This is the usual way to prove convergence when the limit is not known in advance. For example, the sequence aₙ = (1 + 1/n)ⁿ is increasing and bounded above by 3, so it converges; its limit defines e ≈ 2.718. A recursively defined sequence such as a₁ = 1, aₙ₊₁ = √(2 + aₙ) can be shown by induction to be increasing and bounded by 2, so it converges, and its limit L satisfies L = √(2 + L), giving L = 2.
A subsequence takes terms of the original sequence in order, skipping some. If a sequence converges, every subsequence converges to the same limit; finding two subsequences with different limits proves divergence. The Bolzano–Weierstrass theorem for sequences states that every bounded sequence has a convergent subsequence. The limit superior, lim sup aₙ, is the largest limit of any subsequence, and the limit inferior the smallest; a bounded sequence converges if and only if they are equal. For (−1)ⁿ(1 + 1/n), lim sup = 1 and lim inf = −1.
A sequence is Cauchy if its terms become arbitrarily close to each other: for every ε > 0 there is an N such that |aₘ − aₙ| < ε for all m, n ≥ N. In R, a sequence converges if and only if it is Cauchy, which is another expression of completeness; in Q, a Cauchy sequence of rationals approaching √2 has no rational limit. The Cauchy criterion lets us prove convergence without knowing the limit and is the basis of convergence tests for series. Convergence concepts for random variables in probability, such as almost sure convergence and convergence in probability, are built on these definitions applied to sequences of numbers and functions.
A series Σaₙ converges if its sequence of partial sums Sₙ = a₁ + … + aₙ converges. A necessary condition is aₙ → 0; it is not sufficient, as the harmonic series Σ1/n shows: its terms tend to zero but grouping terms (1/2, then 1/3 + 1/4 > 1/2, then 1/5 + … + 1/8 > 1/2, and so on) shows the partial sums grow without bound. The geometric series Σarⁿ⁻¹ converges to a ÷ (1 − r) when |r| < 1 and diverges otherwise. The p-series Σ1/nᵖ converges if and only if p > 1; Σ1/n² converges to π²/6. Telescoping series can be summed exactly: Σ1/(n(n + 1)) = Σ(1/n − 1/(n + 1)) = 1. Differentiating the geometric series gives Σnrⁿ = r ÷ (1 − r)², so Σn/2ⁿ = (1/2) ÷ (1/4) = 2, a sum that appears in the mean of the geometric distribution.
For series of non-negative terms, the comparison test states that if 0 ≤ aₙ ≤ bₙ and Σbₙ converges, then Σaₙ converges, and if Σaₙ diverges so does Σbₙ. The limit comparison test states that if aₙ/bₙ tends to a finite positive limit, the two series converge or diverge together; Σ1/(n² + 3n) behaves like Σ1/n² and converges. The ratio test (d'Alembert) uses L = lim |aₙ₊₁/aₙ|: the series converges absolutely if L < 1, diverges if L > 1, and the test is inconclusive if L = 1, as it is for every p-series. For Σn!/nⁿ the ratio tends to 1/e < 1, so the series converges. The root test (Cauchy) uses lim |aₙ|^(1/n) with the same conclusions and is stronger than the ratio test. The integral test states that if f is positive and decreasing, Σf(n) and ∫₁^∞ f(x)dx converge or diverge together, which proves the p-series result and shows, for example, that Σ1/(n ln n) diverges while Σ1/(n (ln n)²) converges.
A series converges absolutely if Σ|aₙ| converges; absolute convergence implies convergence. A series that converges but not absolutely converges conditionally. The alternating series (Leibniz) test states that if bₙ decreases to zero, Σ(−1)ⁿ⁺¹bₙ converges, and the error after n terms is at most bₙ₊₁. The alternating harmonic series 1 − 1/2 + 1/3 − … converges, to ln 2, but only conditionally. Riemann's rearrangement theorem shows that the terms of a conditionally convergent series can be rearranged to converge to any value or to diverge, whereas absolutely convergent series give the same sum in any order. This is why expectations in probability are defined only when Σ|x|p(x) converges: a discrete random variable whose expectation series converges only conditionally has no well-defined mean.
A power series Σaₙ(x − c)ⁿ is a series whose terms are powers of (x − c). For every power series there is a radius of convergence R, 0 ≤ R ≤ ∞, such that the series converges absolutely for |x − c| < R and diverges for |x − c| > R; at the endpoints |x − c| = R it may do either, and each endpoint must be checked separately. The Cauchy–Hadamard formula gives 1/R = lim sup |aₙ|^(1/n), and when the limit exists, R = lim |aₙ/aₙ₊₁|. Examples: Σxⁿ/n! has R = ∞ (it is eˣ); Σn!xⁿ has R = 0, converging only at x = 0; Σxⁿ/n has R = 1, diverges at x = 1 (harmonic series) and converges at x = −1 (alternating harmonic), so its interval of convergence is [−1, 1); Σ(x − 2)ⁿ/3ⁿ has R = 3 and converges on (−1, 5), diverging at both endpoints because the terms do not tend to zero.
Inside the radius of convergence a power series defines a function that is continuous and infinitely differentiable, and it may be differentiated and integrated term by term without changing the radius (though endpoint behaviour can change). Starting from the geometric series 1/(1 − x) = Σxⁿ for |x| < 1, differentiation gives 1/(1 − x)² = Σnxⁿ⁻¹, and integration of 1/(1 + x) = Σ(−1)ⁿxⁿ gives ln(1 + x) = x − x²/2 + x³/3 − …, valid for −1 < x ≤ 1. Standard expansions to remember are eˣ = Σxⁿ/n!, sin x = x − x³/3! + x⁵/5! − …, cos x = 1 − x²/2! + x⁴/4! − …, all with R = ∞, and the binomial series (1 + x)ᵅ = Σ C(α, n)xⁿ for |x| < 1 with general real α. A power series converges uniformly on every closed interval strictly inside its radius, which justifies these operations.
Generating functions are power series. The probability generating function of a non-negative integer random variable, G(s) = E(sˣ) = Σp(k)sᵏ, converges at least for |s| ≤ 1, and its derivatives at s = 1 give factorial moments: G′(1) = E(X). The moment generating function M(t) = E(eᵗˣ), when it exists in an open interval around zero, has the expansion Σ E(Xⁿ)tⁿ/n!, so moments are read from its coefficients. For the Poisson distribution, G(s) = e^(λ(s − 1)), whose derivative at s = 1 is λ. Knowing where these series converge is what determines whether the moment generating function exists, as it does not for heavy-tailed distributions such as the Cauchy or the lognormal.
The limit of f(x) as x → c is L if for every ε > 0 there is a δ > 0 such that |f(x) − L| < ε whenever 0 < |x − c| < δ. Equivalently, by the sequential criterion, f(xₙ) → L for every sequence xₙ → c with xₙ ≠ c; this form is convenient for proving that a limit does not exist, by finding two sequences giving different limits, as for sin(1/x) at 0. One-sided limits approach from the left or right; the two-sided limit exists if and only if both one-sided limits exist and are equal. A function is continuous at c if lim f(x) = f(c). Sums, products, quotients (with non-zero denominator) and compositions of continuous functions are continuous. Discontinuities are removable (the limit exists but differs from or lacks a function value), jump (one-sided limits exist but differ) or essential (a one-sided limit fails to exist). A monotone function can have only jump discontinuities, and at most countably many of them, which is why a cumulative distribution function, being non-decreasing, has at most countably many jumps; distribution functions are right-continuous.
On a closed bounded interval [a, b], a continuous function is bounded and attains its maximum and minimum (extreme value theorem), and takes every value between f(a) and f(b) (intermediate value theorem). The intermediate value theorem guarantees, for example, that x³ − x − 1 has a root between 1 and 2, because the function is negative at 1 and positive at 2, and it underlies the bisection method for solving equations. These results depend on the interval being closed and bounded: f(x) = 1/x is continuous on (0, 1] but unbounded, and f(x) = x on (0, 1) has no maximum.
A function is uniformly continuous on a set if one δ works for all points: for every ε > 0 there is a δ > 0 such that |f(x) − f(y)| < ε whenever |x − y| < δ, for all x and y in the set. Ordinary continuity allows δ to depend on the point. Every continuous function on a closed bounded interval is uniformly continuous (the Heine–Cantor theorem). On unbounded or open sets this can fail: f(x) = x² is continuous on R but not uniformly continuous, because its slope grows without bound, and f(x) = 1/x is not uniformly continuous on (0, 1), though it is on [1, ∞). A function satisfying a Lipschitz condition |f(x) − f(y)| ≤ K|x − y| is uniformly continuous, and a differentiable function with a bounded derivative is Lipschitz by the mean value theorem; sin x is therefore uniformly continuous on R. Uniform continuity and its relative, uniform convergence of sequences of functions, are the conditions under which limits can be exchanged with integrals and derivatives, an operation statisticians perform when they differentiate under the integral sign to derive properties of estimators.
The derivative of f at c is f′(c) = lim [f(c + h) − f(c)] ÷ h, when the limit exists. Differentiability implies continuity, but not conversely: |x| is continuous at 0 but not differentiable there, because the left and right difference quotients tend to −1 and +1. The rules for sums, products, quotients and compositions (chain rule) follow from the definition. A function with a positive derivative throughout an interval is strictly increasing there; one with a non-negative second derivative is convex, lying below its chords, and Jensen's inequality, E[g(X)] ≥ g(E[X]) for convex g, is one of the most used results in statistics. Derivatives need not be continuous, but they have the intermediate value property (Darboux's theorem).
Rolle's theorem states that if f is continuous on [a, b], differentiable on (a, b) and f(a) = f(b), then f′(c) = 0 for some c in (a, b). For f(x) = x² − 4x + 3 on [1, 3], f(1) = f(3) = 0 and f′(c) = 2c − 4 = 0 at c = 2. The Lagrange mean value theorem removes the requirement f(a) = f(b): there is a c in (a, b) with f′(c) = [f(b) − f(a)] ÷ (b − a), the slope of the chord. For f(x) = x³ on [0, 2], the chord slope is 8 ÷ 2 = 4, and 3c² = 4 gives c = 2/√3 ≈ 1.155. Cauchy's mean value theorem states that for two functions f and g, [f(b) − f(a)]g′(c) = [g(b) − g(a)]f′(c) for some c, and is the basis of L'Hospital's rule. The mean value theorem turns information about derivatives into inequalities: |sin x − sin y| ≤ |x − y| because |cos c| ≤ 1, and a function with zero derivative on an interval is constant.
Taylor's theorem approximates a function near a point by a polynomial. If f has n + 1 derivatives near c, then f(x) = f(c) + f′(c)(x − c) + f″(c)(x − c)²/2! + … + f⁽ⁿ⁾(c)(x − c)ⁿ/n! + Rₙ, where the Lagrange form of the remainder is Rₙ = f⁽ⁿ⁺¹⁾(ξ)(x − c)ⁿ⁺¹/(n + 1)! for some ξ between c and x. With c = 0 the expansion is called Maclaurin's. The remainder bounds the error: approximating e^0.1 by 1 + 0.1 + 0.005 = 1.105 leaves an error of at most e^0.1(0.1)³/6 < 0.0002. In statistics, Taylor expansions give the delta method, which approximates the variance of a function of an estimator, Var[g(T)] ≈ [g′(θ)]² Var(T); they justify the Newton–Raphson algorithm for maximum likelihood; and second-order expansions of the log-likelihood around its maximum give the asymptotic normality of maximum likelihood estimators.
L'Hospital's rule states that if f(x) and g(x) both tend to 0, or both tend to ±∞, as x → c, and f′(x)/g′(x) tends to a limit L (finite or infinite), then f(x)/g(x) also tends to L, provided g′(x) ≠ 0 near c. The point c may be finite or infinite and the limit one-sided. The rule can be applied repeatedly while the form remains indeterminate. Examples: (sin x)/x → cos 0 / 1 = 1 as x → 0; (1 − cos x)/x² → (sin x)/(2x) → 1/2; (eˣ − 1 − x)/x² → (eˣ − 1)/(2x) → 1/2; and (ln x)/x → (1/x)/1 → 0 as x → ∞, showing that logarithms grow more slowly than any power.
Forms other than 0/0 and ∞/∞ are converted to them first. A product of the form 0 · ∞ is rewritten as a quotient: x ln x as x → 0⁺ equals (ln x)/(1/x), which by the rule tends to (1/x)/(−1/x²) = −x → 0. A difference ∞ − ∞ is combined over a common denominator: 1/x − 1/sin x = (sin x − x)/(x sin x) → 0 as x → 0. The exponential forms 1^∞, 0⁰ and ∞⁰ are handled by taking logarithms: for y = (1 + a/x)ˣ, ln y = x ln(1 + a/x) = ln(1 + a/x)/(1/x), which tends to a, so y → eᵃ; with a = 3 the limit is e³ ≈ 20.09. Similarly xˣ → 1 as x → 0⁺, because x ln x → 0. These limits appear in deriving the Poisson limit of the binomial and the exponential limit of geometric waiting times.
The rule requires both conditions: an indeterminate form and the existence of the limit of the ratio of derivatives. Applying it to a determinate form gives wrong answers: (x + 1)/x as x → 0⁺ is +∞, not the ratio of derivatives 1/1. And when the ratio of derivatives has no limit, the original limit may still exist: (x + sin x)/x → 1 as x → ∞, though (1 + cos x)/1 oscillates. Often an algebraic simplification or a Taylor expansion is quicker and safer than repeated differentiation, especially for limits such as (sin x − x)/x³, where the expansion sin x = x − x³/6 + … gives −1/6 at once.
For a bounded function f on [a, b], a partition P divides the interval into subintervals; the upper sum U(P, f) uses the supremum of f on each subinterval and the lower sum L(P, f) the infimum. Refining a partition lowers upper sums and raises lower sums. The function is Riemann integrable if the infimum of upper sums equals the supremum of lower sums; equivalently (Riemann's criterion), for every ε > 0 there is a partition with U − L < ε. Continuous functions on [a, b] are integrable, as are monotone functions and bounded functions with finitely many discontinuities; more generally, a bounded function is Riemann integrable if and only if its set of discontinuities has measure zero (Lebesgue's criterion). Dirichlet's function, equal to 1 on rationals and 0 on irrationals, is not Riemann integrable, since every upper sum is b − a and every lower sum 0; it is integrable in Lebesgue's theory, which measure-theoretic probability uses. Riemann sums Σf(tᵢ)Δxᵢ converge to the integral for integrable functions, which is how integrals are approximated numerically.
The fundamental theorem of calculus states that if f is continuous on [a, b], then F(x) = ∫ₐˣ f(t)dt is differentiable with F′(x) = f(x), and that ∫ₐᵇ f(x)dx = G(b) − G(a) for any antiderivative G. In probability, this is the relation between a density and its distribution function: F′(x) = f(x) where f is continuous. Leibniz's rule for differentiating an integral with variable limits, d/dx ∫ from a(x) to b(x) of f(t)dt = f(b(x))b′(x) − f(a(x))a′(x), is used to find the densities of order statistics and of functions of random variables. Integration by parts, ∫u dv = uv − ∫v du, and substitution are the main techniques; repeated integration by parts gives ∫₀^∞ xⁿe^(−x)dx = n!.
An improper integral has an infinite limit of integration (type I) or an integrand unbounded near a point (type II), and is defined as a limit of proper integrals; it converges if the limit is finite. Key results: ∫₁^∞ x^(−p)dx converges if and only if p > 1 (∫₁^∞ x^(−2)dx = 1), while ∫₀¹ x^(−p)dx converges if and only if p < 1 (∫₀¹ x^(−1/2)dx = 2). Comparison tests work as for series: if 0 ≤ f ≤ g and ∫g converges, so does ∫f. An integral converges absolutely if ∫|f| converges; absolute convergence implies convergence. The integral ∫₀^∞ e^(−x²)dx = √π/2 is improper but convergent, and is the normalising constant behind the normal distribution. Every probability density must integrate to 1 over its support, which requires the improper integral to converge, and moments exist only when ∫|x|ᵏf(x)dx converges: the Cauchy density 1/[π(1 + x²)] integrates to 1, but ∫|x|/[π(1 + x²)]dx diverges, so it has no mean.
The gamma function is defined for α > 0 by Γ(α) = ∫₀^∞ x^(α−1)e^(−x)dx, an improper integral that converges for every positive α (near 0 the integrand behaves like x^(α−1), integrable for α > 0, and at infinity the exponential dominates). Integration by parts gives the recurrence Γ(α + 1) = αΓ(α), and since Γ(1) = 1, Γ(n) = (n − 1)! for positive integers: Γ(5) = 24. A key value is Γ(1/2) = √π, obtained from the Gaussian integral by substituting x = t²; then Γ(3/2) = √π/2 and Γ(5/2) = (3/2)(1/2)√π = 3√π/4 ≈ 1.329. A useful general form, from the substitution y = βx, is ∫₀^∞ x^(α−1)e^(−βx)dx = Γ(α)/βᵅ; for example, ∫₀^∞ x³e^(−2x)dx = Γ(4)/2⁴ = 6/16 = 0.375. This integral is the normalising constant of the gamma distribution, and of the exponential and chi-square distributions that are special cases of it. Stirling's approximation, n! ≈ √(2πn)(n/e)ⁿ, is accurate to about 1% at n = 10 and is used in large-sample approximations.
The beta function is defined for a, b > 0 by B(a, b) = ∫₀¹ x^(a−1)(1 − x)^(b−1)dx. It is symmetric, B(a, b) = B(b, a), and is related to the gamma function by B(a, b) = Γ(a)Γ(b)/Γ(a + b), a result proved by writing the product Γ(a)Γ(b) as a double integral and changing variables. Thus B(2, 3) = Γ(2)Γ(3)/Γ(5) = 1 × 2 / 24 = 1/12, and B(3/2, 3/2) = [Γ(3/2)]²/Γ(3) = (π/4)/2 = π/8. Other forms include B(a, b) = 2∫₀^(π/2) sin^(2a−1)θ cos^(2b−1)θ dθ, which evaluates many trigonometric integrals, and B(a, b) = ∫₀^∞ x^(a−1)/(1 + x)^(a+b) dx.
Gamma and beta functions appear as normalising constants of the gamma, chi-square, beta, Student's t and F distributions, and in their moments. The mean of a beta(a, b) random variable is B(a + 1, b)/B(a, b) = a/(a + b), computed directly from the recurrence. The moments of a gamma(α, β) variable are E(Xᵏ) = Γ(α + k)/[Γ(α)βᵏ], giving mean α/β and variance α/β² in the rate parameterisation. Integrals that look forbidding, such as ∫₀¹ x⁴(1 − x)³dx = B(5, 4) = 4!3!/8! = 1/280, become routine once recognised as beta or gamma integrals, which is why examinations favour them.
For a function f(x, y), the partial derivative ∂f/∂x is the derivative with respect to x holding y fixed. Limits in two variables must hold along every path: f(x, y) = xy/(x² + y²) has no limit at the origin, because along y = mx it equals m/(1 + m²), which depends on m. Existence of partial derivatives does not imply continuity; differentiability, meaning that f is well approximated near a point by a linear function, does, and continuous partial derivatives guarantee differentiability. If the mixed second partials are continuous, they are equal (Schwarz's or Young's theorem): f_xy = f_yx. The gradient ∇f collects the first partial derivatives and points in the direction of steepest increase. The chain rule for z = f(x(t), y(t)) gives dz/dt = f_x x′ + f_y y′. The Jacobian of a transformation (u, v) = (u(x, y), v(x, y)) is the determinant of the matrix of partial derivatives, ∂(u, v)/∂(x, y); it measures how the transformation scales areas and must be non-zero for the transformation to be locally invertible.
At an interior local maximum or minimum of a differentiable function, all first partial derivatives are zero; such points are called critical or stationary points. The second-derivative test in two variables uses D = f_xx f_yy − (f_xy)² at the critical point: if D > 0 and f_xx > 0 the point is a local minimum; if D > 0 and f_xx < 0, a local maximum; if D < 0, a saddle point; if D = 0, the test is inconclusive. More generally, the Hessian matrix of second derivatives is positive definite at a strict local minimum and negative definite at a strict local maximum. For f(x, y) = x² + y² − 2x − 4y, setting f_x = 2x − 2 = 0 and f_y = 2y − 4 = 0 gives (1, 2), with D = 4 > 0 and f_xx = 2 > 0, a minimum of value −5. Maximum likelihood estimation solves exactly this kind of problem: the score equations set the partial derivatives of the log-likelihood to zero, and a negative definite Hessian confirms a maximum; the negative of the expected Hessian is the Fisher information.
To find extrema of f(x, y) subject to a constraint g(x, y) = c, the method of Lagrange multipliers sets ∇f = λ∇g together with the constraint. At an extremum the level curve of f is tangent to the constraint curve, so their gradients are parallel. To maximise xy subject to x + y = 10: y = λ and x = λ, so x = y = 5 and the maximum is 25. With several constraints, one multiplier is introduced for each. In statistics, Lagrange multipliers give the maximum likelihood estimates of multinomial probabilities under the constraint that they sum to one (the answer is the observed proportions), optimum allocation of a sample among strata to minimise variance for a fixed cost (Neyman and cost-optimal allocation), and the derivation of best linear unbiased estimators.
A double integral over a rectangle or a more general region is defined through Riemann sums over small rectangles and is evaluated, under mild conditions, as an iterated integral (Fubini's theorem): ∫∫_R f dA = ∫ₐᵇ [∫ f(x, y)dy] dx, with inner limits that may depend on x. Over the triangle 0 ≤ y ≤ x ≤ 1, ∫₀¹∫₀ˣ xy dy dx = ∫₀¹ x · x²/2 dx = 1/8. Reversing the order of integration requires redescribing the region: the same triangle is y ≤ x ≤ 1 for 0 ≤ y ≤ 1. Choosing the order well can turn an impossible integral into an easy one, as with ∫₀¹∫ᵧ¹ e^(x²)dx dy, which becomes ∫₀¹ x e^(x²) dx = (e − 1)/2 after reversal. In probability, joint densities are integrated over regions to find probabilities such as P(X < Y), and marginal densities are obtained by integrating out one variable.
When variables are transformed, (x, y) = (x(u, v), y(u, v)), the area element changes by the absolute value of the Jacobian: dx dy = |∂(x, y)/∂(u, v)| du dv. For polar coordinates, x = r cos θ and y = r sin θ, the Jacobian is r, so dx dy = r dr dθ. This gives the Gaussian integral: ∫∫ over R² of e^(−(x² + y²))dx dy = ∫₀^(2π)∫₀^∞ e^(−r²) r dr dθ = 2π × 1/2 = π, so ∫ from −∞ to ∞ of e^(−x²)dx = √π. For the linear transformation u = x + y, v = x − y, the Jacobian ∂(u, v)/∂(x, y) = −2, so |∂(x, y)/∂(u, v)| = 1/2. In three dimensions, spherical coordinates give dV = r² sin φ dr dφ dθ, from which the volume of a ball of radius a is 4πa³/3, and cylindrical coordinates give dV = r dr dθ dz.
The change-of-variables formula is the standard method for finding the joint density of functions of random variables. If (U, V) is a one-to-one differentiable transformation of (X, Y) with joint density f_X,Y, then the joint density of (U, V) is f_U,V(u, v) = f_X,Y(x(u, v), y(u, v)) |∂(x, y)/∂(u, v)|. Applying it to two independent standard normal variables with polar coordinates shows that R² is exponential and the angle uniform, the basis of the Box–Muller method for generating normal random numbers. Applying it to independent gamma variables X and Y with U = X + Y and V = X/(X + Y) shows that U is gamma and V is beta, independently, which is one route to the relation B(a, b) = Γ(a)Γ(b)/Γ(a + b). Derivations of the t and F distributions use the same technique, so fluency with Jacobians is essential for distribution theory.
A statistics student uses real analysis in every course that follows: proving that an estimator is consistent means showing that a sequence converges; checking that a density is valid means evaluating an improper integral; finding moments uses gamma and beta integrals; deriving the distribution of a transformed variable needs a Jacobian; and maximum likelihood needs partial derivatives, the Hessian and sometimes Lagrange multipliers. In entrance examinations, the student must decide quickly whether a series converges, find a radius of convergence, evaluate a limit by L'Hospital's rule or a Taylor expansion, and recognise an integral as a beta or gamma function.
A statistician five years into practice uses analysis less explicitly but constantly: judging whether an iterative estimation algorithm will converge, choosing transformations and checking that the delta-method approximations used for standard errors are reasonable, understanding why a model's likelihood has no maximum in some data sets, and reading methods papers whose results rest on limit theorems. When software warns of non-convergence or a singular Hessian, the analyst who understands the underlying mathematics can diagnose the problem rather than guess.
A senior statistician with twenty-five years of experience reviews the methods of others, decides whether an asymptotic approximation is trustworthy for the sample sizes in agricultural trials and surveys, and judges whether new methods proposed by vendors or collaborators rest on sound mathematics. Such a person sets standards for analysis in an institute or agency, teaches younger colleagues to check assumptions such as the existence of moments, and explains clearly to scientists why some apparently simple quantities, such as the mean of a ratio, may not exist.
A researcher with a PhD and long experience uses real analysis as the working language of theory: proving consistency and asymptotic normality of new estimators, establishing convergence of algorithms such as EM and Markov chain Monte Carlo, bounding approximation errors with Taylor remainders, and moving into measure-theoretic probability where Riemann integration gives way to Lebesgue integration. Such researchers set and mark the analysis questions of admission and fellowship examinations and supervise theses whose central results are limit theorems.
This manual covers real analysis to the level of undergraduate statistics and graduate entrance examinations. It does not cover metric spaces in general, uniform convergence of sequences of functions in depth, the Riemann–Stieltjes integral, Lebesgue measure and integration, or Fourier series, which later courses in probability and asymptotic theory require. Proofs are sketched rather than given in full; students preparing for proof-based examinations should work through a standard analysis text.
Original practice questions written to the syllabus and pattern of the exams named in each question's examTags. They are not past papers and do not reproduce any board's questions.
These notes are free to read here, with no account.
What the Laisha app adds:
Free for ten days, then a subscription. We would rather write that here than have you find it later.