Agricultural Statistics & Data Science · Time series, forecasting and modelling · also Impact Evaluation Methods, Advanced Econometric Methods for Agriculture
Did the scheme, technology or variety cause the change? Potential outcomes, matching and weighting, difference-in-differences, instrumental variables, regression discontinuity and synthetic control, panel data models, quantile regression, and stochastic frontier and DEA efficiency analysis. Classical econometrics (OLS violations, simultaneous equations) is in the Agribusiness track and is assumed here.
In the exams: Not an explicit unit in AICE-JRF Ag Stat; AICE-JRF Ag Econ research methodology (present); RBI Grade B DSIM (reported); research and evaluation roles (applied)
Agricultural policy and research keep asking causal questions. Did a soil health card scheme raise yields, or did the farmers who used it already farm better? Did a new variety raise income, or did it spread first in irrigated villages? Did a market reform raise farm-gate prices? A regression of outcome on treatment answers a different question, how outcomes differ between those who were and were not treated, and the two answers coincide only under conditions that rarely hold by accident. Causal inference is the set of designs and estimators that make those conditions explicit and, where possible, satisfy them.
This subject starts from the potential-outcomes framework and the identification problem, then covers the main designs: randomised evaluation, matching and weighting on the propensity score, difference-in-differences and event studies, instrumental variables, regression discontinuity and synthetic control. It then treats panel data models, which use repeated observations of the same farms, districts or countries to control for unobserved, time-invariant differences, with an introduction to dynamic panels estimated by GMM. Quantile regression shows how effects differ across the distribution of outcomes. The last two chapters cover efficiency analysis, stochastic frontier production functions and data envelopment analysis, which measure how far farms fall short of the best observed practice.
The classical regression model and its violations (heteroscedasticity, autocorrelation, endogeneity, simultaneous equations) are taught in the Agribusiness track's Econometrics subject and in this track's Linear Models and Regression; they are assumed here. In examinations these topics appear in econometrics and research-methodology papers, such as central-bank and statistical-service papers and agricultural economics PhD entrance; in practice they are the toolkit of every impact evaluation of an agricultural scheme and every farm-efficiency study.
For each unit i (a farm, a village) and a binary treatment D (adopting drip irrigation, receiving a subsidy), define two potential outcomes: Y(i,1), the outcome if treated, and Y(i,0), the outcome if not treated. The causal effect for unit i is Y(i,1) - Y(i,0). We observe only one of the two: Y(i) = D(i) Y(i,1) + (1 - D(i)) Y(i,0). The unobserved one is the counterfactual. This is the fundamental problem of causal inference: individual effects can never be observed directly, so we estimate averages. The average treatment effect is ATE = E[Y(1) - Y(0)] over the whole population; the average treatment effect on the treated is ATT = E[Y(1) - Y(0) | D = 1], usually the policy-relevant quantity for a voluntary scheme; and the effect on the untreated (ATU) concerns extending a scheme to those not yet in it. The framework, associated with Neyman and Rubin, requires the stable unit treatment value assumption (SUTVA): one unit's outcome does not depend on others' treatment (no spillovers), and there is only one version of the treatment. Spillovers, such as neighbours copying a demonstrated practice, violate SUTVA and must be designed for.
The naive comparison of mean outcomes between treated and untreated units decomposes as E[Y | D = 1] - E[Y | D = 0] = ATT + {E[Y(0) | D = 1] - E[Y(0) | D = 0]}. The term in braces is selection bias: the difference the two groups would have shown even without treatment. Farmers who adopt a new technology are often better educated, richer and on better land, so their untreated yields would also have been higher; the naive comparison then overstates the effect. Selection can also go the other way: a relief scheme targeted at drought-hit villages will show worse outcomes among recipients, even if it helped. Every causal design is a strategy to eliminate this bias by an assumption that makes the counterfactual mean estimable.
Each design rests on an identifying assumption. Randomisation guarantees that treatment is independent of potential outcomes. Selection on observables (unconfoundedness, the conditional independence assumption) assumes that, given observed covariates X, treatment is as good as random; it also needs overlap (common support), meaning every type of unit has a positive probability of being treated and untreated. Difference-in-differences assumes parallel trends; instrumental variables assume a variable that shifts treatment but affects the outcome only through it; regression discontinuity assumes continuity of potential outcomes at a cut-off. Directed acyclic graphs (DAGs) help decide which variables to control for. A confounder, a common cause of treatment and outcome, must be controlled. A mediator, on the path from treatment to outcome, should not be controlled when the total effect is wanted. A collider, a common effect of treatment and outcome, must not be controlled, because conditioning on it creates a spurious association. Controlling for every available variable is therefore not a safe default.
When treatment is assigned at random, it is independent of both potential outcomes, so E[Y(0) | D = 1] = E[Y(0) | D = 0] and selection bias is zero in expectation. The difference in means then estimates the ATE without bias, and regression with pre-treatment covariates can improve precision without introducing bias. Field experiments in agricultural economics randomise information campaigns, input subsidies, credit and insurance offers, extension methods and new technologies among farmers or villages. Stratified (blocked) randomisation, assigning treatment within strata such as districts or farm-size classes, guarantees balance on those variables and improves precision, the same idea as blocking in agricultural field experiments. A balance table comparing baseline characteristics across arms checks that randomisation was carried out correctly.
Many agricultural interventions operate at village or group level, and spillovers between neighbouring farmers make individual randomisation impossible, so whole clusters are randomised. Outcomes within a cluster are correlated, measured by the intra-cluster correlation rho, and this inflates the variance of the estimated effect by the design effect, 1 + (m - 1) rho, where m is the number of units sampled per cluster. With 20 farmers per village and rho = 0.1, the design effect is 2.9: the study needs almost three times as many farmers as an individually randomised one for the same precision. Adding clusters raises power much more than adding farmers per cluster. Before a study, a power calculation sets the minimum detectable effect: for a two-arm trial with equal allocation, it is about (z(1 - alpha/2) + z(1 - beta)) times the standard error of the difference, roughly 2.8 standard errors for 5 per cent two-sided significance and 80 per cent power. Standard errors in the analysis must be clustered at the level of randomisation.
Not everyone offered a treatment takes it up, and some in the control group may obtain it elsewhere. The comparison of groups as randomised gives the intention-to-treat (ITT) effect, the effect of offering the programme, which is unbiased whatever the take-up. The effect of actually receiving treatment can be recovered by using random assignment as an instrument (Chapter 5): with one-sided non-compliance, the effect on the treated is ITT divided by the take-up rate. Attrition, losing respondents between baseline and follow-up, biases results if it differs between arms or is related to outcomes; bounds (Lee bounds) give a range under plausible assumptions. Spillovers to control units bias the comparison towards zero; designs that randomise the share of treated farmers within villages can measure them.
Randomised evaluations answer a specific question in a specific place and time. External validity, whether the effect holds elsewhere, depends on the mechanism: a fertiliser subsidy's effect depends on soils, rainfall, prices and the availability of credit. Many agricultural questions cannot be randomised: the effect of rainfall, of a national price policy, of irrigation canals built decades ago, or of land reform. Experiments may also be too small to detect plausible effects on noisy outcomes such as yield, may change behaviour simply by being observed (Hawthorne effects), and raise ethical questions about withholding a beneficial treatment. Experimental and observational designs are therefore complements, and the rest of this subject covers the observational designs that apply when randomisation is impossible.
Under selection on observables, comparing treated and untreated units with the same covariates X removes selection bias. With many covariates, exact matching is impossible. Rosenbaum and Rubin (1983) showed that the propensity score, e(X) = P(D = 1 | X), the probability of treatment given covariates, is a balancing score: conditional on e(X), the distribution of X is the same in treated and untreated groups, and if treatment is unconfounded given X it is unconfounded given e(X). This reduces matching on many variables to matching on one. The score is usually estimated by logit or probit regression of treatment on pre-treatment covariates; flexible methods such as boosted trees can also be used. Only variables measured before treatment, and not affected by it, may enter; including outcomes or post-treatment variables destroys the design.
Matching estimators pair each treated unit with one or more untreated units with similar propensity scores, and estimate the ATT as the mean difference in outcomes. Nearest-neighbour matching uses the closest control, with or without replacement; a caliper discards matches whose scores differ by more than a set amount (a common choice is 0.2 standard deviations of the logit of the score). Radius matching uses all controls within the caliper; kernel matching weights all controls by their closeness. Common support is imposed by dropping treated units whose scores lie outside the range of the controls; the estimate then applies only to the units that remain, which must be reported. After matching, balance is checked by comparing covariate means: the standardised mean difference, the difference in means divided by a pooled standard deviation, should be small (below 0.1 is a common rule of thumb). Hypothesis tests of balance are not appropriate, because they depend on sample size.
Inverse probability weighting (IPW) uses the score as a weight instead of matching. To estimate the ATE, each treated unit is weighted by 1/e(X) and each untreated unit by 1/(1 - e(X)), creating a pseudo-population in which treatment is independent of X. For the ATT, treated units get weight 1 and untreated units e(X)/(1 - e(X)), making the controls resemble the treated. Scores near 0 or 1 give extreme weights and unstable estimates, a sign of poor overlap; weights are sometimes trimmed or stabilised. Doubly robust estimators, such as augmented IPW, combine a model for the outcome with a model for the propensity score and remain consistent if either model is correctly specified.
Propensity score methods adjust only for observed covariates. If an unobserved variable, such as managerial ability or access to an influential village leader, affects both adoption and yield, the estimate remains biased, no matter how good the balance on observed variables looks. Matching is not a remedy for unobserved confounding; it is a careful way of using the selection-on-observables assumption. Sensitivity analysis, for example Rosenbaum bounds, asks how strong an unobserved confounder would have to be to overturn the conclusion. Combining matching with difference-in-differences (matching on baseline characteristics and then differencing) relaxes the assumption to unobservables that are constant over time.
Difference-in-differences (DiD) compares the change in outcome over time in a treated group with the change in an untreated group. With two groups and two periods, the estimate is (Y(treated, after) - Y(treated, before)) - (Y(control, after) - Y(control, before)). The first difference removes fixed differences between units; the second removes common shocks that affect both groups, such as a drought year or a national price change. The key assumption is parallel trends: without treatment, the outcome would have changed by the same amount in both groups. The design is natural for schemes introduced in some districts and not others, or at a known date. In regression form, Y(it) = alpha + beta Treat(i) + gamma Post(t) + delta Treat(i) x Post(t) + error, and delta is the DiD estimate; with many units and periods, unit and time fixed effects replace Treat and Post.
Parallel trends cannot be tested for the post-treatment period, but pre-treatment trends can be examined. An event-study regression replaces the single interaction with a set of indicators for each period relative to treatment (leads before and lags after), omitting one pre-treatment period as the reference. Lead coefficients near zero support (though do not prove) parallel trends; significant leads suggest anticipation or different pre-existing trends. Lag coefficients show how the effect evolves over time, for example a yield effect that builds over several seasons as farmers learn a new practice. Plotting the coefficients with confidence intervals is standard. Group-specific linear trends are sometimes added, but they can absorb part of the treatment effect and should be used with care.
When units adopt treatment at different times, the usual two-way fixed-effects (TWFE) regression is a weighted average of many two-by-two comparisons, some of which use already-treated units as controls for later-treated ones. Goodman-Bacon's decomposition showed that when effects change over time, these comparisons can receive negative weights and the TWFE estimate can be badly biased, even of the wrong sign. Newer estimators avoid this by comparing each adoption cohort only with not-yet-treated or never-treated units and then aggregating; examples include the methods of Callaway and Sant'Anna, of Sun and Abraham, and of de Chaisemartin and D'Haultfoeuille. For schemes rolled out across districts over several years, as many agricultural programmes are, these estimators should be preferred to a single TWFE coefficient.
Outcomes of the same unit over time are serially correlated, and Bertrand, Duflo and Mullainathan showed that ignoring this makes DiD standard errors far too small. Standard errors should be clustered at the level at which treatment is assigned (for example the district), and with few clusters (below about 30 to 50) wild cluster bootstrap methods are preferred. Triple differences (DDD) add a third comparison, for example crops covered and not covered by a scheme within treated and untreated districts, relaxing the parallel-trends assumption. Changes in the composition of the groups over time, such as migration of farmers into treated districts, and spillovers to control districts threaten the design and should be examined.
When treatment is correlated with unobserved determinants of the outcome (endogeneity), ordinary least squares is biased. An instrument Z is a variable that shifts treatment but is otherwise unrelated to the outcome. It must satisfy relevance, a non-zero effect on the treatment, which can be tested; independence, being as good as randomly assigned; and the exclusion restriction, affecting the outcome only through the treatment, which cannot be tested and must be argued. In agriculture, candidate instruments include random offers or encouragement (lotteries for subsidies), distance to an input dealer or extension centre, rainfall shocks, and policy rules; each needs a defence of exclusion, for example distance to a market may affect outcomes through prices as well as through adoption.
With a binary instrument and a binary treatment, the Wald estimator is the effect of Z on Y divided by the effect of Z on D: beta(IV) = [E(Y | Z = 1) - E(Y | Z = 0)] / [E(D | Z = 1) - E(D | Z = 0)], the reduced form over the first stage. If a lottery offer raises yields by 0.3 tonnes per hectare on average and raises adoption by 40 percentage points, the IV estimate is 0.3/0.4 = 0.75 tonnes per hectare for those induced to adopt. With continuous variables and covariates, two-stage least squares (2SLS) regresses D on Z and covariates in the first stage, then regresses Y on the fitted values of D and covariates in the second. The second-stage standard errors must be computed from the correct residuals, as software does; manual two-step fitting gives wrong standard errors. With more instruments than endogenous variables, overidentification tests (Sargan or Hansen J) check whether instruments agree with each other, though agreement does not prove validity.
If the instrument barely moves the treatment, IV estimates are biased towards OLS, their sampling distribution is far from normal and confidence intervals are unreliable. The first-stage F-statistic for excluded instruments measures strength; Staiger and Stock's rule of thumb treats F below 10 as weak, and later work suggests higher thresholds for reliable inference. With weak instruments, inference robust to weakness (the Anderson-Rubin test) should be used. A small violation of exclusion is also magnified when the first stage is weak, because the bias is divided by a small first-stage effect.
When treatment effects differ across units, Imbens and Angrist (1994) showed what IV estimates. Units fall into compliers (treated if and only if Z = 1), always-takers, never-takers and defiers (who do the opposite of Z). Under monotonicity, no defiers, the IV estimand is the local average treatment effect (LATE): the average effect for compliers. This is a well-defined causal effect, but for a particular subgroup, those whose adoption responds to the instrument, which may differ from the population of interest. A subsidy lottery identifies the effect for farmers who adopt only when subsidised, not for those who would adopt anyway. Different instruments can therefore give different, equally valid, LATEs.
Many agricultural programmes assign benefits by a rule on a continuous score: farmers with up to 2 hectares are eligible for a scheme; villages above a population threshold get a market yard; districts below a rainfall percentage are declared drought-affected. Regression discontinuity (RD) compares units just above and just below the cut-off. In a sharp design, treatment switches from 0 to 1 exactly at the cut-off, and the effect at the cut-off is the jump in the expected outcome there. In a fuzzy design, the probability of treatment jumps but not from 0 to 1 (not every eligible farmer enrols); the effect is the jump in the outcome divided by the jump in treatment probability, an IV estimate using eligibility as the instrument, and it is a LATE for compliers at the cut-off.
The identifying assumption is that potential outcomes change smoothly with the score at the cut-off, so that any jump in outcomes is due to treatment. The estimate is local: it applies to units near the cut-off and may not generalise to farms far from it. Estimation uses local linear (or low-order polynomial) regression on each side within a bandwidth around the cut-off, usually with a triangular kernel giving more weight to observations near it. Bandwidth choice trades bias (wide bandwidth) against variance (narrow bandwidth); data-driven optimal bandwidths and robust bias-corrected confidence intervals (associated with Imbens and Kalyanaraman and with Calonico, Cattaneo and Titiunik) are standard. High-order global polynomials fitted to the whole range are discouraged, because they give noisy and misleading estimates at the boundary.
The design fails if units can manipulate their score to fall on the favourable side, for example landholdings re-registered just under the eligibility limit. A density test (McCrary's test, or its refinements) checks for bunching of observations just on one side of the cut-off. Pre-treatment covariates should show no jump at the cut-off; a jump suggests sorting. Placebo cut-offs, where no treatment change occurs, should show no effect. Results should be shown for several bandwidths, and a plot of binned means of the outcome against the score, with fitted lines on each side, should accompany every RD analysis. Where several programmes use the same cut-off, the estimated jump combines their effects.
Some policies affect a single unit or very few: one state introduces a new procurement policy, one country bans an export, one district pilots digital land records. DiD needs a comparison group with parallel trends, but no single untreated unit may resemble the treated one. The synthetic control method of Abadie and Gardeazabal (2003) and Abadie, Diamond and Hainmueller (2010) builds a comparison as a weighted average of untreated units (the donor pool) chosen so that the synthetic unit closely reproduces the treated unit's outcome path and key predictors before treatment. The effect in each post-treatment period is the gap between the treated unit's outcome and the synthetic control's.
The weights are non-negative and sum to one, so the synthetic control is a convex combination of donors and does not extrapolate outside their range; usually only a few donors receive positive weight, which makes the comparison transparent. Weights are chosen to minimise the distance between treated and synthetic pre-treatment characteristics (lagged outcomes and covariates), with a matrix of importance weights for the predictors. The quality of the pre-treatment fit, summarised by the root mean squared prediction error (RMSPE) over the pre-treatment period, is essential: a poor fit means the method has not found a credible counterfactual. Long pre-treatment periods give more confidence that the synthetic control tracks the underlying factors, not just noise.
With one treated unit, standard sampling-based inference does not apply. Inference is by placebo (permutation) tests: the method is applied to each donor in turn as if it had been treated, and the treated unit's effect is compared with the distribution of placebo effects. A common statistic is the ratio of post-treatment to pre-treatment RMSPE, which accounts for placebos with poor pre-period fit; the rank of the treated unit among all units gives a permutation p-value. Placebos in time, pretending treatment began earlier, check for spurious effects. The method requires that donors are not affected by the treatment (no spillovers, which are common when one state's policy changes trade flows to its neighbours) and that the treated unit is not extreme relative to the donors. Extensions include augmented and synthetic DiD estimators that combine the ideas of synthetic control and difference-in-differences.
Panel (longitudinal) data follow the same units over time: farm households in repeated survey rounds, districts over years, countries over decades. The basic model is y(it) = x(it)'beta + a(i) + u(it), where a(i) is an unobserved unit-specific effect, constant over time, such as soil quality or management ability, and u(it) the idiosyncratic error. If a(i) is correlated with x(it), pooled OLS (ignoring a(i)) is biased: farmers on better land use more fertiliser, so the fertiliser coefficient picks up land quality. The great advantage of panel data is that time-invariant unobservables can be removed without being measured. A balanced panel has every unit observed in every period; unbalanced panels are common in farm surveys and can be handled, provided attrition is not related to the errors.
The fixed-effects (within) estimator removes a(i) by subtracting each unit's time average from its observations: y(it) - y-bar(i) = (x(it) - x-bar(i))'beta + (u(it) - u-bar(i)). OLS on the demeaned data gives beta-hat(FE), which is consistent even when a(i) is correlated with the regressors. It is identical to least-squares dummy variables (LSDV), a regression with one dummy per unit. First differencing, y(it) - y(i,t-1), also removes a(i); with two periods it gives exactly the same estimate as the within estimator, but with more periods the two differ in efficiency depending on the error structure. Fixed effects cannot estimate the effects of variables that do not change over time (soil type, caste, distance to the state capital), because they are absorbed by a(i), and they rely only on within-unit variation, which may be small and measured with error. Time fixed effects (one dummy per period) remove common shocks; two-way fixed effects include both.
The random-effects model treats a(i) as a random variable uncorrelated with all regressors in all periods. The composite error a(i) + u(it) is then serially correlated within units, and feasible generalised least squares (GLS) gives efficient estimates. The RE estimator is a weighted average of within and between variation: it quasi-demeans the data by subtracting a fraction theta of the unit mean, with theta = 1 - sqrt(sigma-u squared / (sigma-u squared + T sigma-a squared)); theta near 1 makes RE close to FE, theta near 0 close to pooled OLS. RE can estimate the effects of time-invariant variables. Its assumption of no correlation between a(i) and the regressors is strong and often implausible in agricultural data. The Breusch-Pagan Lagrange multiplier test compares RE with pooled OLS (null: no unit effect).
The Hausman test compares FE and RE estimates. Under the null hypothesis that a(i) is uncorrelated with the regressors, both are consistent and RE is efficient; under the alternative, only FE is consistent. The statistic H = (b(FE) - b(RE))' [V(FE) - V(RE)] inverse (b(FE) - b(RE)) follows chi-square with as many degrees of freedom as compared coefficients. A significant H rejects RE in favour of FE. The standard form assumes RE is fully efficient, which fails with heteroscedastic or serially correlated errors; a robust version uses the Mundlak approach, adding unit means of the time-varying regressors to the RE model and testing their joint significance. In all panel models, standard errors should be clustered by unit to allow for serial correlation. The correlated random effects (Mundlak) model also allows estimation of time-invariant effects while controlling for correlation.
Many agricultural outcomes are persistent: this year's herd size, area under a crop or farm income depends on last year's. A dynamic panel model includes the lagged dependent variable: y(it) = rho y(i,t-1) + x(it)'beta + a(i) + u(it). This creates a problem for fixed effects. The demeaned lagged outcome is correlated with the demeaned error, because the unit mean of the errors contains u(i,t-1), which also enters y(i,t-1). The within estimator of rho is then biased, typically downwards for positive rho, with a bias of order 1/T, the Nickell bias. In panels with many units and few periods, which are typical of farm surveys (T of 3 to 10), the bias is serious; it disappears only as T grows. Pooled OLS is biased in the other direction, because y(i,t-1) is correlated with a(i), so the two estimates often bracket the true rho.
Anderson and Hsiao proposed first differencing to remove a(i) and then instrumenting the differenced lagged outcome, which is correlated with the differenced error, by deeper lags such as y(i,t-2), which are uncorrelated with the differenced error if the u(it) are not serially correlated. Arellano and Bond (1991) generalised this into difference GMM, which uses all available lagged levels (y(i,t-2), y(i,t-3), and so on) as instruments in each period, combined efficiently by the generalised method of moments. Other regressors are classified as strictly exogenous, predetermined or endogenous, which determines which of their lags are valid instruments.
When the series is highly persistent (rho close to 1), lagged levels are weak instruments for first differences, and difference GMM performs poorly. Blundell and Bond's system GMM adds equations in levels, instrumented by lagged differences, under an additional assumption about initial conditions; it is usually more efficient in persistent series. Two diagnostics are always reported. The Arellano-Bond test for serial correlation in the differenced errors should find first-order correlation (expected by construction) but no second-order correlation; second-order correlation means y(i,t-2) is not a valid instrument. The Hansen J test of overidentifying restrictions checks the joint validity of the instruments. Using too many instruments (instrument proliferation, since their number grows with the square of T) overfits the endogenous variables and weakens the Hansen test, making it accept invalid instruments; the number of instruments should be reported and limited, for example by collapsing the instrument matrix or restricting lag depth.
Least-squares regression models the conditional mean of the outcome. But a policy may affect poor and rich farmers differently, a variety may raise yields more in bad years than in good, and the risk of very low yields may matter more than the average. Quantile regression, introduced by Koenker and Bassett (1978), models conditional quantiles: the tau-th quantile of y given x, Q(tau | x) = x'beta(tau), for any tau between 0 and 1. Fitting a set of quantiles (say 0.1, 0.25, 0.5, 0.75, 0.9) shows how the whole conditional distribution shifts and spreads with x. A coefficient that increases with tau means the predictor spreads the distribution upward, for example irrigation raising the upper tail of yields more than the lower.
The estimate beta(tau) minimises the sum over observations of rho(tau)(y(i) - x(i)'beta), where the check function rho(tau)(u) = u(tau - I(u < 0)) weights positive residuals by tau and negative residuals by 1 - tau. For tau = 0.5 this is least absolute deviations (LAD), median regression, which is robust to outliers in y, unlike least squares. The problem is a linear programme and is solved efficiently. Quantile regression makes no assumption about the error distribution and naturally handles heteroscedasticity, which in least squares appears as non-constant variance and here appears as different slopes at different quantiles. Quantiles are equivariant to monotone transformations: the median of log y is the log of the median of y, which does not hold for means. Standard errors are obtained by bootstrap, by rank-score inversion or by kernel estimates of the error density; with clustered data, cluster bootstrap is used.
A conditional quantile coefficient describes how the tau-th quantile of y given x changes with x; it is not the effect on the farmers who happen to be at that quantile, because a treatment can change farmers' ranks. Quantile treatment effects in randomised or quasi-experimental designs compare quantiles of the treated and control outcome distributions. Unconditional quantile regression (based on the recentred influence function) estimates effects on quantiles of the overall distribution, which is often the policy question (does a scheme raise the 10th percentile of farm incomes?). Agricultural uses include yield response to inputs at different points of the yield distribution, crop insurance (the lower quantiles of yield are what insurance covers), price transmission at high and low prices, and determinants of food expenditure across the income distribution. Crossing of estimated quantile curves, where a lower quantile exceeds a higher one in some region, signals misspecification or small samples.
Following Farrell (1957), a farm is technically efficient if it produces the maximum output possible from its inputs (or uses the minimum inputs for its output), given the technology. Technical efficiency (TE) is the ratio of actual output to frontier output, between 0 and 1. Allocative efficiency concerns using inputs in the cost-minimising proportions given their prices, and economic (cost) efficiency is the product of technical and allocative efficiency. Measuring efficiency requires a frontier, the best-practice boundary, which must be estimated from data. Two families of methods do this: econometric stochastic frontier analysis (SFA) and mathematical-programming data envelopment analysis (DEA, Chapter 12). Efficiency is relative to the sample: a farm judged fully efficient may still be far from what is technically possible elsewhere.
Aigner, Lovell and Schmidt (1977) and Meeusen and van den Broeck (1977) proposed the stochastic frontier production function ln y(i) = f(x(i); beta) + v(i) - u(i). The composed error has two parts: v(i), symmetric noise (weather, measurement error), usually normal; and u(i), a non-negative term representing technical inefficiency, usually half-normal, exponential or truncated normal. Separating noise from inefficiency is the key advantage over deterministic frontiers, which attribute every shortfall to inefficiency. Technical efficiency of farm i is TE(i) = exp(-u(i)). The frontier f is commonly Cobb-Douglas (log output linear in log inputs) or translog, which adds squared and cross-product terms of log inputs; a likelihood-ratio test chooses between them. Parameters are estimated by maximum likelihood. Because u cannot be observed separately from v, farm-level inefficiency is predicted by its conditional expectation given the composed residual, E[u(i) | v(i) - u(i)], the Jondrow, Lovell, Materov and Schmidt (JLMS) predictor, or by Battese and Coelli's direct predictor of E[exp(-u) | e].
Two parameterisations summarise the variance split: lambda = sigma-u / sigma-v, and gamma = sigma-u squared / (sigma-u squared + sigma-v squared), which lies between 0 and 1. If gamma is zero, there is no inefficiency and the model reduces to an average production function estimated by OLS. Testing gamma = 0 is non-standard because it lies on the boundary of the parameter space: the likelihood-ratio statistic follows a mixed chi-square distribution, and the critical values of Kodde and Palm are used rather than ordinary chi-square tables. To explain why some farms are less efficient, Battese and Coelli (1995) let the mean of the truncated-normal u depend on farm characteristics (education, extension contact, credit, farm size) and estimated everything in one step. The older two-step approach, estimating efficiencies and then regressing them on characteristics, is biased because the first step assumes the characteristics are unrelated to inefficiency. Panel versions allow inefficiency to vary over time, and newer models separate persistent from transient inefficiency and unobserved heterogeneity.
Estimated inefficiency is whatever the model does not explain: omitted inputs (soil quality, irrigation timing), unobserved weather differences between farms and differences in technology can all appear as inefficiency. Results are sensitive to the assumed distribution of u and to the functional form. Endogeneity of inputs, chosen by farmers who know their own productivity, can bias the frontier. Efficiency scores should therefore be interpreted with care, compared across specifications, and used to rank or group farms rather than as precise measures of waste. Cost and profit frontiers, with u entering with the opposite sign in a cost function, extend the approach to economic efficiency.
Data envelopment analysis builds a piecewise-linear frontier that envelops the observed input-output combinations of decision-making units (DMUs: farms, dairy cooperatives, irrigation schemes) using linear programming, without assuming a functional form or an error distribution. Charnes, Cooper and Rhodes (1978) introduced the constant-returns-to-scale (CRS) model, known as CCR; Banker, Charnes and Cooper (1984) introduced the variable-returns-to-scale (VRS) model, BCC. For each DMU, a linear programme finds the largest proportional reduction of its inputs (input orientation) or the largest proportional expansion of its outputs (output orientation) such that the resulting point is still achievable as a combination of observed DMUs. Efficient DMUs lie on the frontier with a score of 1; others score below 1 (input orientation) and are compared with their peers, the efficient DMUs that define the relevant part of the frontier. DEA handles multiple inputs and multiple outputs naturally, which is useful for farms producing crops and livestock together.
Under VRS the frontier hugs the data more closely, so VRS technical efficiency scores are at least as high as CRS scores. The ratio TE(CRS) / TE(VRS) is scale efficiency, showing how much of a DMU's inefficiency comes from operating at a non-optimal size; whether the DMU lies in the region of increasing or decreasing returns can be determined from a further non-increasing-returns model. With one input and one output under CRS, a DMU's efficiency is simply its output-input ratio divided by the highest ratio in the sample. Radial efficiency scores may leave slacks: even after proportional reduction of all inputs, one input may still be reducible without lowering output. Slacks should be reported alongside the scores, since a DMU with a score of 1 but non-zero slacks is not fully efficient in the Pareto-Koopmans sense.
DEA's strengths are flexibility and the absence of distributional assumptions, and its peer comparisons give practical benchmarks: a farm can be shown which similar farms do better and by how much. Its main weakness is that it is deterministic: all deviation from the frontier is treated as inefficiency, so measurement error, outliers and weather shocks can distort the frontier and everyone's scores. Because a single unusual DMU can define the frontier, outlier detection is important. Scores depend on the sample and on the number of inputs and outputs: with many variables relative to the number of DMUs, many units appear efficient by default (a common rule of thumb asks for at least three times as many DMUs as inputs plus outputs). Simar and Wilson's bootstrap methods give bias-corrected scores and confidence intervals, and their two-stage procedure gives valid inference when explaining scores by farm characteristics, where naive second-stage regressions are invalid. The Malmquist productivity index uses DEA distance functions over time to decompose productivity change into efficiency change (catching up) and technical change (frontier shift).
A student must be able to write the potential-outcomes decomposition of a naive difference, compute a DiD estimate from four means, a Wald IV estimate from a reduced form and first stage, a fuzzy RD ratio, a design effect, IPW weights, a Hausman decision and a DEA or SFA efficiency score, and explain each design's identifying assumption and how it can fail. In a thesis the student evaluates a scheme or measures farm efficiency and defends the design with balance tables, pre-trend plots, first-stage statistics or density tests.
Five years into work in an evaluation unit, a research institute or a development agency, the practitioner designs baseline and follow-up surveys, runs power calculations, chooses between randomisation and quasi-experimental designs, implements matching, DiD with modern staggered-adoption estimators and IV, estimates farm efficiency, and writes reports that state assumptions plainly. The practitioner knows that most scheme evaluations fail on design, not on estimation.
A senior evaluator or head of a monitoring and evaluation division with 25 years of experience decides which programmes are evaluated and how, negotiates phased roll-outs that allow credible comparison, judges the evidence presented by others, and translates mixed results into decisions on scaling up, redesigning or stopping schemes. The questions at this level concern external validity, spillovers, political feasibility of randomisation and the cost of evaluation relative to the programme.
A professor or research leader works on heterogeneous and dynamic treatment effects, staggered-adoption DiD, machine-learning methods for causal inference, partial identification and sensitivity analysis, stochastic frontier models with heterogeneity and endogeneity, and efficiency and productivity measurement in smallholder agriculture. The research leader supervises theses, reviews papers whose causal claims rest on untested assumptions, and advises governments on evaluation policy.
This manual introduces but does not develop machine-learning methods for causal inference (double machine learning, causal forests), mediation analysis, partial identification, structural models, spatial spillover designs, and advanced stochastic frontier models with endogeneity. Rules of thumb quoted (standardised differences below 0.1, first-stage F above 10, at least three DMUs per DEA variable) are conventions from the literature, not tests. Classical econometrics is assumed from the Agribusiness Econometrics subject.
Original practice questions written to the syllabus and pattern of the exams named in each question's examTags. They are not past papers and do not reproduce any board's questions. Every numerical answer was computed with a script; the values needed are stated in the question.
These notes are free to read here, with no account.
What the Laisha app adds:
Free for ten days, then a subscription. We would rather write that here than have you find it later.