Agricultural Statistics & Data Science · Sample surveys and official agricultural statistics · also SAE, Model-assisted survey sampling
Producing district and block estimates that a national sample is too thin to support directly: synthetic and composite estimators, the Fay-Herriot and nested-error models, empirical Bayes and hierarchical Bayes, model-assisted calibration and integration with census and remote-sensing data. A research strength of ICAR-IASRI.
In the exams: Not an explicit unit in AICE-JRF Ag Stat; PhD course and IASRI research area; national statistical offices (applied)
Surveys designed for state or national estimates are routinely asked for district, block or village figures, where samples are too small for reliable direct estimates. Small area estimation (SAE) answers this need by combining survey data with auxiliary information through statistical models. This subject covers why direct estimators fail for small domains; synthetic and composite estimators; the area-level Fay-Herriot model; the unit-level nested-error (Battese-Harter-Fuller) model; EBLUP and its mean squared error; empirical and hierarchical Bayes methods; calibration and the GREG estimator; model-based versus design-based inference; applications to crop yield, poverty and food insecurity; and the combination of survey, census and satellite data.
It builds on sampling-theory-basic and sampling-theory-advanced (direct estimators, weights, design-based variances), on the mixed-models subject (random effects, REML, BLUP) and on the crop estimation surveys subject (crop-cutting experiments and the need for small-area yields in insurance). Those topics are referred to rather than repeated.
SAE is not an explicit unit of the ICAR AICE-JRF/SRF Agricultural Statistics syllabus but is a standard PhD course and an ICAR-IASRI research area, and it is increasingly used by national statistical offices. Questions test the shrinkage factors and EBLUPs of the Fay-Herriot and nested-error models, synthetic and composite estimators, the components of MSE, GREG and calibration, and the distinction between design-based and model-based inference.
A survey designed to give reliable state or national estimates is often asked for estimates for smaller domains: districts, blocks, villages, farm-size classes within districts, or crop-district combinations. A direct estimator uses only the sample units that fall in the domain, weighted by their design weights, as in the sampling subjects of this track. Its variance is roughly inversely proportional to the domain sample size n_d. If the coefficient of variation of the variable is 50% per unit and only 10 units fall in a district, the direct estimate of the district mean has a CV of about 0.5 / sqrt(10) = 15.8%, and many districts may have fewer units or none at all.
Small area estimation (SAE) borrows strength from other areas, other times or auxiliary information, using models that link the areas. The resulting indirect estimators trade a small amount of bias for a large reduction in variance, so their mean squared error (MSE) is smaller than that of the direct estimator for areas with little sample. The price is reliance on a model whose adequacy must be checked. SAE grew from demographic and agricultural needs in the 1970s and 1980s; the books by Rao (2003) and Rao and Molina (2015) are the standard references, and in India ICAR-IASRI has developed SAE methods for crop yield, livestock and socio-economic indicators.
Consider a state survey of 2,000 farm households spread over 40 districts, about 50 per district on average but with some districts having only 15. For a variable with CV 0.8 per household, a district with 50 households has a direct CV of 0.8 / sqrt(50) = 11%, acceptable for many uses, but a district with 15 has 0.8 / sqrt(15) = 21%, and the block level (a few households per block) is hopeless. Doubling the whole survey would cut CVs only by a factor of about 1.4. Increasing the sample enough for direct block estimates is usually unaffordable, which is why model-based methods are used.
Domains can be planned at the design stage, by stratifying on them and allocating enough sample to each, or unplanned, cutting across strata. Planned domains are the first defence against small samples: if district estimates are a known requirement, the survey can be stratified by district and allocated to give each district a minimum sample, at the cost of national precision. SAE is needed for the domains that cannot all be planned, such as blocks, villages or cross-classifications.
Statistical offices often publish direct estimates only when their CV is below a threshold (commonly in the range of 20 to 30%, with lower thresholds for key indicators) and flag or suppress others. Such rules make clear why small-area demands cannot be met by direct estimation and provide a benchmark against which model-based estimates are judged: an SAE estimate is useful when its estimated root MSE, relative to the estimate, falls within the acceptable range.
District and block estimates are used to allocate funds, target programmes, monitor schemes, plan input supplies and settle insurance claims. Each use has its own tolerance for error: broad allocation can accept moderate errors, whereas legal entitlements need high accuracy and transparency. Understanding the use helps choose between direct estimates with larger samples, model-based estimates, or a combination.
Borrowing strength means using information from outside an area to improve its estimate. The information can come from other areas through a common model (cross-sectional borrowing), from earlier survey rounds (temporal borrowing), from neighbouring areas (spatial borrowing), or from auxiliary variables known for every area. Each form rests on an assumption of similarity that should be justified substantively and checked statistically.
To obtain a direct CV of 10% for a variable with a per-unit CV of 0.8, each district would need about (0.8/0.1) squared = 64 units; for 600 blocks, that would be 38,400 units, far beyond most budgets. With SAE, a few thousand well-spread units and good covariates can give block estimates with acceptable MSE for many purposes. Calculations of this kind justify investment in covariates and models rather than only in sample size.
A synthetic estimator assumes that small areas share characteristics with a larger area for which reliable estimates exist. If the population is divided into groups g (for example farm-size classes) and the large-area direct estimates of group means Y-bar_g are reliable, the synthetic estimator of the total for area d is sum over g of N_dg Y-bar-hat_g, where N_dg is the known number of units of group g in area d (for example from a census). Its variance is small because it uses large-area estimates, but it is biased whenever the area's group means differ from the large-area group means, and the bias does not shrink with sample size. Synthetic estimators were used early for disability prevalence and agricultural indicators.
A composite estimator is a weighted average of the direct and synthetic estimators: Y-hat_d = phi_d Y-hat_d(direct) + (1 - phi_d) Y-hat_d(synthetic), with 0 <= phi_d <= 1. The optimal weight minimises the MSE and gives more weight to the direct estimator when the area's sample is large and to the synthetic estimator when it is small. Because the optimal weight depends on unknown quantities, simple sample-size-dependent rules were proposed (for example by Drew, Singh and Choudhry), giving phi_d = 1 when the area's sample is at least its expected share and a smaller value otherwise. The model-based estimators of the following chapters can be seen as composite estimators whose weights come from a model.
Suppose a district's direct estimate is 52 with variance 20 and the synthetic estimate is 44 with an estimated squared bias plus variance of 30. Treating the two as independent, the weight that minimises the MSE of the combination is 30/(20 + 30) = 0.6 for the direct estimate, giving 0.6 x 52 + 0.4 x 44 = 48.8. The composite estimate lies between the two, nearer the more reliable one. In practice the synthetic estimator's bias is unknown, which is why model-based weights are preferred.
A district with an unusual crop mix, irrigation level or market access may differ systematically from the state averages on which the synthetic estimator relies. Synthetic estimates for such districts are biased, and the bias is invisible in their small variance. Model-based estimators that keep part of the direct information and include district covariates reduce this risk, and diagnostics comparing synthetic and direct values for well-sampled districts reveal it.
Structure-preserving estimation (SPREE) updates a detailed cross-classification from an old census to the present using current survey margins, preserving the association structure of the census while matching new totals. It is a synthetic method useful when only marginal totals are reliable from the survey, for example updating crop-by-district tables between agricultural censuses.
A regression-synthetic estimator predicts each area's mean from a model fitted to all the sample, x_d' beta-hat, ignoring the area's own data. It is the extreme of shrinkage (gamma = 0) and is what Fay-Herriot and nested-error EBLUPs reduce to for areas with very imprecise or no direct estimates. Comparing regression-synthetic, direct and EBLUP values for each area is a useful diagnostic table.
Synthetic and composite estimators were simple to compute and easy to explain, which suited statistical offices before mixed-model software was widespread. Their main limitation, the lack of a model-based measure of bias and MSE for each area, is what the model-based methods of the following chapters address.
Fay and Herriot (1979), estimating per-capita income for small places in the United States, proposed a model for area-level direct estimates. For area d, the direct estimate theta-hat_d equals the true value theta_d plus sampling error e_d with known (or separately estimated) variance psi_d: theta-hat_d = theta_d + e_d. The true values follow a linear model on area-level auxiliary variables x_d (from a census, administrative data or remote sensing): theta_d = x_d' beta + v_d, with area random effects v_d of variance sigma squared v. Combining the two gives a linear mixed model of the kind treated in the mixed-models subject, with heterogeneous known error variances.
The best linear unbiased predictor of theta_d is theta-tilde_d = gamma_d theta-hat_d + (1 - gamma_d) x_d' beta-hat, with shrinkage factor gamma_d = sigma squared v / (sigma squared v + psi_d), where beta-hat is the GLS estimate. Replacing sigma squared v by an estimate (by the method of moments, ML or REML) gives the empirical BLUP (EBLUP). Areas with precise direct estimates (small psi_d) keep most of their direct value; areas with imprecise ones are pulled towards the regression prediction. For example, with sigma squared v = 4 and psi_d = 12, gamma_d = 0.25; if the direct estimate is 50 and the regression prediction is 40, the EBLUP is 0.25 x 50 + 0.75 x 40 = 42.5. The leading term of its MSE is gamma_d psi_d = 3, a quarter of the direct estimator's variance of 12. The Fay-Herriot model needs only area-level data, which is why it is used widely when unit-level data cannot be linked to auxiliary information; its weaknesses are the assumption that psi_d is known (in practice it is a smoothed estimate) and the reliance on a linear model for theta_d.
Consider an area whose direct estimate has a small sampling variance psi_d = 2, while sigma squared v = 6. Then gamma_d = 6/8 = 0.75, so the EBLUP keeps three-quarters of the direct estimate, and the leading MSE term is 0.75 x 2 = 1.5, only slightly below the direct variance. Shrinkage is strong only where the direct estimate is weak; well-sampled areas change little, which is a reassuring property for users.
The sampling variances psi_d are treated as known but are themselves estimated from the survey, often unstably for small samples. Common practice is to smooth them with a model relating variance to sample size and the estimate (generalised variance functions) before fitting the Fay-Herriot model. Ignoring the uncertainty in psi_d leads to understated MSEs, which hierarchical Bayes models with unknown sampling variances can address.
The area-level variance sigma squared v can be estimated by the Prasad-Rao or Fay-Herriot moment methods, by ML or by REML, as in the mixed-models subject. Estimates of zero occur when covariates explain nearly all between-area variation; then the EBLUP equals the regression-synthetic estimate for every area, which may be unrealistic, and adjusted likelihood methods that avoid zero estimates have been proposed.
Good Fay-Herriot covariates are available for every area, measured consistently and strongly related to the target. Candidates are screened by their correlation with direct estimates in well-sampled areas and by model selection criteria, as in the linear models subject, while avoiding variables that are themselves estimated with large errors. A covariate that explains much of the between-area variation reduces sigma squared v and improves all EBLUPs.
Shrinkage pulls unusual direct estimates towards the regression line. This is desirable when the unusual values are due to sampling noise, but it can mask genuine outliers, such as a district suffering a local drought. Robust EBLUP methods limit the influence of outlying areas, and reports should flag areas where direct and model estimates differ greatly so that users can investigate.
Standardised residuals (theta-hat_d - x_d' beta-hat)/sqrt(sigma squared v + psi_d) should look approximately standard normal; plots against covariates and fitted values reveal non-linearity; and a comparison of the sum of squared standardised residuals with its expected value checks the variance assumptions. Influential areas, those with large leverage or residuals, deserve examination.
When auxiliary variables are available for each sampled unit and their means are known for each area, a unit-level model uses more information. Battese, Harter and Fuller (1988) estimated corn and soybean areas for counties in Iowa by combining farm-segment survey data with satellite (LANDSAT) pixel counts classified as corn or soybeans. Their nested-error regression model is y_dj = x_dj' beta + v_d + e_dj, where v_d is the area random effect with variance sigma squared v and e_dj the unit error with variance sigma squared e. It is a linear mixed model with a random intercept for each area.
Under this model the EBLUP of the area mean is approximately gamma_d [y-bar_d - x-bar_d' beta-hat] + X-bar_d' beta-hat, where y-bar_d and x-bar_d are sample means in area d, X-bar_d is the known population mean of the auxiliary variables, and gamma_d = sigma squared v / (sigma squared v + sigma squared e / n_d). For example, with sigma squared v = 2, sigma squared e = 20 and n_d = 10, gamma_d = 2 / (2 + 2) = 0.5, so half the area's own residual is retained. Areas with no sample receive the synthetic regression prediction X-bar_d' beta-hat. The method requires the population means of the auxiliary variables for each area (from a census, a register or a full-coverage satellite image) and unit-level linkage of survey data with the same variables.
Unit-level models use more information and can be more efficient, and they allow estimation of non-linear quantities (such as poverty rates) by simulation. Area-level models are easier when only aggregate auxiliary data exist, they automatically incorporate the survey design through the direct estimates and their variances, and they are less affected by unit-level model failures. Survey weights and informative sampling are a concern for unit-level models; pseudo-EBLUP methods incorporate weights.
With sigma squared v = 2 and sigma squared e = 20, an area with 40 sampled units has gamma_d = 2/(2 + 0.5) = 0.8, while an area with 10 units has gamma_d = 0.5, as in the main text. If the second area's sample mean exceeds its regression prediction by 3 units, the EBLUP adds 0.5 x 3 = 1.5 to the synthetic prediction X-bar_d' beta-hat. The comparison shows how the weight on the area's own data rises with its sample size.
In the Battese-Harter-Fuller study, segments sampled in each county were enumerated for corn and soybean area by the USDA June survey, and the number of satellite pixels classified as each crop was known for every segment in the county. The nested-error model related segment crop area to pixel counts with a county random effect, and the EBLUPs gave county estimates much more precise than the direct survey estimates, a classic demonstration of combining surveys with remote sensing.
If selection probabilities depend on the response even after conditioning on covariates (for example larger farms selected with higher probability and larger farms having higher yields), unit-level models that ignore the weights are biased. Remedies include adding the size measure as a covariate, using pseudo-EBLUPs that incorporate survey weights, or using area-level models built on design-consistent direct estimates.
Areas with no sampled units receive the regression-synthetic prediction, whose MSE includes the full area-effect variance sigma squared v plus the error in beta-hat. These estimates are usually less precise than those for sampled areas and should be flagged. Spatial models can improve them by borrowing from neighbouring sampled areas.
When several related variables are estimated (for example production of several crops), multivariate nested-error models exploit their correlations. Two-fold models add random effects for sub-areas within areas (such as villages within districts), allowing estimates at both levels with appropriate borrowing from each.
Unit-level models can include both unit-level covariates (farm size, irrigation) and area-level covariates (district rainfall, market access) whose values are the same for all units in an area. Area-level covariates help explain between-area variation and reduce sigma squared v, which increases the precision of EBLUPs for areas with small samples.
The MSE of an EBLUP has three parts. The first, g1, is the MSE of the BLUP if the model parameters were known: gamma_d psi_d in the Fay-Herriot model. The second, g2, reflects estimation of beta; the third, g3, reflects estimation of the variance component. Prasad and Rao (1990) derived the second-order approximation MSE approximately g1 + g2 + g3 and an estimator g1 + g2 + 2g3 that is approximately unbiased. For areas with small samples, g1 is much smaller than the direct variance, which is the gain from borrowing strength; g2 and g3 are usually small when there are many areas.
Jackknife and parametric bootstrap MSE estimators are alternatives that extend to complex models. Benchmarking adjusts EBLUPs so that they add up to the reliable direct estimate for the larger area (for example the state total), which users expect; it introduces a small extra error. Model diagnostics (residual plots of standardised direct-minus-predicted values, checks of normality of random effects, tests of covariates) and comparison of EBLUPs with direct estimates for areas with large samples are essential before release. The reported precision should be the estimated MSE, not the model variance alone.
The estimated MSE of an EBLUP is an average over the model and the design; for a particular area whose characteristics depart from the model, the true error may be larger. Confidence intervals based on EBLUP MSEs therefore have approximately correct coverage on average over areas rather than for each area. Users comparing two areas should look at both estimates and their MSEs, and avoid fine rankings.
Ratio benchmarking multiplies all area EBLUPs by the ratio of the reliable state direct estimate to the sum of the EBLUPs; difference benchmarking adds a share of the discrepancy to each area. Benchmarking protects against overall model misspecification and makes published figures consistent, at the cost of a small increase in MSE, which bootstrap methods can include.
In the parametric bootstrap, many populations are generated from the fitted model, samples are drawn, the model is refitted, and the spread of the bootstrap EBLUPs around the generated true values estimates the MSE. It handles complex models where analytical approximations are hard, at the cost of computation, and can include benchmarking and transformations.
For areas with no sample, the leading MSE term is sigma squared v rather than gamma psi, often much larger, plus the error in estimating beta. Reported MSEs should therefore differ markedly between sampled and unsampled areas, and users should be warned when many estimates come from unsampled areas.
Intervals of the form EBLUP plus or minus 1.96 times the estimated root MSE have approximately 95% coverage on average across areas when the model holds and the number of areas is large. With few areas, or when the variance component is near zero, coverage can fall below nominal levels, and bootstrap or Bayesian intervals are preferred.
Under normality, the Fay-Herriot model has a Bayesian reading: theta_d has a prior distribution N(x_d' beta, sigma squared v) and the direct estimate has likelihood N(theta_d, psi_d), so the posterior mean is exactly the BLUP. Empirical Bayes (EB) replaces the unknown prior parameters with estimates from the data, giving the EBLUP for normal linear models. EB extends naturally to non-normal data: for proportions (such as the proportion of households food insecure), logistic mixed models give EB estimates; for counts (cases of a disease), Poisson-gamma or Poisson-lognormal models.
Hierarchical Bayes (HB) puts prior distributions on all parameters, including beta and sigma squared v, and computes posterior distributions by Markov chain Monte Carlo, as introduced in the Bayesian inference subject. Posterior means estimate the small-area values and posterior variances measure uncertainty, automatically including the uncertainty in the variance components that EBLUP methods must approximate. HB handles complex models (spatial and temporal correlation, multiple outcomes, unknown sampling variances, benchmarking constraints) and small numbers of areas well, at the cost of computation and the need to specify and check priors. In practice EB and HB estimates are often close for linear models with many areas.
If theta_d has prior N(40, 4) and the direct estimate 50 has sampling variance 12, the posterior mean is (50/12 + 40/4)/(1/12 + 1/4) = 42.5, exactly the BLUP of chapter 3, and the posterior variance is 1/(1/12 + 1/4) = 3. Empirical Bayes estimates the prior from all areas; hierarchical Bayes adds a prior for the prior parameters and averages over their uncertainty.
For a proportion such as the share of households food insecure, direct estimates near 0 or 1 have unstable variances, and the normal Fay-Herriot model can give estimates outside [0, 1]. Area-level models on a transformed scale (arcsine or logit) or unit-level logistic mixed models avoid this. HB beta-binomial or logit-normal models are common in poverty and nutrition mapping.
Posterior predictive checks compare replicated data simulated from the posterior with the observed direct estimates; systematic discrepancies indicate model failure. Sensitivity to priors, especially for variance components with few areas, should be examined by refitting with alternative reasonable priors.
Small area models are fitted with general mixed-model and Bayesian software and with specialised packages in R (for example packages implementing Fay-Herriot, nested-error and empirical best predictors with MSE estimation). Whatever the software, the analyst must understand the model, check its assumptions and report the estimation choices.
With many areas and a linear model, EB (EBLUP) and HB usually give similar point estimates; HB gives better uncertainty measures when areas are few or variance components small. For non-linear models, complex structures or when full posterior distributions are needed (for example for ranking areas), HB is usually more convenient.
Design-based estimators can also use auxiliary information. The generalised regression (GREG) estimator of a total is Y-hat_GREG = Y-hat_HT + (X - X-hat_HT)' B-hat, where Y-hat_HT and X-hat_HT are Horvitz-Thompson estimators, X the known population totals of the auxiliary variables and B-hat a weighted regression coefficient. It is model-assisted: a regression model motivates it, but its approximate design-unbiasedness does not depend on the model being true. The ratio and regression estimators of Sampling Theory I are special cases.
Deville and Sarndal (1992) showed that GREG can be written as a weighted sum with calibrated weights w_i, chosen to be as close as possible to the design weights d_i (in a chosen distance) subject to the calibration equations sum of w_i x_i = X. With the chi-square distance, calibration gives GREG; other distances give raking and bounded weights. Statistical offices use calibration to make survey weights consistent with census totals, which also reduces non-response bias (sampling theory II subject).
The domain GREG applies the same idea to each small area, using area-level auxiliary totals. It remains approximately design-unbiased but, like any direct estimator, its variance grows as the area sample shrinks; for very small samples, model-based EBLUPs are more precise. GREG and EBLUP therefore complement each other: GREG for domains with adequate samples, EBLUP for the rest.
Suppose a sample gives an HT estimate of total crop area 9,500 ha and of total production 28,000 t, while the area is known from land records to be 10,000 ha. With an estimated regression slope of 3 t/ha, GREG adjusts the production estimate to 28,000 + (10,000 - 9,500) x 3 = 29,500 t. The adjustment corrects for the sample's under-representation of area and, if production is closely related to area, reduces variance substantially.
Calibration can produce negative or very large weights when constraints are many or samples small. Bounded calibration methods restrict weights to a range (for example between half and three times the design weight) while meeting the constraints, and choosing a moderate number of well-measured calibration variables keeps weights stable. Calibrated weights are then used for all estimates from the survey, ensuring consistency with known totals.
When survey non-response is related to variables with known totals, calibration to those totals reduces non-response bias as well as variance, which is why statistical offices calibrate final weights to census age-sex or farm-size distributions. The sampling theory II subject discusses non-response weighting; calibration integrates it with the use of auxiliary information.
Calibrating weights to totals for each small area is possible only when area totals of the auxiliary variables are known and the area sample is large enough to support the constraints. When it is not, calibration at a higher level combined with model-based small-area estimation is the practical compromise.
GREG uses a model only to construct weights and remains design-consistent even if the model is wrong; EBLUP relies on the model for its properties. When area samples are adequate, GREG is safer; when they are small, EBLUP's gain in precision usually outweighs its model dependence. Some methods (model-calibration, pseudo-EBLUP) combine features of both.
In design-based inference, population values are fixed and randomness comes only from the sampling design; estimators are judged by their design bias and variance over repeated samples. In model-based inference, population values are realisations of a superpopulation model, and estimators are judged under the model. Model-assisted inference uses models to construct estimators (such as GREG) but evaluates them under the design. Sampling theory for national estimates is largely design-based; small area estimation is largely model-based, because designs do not provide enough information for small domains.
Model-based small-area estimates are only as good as their models. If the model omits an important area characteristic, estimates for areas that are unusual in that respect are biased towards the regression line, and their estimated MSE may understate the error. Good practice includes careful choice of covariates, diagnostics, comparison with direct estimates where samples are adequate, benchmarking, validation against census or administrative values when available, clear documentation of methods, and labelling estimates as model-based with their MSE. Users should be told that model-based estimates are not suitable for ranking areas finely when their confidence intervals overlap widely.
Suppose a district has a direct estimate of 30 with a CV of 35% and an EBLUP of 26 with an estimated relative root MSE of 12%. The EBLUP is more precise under the model, but if the district differs from the model in an unmeasured way, its true error could be larger. If the census later shows the true value was 29, the direct estimate was closer in this case, illustrating that model-based gains hold on average across areas, not for every area.
Publications should state whether each figure is direct or model-based, describe the model and covariates, give MSE estimates and explain benchmarking. Labelling builds trust and helps users decide how to use the figures, for example for broad targeting rather than legal entitlements.
Many modern methods are hybrids: design-consistent direct estimates feed area-level models; survey weights enter unit-level models; MSEs are evaluated both under the model and by design-based simulation. Statistical agencies increasingly accept model-based estimates when hybrid methods show they are robust.
When small-area estimates determine resource allocation, errors have consequences for real communities. Transparent methods, published MSEs, opportunities for review and periodic validation are not only good statistics but fairness to the areas affected.
Simulation studies that repeatedly draw samples from a real or realistic population, compute model-based estimates each time and compare them with the known area values show how the estimators perform under the actual design. Such design-based evaluation is the standard way to compare SAE methods before adopting them.
Crop yields are needed for small administrative units (blocks, villages or village panchayats) for crop insurance, planning and targeting, while crop-cutting experiments (crop estimation surveys subject) give reliable estimates only at district level. SAE combines the CCE yields with auxiliary data available for all small areas: satellite vegetation indices, weather variables, irrigated area and previous yields. Area-level Fay-Herriot models with district or state CCE direct estimates, and unit-level models with field-level CCE yields linked to satellite indices, have both been applied; ICAR-IASRI has published SAE studies for crop yield at block level. The technology-assisted yield estimation for insurance discussed in the crop estimation subject is essentially an application of these ideas.
Poverty mapping, popularised by the World Bank method of Elbers, Lanjouw and Lanjouw (2003), fits a unit-level model of household consumption on variables common to a household survey and the population census, then simulates consumption for every census household to estimate poverty rates for small areas. Later work (for example the empirical best predictor of Molina and Rao) corrected some weaknesses in the treatment of area effects. Food insecurity prevalence, measured by scales such as the FAO Food Insecurity Experience Scale, and child undernutrition indicators are also estimated for districts and sub-districts by SAE, combining surveys with census, administrative and geospatial data. Such maps guide the targeting of food and nutrition programmes.
Under area-yield insurance, payouts depend on unit-level yields estimated with few crop-cutting experiments, as the crop estimation subject explains. SAE can improve these estimates by borrowing from satellite indices, weather and neighbouring units, and by giving MSEs that quantify the uncertainty in payouts. Because payouts are legal entitlements, the methods must be transparent, documented and acceptable to insurers, governments and farmers, which is why combinations of CCEs and model estimates with stated weights are used rather than purely model-based figures.
Where a census or complete administrative records exist for a year, small-area estimates from survey data alone can be compared with the true values to assess accuracy, choose models and calibrate MSE estimators. Such validation studies are the strongest evidence for adopting SAE in official statistics.
Beyond crop yield, SAE has been applied to livestock numbers and products, fertiliser use, adoption of technologies and agricultural household incomes, typically with census counts, administrative records and remote-sensing covariates. The same steps apply: define areas and indicators, assemble covariates, choose models, estimate MSE and validate.
Food insecurity experience scales are collected in national surveys at sample sizes adequate for national and regional estimates. Mapping them at district level uses area-level models with covariates such as poverty estimates, agricultural productivity, prices and climate shocks, with logit transformations to keep estimates within the valid range.
Census-survey poverty mapping assumes that the consumption model fitted on survey households applies to census households, that common variables are measured comparably in both sources, and that area effects are properly modelled. When these assumptions fail, standard errors can be understated. Validation, sensitivity analysis and transparent reporting of model choices are therefore part of credible poverty maps.
Maps of small-area estimates are powerful but can mislead when colour classes hide large uncertainty. Pairing estimate maps with maps of relative root MSE, or shading only areas whose estimates are reliable enough, helps users read them correctly.
The success of SAE depends on auxiliary variables that are strongly related to the target variable and available for all areas: population and agricultural census variables, administrative records (land records, irrigation, input sales, scheme enrolments), and geospatial data (satellite vegetation indices, land cover, night-time lights, climate grids, accessibility). Census variables can be outdated; administrative records can have coverage gaps; satellite indices can be affected by clouds and classification error. Measurement error in covariates attenuates their predictive value and can bias estimates, and models that account for it have been developed.
Neighbouring areas tend to be similar, and the same area tends to be similar over time. Spatial Fay-Herriot models allow area effects to be correlated with neighbours (for example through simultaneous autoregressive models), and spatio-temporal models such as that of Rao and Yu borrow from previous rounds of a repeated survey. These extensions improve precision when spatial or temporal correlation is strong and the covariates explain only part of the variation.
An operational SAE system defines the target indicators and areas, assembles and harmonises auxiliary data, fits and checks models, estimates MSEs, benchmarks to official direct estimates, validates against external data, and publishes estimates with clear quality indicators. Statistical offices increasingly treat SAE as a routine part of official statistics, with documentation and review comparable to that of direct survey estimates.
Satellite night-time light intensity correlates with economic activity and electrification and has been used as a covariate for poverty and income in small areas, alongside land cover, vegetation indices, road density and distance to markets. Such covariates are available for every area at low cost but must be tested for each indicator and region, since their relationships with welfare vary.
Combining sources requires matching geographic boundaries (which change over time), harmonising definitions and reference periods, and handling missing data in covariates. A small error in matching survey clusters to the wrong area can distort unit-level models. Data integration is often the most time-consuming part of an SAE project and deserves the same quality control as the survey itself.
Censuses are decennial or quinquennial, while estimates are needed every year. Models that borrow strength over time, using previous rounds of a survey and annually updated covariates such as satellite indices and administrative counts, keep small-area estimates current between censuses. Their quality should be monitored against each new census.
Satellite indices are treated as covariates whose relationship with the target is estimated from survey data, not as direct measurements of yield or welfare. This guards against biases from classification error and changing relationships between seasons, and makes clear that ground data remain essential.
The smaller the areas, the fewer sample units each contains and the more the estimates depend on the model; the larger the areas, the less useful they are for local decisions. Agreement between producers and users on the area level (district, block or village) at the outset prevents producing estimates that are either too uncertain or too coarse.
PhD students in agricultural statistics take SAE as a course and often as a thesis topic: new models for crop yield, livestock or socio-economic indicators at block or district level. They must fit Fay-Herriot and nested-error models, compute EBLUPs and MSEs, and compare model-based with direct estimates.
A five-year statistician in a statistical office or research institute produces district and block estimates: assembling census, administrative and satellite covariates, fitting and diagnosing models, estimating MSE, benchmarking to state totals, and preparing maps and documentation. They advise users on what precision the estimates support.
A senior methodologist decides when official statistics may include model-based small-area estimates, sets quality standards and validation procedures, and negotiates access to administrative and geospatial data. They explain to policy makers the difference between direct and model-based figures.
A PhD with long experience develops SAE methodology (models for non-normal data, spatial and temporal models, measurement-error models, MSE estimation) and advises national and international agencies on SAE systems.
MSE formulas beyond the leading term are described, not derived. Non-linear indicators by simulation (empirical best predictors), measurement-error models and Bayesian computation are outlined only. Specific ICAR-IASRI applications are referred to generally without reproducing their results. Software is not covered in detail.
Original practice questions written to the syllabus and pattern of the exams named in each question's examTags. They are not past papers and do not reproduce any board's questions. Every numerical answer was computed with a script; the values needed are stated in the question.
These notes are free to read here, with no account.
What the Laisha app adds:
Free for ten days, then a subscription. We would rather write that here than have you find it later.