What is sample design and estimation?
Sample design covers all aspects of how the samples in our surveys are specified and selected.
The design of samples is a particularly important aspect of survey methodology, and provides a basis for the sound measurement of economic and social phenomena from surveys of businesses and households.
Topics include sample frames, which list the businesses and household addresses we might select for a survey; types of sampling (for example, stratified, clustered - covering how we group businesses and households when sampling), and specification of sample sizes, with implications for costs and quality.
Finally, there is the selection mechanism used for choosing, randomly, the specific units that will make up any given survey sample.
Estimation is the method by which we produce aggregate estimates about characteristics of the population at large, from the responses received from those people and businesses selected in the survey samples.
There are many different ways of producing estimates (through weighting, for example), and we have to ensure the methods used produce good quality estimates, and that these represent the whole population without bias.
As well as the estimates themselves, we also calculate standard errors, which measure the statistical precision of the aggregate estimates.
Aims of the Sample Design & Estimation Centre
to support production of official statistics, at ONS and elsewhere in the Government Statistical Service, through the specification and implementation of appropriate sample design and estimation methodology
to optimise the quality of statistical outputs taking account of resources through research into, and application of, effective sampling and estimation methods
to promote integration, harmonisation and improvements in consistency of ONS outputs through the use of, for example, common methods
We conduct research and provide consultancy in a number of areas, including:
use of sampling frames
design of sampling schemes
minimisation and distribution of the burden on respondents
weighting and estimation, including for non-response
use of auxiliary information to improve the efficiency of sampling and estimation
detection and treatment of outliers in surveys
estimation of variances and other quality measures
use of administrative data in estimation
Sample design and estimation (SD&E) is split into three separate work areas (Branches): social surveys, business surveys and the census.
SD&E(Social) Branch designs sampling procedures for the social surveys and works closely with the Sampling Implementation Unit in ONS, which is responsible for drawing the samples.
SD&E(Social) also develops the weighting and estimation strategies and often implements these for survey output areas when the design is particularly complex.
SD&E(Social) provides support services for both the regular surveys that ONS runs, such as the Labour Force Survey, and for other, ad-hoc surveys carried out by ONS, sometimes for other government departments or those won by tender.
The vast majority of ONS social surveys are concerned with estimating characteristics of people and households, typically of the private residential population.
Samples of the private residential population are drawn from Royal Mail’s Postcode Address file (PAF).
Addresses are drawn from the frame using a systematic sampling procedure, and include provision for more complex designs that include stratification and multiple stages of selection (clustering). Weighting is carried out through calibrating to totals based on ONS’s mid-year population estimates.
SD&E(Social) derives methods that enable the survey output areas to produce standard errors that take account of the complex survey designs, and go beyond the limits of generic estimation software. We also provide general support and quality assurance.
Recent SD&E(Social) work has included:
design for the weighting methodology for the Integrated Household Survey, in which separate ONS surveys that ask some core questions in common are pooled, enabling production of more precise estimates.
new, efficient statistical computing algorithms have been developed to estimate standard errors given complex survey designs, where these had previously been unavailable or impractical with large data sets.
developing methodology for weighting longitudinal surveys, which enables ONS to develop new estimates and outputs:
- Survey of Income and Living Conditions (SILC)
- Wealth and Assets
- Life Opportunities Survey
informing the re-design of the sample for the International Passenger Survey, helping to improve estimates of migration
we are currently working on various initiatives to understand response patterns in social surveys, and to update non-response weighting procedures using information from the census, with the aim of reducing the potential for non-response bias
supporting the merger between SILC and the Family Resources Survey, contributing to efficiencies at ONS
changes in estimation methods were developed, as necessary, to allow some revised sampling procedures to be implemented in the LFS from mid-2010 as part of an efficiencies programme
support has been given across the social surveys for the implemention for the revised (2010) version of the Social Occupational Classification (SOC)
The role of SD&E(Business) is to maintain and improve the sampling and estimation methods used in ONS business surveys as well as developing new methods to fulfil changing user requirements.
Response to most business surveys is mandatory and the outputs from data collected are used to produce economic statistics that inform central government’s decision making.
Examples of economic statistics include earnings, workforce jobs, retail sales, inflation, and gross domestic product (GDP).
Most business surveys sample from the Interdepartmental Business Register, that is updated continuously from administrative sources and an annual survey.
Part of the role of SD&E(Business) is to support ONS’s register unit in maintaining the business register, in particular with regard to unbiased updating methods.
We support survey output areas during results processing and work closely with them when developing new methods and quality assure implementation.
Recent SD&E(Business) work has included:
use of administrative data in estimation – ONS is taking the lead in a workpackage and participating in another in a major European-wide project on the use of administrative data
electronic data collection – investigating estimation methods for variables not available via Real Time Information or direct data feed from businesses
dealing with births and deaths in small financial surveys and addressing sampling frame coverage problems
investigate the potential impact on quality of a reduction of sampled locations for the CPI
use of resampling methods to estimate variances of complex estimators
minimisation and distribution of the burden on respondents
SD&E(Census) supports all sampling aspects of the census coverage survey and estimation methods used in the census. Further details can be found on the census webpage.