FOI reference: FOI-2026-3479
You asked
I am writing to request information under the Freedom of Information Act 2000 regarding the methodology and criteria used by the ONS to select businesses for its mandatory economic surveys (e.g., MBS, ABS, BRES).
Specifically, I am looking for detailed information on the following:
- Selection "With Certainty" Thresholds: Please provide the current employment and/or turnover thresholds used to classify a business as a "certainty case" (where selection is guaranteed rather than random) for the major economic surveys.
- Stratification by Industry (SIC Codes): How does the ONS determine the sampling fraction for specific Standard Industrial Classification (SIC) codes? Specifically, are there "niche" industries where the population size is so small that all registered businesses are sampled annually?
- Rotation and "Holiday" Policies: What are the formal rules regarding "survey holidays" for Small and Medium Enterprises (SMEs)? Please provide the specific duration a business is expected to remain in a sample and the mandatory minimum period they should be "rotated out" before being eligible for re-selection.
- Selection Overlap: What mechanisms are in place to prevent a single SME from being selected for multiple different ONS surveys simultaneously?
- Exemption Criteria: What are the specific criteria or internal guidelines used to assess requests from businesses to be removed from a sampling frame due to "undue burden"?
We said
Thank you for your request.
Many of the outputs created from data collected via our major economic surveys are important components of the National Accounts and crucial economic indicators such as Gross Domestic Product (GDP) used by government to set and monitor economic policy. It is therefore essential that these outputs - and the associated design of the sample (including stratification, sampling fractions, rotation, overlap and selection of important businesses with certainty etc) - are of the highest quality possible. For this reason, major economic surveys are mandatory under the Statistics of Trade Act, and are managed according to the principles of the Code of Practice for Statistics.
1. Selection "With Certainty" Thresholds
Each survey has different stratification, based on the requirements of the outputs. Businesses are generally selected within strata which are defined at varying levels of region, industry classification and size (this can be defined in terms of employment, turnover, assets etc). This means the standard census (selection with certainty) thresholds can be different between and within surveys, depending on the industry and region the business is in. Further details can be found in the Quality and Methodology Information (QMI) document for a particular survey. As the largest businesses generally contribute the most to the estimates due to the skewed nature of the distribution of economic variables such as turnover, it is very important that these businesses are continually selected for our major surveys to ensure quality.
Using the Annual Business Survey (ABS) as an example, details of the thresholds can be found in section 3.2 of the ABS technical report. All businesses with 250+ employment are selected with certainty, except for the following industries, where all businesses with 1000+ employment are selected with certainty and for those with 100-999 employment, 50% of the population are selected randomly:
event catering activities (SIC2007: 56210)
market research and public opinion polling (73200)
other activities of employment placement agencies (78109)
temporary employment agency activities (78200)
private security activities (80100)
general cleaning of buildings (81210)
other building and industrial cleaning activities (81.22)
In addition, businesses with low employment but high turnover are also included in the ABS sample with certainty - as they are important for the quality of the estimates - and sent questionnaires, as follows:
10-49 employment and £100 million + in production (SIC divisions 01-39)
10-49 employment and £75 million+ in non-production (SIC divisions 40+)
50-99 employment and £100 million+ in all sectors
100-249 employment and £150 million+ in all sectors
0<10 employment and £100 million+ in Wholesale (division 46) & Property (division 68) and in an enterprise group
0<10 employment and £50 million + in all sectors excluding 46 and 68 and in an enterprise group
There are also specific businesses that are selected with certainty for most surveys as they are important for the estimates for various other reasons - such as those in niche industries and/or strata with small populations.
2. Stratification by Industry (SIC Codes)
Sampling fractions in each stratum - which are generally defined in terms of region, industry and size and can also be defined at different levels of SIC aggregation within a survey - are determined based on the sample size and the quality requirements (e.g. Coefficients of Variation) of the estimates of different variables at different levels of aggregation, depending on the specific output(s). Depending on the requirements, there are various ways to calculate what the sampling fractions in each stratum need to be to meet the quality requirements. As an example, if the requirement is to minimise the variance of the population estimate of one variable at the overall level of the survey, the methods of Neyman (1934) or Wright (2017) could be used. This optimisation process can result in more census strata than solely those that are defined by the thresholds mentioned above.
During this process, minimum sample sizes per stratum are specified, to ensure enough responses to obtain quality estimates and to reduce the likelihood of bias in the estimates. This can result in strata with small population sizes (e.g. specialist industries) being fully enumerated (i.e. a census stratum).
The sampling fractions are reviewed and updated on a regular basis to meet changing user needs and/or to account for changes in the population characteristics.
3. Rotation and "Holiday" Policies
ONS follows the principle that contact with small businesses should be kept to a minimum where possible, as statistical surveys create a proportionately higher burden for them compared with large enterprises.
Small businesses (those with 0 to 9 employment) are covered by the Osmotherly rule which restricts the burden placed upon them.
This rule guarantees small businesses will:
not be simultaneously selected for more than one survey
remain in that survey for only a limited time span (for monthly surveys this is 15 months; for quarterly surveys this is 5 quarters)
have a 'survey holiday' of at least three years from most surveys once the time in the survey has completed and while the business remains small
There are no guarantees made for businesses which have greater than 9 employment. However, for monthly surveys where the business has greater than 9 employment and is not selected with certainty, the period for which a business is expected to be in the survey sample before being rotated out (this is called the rotation period) is usually 27 months. For quarterly surveys, this period is not standard across surveys. For some surveys it is five quarters and for some it is 9 quarters.
Businesses can remain in the survey longer or shorter than the specified rotation period due to businesses starting/ceasing trading and therefore appearing or disappearing from the stratum and due to movements of businesses from one stratum to another due to changes in the stratification variables, such as their industry classification or employment levels.
There are some exceptions to the Osmotherly rules which relate to certain financial surveys and to surveys run in Northern Ireland, but the vast majority of small businesses (those with 0 to 9 employment) are covered by Osmotherly.
4. Selection Overlap
For businesses with 0 to 9 employment, it is guaranteed that they will be selected for only one survey when they are selected into a survey, other than for the exceptions referred to above. There are no mechanisms in place to prevent selection for multiple ONS surveys if the business has greater than 9 employment.
5. Exemption Criteria
There are no specific statutory criteria or guidelines under the Statistics of Trade Act 1947 for removing a business from a sampling frame or a specific survey on the grounds of "undue burden".
Where a business raises concerns about the burden of participation, these are considered on a case-by-case basis. This typically involves reviewing the business's recent survey selection history and the nature of the concern raised, to understand whether the level of contact is appropriate and proportionate.
The Office for National Statistics seeks to balance the need to produce high-quality economic statistics with the need to minimise burden on respondents. Any actions taken as a result of respondent feedback are informed by this balance and by operational considerations, rather than by fixed thresholds or automatic exemptions.
References
Neyman, J. (1934). "On the two different aspects of the representative method: The method of stratified sampling and the method of purposive selection". Journal of the Royal Statistical Society. 97 (4): 558-625.
Wright, T. (2017). "Exact optimal sample allocation: More efficient than Neyman". Statistics & Probability Letters, Elsevier, vol. 129(C), pages 50-57
Osmotherly, E et al. (1996). https://analysisfunction.civilservice.gov.uk/wp-content/uploads/2018/02/Osmotherly-Report.pdf