1. Methodology background


 National Statistic   
 Survey name  Annual Business Survey
 Frequency  Annual
 How compiled  Sample based survey
 Geographic coverage  UK
 Sample size  73,000
 Last revised  24 August 2018

Back to table of contents

2. Executive summary

The Annual Business Survey (ABS), formerly known as the Annual Business Inquiry - part 2 (ABI/2), is an annual survey of businesses covering the production, construction, distribution and service industries, which represents approximately two-thirds of the UK economy, by gross value added (GVA).

Every year, ABS questionnaires are sent by Office for National Statistics (ONS) to around 62,000 businesses in Great Britain, and by the Northern Ireland Statistics and Research Agency (NISRA) to around 11,000 businesses in Northern Ireland.

ABS is the largest business survey conducted by us in terms of the combined number of respondents and variables it covers (62,000 Great Britain questionnaires, with around 600 different questions asked). It is the main resource for understanding the detailed structure and performance of businesses across the UK, and is a large contributor of business information to the UK National Accounts.

ABS provides a number of high-level indicators of economic activity, such as the total value of sales and work completed by businesses, the value of purchases of goods, materials and services, and total employment costs. The data published are estimates of totals for the calendar year, January to December.

The contribution of different industries to the overall value of economic activity can be assessed. Although estimates of employment from each company are not collected by the ABS, it is possible to get a measure of value added and costs per head to allow better comparison between industrial sectors of different sizes. This can be carried out using employment estimates from the Business Register Employment Survey (BRES), which are published alongside the ABS data in the ABS releases.

The indicators in the ABS publications are collected and presented as monetary values or counts, for example, approximate gross value added (aGVA) and numbers of enterprises. They are essentially a snapshot of UK business activity, and can be used to understand the level of the contributions to the UK economy from different sectors of the economy at any one time. The statistics produced are referred to as structural business statistics.

The ABS publishes estimates at national and English region and UK country level, for the UK non-financial business economy and industry breakdowns down to the detailed industry class four-digit UK Standard Industrial Classification 2007: SIC 2007 level. Further breakdowns by industry and geography, and additional variables are available on request from the ABS Special Analysis team.

ABS outputs may be used to answer questions such as:

  • how much wealth has been created in a particular industry?

  • has there been a shift in activity from one industrial sector to another, and which industry groups, classes or sub-classes are driving the change?

  • are any industries particularly dominant in specific regions or countries of the UK and are there structural changes over time?

  • how productive is a particular industry, such as the chemicals sector, and what is its operating profitability?

Full details on the background and history, uses and users, and concepts and statistical methods underlying the ABS can be found in the ABS technical report.

Back to table of contents

3. Summary of the Annual Business Survey

What it measures

The Annual Business Survey (ABS) measures business and financial information from UK businesses, including total turnover, total employment costs, total purchases, capital expenditure, stocks and other aggregates. Variables derived from these statistics, such as approximate gross value added at basic prices are also published by the ABS. Detailed industry and geographical breakdowns are available.

Frequency

Annual – estimates are made for calendar years.

Period available

Data are available from 1997.

Between 1997 and 2008, ABS estimates were published on the UK Standard Industrial Classification 2003: SIC 2003 system. In 2008, this system was updated to meet European requirements, and estimates from 2008 are classified by SIC 2007.

Estimates from the 1997 to 2008 time series, converted from SIC 2003 to SIC 2007 are available on the ABS web pages.

Sample frame

The ABS sample frame is the Inter-Departmental Business Register (IDBR). The IDBR covers businesses in all parts of the economy, except those that are not registered for Value Added Tax (VAT) or Pay As You Earn (PAYE), which includes very small businesses, the self-employed, those without employees, and those with low turnover. Some non-profit making organisations are also not registered on the IDBR. There are 2.6 million businesses on the IDBR, covering nearly 99% of UK economic activity. It is used by government departments, including us, as the sampling frame for most business surveys. The ABS draws its sample from the 2.4 million businesses that are in scope for the survey.

In 2015, as described in Improving the Coverage of the Standard Business Survey Population the coverage of the ONS Standard Business Survey Population was expanded to include a population of solely PAYE-based businesses. This increased the population by approximately 92,000 businesses.

Sample size

The ABS samples approximately 62,000 businesses across Great Britain. Northern Ireland Statistics and Research Agency (NISRA) supplement the Great Britain sample with a sample of around 11,000 businesses from Northern Ireland.

Sample design

Stratified random sample, where the strata are defined by Standard Industrial Classification (SIC), UK country and employment size.

Imputation

Imputation is mainly carried out for non-responding businesses with an employment of 250 or more, and businesses with smaller employment but high turnover. For other businesses, imputation is not carried out, and totals are estimated using weights adjusted for non-response. To calculate imputed values, ratio imputation is used. This uses, where available, the previous returned value for the business with an estimated growth applied, derived from the returned values of similar businesses.

Estimation

Estimation is carried out using standard statistical techniques for stratified random sampling. Each sampled business is weighted by its design weight (a-weight) and a calibration factor, which is calculated using ratio estimation (g-weight). The a-weight represents the number of similar businesses that each sampled business represents. The g-weight improves the precision of the estimates and corrects for any imbalance in the selected sample by taking account of characteristics of the businesses that were randomly selected. Weights are updated annually.

Outliers

Businesses with atypical values compared with other businesses in their industry and employment size are treated as outliers, using a post-stratification method. Values returned by businesses that are identified as outliers are given a weight of one in estimation calculations, that is, they represent only themselves.

Coverage

The ABS estimates cover the UK non-financial business economy. This is approximately two-thirds of the whole UK economy, by gross value added (GVA). The SIC 2007 industries that are included in the survey are:

  • agriculture (support activities 01.6 and hunting, trapping and related service activities 01.7), forestry and fishing – part of section A

  • production industries – sections B to E

  • construction industries – section F

  • distribution industries – section G

  • other service industries – sections H, I, J, K (insurance and reinsurance, groups 65.1 and 65.2 only), L, M, N, P (excludes public sector), Q (excludes public sector and medical and dental practice activities, group 86.2), R, S

ABS covers the insurance and reinsurance parts of the financial and insurance sector (groups 65.1 and 65.2 in section K). However, data for this industry have remained experimental and, due to ongoing volatility, it was decided to remove it from the ABS 2012 provisional release onwards.

Publication schedule

  • November – Provisional national results (for previous calendar year)

  • April or May – Revised national results (including revision of the previous survey year)

  • April or May – Revised regional results (including revision of the previous survey year)

Release dates are published on the GOV.UK release calendar.

Back to table of contents

4. Output quality

This report provides a range of information that describes the quality of the output and details any points that should be noted when using the output.

We have developed Guidelines for measuring statistical quality; these are based upon the five European Statistical System (ESS) quality dimensions. This report addresses these quality dimensions and other important quality characteristics, which are:

  • relevance timeliness and punctuality

  • coherence and comparability

  • accuracy output quality trade-offs

  • assessment of user needs

  • perceptions accessibility and clarity

More information is provided about these quality dimensions in the following sections.

Back to table of contents

5. About the output

The Annual Business Survey (ABS) meets the quality standards required by its UK and legal obligations for the collection and delivery of business statistics as described in this section. However, the ABS team also seeks to continually improve the quality of ABS statistics and services for its wider users, informed by the collection and analysis of feedback from users.

Relevance

(The degree to which the statistical product meets user needs for both coverage and content.)

The ABS collects data that fulfils the UK’s obligations under the Structural Business Statistics (SBS) Regulation of the European Union, established by the Council Regulation (EC, Euratom) Number 58/97 of 20 December 1996. This established a common framework for the production of European Community statistics on the structure, activity, competitiveness and performance of businesses in the Community. It was recast (Regulation (EC) Number 295/2008) for the 2008 survey with new demands and the move to Standard Industrial Classification 2007. Data are transferred to Eurostat and used for policy monitoring and formulation by the EU, and as a source for annual structural statistics.

The ABS also meets its requirements as a statutory annual survey conducted under the Statistics of Trade Act 1947, for the production of the UK National Accounts (for the compilation of supply and use tables, which provide a framework for understanding and analysing the interdependence of industries in the UK), and to give weights for index aggregation and turnover deflation for the indices of services and production.

Main users of the output include:

  • national accounts – for the compilation of supply and use tables, which show the sales and purchases relationships between consumers and producers by industry

  • Indices of services and production – to calculate the weights used to produce the indices, and to calculate the deflation of turnover

  • Eurostat – to meet the Structural Business Statistics Regulation requirements for annual structural statistics to inform and monitor European Union policy

  • the Scottish government and the Welsh government – to calculate the Scottish and Welsh indices of production, to produce Scottish and Welsh supply and use tables, to inform and monitor policy

  • the Department for Business, Energy and Industrial Strategy – to assess the structure and performance of UK industries

  • the Inter-Departmental Business Register team – to update information on UK businesses held on the register

  • Department for International Trade – assessing the impact of importers and exporters in UK industries

  • local authorities – for economic research, planning purposes, lobbying and economic strategy

  • development business consultants – to understand trends in industry sectors and UK regions

  • marketing experts – demographic mapping and market segmentation analysts, to carry out detailed analysis of UK business using microdata

  • other local and national government departments and bodies, businesses, academics and the general public

Timeliness and punctuality

(Timeliness refers to the lapse of time between publication and the period to which the data refer. Punctuality refers to the time lag between the actual and planned dates of publication.)

The ABS publishes its releases annually. The GOV.UK release calendar publishes the release dates 12 months in advance. The ABS has consistently met the target publication deadlines. If there are any changes to the preannounced release schedule, public attention will be drawn to the change and the reasons for the change will be explained fully at the same time, as set out in the Code of Practice for Statistics.

In 2018, a view was taken to change the publication dates of the national and regional revised results. Bringing forward the release dates means that data are consistent with data used within the annual supply use balancing process. Furthermore, from the publication of the Annual Business Survey (ABS) 2017 inquiry onwards, the bulletin will form part of a combined ABS release of both the UK and regional data. The first release of the combined bulletin is anticipated to be around April to May 2019.

ABS national provisional results are published 11 months after the end of the calendar year reference period, revised national results are published 16 to 17 months after the end of the reference period, and the final revised results are published 28 to 29 months after the end of the reference period.

The long time lag between the collection and publication of ABS results reflects the size and complexity of the survey. Detailed breakdowns by industry and geography require detailed responses from a large number of businesses (62,000 businesses answering questions drawn from a possible 600). It is not possible to collect this large quantity of data more frequently, as this would place an unacceptable level of burden on businesses, and would require a prohibitively large resource to process results. Therefore, the ABS is used to explore the UK non-financial business economy in detail, but we also publish more timely, but less detailed, short-term indicators (for example, the monthly retail sales release). In this way, the need for detailed analysis as well as timeliness is met.

Back to table of contents

6. How the output is created

The Annual Business Survey (ABS) covers the UK non-financial business economy, which represents around two-thirds of the whole economy of the UK by gross value added. The Standard Industrial Classification 2007 industries covered by ABS are:

  • agriculture (support activities 01.6 and hunting, trapping and related service activities 01.7), forestry and fishing – part of section A

  • production industries – sections B to E

  • construction industries – section F

  • distribution industries – section G

  • other service industries – sections H, I, J, K (insurance and reinsurance, groups 65.1 and 65.2 only), L, M, N, P (excludes public sector), Q (excludes public sector and medical and dental practice activities, group 86.2), R, S

ABS covers the insurance and reinsurance parts of the financial and insurance sector (groups 65.1 and 65.2 in section K). However, data for this industry have remained experimental and, due to ongoing volatility, we have decided to remove it from the ABS 2012 provisional release and future ABS releases while a more detailed quality assessment is undertaken. This does not affect other industries.

The main areas excluded are:

  • agriculture – section A (crop and animal production, groups 01.1, 01.2, 01.3, 01.4 and 01.5)

  • financial activities – section K (groups 64, 65.3, 66)

  • public administration and defence – section O

  • activities of households as employers; undifferentiated goods and services-producing activities of households for own use – section T

  • activities of extraterritorial organisations and bodies – section U

Employment data are not collected by the ABS. The employment variables published by the ABS are collected by the Business Register and Employment Survey (BRES). BRES is optimised for the collection of employment data from local units, and uses different sample and questionnaire designs, and different validation and estimation methodologies to the ABS. The sampling frames are also created at different points in time. This means that care must be taken with the interpretation of measures calculated using the ABS variables and the BRES employment variables, for example, “per number of people in employment” measures of productivity.

Sample design

Data are collected for the ABS by Office for National Statistics from around 62,000 businesses in Great Britain, and by the Northern Ireland Statistics and Research Agency (NISRA) from around another 11,000 businesses in Northern Ireland.

Sample selection is carried out using a stratified random sample design. Groups of businesses (called cells) are defined by three criteria: employment size band; industry (Standard Industrial Classification 2007); and UK country. There are around 4,800 of these cells in the ABS design.

Sample selection occurs independently for each cell. When the sample is designed, the size of the sample in each cell is determined by an algorithm that distributes the sample amongst the cells to give the lowest estimated variance (uncertainty). This design is significantly more efficient (that is, it gives a much more accurate estimate for the same sample size) than a simple, unstratified random sample, or a census with a poor response rate.

A sample re-optimisation is carried out every five years to improve the efficiency of the sample estimation and reduce sampling variability as part of the regular process to improve estimates.

Imputation

Imputation techniques are used to estimate the value of the missing data due to non-response.

Imputation is carried out for large businesses with an employment of 250 or more, and for businesses with low employment but high turnover. To calculate imputed values, ratio imputation is used. This uses the returned values of businesses within a similar industry and with a similar size to estimate the value of missing responses.

For non-responding businesses with fewer than 250 employment, and without a high turnover, imputation is not carried out and totals are estimated using adjusted weights.

Estimation

In order to calculate estimates of totals for an entire population from data collected from a sample, ABS uses standard statistical weighting methods. Essentially the results received from the sample are multiplied by two weights.

The a-weight, also known as the design weight, that accounts for the fraction of the population in a particular stratum that the sample represents for that stratum. So, for example, if one business out of every five is selected in a particular stratum, each selected business will have an a-weight of five, as it “represents” five business in the population.

The g-weight, or calibration factor, makes a correction for any imbalance in the sample. For example, in a random selection of five businesses out of a population of 10, it is possible that the five businesses selected have, by chance, higher values for the variables of interest than the non-sampled businesses. If no correction is made, the population total would be over-estimated due to the variability in the population.

Auxiliary information, that is, information not collected by the survey, but already available for every business, that is statistically related to the variable of interest, is used to correct for this effect. The ratio of the actual population total for the auxiliary variable to the population total estimated from the sample’s auxiliary variables is calculated, and this is called the g-weight. For ABS, the auxiliary variables are the Inter-Departmental Business Register employment and turnover, with the choice dependent on the variable being estimated. Due to some strata containing small numbers of businesses, g-weights are calculated within groups of strata; strata representing the three smallest size bands are typically collapsed to form a g-weight band whereas strata representing larger size bands are not collapsed.

Outliers

For the ABS, outliers are defined as those returned values that are atypical when compared with similar businesses, and also have a large impact on estimated totals. The method ABS uses to treat outliers is known as the post-stratification method. In this method, the weights of the outliers are reduced to one, so that they do not have a large distorting effect on the estimates. The weights of other businesses in the same cell as the outlier are then recalculated.

Statistical disclosure

Statistical disclosure control methodology is applied to ABS data. This is to make sure that information attributable to an individual or individual organisation is not identifiable in any published outputs. The Code of Practice for Statistics and specifically the Principle on Confidentiality set out practices for how individual data are protected from disclosure. The Principle includes the statement that our outputs should “ensure that official statistics do not reveal the identity of an individual or organisation or any private information relating to them, taking into account other relevant sources of information”. More information can be found in National Statistician’s Guidance: Confidentiality of Official Statistics and also on our disclosure control pages.

The ABS Technical Report

The ABS technical report describes the procedures used by us to produce the Annual Business Survey. The report is aimed at users who want to know more about the background and history, uses and users, and concepts and statistical methods underlying the survey. It includes information about questionnaire development, sample design, data collection, results processing, publications and quality issues.

Further enquiries about ABS can be addressed to the ABS team at abs@ons.gov.uk, or, to engage in discussion about the ABS and to share information with other users or producers of business statistics, visit the Business and Trade Statistics Community on the Royal Statistical Society’s StatsUserNet discussion forum.

Back to table of contents

7. Validation and quality assurance

Accuracy

(The closeness between an estimated result and the true value.)

The Annual Business Survey (ABS) meets its legal requirements for statistical accuracy. However, as in all surveys, the estimates from the ABS are subject to various sources of error. The total error in a survey estimate is the difference between the estimate derived from the data collected and the true (unknown) value for the population. The total error consists of two main elements; the sampling error and the non-sampling error. The ABS was designed to minimise both these errors.

Sampling error

The sampling error is the error that arises because the estimate is based on a survey rather than a census of the population. The results obtained for any single sample may, by chance, vary from the true values for the population but the variation would be expected to be zero on average over a number of repeats of the survey.

The standard error gives an indication of the magnitude of the sampling error. We expect 95% of our estimates for a variable to be within two standard errors of the true unknown value for the population. The closer the standard error to zero, the more precise the estimate.

The coefficient of variation is the standard error of a variable divided by the survey estimate, and it is used to compare the relative precision across surveys or variables. The closer the coefficient of variation is to zero, the more precise the estimate in percentage terms.

Sampling errors for the ABS are available down to four-digit Standard Industrial Classification 2007 class level for the following variables:

  • total turnover

  • approximate gross value added at basic prices

  • total purchases of goods and services

  • total net capital expenditure

Non-sampling error

Non-sampling errors are not easy to quantify but can be caused by coverage issues, measurement, processing and non-response. The response rate gives an indication of the likely impact of non-response error on the survey estimates.

The main non-sampling error quality issues for the ABS are listed in this section.

Response accuracy – it is difficult to accurately quantify the effect of response inaccuracy. Questionnaires are tailored to industrial sector, so that businesses are only asked to respond to questions relevant to their industry. This helps to reduce inaccuracy. In addition, ABS has a rolling programme of questionnaire reviews, to improve and clarify the survey questions and supporting notes, and hence to help respondents complete the survey more accurately. The Annual Business Survey Questionnaire Review reports on the June 2011 review of the ABS catering and services industries questionnaires. Upon receipt of the questionnaires, responses are validated and edited where necessary. This process involves automatic totalling and rounding, date tests, and selective editing using the SELEKT tool. For further information on editing, see Section 5.1 of the ABS technical report.

ABS calendar year results – ABS results are published for calendar years. However, in order to reduce the burden on respondents, businesses have, and some use, the option to return data for their business year-end, covering any 12-month period up to and including the end of the financial year that follows the end of the calendar year. It is possible that, particularly if the economy is undergoing a period of rapid change such as during an economic downturn, the different reporting periods could introduce some bias.

Regional apportionment – data are collected by ABS at the reporting unit level. These data are then divided amongst the businesses’ local sub-units (“apportioned”) to produce the regional results. This means that the regional results are less accurate than if the data were collected at the sub-unit level, but the burden on businesses and resource costs of carrying out the ABS at the local unit level are prohibitive. At present, employment information from the Business Register and Employment Survey (BRES), which is collected at the sub-unit level, is used to apportion national results to regional estimates. Apportionment of Financial Variables Using BRES Local Unit Turnover Data describes the analysis of proposals for apportioning ABS UK financial data to the UK regions using BRES local unit turnover data, which may be a better proxy for most financial variables than employment.

Industry classification in the Inter-Departmental Business Register – industry re-classification of a business can occur due to a relatively small change to the nature of its operation, and this can have a significant effect on ABS estimates by industry. In addition, the correction of misclassification of businesses can lead to bias, particularly where there is systematic movement from one industry to another. This is because, where classification updates are identified via survey returns, it is only units in the survey sample that are updated. Where a survey does not cover the whole business population, such as the ABS, re-classification can lead to units moving out of the sample, but never into it. In the ABS, this effect is likely to be small, and is corrected for by adjusting the weights of the businesses that remain in the sample.

More detailed information on these and other quality issues is available in the ABS technical report.

Coherence and comparability

(Coherence is the degree to which data that are derived from different sources or methods, but refer to the same topic, are similar. Comparability is the degree to which data can be compared over time and domain, for example, geographic level.)

We produce other business statistics in addition to the ABS, some of which publish the same or similar variables. The Government Statistical Service (GSS) Business Statistics Interactive User Guide allows users to find out what official business statistics are available, and provides information that allows users to choose the right data for their needs. The guide is suitable for non-expert users, and users who are not familiar with the full range of business statistics available. The interactive user guide covers around 200 variables from around 40 different sources.

There are differences between the ABS approximate measure of gross value added (GVA) at basic prices and the measure published by national accounts. National accounts carry out scope adjustments, coverage adjustments, conceptual and value adjustments such as subtracting taxes and adding subsidies not included in the ABS measure, quality adjustments and coherence adjustments. The national accounts estimate of GVA uses input from a number of sources, and covers the whole UK economy, whereas ABS does not include some parts of the agriculture and financial activities sectors, or public administration and defence. ABS total GVA is approximately two-thirds of the national accounts whole economy GVA, because of these differences in scope, coverage and calculation.

No real (inflation-adjusted) estimates of regional GVA are published in the national accounts, however, nominal (non-inflation-adjusted) regional GVA and approximate regional GVA at basic prices are published by regional accounts and ABS respectively.

Further discussion of these and other issues is presented in the ABS technical report.

The ABS is designed in accordance with Eurostat regulations (Regulation (EC) Number 295/2008) to ensure comparability across European Union member states. An important aspect of this is the use of the UK’s Standard Industrial Classification of Economic Activities system (SIC), which is consistent with the European Union’s NACE system of industry classification.

The UK is required by European legislation to revise the SIC in parallel with NACE so that both systems remain identical down to and including the four-digit class level. In the UK SIC, a further breakdown is provided for certain classes by the addition of a five-digit sub-class level. Both the UK SIC 2007 and NACE (Revision 2) are completely consistent with the fourth revision of the UN’s International Standard Industrial Classification of all Economic Activities (ISIC Revision 4). Results for ABS are available on the SIC 2003 system for the reference years 1995 to 2007.

However, following the 2007 review, SIC 2003 was updated to SIC 2007, to reflect changes to the structure of the European economy, for example, the growth in technology industries. As a result, ABS estimates from reference year 2008 onwards are published classified by SIC 2007, and these are not directly comparable with the earlier results published on SIC 2003. In response to user feedback, the ABS team published the 1997 to 2007 time series converted from SIC 2003 to SIC 2007. Converted values for all the variables in the national and regional results are available, for industry levels down to industry division (two-digit SIC).

Back to table of contents

8. Other information

Assessment of user needs and perceptions

(The processes for finding out about uses and users, and their views on the statistical products.)

The Annual Business Survey (ABS) team welcomes feedback from users through the Business and Trade Statistics Community on the Royal Statistical Society’s StatsUserNet. This is a forum to promote dialogue, share information and maintain close liaison between the producers and users of official business and trade statistics. Feedback, comments and requests are also sought from users of the ABS Special Analysis service and through the ABS Government User Group.

A summary of user feedback on all aspects of the ABS, together with the ABS team’s plans to meet any identified unmet user need, and a summary of the ABS user engagement plan can be found in Responding to ABS user needs (PDF, 223 KB).

Back to table of contents

9. Sources for further information or advice

Accessibility and clarity

(Accessibility is the ease with which users are able to access the data, also reflecting the format(s) in which the data are available and the availability of supporting information. Clarity refers to the quality and sufficiency of the metadata, illustrations and accompanying advice.)

Our recommended format for accessible content is a combination of HTML web pages for narrative, charts and graphs, with data being provided in usable formats such as CSV and Excel. Our website also offers users the option to download the narrative in PDF format. In some instances other software may be used, or may be available on request. Available formats for content published on our website but not produced by us, or referenced on the our website but stored elsewhere, may vary. For further information please refer to the contact details at the beginning of this report.

For information regarding conditions of access to data, please refer to the following links:

The Annual Business Survey (ABS) publications and data are available free of charge, for reference years 2009 to the present. Releases of the Annual Business Inquiry - part 2 (ABI/2), from 1995 to 2008 and methodology information are also available.

ABS releases include a quality and methodology section to aid user understanding and interpretation of ABS estimates.

The ABS technical report covers all aspects of the production and publication of ABS estimates.

Further enquiries about ABS can be addressed to the ABS team at abs@ons.gov.uk, or, to engage in discussion about the ABS and to share information with other users or producers of business statistics, visit the Business and Trade Statistics Community on the Royal Statistical Society’s StatsUserNet discussion forum.

Back to table of contents