1. Methodology background


 National Statistic   
 Survey name  Vacancy Survey QMI
 Frequency  Monthly
 How compiled  Sample based survey
 Geographic coverage  UK
 Sample size  6,000
 Last revised  6 November 2012

Back to table of contents

2. Overview

  • measures the number of vacancies that are actively seeking recruits from outside organisations
  • questionnaires are sent to a sample of approximately 6,000 businesses every month
  • adopted as a National Statistic in June 2003
  • uses the Inter-Departmental Business Register (IDBR) as the sampling frame
  • historic data available on a monthly basis from June 2001

The Vacancy Survey (VS) is a statutory, monthly survey of businesses. The survey asks a single question: how many job vacancies did a business have in total (on a specified date) for which they were actively seeking recruits from outside their organisation. Results from the survey cover all sectors of the UK economy and all industries, with the exception of employment agencies and agriculture, forestry and fishing. The headline series are based on 3 month moving averages, by type of industry and by employment count. Vacancy statistics are not available by region.

The Inter-Departmental Business Register (IDBR) is used as the sampling frame. The total sample is approximately 6,000 businesses per month, with approximately 1,300 large businesses included every month and the remaining 4,700 consisting of smaller enterprises randomly sampled on a quarterly basis.

Vacancy statistics are published in the Labour Market statistical bulletin, usually within 6 weeks of the reference date of the survey. Short term statistics provide important information about the demand for labour and can provide an early indication of what is happening in the labour market. They are used for monitoring and policy making purposes by UK government departments and Eurostat. They are also used by businesses and analysts to gauge the strength of the labour market.

Back to table of contents

3. Executive summary

This report is part of a rolling programme of quality reports being introduced by the Office for National Statistics (ONS). Quality and Methodology Information (QMI) is an overview that pulls together important qualitative information on the various dimensions of quality as well as providing a summary of methods used to compile the output.

This report relates to the Vacancy Survey (VS). This survey is a statutory, monthly survey of businesses, which asks the single question, how many job vacancies did a business have in total (on a specified date) for which they were actively seeking recruits from outside their organisation.

Questionnaires are sent to a sample of approximately 6,000 businesses every month. Results from the survey cover all sectors of the UK economy and all industries with the exception of employment agencies, and agriculture, forestry and fishing. The headline series are based on 3-month moving averages, by type of industry and by employment count. Vacancy statistics are not available by region. Results of the survey were adopted as National Statistics in June 2003; back series are available from June 2001.

Vacancy statistics are published in the Labour market statistical bulletin usually within 6 weeks of the reference date of the survey. Short-term vacancy statistics provide important information about the demand for labour. They can give an early indication of what is happening in the labour market as the advertisement of a vacancy is one of the first tangible signs of an employer intending to recruit.

This report contains the following sections:

  • Output quality
  • About the output
  • How the output is created
  • Validation and quality assurance
  • Concepts and definitions
  • Other information, relating to quality trade-offs and user needs
  • Sources for further information or advice
Back to table of contents

4. Output quality

This report provides a range of information that describes the quality of the output and details any points that should be noted when using the output.

We have developed Guidelines for Measuring Statistical Quality; these are based upon the five European Statistical System (ESS) Quality Dimensions. This report addresses the quality dimensions and important quality characteristics, which are:

  • relevance
  • timeliness and punctuality
  • comparability and coherence
  • accuracy
  • output quality trade-offs
  • assessment of user needs and perceptions
  • accessibility and clarity

    More information is provided about these quality dimensions in the following sections.

Back to table of contents

5. About the output

Relevance

(The degree to which the statistical outputs meet users’ needs.)

The Vacancy Survey produces monthly estimates of job vacancies across the whole economy. Questionnaires are sent to a sample of approximately 6,000 businesses every month, approached mainly via head offices.

The survey covers all sectors of the economy and all industries in England, Scotland and Wales (Great Britain) with the exception of employment agencies (to avoid double-counting of vacancies) and agriculture, forestry and fishing (because of the disproportionate costs involved as these industries mainly consist of very small businesses with few vacancies). Northern Ireland businesses are not approached because of the risk of overlap with other surveys conducted by Northern Ireland departments.

Estimates for UK are derived by weighting up the data for Great Britain using employment estimates (Northern Ireland accounts for around 3% of UK employment). The headline series are based on 3-month moving averages, by type of industry and by employment count. Vacancy statistics are not available by region.

What it measures

Number of vacancies, across the UK, for which recruits are being actively sought from outside of organisations.

Frequency

Monthly.

Sample size

Approximately 6,000.

Periods available

Monthly from June 2001.

Sample frame Inter-Departmental Business Register (IDBR).

Sample design

Stratified random sample with 100% coverage of businesses with employment (as recorded on the IDBR) above a threshold that varies by industry.

Weighting

In strata without 100% sampling each business represents a number of similar businesses, based on the number of employees and the Standard Industrial Classification 2007: SIC 2007. The design weights are adjusted using the ratio model, where register employment is the auxiliary variable (ratio estimation). Weights are updated monthly.

Estimation

Results are produced using a ratio estimator, which is one of the standard estimators used by Office for National Statistics (ONS). Businesses with an employment of one are not sampled and hence their contribution to the total estimate is obtained using the ratio model based on businesses with an employment between two and nine.

Imputation

A link factor is calculated and applied to previous returns for each non-responder. The original construction for a never-responding business is calculated from a ratio (calculated from other respondent values in the same strata) being applied to the register employment. For subsequent periods, imputed values will be based on movements in similar-sized businesses.

Outliers

Outliers are detected manually and are treated by setting their estimation weight to one, thereby representing only themselves.

Seasonal adjustment

The series are seasonally adjusted in X-12-ARIMA using a multiplicative model.

Vacancy statistics provide important information about the demand for labour and give an early indication of what is happening in the labour market. They are used for monitoring and policy-making purposes by UK government departments and Eurostat. Examples are:

  • HM Treasury – vacancy statistics are used by HM Treasury to assess labour market conditions and are an important indicator in the macroeconomic assessment

  • Bank of England – the Bank’s Monetary Policy Committee uses vacancy statistics in the assessment of the labour market, that is, the balance between demand and supply and the extent of inflationary pressures in the economy

  • Department for Work and Pensions (DWP) – whole economy estimates allow DWP to give some context to the Jobcentre Plus estimates

  • Department for Business, Energy and Industrial Strategy – the levels of vacancies are tracked as one of the indicators of the balance of supply and demand for skills

  • Eurostat – Eurostat receives quarterly job vacancy estimates in order to monitor short-term changes in job vacancies by economic activity and to draw comparisons with other countries

In 2002, Eurostat agreed with member states that the collection of vacancy statistics should be carried out under a gentlemens’ agreement. In 2008 a new regulation, EC Number 453/2008 was passed making the collection a mandatory requirement from 2010. The UK has played an active part in the task force to develop the regulation, which the UK Vacancy Survey fully meets.

Vacancy statistics are also used by businesses and analysts to gauge the strength of the labour market.

A summary of users’ views can be found later in this report in the “Assessment of user needs and perceptions” section, which is located under “Other information”.

Timeliness and punctuality

Timeliness refers to the lapse of time between publication and the period to which the data refer. Punctuality refers to the gap between planned and actual publication dates.

The Vacancy Survey reference date falls on the first Friday of the month, unless this is the first day of the month. In this case, the reference date moves to the second Friday of the month. Data collection for each period closes down on the 16th working day of the calendar month after the survey reference date. Vacancy statistics are then published in the Labour market statistical bulletin usually between 6 and 7 weeks after the reference date of the survey.

We publish the latest month's vacancy estimates along with revisions to the previous 3 months’ estimates and figures for the same period a year ago. In April of each year, revisions for the previous 3 years (38 months) are published and are pre-announced in the March statistical bulletin.

In the unlikely event of a change to the release schedule, an announcement will be made at least 2 weeks in advance, as set out in the Code of Practice for Official Statistics.

For more details on related releases, the GOV.UK release calendar provides 12 months’ advanced notice of release dates. If there are any changes to the pre-announced release schedule, public attention will be drawn to the change and the reasons for the change will be explained fully at the same time, as set out in the Code of Practice for Official Statistics.

Back to table of contents

6. How the output is created

Coverage

The survey covers all sectors of the economy and all industries in England, Scotland and Wales (with the exception of agriculture, forestry and fishing, and recruitment agencies). Results for Great Britain are adjusted to cover Northern Ireland using employment estimates. The survey records how many job vacancies a business has in total (on a specified date) for which they are actively seeking recruits from outside their organisation.

Sample

The total sample is approximately 6,000 businesses per month. Approximately 1,300 large businesses are included in the survey every month. The remaining 4,700 are smaller enterprises sampled randomly on a quarterly basis and are kept in the survey for 5 to 9 quarters (3-month periods) depending on the size of the business.

Businesses are grouped into five employment size-bands: 2 to 9, 10 to 49, 50 to 249, 250 to 2,499 and 2,500 and over. Following the 2009 Triennial Review it was decided to use modelling rather than sampling for enterprises with a register employment of one. After detailed analysis it was found that the number of returns greater than zero from these companies was very small and was placing an unnecessary burden on these small companies.

In common with most other business surveys conducted by Office for National Statistics (ONS), the sampling frame is the Inter-Departmental Business Register (IDBR). The IDBR stores details of about 2 million businesses registered in the UK and is maintained largely by updates from Her Majesty’s Revenue and Customs, Companies House and surveys conducted by ONS specifically geared to maintaining the IDBR.

Data collection

The data collection method used for 99% of businesses is an automated Telephone Data Entry system (TDE). Businesses return their data to us using key presses on a telephone rather than returning a paper questionnaire. A small number of businesses have elected to receive a paper questionnaire, which is returned in the pre-paid envelope provided.

Respondents have 7 days to make a return to us, if no response is received a further paper reminder is despatched to elicit a response without delay. Targeted selective editing was implemented into the survey in April 2012. This editing approach identifies potential errors in returned values that are having the greatest impact on survey estimates. Potential errors are followed up with respondents where further explanation is required.

Weighting

As it is not possible to survey every business in the population it is necessary to weight the data to provide estimates for the full population. In strata that are not fully enumerated, each business represents a number of similar businesses, based on the number of employees and the Standard Industrial Classification 2007: SIC 2007. Weights are updated monthly.

Estimation

Results are produced using a ratio estimator, which is one of the standard estimators we use. Ratio estimation uses the ratio between the survey variable (number of vacancies) and an auxiliary variable (IDBR employment) to make estimates for non-sampled units. Although the correlation between the number of vacancies and employment is not strong, use of the ratio estimator gives slightly more precise results for the whole economy than would come about through use of a simple expansion estimator. (Within stratum, expansion estimation is a popular method of estimating totals in a stratified finite population. However, if by chance several unusually large observations should fall in the sample, then the expansion estimator may grossly overestimate population totals.)

Imputation

Values for businesses that do not respond are imputed. Imputation is based on the pattern of responses for similar businesses. A link factor is calculated from responses received and applied to previous returns for each non-responder. The original construction for a never-responding business is calculated from a ratio (calculated from other respondent values in the same strata) being applied to the register employment. For subsequent periods, imputed values are based on movements in similar-sized businesses.

Vacancy ratio

The vacancy ratio is an estimate of the number of vacancies per 100 jobs, using employee jobs estimates from the Short-Term Employment Surveys (STES).

Seasonal adjustment

The Vacancy Survey data show strong seasonal patterns, with vacancies peaking around September to October and dropping around January each year. Seasonal adjustment is the process of estimating and removing effects that are caused by the time of the year and arrangement of the calendar. The parameter settings used for the process of seasonal adjustment are derived using X-12-ARIMA. The monthly series and 3-month moving average series by industry and by size of business are seasonally adjusted directly to give the best possible seasonal adjustment. These seasonally adjusted time series are quality-assured on an annual basis.

Statistical disclosure

Statistical disclosure control methodology is applied to survey data. The Code of Practice for Official Statistics and specifically the Principle on Confidentiality set out practices for how we protect data from being disclosed. The Principle includes the statement that our outputs should “ensure that official statistics do not reveal the identity of an individual or organisation, or any private information relating to them, taking into account other relevant sources of information”. More information can be found in National Statistician ’s Guidance: Confidentiality of Official Statistics.

Published data

The monthly vacancy headline estimate is based on a 3-month moving average, by type of industry and by size of business (in terms of employment). Vacancy statistics are published in the Labour market statistical bulletin.

Back to table of contents

7. Validation and quality assurance

Accuracy

(The degree of closeness between an estimate and the true value.)

Estimates from this survey are subject to various sources of error that can be categorised as sampling error and non-sampling error.

Sampling error

Sampling error occurs because estimates are based on a sample rather than a census. The Vacancy Survey estimates the size of this error by estimating standard errors and coefficients of variation.

For the monthly estimates of total vacancies, the standard errors for the level are around 20,000, or 3% expressed as a coefficient of variation (as a percentage of the estimate for the number of vacancies). This yields a confidence interval of plus or minus 40,000. For the 3-month rolling average of vacancies, the standard errors are around 10,000 (1.5% expressed as a coefficient of variation), giving a confidence interval of plus or minus 20,000. For a 3-month average of vacancies, for a typical industrial sector, the coefficient of variation is around 10%.

Non-sampling error

Non-sampling errors are not easy to quantify and include errors of coverage, measurement, processing and non-response. The Vacancy Survey usually exceeds a target monthly response rate of 80%. In 2011, the average response rate for the survey was 81.6%.

Revisions are analysed over time in terms of their size, direction and volatility. This is to monitor the accuracy of various statistical processes, such as imputations for non-response, which can be replaced by late responses.

Revisions analyses for the Vacancy Survey can be found in Implementation of SIC 2007 for the Vacancy Survey11.

Comparability and coherence

(Comparability is the degree to which data can be compared over time and domain, for example, geographic level. Coherence is the degree to which data that are derived from different sources or methods, but refer to the same topic, are similar.)

The survey uses a simple “one number” questionnaire, asking for the total number of vacancies on a particular date near the beginning of each month. The data are available either broken down by industry to 18 industrial groupings or broken down by size of enterprise.

The Vacancy Survey covers most of the UK economy, the only exception being agriculture, forestry and fishing. Data are available from June 2001 and have been classified by industry using Standard Industrial Classification 2007: SIC 2007.

Comparability with statistics from other EU countries has been considered in the development of the new regulation. A methodological comparison of vacancies data supplied to Eurostat can be found in Annex I of the Vacancy Survey Triennial Review.

Statistics of jobcentre vacancies notified by employers to Jobcentre Plus are collected by Department for Work and Pensions from the Jobcentre Plus administrative system. These provide analyses of vacancies in more detail than is possible using the ONS survey, such as statistics by local area, occupation and duration of vacancies, as well as by industry. However, only a proportion of all vacancies are notified.

The jobcentre figures can be expressed in terms of the inflow of newly-notified vacancies over a period or the number (or stock) of unfilled vacancies on a specified date each month. A variety of statistics are available on Nomis®. However, these figures have been subject to various administrative distortions in recent years. Recent changes in these figures would not necessarily signify developments in the labour market. For these reasons the jobcentre data do not provide a suitable alternative source to the Vacancy Survey. An article on publication of the jobcentre figures is available.

Back to table of contents

8. Concepts and definitions

(Concepts and definitions describe the legislation governing the output and a description of the classifications used in the output.)

International regulations and UK legislation

The Vacancy Survey is a legal requirement of the European Union. Eurostat compile and disseminate European-level estimates of vacancies. In 2002, Eurostat agreed with member states that the collection of vacancy statistics should be carried out under a gentlemens’ agreement. In 2008 a new regulation, EC No 453/2008, was passed making the collection a mandatory requirement from 2010. The UK has played an active role in the task force to develop the regulation, which the UK Vacancy Survey fully meets.

Nationally and internationally agreed definitions and standards used

The Vacancy Survey questionnaire contains just one question:

On “specified date” what was the number of job vacancies for which you were actively seeking recruits from outside your business or organisation?

This question meets the internationally agreed definition common to all European vacancy surveys.

Collection of the Vacancy Survey data from January 2010 is based on the latest definition of the Standard Industrial Classification 2007: SIC 2007. The same classification is used across the EU and impacts on a wide range of statistics produced by ONS. The article Implementation of SIC 2007 for the Vacancy Survey details the work carried out to move the survey onto the new classification.

Back to table of contents

9. Other information

Output quality trade-offs

(Trade-offs are the extent to which different dimensions of quality are balanced against each other.)

The Vacancy Survey is one of the most timely labour market releases, with survey estimates published usually within 6 weeks of the reference date. While this timetable does not have a significant impact on quality in terms of response information, there is still scope for revisions to data in later months to take account of late information from respondents or corrections for misreporting.

Each month we publish the latest month’s estimates along with revisions to the previous 3 months’ estimates and figures for the same period a year ago. In April each year, revisions for the previous 3 years (38 months) are published.

Assessment of user needs and perceptions

(The processes for finding out about users and uses, and their views on the statistical products.)

The most recent Triennial Review incorporated a survey of users’ views. Fifty-nine questionnaires were sent out to a list of known users, mostly within government departments.

A range of comments were received about the survey. Most centred on our ability to provide additional information about advertised vacancies on a monthly basis. However, these responses seldom took account of the additional burden that would be placed on businesses if we were to undertake these additional requirements or the cost of change. The most common requests were for information on regional breakdowns, occupational breakdowns and a split between full-time and part-time vacancies. One user also requested extra metadata to accompany survey results, to provide a more detailed understanding of how the outputs are derived.

Based on the responses to the survey, we will investigate fully the cost and feasibility of producing more granular estimates of vacancies on a monthly basis, including estimates by region. This investigation will take account of the burden on users, most notably those with a small employment (as recorded on the Inter- Departmental Business Register. It is, unfortunately, likely that a change of this nature will not be feasible, however, we need to be able to explain clearly to important users of the survey the main reasons why changes of this nature are not possible, both in terms of cost and burden on business.

This QMI is an example of our intention to improve the availability and accessibility of information publicly available pertaining to the Vacancy Survey.

Back to table of contents

10. Sources for further information or advice

Accessibility and clarity

(Accessibility is the ease with which users are able to access the data, also reflecting the format in which the data are available and the availability of supporting information. Clarity refers to the quality and sufficiency of the release details, illustrations and accompanying advice.)

Our recommended format for accessible content is a combination of HTML webpages for narrative, charts and graphs, with data being provided in usable formats such as CSV and Excel. Our website also offers users the option to download the narrative in PDF format. In some instances other software may be used, or may be available on request. Available formats for content published on our website but not produced by us, or referenced on our website but stored elsewhere, may vary. For further information please refer to the contact details at the beginning of this report.

For information regarding conditions of access to data, please refer to the following links:

Currently, investigations are underway into the feasibility of adding Vacancy Survey microdata to the Virtual Microdata Laboratory (VML). The VML is a facility within the Office for National Statistics (ONS), which enables access to restricted microdata for research purposes.

Researchers from government and academia use the VML to carry out research on ONS surveys and other confidential datasets.

In addition to this Quality and Methodology Information, Quality and methodology information relevant to each release is available in the background notes of the Labour market statistical bulletin.

Useful links

Vacancy survey data tables can be found in the Labour market statistical bulletin:

There have been a number of methodological articles on vacancy statistics published during the last few years. These articles can be accessed at:

Back to table of contents

Contact details for this Methodology