The purpose of this short paper is to draw the attention of key users to a methodological change that will be made to Workforce Jobs (WFJ) estimates by region and industry (section level) on a quarterly basis from March 2012 (the next publication of estimates).
The change in methodology will be backdated and as a result there will be revisions to previously published estimates.
WFJ is a quarterly measure of the number of jobs in the United Kingdom (UK) and is the preferred measure of the change in jobs by industry.
It is a compound source that draws on a range of employer surveys, household surveys and administrative sources.
WFJ is the sum of employee jobs (EJ) measured primarily by employer surveys (predominantly the Short-Term Employment Surveys (STES) and the Quarterly Public Sector Employment Survey (QPSES)), self-employment jobs (SEJ) from the Labour Force Survey (LFS), and government-supported trainees (GST) and Her Majesty’s Forces (HMF) from administrative sources.
A variety of outputs by industry, region, gender and full/part-time status are produced for a range of publications and users.
A fundamental redevelopment of WFJ sources, classifications, methods and systems was recently undertaken and is explained clearly in the article ‘Revisions to Workforce Jobs’ (Barford 2010).
One of the key changes highlighted in this article was the replacement of a matched-pairs estimator with a point-in-time ratio estimator, ONS’s standard method.
This change was aimed at removing the bias caused by the matched-pairs method. A matched-pairs method tends to underestimate change over time, as it excludes the births and deaths of businesses in the sample.
In essence, only those businesses sampled in two consecutive periods are used to produce estimates of change.
This bias used to cause large revisions when the STES series were benchmarked retrospectively to Business Register Employment Survey (BRES) estimates.
BRES is an annual survey which selects a larger sample and also uses a point-in-time ratio estimator. The point-in-time estimator includes all sampled businesses in each and every period, which reduces the bias over-time.
The trade-off is an increase in volatility caused by the inclusion of the rotated part of the sample for small and medium sized businesses. Sample rotation spreads the administrative burden; ensuring businesses are selected for a limited number of periods.
Unfortunately, the degree of volatility of regional estimates at an industry level has been greater than anticipated and in general has been met unfavourably by users, particularly those that are interested in regional data.
There are a number of instances, for example, whereby businesses have been rotated in to a region and served to distort the level of jobs for a particular industry, usually for a period of five quarters, which is the length of time that a small or medium sized business remains in the sample.
Notification of change
Based on user feedback and in consultation with key stakeholders ONS have been working hard over the past nine months to make improvements to the method of estimating the number of jobs by region and industry.
These changes will be reflected in the March edition of the Labour Market Statistical Bulletin (reference period 2011 Q4).
Overview of the current methodology and details of the proposed change
Some of the strata that form the 'STES' sample contain very few units compared to the overall population, and hence require very large associated design weights (the ratio of business count from the population and business count from the sample).
A proportion of these units are rotated each quarter. Since there is no regional dimension to the sample design (it is at GB level, which is standard for ONS surveys), the number and type of units being rotated in or out of any 'regional industry' will vary over time.
Thus for any period, one or some of the highly-weighted units may be rotated in or out of a particular regional industry without being balanced by a similar movement in the other direction. This can lead to extreme volatility.
To account for this, estimates are calibrated to known regional employment totals from the sampling frame (the Inter-Departmental Business Register or ‘IDBR’).
The current methodology uses a two-way calibration with respect to marginal totals of industry section and region; weights are calculated so that in each section and each region, the estimate of total employment is equal to the total register employment.
However, results over the last seven quarters suggest that in some cases this level of calibration is too broad to produce reliable estimates of region by industry section.
Our proposed method for improving this situation involves calibrating or ‘controlling’ estimates to a more ‘targeted’ total based on the IDBR.
Weights are now calculated so that in each ‘regional section’ (the cross classification of region and industry) the estimate of total employment is equal to the total register employment.
As a consequence, new estimates at a region by industry section level should be driven more by differences between returned and registered survey data than the current version and thus less volatile.
This change will be worked back to 2008 Q3, which is consistent with the fundamental redevelopment outlined above.
Sampling errors and CVs
Approximate sampling errors at the regional section level have been produced. The sampling errors are smaller using the proposed method.
Regional estimates below section level
The methodology change also necessitates a change to the production process of regional series by lower level Standard Industrial Classification (it should be noted that the discussed volatility at the level of industry section by region is further exacerbated in lower level industry series on the current basis).
Instead of these estimates being routinely produced by ONS, users will be recommended to take regional estimates from BRES (Business Register & Employment Survey) for these lower level industries, calculate the proportions relative to their industry section and apply these to the improved quarterly Employee Jobs estimate of region by industry section.1
To alleviate concerns regarding the timeliness of BRES and applying a fixed annual ratio to quarterly estimates, it is important to note that the current Workforce Jobs methodology makes heavy use of the IDBR (Inter-Departmental Business Register) to apportion sample returns to local units, which in many cases is less timely than BRES.
As such, the proposed method is not likely to reduce the quality of regional estimates at this lower level of aggregation.
It is important to note that estimates at UK and GB level (at all levels of industry) are not affected.
Revised Workforce Jobs estimates will be published on 14 March 2012.
These estimates will take account of benchmarking to the latest estimates produced from the annual Business Register and Employment Survey (BRES) as well as the new proposed methodology.
The release of data on 14 March 2012 will be accompanied by a methodological article detailing the changes that have been made to the process of estimation in full.
For further information about the proposed methodological change to Workforce Jobs please contact, in the first instance, Steffan Hess (firstname.lastname@example.org; 01633 456714) or David Matthews (email@example.com; 01633 456756).
Barford N (2010) ‘Revisions to Workforce Jobs
(268.7 Kb Pdf)
’, Economic & Labour Market Review, Vol 4 No 9.
1 Data are not published on the ONS website at this level, but are currently made available on request. This affects a small number of users who have all been contacted separately.