1. Overview

This publication is the next in our series of labour market transformation updates. It follows our Labour market transformation - update on progress and plans: September 2022 article and notes our transformation progress and achievements since then.

You can also find a summary of previous updates in our Making everybody count: How we're transforming the Labour Force Survey blog post.

Back to table of contents

2. Our transformation journey

Section 4 of our September update detailed our transformation journey over the last few years and includes our current timeline for the transformation of the Labour Force Survey (LFS). 

As part of that, in November 2022, we introduced our “knock to nudge” field mode as part of our adaptive survey design for the Transformed Labour Force Survey (TLFS). This was specifically aimed at encouraging response from households in under-represented areas by using targeted face-to-face engagement on the doorstep.

The resulting data will be closely evaluated to determine the impact of the knock-to-nudge visits on the demographic makeup, and specifically how effective face-to-face engagement is at reducing bias. This evaluation will also feed into decisions about whether further design changes are required, such as the introduction of face-to-face interviewing.

November 2022 also saw the publication of our first iteration of our Transformed Labour Force Survey - user guidance. This is a suite of guidance documents to help our users familiarise themselves with the TLFS and get a better understanding of how it differs from the current LFS. We also held walkthrough sessions with users to discuss the content.

Further guidance relating to derived variables, and further updates to current guidance will be published later in 2023. 

In addition to improving the methods of collection on the TLFS, we are also improving the methods we apply to the data further to improve accuracy.

Imputation

We are making changes to the way we account for missing data. Individual variables that contribute to derived measures will be imputed rather than the limited number of derived variables imputed for the LFS. This should bring greater accuracy by making use of information within the data for each component, including previous responses where appropriate, and maintain consistency between variables in the microdata. To do this, we will need either donor-based or machine learning imputation methods, similar to the alternative methods recently introduced on the LFS. Full requirements are being specified, and the final method will be developed into a sustainable imputation pipeline.

Weighting

Weighting methods, while broadly similar to the LFS, will be developed into a sustainable pipeline. We are revisiting adjustments introduced during the pandemic (tenure distribution growth rate and UK, EU, or other country of birth growth rate based on HM Revenue and Customs (HMRC) Real Time Information (RTI) data). While effective at the time, the rates are not timely enough for ongoing production in the TLFS. We are exploring the impact of removing these adjustments from the weighting process or replacing with alternative data sources. 

Model-based methods

Using the monthly TLFS datasets, we are aiming to be able to produce more timely and precise labour market statistics by main domain breakdowns (regions, age groups, sex).

Longitudinal

Dataset investigations are also underway to assess the effect of sample size differences between the LFS and the TLFS and how this will affect weighting of longitudinal datasets, particularly the effect this will have on precision of estimates.

Analysing the data collected following the introduction of "knock to nudge" and the impact of improved methodology over the coming months is one of our main priorities. This will be an important consideration in deciding whether to make the TLFS the primary source of official estimates of the UK labour market. We have been working towards switching to the TLFS in September 2023, but we continue to review our plans. As more data and analysis become available, we will have a clearer view on whether this is feasible. 

Back to table of contents

3. Governance of the transformation process

The development of the Transformed Labour Force Survey (TLFS) is a collaboration between Social Survey Transformation (SST) and the Economic, Social and Environmental Statistics Group (ESEG) within the Office for National Statistics (ONS) and is governed by a series of meetings. These include a steering group, comprising ONS representatives and representatives from external government departments and devolved administrations. These meetings also include programme and portfolio boards that use existing ONS governance frameworks to make decisions. While positively demonstrating the seniority of oversight and decision-making for TLFS, they also ensure the relevant discussion and prioritisation takes place in an open, transparent reporting manner. 

The Office for Statistics Regulation (OSR) is conducting a review of the transformation of the Labour Force Survey. This work ensures that practices are followed as outlined in the Code of Practice for Statistics. Compliance with the Code of Practice gives users confidence that published government statistics comply with the pillars of trustworthiness, quality and value.

Its initial findings are positive, recognising and fully supporting our ambitions to improve the quality, coverage, adaptability and responsiveness of the Labour Force Survey (LFS) to maximise its relevance and meet changing user needs. We welcome these early findings and recommendations and have responded to the OSR, noting our plans for suggested areas of development and ongoing engagement.

Back to table of contents

4. Annual Population Survey

The Annual Population Survey (APS) will change as a result of the transformation. The APS is a derivation of the Labour Force Survey (LFS), where multiple waves of LFS data are combined to create a larger dataset. This can be used to create more accurate estimates or slightly more detailed breakdowns than can be done with the quarterly files, but at the expense of being on an annual basis rather than quarterly. 

The APS will continue to be produced using the current LFS as normal, with the last dataset on current basis covering the period July 2022 to June 2023 on both a person basis and a household basis. The July to June dataset is scheduled for release in September 2023. We aim to produce an equivalent to the APS using data from the transformed LFS once enough data (so, four full quarters) are collected. This would cover the period between July 2023 to June 2024. We are currently investigating whether it is possible to create an APS file composed of a mixture of current and transformed data for the period when we transition from LFS to Transformed Labour Force Survey (TLFS) data.

Back to table of contents

5. Use of administrative data

As part of our journey to transform labour market statistics, we aim to make greater use of administrative data. To achieve this, we are working to include Pay As You Earn (PAYE) Real Time Information (RTI) data in our statistical output, as previously outlined in our Labour market transformation - update on progress and plans: September 2022 article. During 2022 we researched the opportunities provided by the dataset and now have a plan and priorities for development throughout the next few years. 

Our first aim is to establish a benchmark against the existing HM Revenue and Customs (HMRC) Earnings and employment from Pay As You Earn Real Time Information, UK. We will use this to test our methods and development. This will provide the information needed to make decisions about future development.

Providing the quality checks are favourable, we will then focus on enhancing the output for our Average weekly earnings in Great Britain bulletin. Opportunities include providing earnings statistics at a more granular level for industry, geography, and sector, as well as improving the accuracy of earnings data by including earnings data for smaller businesses.

There are many more opportunities to be identified and we will be conducting further research as we proceed with the project. We are continually mapping our findings to the aims of the project. The following diagram contains the current view.

Back to table of contents

6. Future developments

We continue our regular engagement with users through established channels such as our steering group, user groups, and bilateral engagement with various departments. We also have our dedicated labour market transformation inbox, where users can contact us regarding any aspect of the transformation project.

Alongside the ongoing analysis of the Transformed Labour Force Survey (TLFS) data and development of improved methods, we are committed to sharing outcomes of the analysis with users, ahead of the TLFS becoming the primary source of official estimates of the UK labour market. It is important we fully understand the impact of introducing "knock to nudge" and improved methods, and this will need to have matured before we are able to share with our users more widely. We expect to have firmer plans of when this will be, and how we plan to share this with our users, in the next transformation update.

We will be attending the Labour Force and Annual Population Surveys User Conference in February 2023, where presentations will be given on both the current Labour Force Survey (LFS) and Annual Population Survey (APS) and the TLFS. 

We welcome your feedback on this latest update and our plans, please email labour.market.transformation@ons.gov.uk to tell us what you think.

Back to table of contents

7. Cite this article

Office for National Statistics (ONS), released 24 January 2023, ONS website, article, Labour market transformation - update on progress and plans: January 2023

Back to table of contents

Contact details for this Article

Leigh Skuse
labour.market.transformation@ons.gov.uk
Telephone: +44 1633 456216