Labour market transformation – update on progress and plans: November 2023

This publication is the next in our series of labour market transformation updates. It follows our Labour market transformation - update on progress and plans: May 2023 article and notes our progress and achievements since then. You can also find an accompanying summary of previous updates we have shared about transforming the labour force survey.

Following the suspension of publications using Labour Force Survey (LFS) data in October, we have set out our planned improvements and aim to reintroduce LFS data in December. This is running in parallel with the transformation work. However, the move to the Transformed Labour Force Survey (TLFS) is the long-term solution for providing high-quality and granular estimates of changes in the labour market.

The ambition of transforming the Labour Force Survey (LFS) is to enable a more adaptive and responsive survey to meet user needs and maintain the quality of our  labour market statistics. The Transformed Labour Force Survey (TLFS) being online-first has enabled us to increase the sample size, improve response rates and better target respondents to improve representativeness. This will allow us to provide greater insight into the UK labour market and better inform decision-making at both  national and local levels.

We have been dual-running the LFS and TLFS since February 2022. In ourLabour market transformation - update on progress and plans: May 2023 article, we noted that all collection methods (online, telephone and knock-to-nudge) had been in place for a full calendar quarter. We now have three calendar quarters of data and have been progressing our quality assurance and analysis throughout the year.

We assess statistical quality against a number of criteria, which we shared in our previous article. Some of these criteria are to reduce bias, reduce attrition and increase response in the TLFS. Assessing the survey performance throughout this year has shown encouraging results.

Variability in response rates across region and country, Index of Multiple Deprivation (IMD) and Output Area Classification (OAC) for Quarter 2 (Apr to June) 2023 were all reduced on the TLFS compared with the LFS. This indicates more representative data with reduced bias. A greater proportion of households in the longitudinal sample also responded at Wave 5 on the TLFS compared with the LFS. This is a positive step towards reducing attrition, although greater drop-off of Wave 1 respondents suggests more could be done to retain responding households across the waves. The TLFS has higher overall response rates at every wave compared with the LFS. Average overall response at Wave 1 are 39% and quarterly Great Britain person datasets contain roughly three times as many individuals as the LFS.

Quality assurance will continue to be our focus for the remainder of this year. This will enable us to continue thoroughly analysing the data from the TLFS and assessing against our quality criteria. In addition, incremental survey improvements have continued to be made to the TLFS throughout 2023. Some of these are planned improvements and others have been informed as a result of the ongoing quality assurance and analysis. We will begin to share early results from the survey once we have consistent data covering several months.

Continuing this analysis is critical so we can take an informed decision of the most appropriate time to move to using the TLFS as the lead survey measure for our labour market and productivity outputs. We continue to work towards the aim of using the TLFS in place of the LFS from March 2024; a list of the publications that will use the TLFS can be found in our previous article. The dates on which releases use the TLFS for the first time may change, as we are dependent on ensuring quality criteria targets are met before any transition. We will continue to keep users updated.

We release several different types of microdata based upon the Labour Force Survey (LFS) and intend to continue delivery with the Transformed Labour Force Survey (TLFS). LFS datasets include quarterly person-basis, quarterly household-basis, annual person-basis, annual household-basis, a three-year pooled person-basis, and two longitudinal files (two-quarter and five-quarter). The annual files are known as the Annual Population Survey (APS).

We aim to continue releasing similar quarterly, annual and longitudinal datasets based on TLFS data, following the same publication schedule as the usual labour market statistics. The files will be in a CSV format rather than SPSS. This should make them more easily compatible with a wider range of software. The naming convention used for variables will also be noticeably different in the TLFS, but the files will continue to contain all available records and variables, as well as associated weights and derived variables. The final APS files based entirely on LFS data are planned to cover the period January to December 2023. Following this, we will continue to produce annual files but based on the TLFS, with the first covering the period April 2023 to March 2024.

Work on the production and contents of these datasets continues through the TLFS development process, incorporating feedback provided through our peer review process. Further information about the microdata can be found in our TLFS user guidance.

In recognition of its importance and aligned with the User engagement strategy for statistics, the Transformed Labour Force Survey (TLFS) has an engagement strategy that underpins the transformation of labour market statistics. Its primary aim is to ensure that the transformation takes place in accordance with the Code of Practice for Statistics.

The code is based on three pillars - trustworthiness, quality and value. Within these pillars, there are required practices when producing and releasing official statistics. Effective user engagement is part of all the pillars and is important to successful transformation programmes.

As quality is one of the three pillars, we have initiated an expert peer review process to support and assure the appropriate content and quality of the survey alongside our internal process. We continue to engage with a core group of external users, including other government departments, the devolved administrations and some economic experts. In July, we shared January to March 2023 TLFS data and supplementary information. In October, we were able to share further data points covering October 2022 to June 2023, alongside supplementary information on any changes since the first iteration of the data shared.

The purpose of the data sharing exercise is to gain valuable feedback from some of our users comparing TLFS results with existing results and other available data sources. This helps us to ensure we are able to use the data to continue producing existing publications, further strengthening our quality assurance. Through our ongoing engagement with the peer review group, we have received useful feedback, covering a range of topics and technical themes.

The peer review process is iterative and more data will be shared with users as the transformation progresses.

In our Labour market transformation -- update on progress and plans: May 2023 article we noted changes in the way we are collecting data in different UK geographies in collaboration with our colleagues in the Northern Ireland Statistics and Research Agency (NISRA).

Historically, the data collection of the Labour Force Survey (LFS) in Northern Ireland (NI) has been conducted by NISRA, with the Office for National Statistics (ONS) processing the data before passing it back to NISRA to facilitate the publishing of results for NI.

As part of its work to transform the LFS in Northern Ireland, NISRA have been preparing to take over responsibility for the processing of the LFS data in NI, which will continue to run through to summer 2024. In parallel, NISRA have been developing a replacement for the LFS, known as the NI Labour Market Survey (LMS), which will also be an online survey. In time, this will be the primary way of delivering labour market indicators for NI and feed into the corresponding UK statistics.

NISRA and the ONS have created a dedicated team, with subject matter experts from both organisations working to ensure smooth delivery of UK labour market statistics throughout the transformation.

We will continue our regular engagement with users through established channels, including our steering group, user groups, and bilateral engagement with various departments. We will also continue to publish regular transformation updates and expect the next update to be early in 2024.

We also aim to update the Transformed Labour Force Survey (TLFS) user guidance in January 2024 with the latest details on the design and additional information about questionnaire content.

We welcome your feedback on this latest update and our plans. Please email labour.market.transformation@ons.gov.uk to tell us what you think.