Labour market transformation – update on progress and plans: December 2024

Labour Force Survey and Annual Population Survey

• We continue to address Labour Force Survey (LFS) quality issues through targeting improvements across data collection, methodology and communication about how LFS data should currently be used.

• The recovery plan implemented into the Labour Force Survey (LFS) from late 2023 continues to improve response levels and the size of datasets, though the changes have led to some data instability, as the changes take time to progress through waves of data collection into outputs.

• LFS quarterly person data have been reweighted back to 2019 using updated population numbers; we have published an article showing the impact, and full results will be released on 17 December as part of the regular Labour Market publication.

• Reweighted two-quarter longitudinal LFS person outputs are expected in Spring 2025, and a more complete reweighting of all LFS and Annual Population Survey (APS) data expected to follow during 2025 and 2026.

• Following discussions with users about quality of LFS and APS, we continue to increase fieldwork capacity, undertaking further assessments of potential bias in the outputs, and bringing together survey and administrative data sources to help build a comprehensive understanding of the labour market.

Transformed Labour Force Survey

• The focus of activity on the Transformed Labour Force Survey (TLFS) has been to undertake tests to improve the quality of the data, investigating the effects of a shortened questionnaire and specific changes to the content; early indications show positive outcomes from these tests, notably some reduction in partial responses.

• TLFS remains the long-term solution for collecting labour market data, and we expect to deliver a more detailed evaluation early next year to inform Spring 2025 decisions on the future design of the TLFS.

The Labour Force Survey (LFS) remains the lead source of labour market outputs and as such we are continuing to address quality issues through targeting improvements across data collection, methodology and communication about how LFS data should currently be used. The design of the survey and complexity of the issues mean that it will take time for the impact of any improvements to be fully realised.

Improvements to the achieved sample

The recovery plan implemented into the Labour Force Survey (LFS) from late 2023 is explained in our Labour Force Survey: planned improvements and its reintroduction methodology. The recovery plan focused on increasing the achieved sample size and continues to improve response rates, and subsequently to increase the size of datasets available to analysts. In the period from April to June 2023 before the sample size reduction, we achieved 36,526 individual responses to the LFS (GB excluding imputation). Following the reduction, this dropped to 29,220 individual responses for the period July to September 2023. This then increased to 42,398 in July to September 2024, the latest period, where the effects of the sample increase implemented in January 2024 can be seen in Wave 3 of the survey design. We expect the LFS dataset size to increase further in coming periods, as the effects of changes to the survey progress through the waves of survey collection.

During this period, there may be more instability in LFS estimates as they are likely to be affected by these collection changes, in addition to any underlying changes in the labour market.

While interviewer resource was being increased, the focus of the recovery was predominantly an increase of the Wave 1 sample implemented from January 2024. As the increased sample has fed through to later waves, and to return dataset sizes to levels achieved prior to the coronavirus (COVID-19) pandemic, we are now widening the focus of fieldwork to follow-up waves with additional interviewer resource being recruited. This is expected to increase the achieved sample across all five waves and reduce bias by revisiting previously unproductive cases (known as "non-contacts").

Reweighting with new population totals

In our Labour market transformation – update on progress and plans: July 2024 article, we explained our plan to reweight the LFS person data back to 2019 using the latest national-level population projections and Census-based estimates of the Scottish population. Further information on the impact of the revised weights on labour market estimates has been released alongside this update, with the release of the reweighted data following as part of the regular Labour Market publication on 17 December. The reweighting of outputs will also be applied to the two-quarter longitudinal LFS person outputs, which are expected in spring 2025.

A full reweighting covering all datasets from the LFS and APS is planned to start in Spring 2025, pending the publication of our new Subnational population projections: 2022-based release. We will continue to engage with users on their priorities for reweighting to shape our plans and will communicate more details in the new year.

Assessment of LFS bias

We regularly update our assessment of bias in the LFS, given the substantial collection changes taking place and the potential impact on data quality.

The most recent Stakeholder Advisory Panel on Labour Market Statistics on 24 October (chaired by Professor Jonathan Portes) explored how the issues affecting the LFS might be contributing to bias in outputs. These challenges include:

• increasing difficulty in accessing gated properties

• unwillingness to share personal information

• unwillingness to allow interviewers in homes

The Panel will discuss this further in a special session later this month to which the technical engagement group and our academic experts, Professors Ray Chambers and James Brown, have also been invited. We will use this session to assess whether any further methodological actions are necessary.

Coherence of labour market data and administrative sources

As described in our labour market releases in recent months, there are ongoing challenges in assessing the coherence between labour market statistics. With the LFS data now reweighted, we plan to update our analysis on coherence between the LFS and Workforce Jobs, with an article summarising the results planned for 2025.

We are actively pursuing other data sources and sharing of data across government to use alongside survey sources.

Working in partnership with important stakeholders, and learning from other countries, we are refreshing our research plans to explore how administrative sources can be used alongside our survey sources to build a more complete understanding of the labour market. This includes using the data for quality assurance, to inform methods, or to complement survey content.

One example of work we are currently doing is using the regular supply of PAYE RTI data that the Office for National Statistics (ONS) now receives to link to LFS (and TLFS) to further build our understanding of LFS data quality.

We are also exploring the work needed to produce labour market accounts that provide a comprehensive understanding of the UK's labour market, by using the optimum balance of survey and administrative data.

Further developments and research with surveys

The Independent Review of the UK Statistics Authority (the Authority) conducted by Professor Denise Lievesley earlier this year recommended that we explore the possibility of mandating responses for some important surveys. Mandating would align with the Census and business surveys, although mandating responses to the LFS would require legislation. This is not something the ONS can consider alone, but we will take forward further discussions to understand the benefits and drawback of mandating surveys.

Given the strategic challenges that surveys are facing, we are establishing a new Surveys Innovation Hub, expanding our portfolio of survey research, and continuing to work closely with the Economic and Social Research Council (ESRC) funded Survey Futures project. On Survey Futures, we are conducting a suite of research to improve our ability to reach the public, build trust and gain consent to overcome barriers to completion of our household surveys. This includes exploring the use of alternative data sources in our end-to-end survey design, such as improved usage of name and address databases, and adapting our samples and operational efforts on contacting those population sub-groups that are typically less likely to engage in our surveys.

The Annual Population Survey (APS) is built making use of Labour Force Survey (LFS) responses and additional sample boosts. It will therefore also benefit from the improvements made to the LFS mentioned in previous sections, but over longer timescales.

In line with the decision to suspend the National Statistics accreditation of our LFS outputs in February 2024, and since the reweighting has not yet been applied to APS datasets, on 9 October 2024 we proposed that the Office for Statistics Regulation temporarily suspend the National Statistics accreditation for our APS-based outputs. Outputs are currently labelled official statistics in development.

With a view to regaining accreditation, we subsequently held workshops with APS users across government on 5 November and 7 November 2024, and later conversations with other user groups. This provided a platform for users to engage and share their assessments of the quality of APS outputs, the sample sizes and approaches to data collection, and to discuss possible mitigation activities and actions that the ONS could be taking to address quality concerns. Additional analysis, by analytical and methodological colleagues, is also in progress into specific aspects of survey quality, including attrition, bias, and approaches to weighting.

We will continue to inform users about how these developments and changes will support our survey outputs.

The Transformed Labour Force Survey (TLFS) remains the long-term solution to collecting labour market data. Work is progressing to resolve the current quality issues with the data. These are:

• bias

• missingness

• collection of complex variables, such as industry and occupation

Shorter TLFS tests

Following the plans we presented in our Labour market transformation – update on progress and plans: July 2024, in October and November 2024, we ran tests to identify:

• the impact of shortening the TLFS questionnaire on the levels of response to the survey

• the proportion of partial individual and household responses

• the impact of two important changes to the questionnaire content

Alongside normal operations of the TLFS, we also invited 30,000 households in England to take part in a two-week online-only data collection exercise, split into three treatment groups. Households in the first treatment group were asked to complete a shorter questionnaire more focused on labour market topics, aiming to reduce the average household completion time from nearly half an hour to about 15 minutes. The second treatment group were given the same shorter questionnaire as the first treatment group, but with the addition of questions on the name and address of the business employing them. These questions were included to test whether it was beneficial to link responses to alternative sources of data for some variables, such as workplace location and industry of employment. As these questions are traditionally difficult for respondents to answer, we wanted to check that including them did not make people more likely to drop out of the survey. The third treatment group was given the same questionnaire as the second treatment group, but with questions about earnings removed to evaluate the specific impact of these questions on rates of drop-off. The test also included the use of a QR code to add to ease of response.

Early indications are that the level of partial responses was reduced when respondents were asked to complete a shorter survey, and that a greater proportion of larger households submitted complete responses.

A more detailed evaluation of these three tests is currently in progress to be discussed with our users in the new year.

TLFS parallel run

While the test activity is being conducted and analysed, TLFS data collection has continued, with updates provided to key users to build our understanding of data quality. This has included an additional calendar quarter of data and the first assessment of the two-quarter employment flow outputs. These new data have shown that while the headline estimates from TLFS are relatively stable, the substantial quality issues that led to the test and design activity remain.

The TLFS data are currently being reweighted in line with changes to the LFS and will be shared with our users for quality assurance in 2025.

Wider TLFS design work and transition timings

Alongside the test evaluation, work is also in progress to develop future TLFS designs that enable both labour market and wider socio-economic requirements to be met, while still accommodating a shorter labour market-focused survey. In part to inform this, the Office for National Statistics (ONS) has recently commissioned Southampton University to conduct an international review of LFS methods to help us stay up to date with best practice in this area.

There are a number of potential scenarios for when the ONS will be able to responsibly transition labour market users from using the LFS to the TLFS, which are dependent on the scale of the design changes that we decide to implement. Transitioning in mid-2025 is now unlikely, given current quality concerns. Aiming to complete this process in 2027 would allow us time to implement the shorter survey with further periods of parallel run. We are continuing to explore options to minimise the timeframe to transition. We will provide an update on timescales in Spring 2025, following engagement on the findings of the test activity with our expert groups.