Labour market transformation – update on progress and plans: April 2024

This article provides an update to users on how we are improving the quality of our labour market statistics. It covers the actions the Office for National Statistics (ONS) is taking to improve the quality of existing Labour Force Survey (LFS) based statistics, while also providing an update on progress to transition to our Transformed Labour Force Survey (TLFS).

We have been implementing a recovery plan for the LFS, after a fall in the number of responses – a problem affecting many countries' statistics institutes – meant that publication of these key metrics had to be suspended in the Autumn. Our wider programme of work is also driven by our move towards an online-first survey, which will enable a larger sample size, more detailed statistics, and reduced overall cost.

Because of the implementation of the recovery plan for the LFS, we have been able to resume publication of these key statistics. This article provides detail on the improvements to survey responses and where we are continuing to take action to further improve quality. It also provides information about how we have undertaken a partial reweight of the LFS datasets to reflect changes in the population. We are also exploring options for further reweighting, given the interest in this from many key users. 

Alongside this, we are developing and rolling out the new TLFS, which asks many more people, in different ways, about their interactions with the labour market.

This update provides the latest on progress where we have seen encouraging increases in response rates and reduced variability across regions. We have also undertaken further work on coherence and comparability of the TLFS data against the LFS and appropriate administrative data sources, and we are refining our acceptance criteria with a view to assuring the quality ahead of transition.

We will receive our second stable quarter of TLFS data in May, allowing us to examine trends for the first time. The article sets out our next steps for analysis of the data, including via an external data share as part of our ongoing expert peer review process. To support our plans to transition, a methodological stocktake of TLFS survey design and response patterns is currently underway with Professor Ray Chambers and Professor James Brown, who are renowned in this field. We are also creating a new stakeholder panel to ensure user needs continue to drive our decisions on labour market statistics.

Finally, we have had the opportunity to reflect on the review published by the Office for Statistics Regulation in March 2024, which made helpful requirements around our work on LFS and TLFS. The design of this report is shaped by their review with specific responses included in Annex A.

In our previous articles, we set out the steps being taken in our Labour Force Survey (LFS) recovery plan. These steps were taken in order to stabilise and improve both achieved sample and the rate of response to the LFS, and therefore to improve the quality of, and confidence in, our published labour market statistics.

To date we have:

1. Returned to face-to-face interviewing at Wave 1 (mid-October 2023 onwards).

2. Ahead of boosting the sample (point 4) – increased recontact rates (November to December 2023).

3. Increased monetary incentives for respondents (November 2023 onwards).

4. Reintroduction of a boost to the LFS sample (January 2024 onwards).

5. Reinstated the majority of LFS-based labour market statistics, reweighted back to July to September 2022 (February 2024 onwards).

Management information from our survey operations suggest that these steps have had the desired effect in increasing both the achieved sample and rate of response to the LFS.

In summary, the number of achieved interviews by the LFS Wave 1 main sample (excluding imputed records) was 2,287 households in March 2024, an increase of 44% compared with the achieved Wave 1 in October 2023 (1,590 households interviewed), noting also that the March 2024 field period was slightly shortened because of Easter. The average number of household interviews achieved per week between January and March 2024 was 1,539 (across all waves), compared with 1,168 between October and December 2023. 

This recent increase in response at Wave 1 may take some time to fully embed itself into results given the wave structure of the LFS. We will keep our users updated with progress on how these improvements are feeding through over the spring and summer. Given the evidence of the effectiveness of the measures introduced to date, we plan to maintain them. 

We have also seen some initial improvements to quality measures around our headline labour market statistics, where the precision of estimates has improved in recent months. However, we also recognise that there might still be some volatility in the three-month movements in our LFS estimates as the increased sample will take some time to feed through to all the sample waves.

It should also be noted that though we have seen some improvement, response rates and achieved sample do remain low by historical standards for the time being. Furthermore, though measures of statistical precision have improved, estimates remain less precise than was historically the case. Therefore, for LFS-based statistics, we continue to advise caution in the interpretation of short-term changes, especially when looking at detailed breakdowns. Users should continue to make use of the full range of labour market data sources when forming a judgement on the UK labour market. We will continue to monitor the quality of our labour market statistics through the coming months, keeping our users apprised of any changes and providing advice on their interpretation and use. Additionally, subject to our usual quality assurance processes, we are looking to reintroduce estimates from both our two-quarter longitudinal and quarterly household datasets in May. Both datasets will be weighted on a basis that is consistent with our reweighted LFS data and, as such, will start new time series until such a time that we can reweight historic data.

For the household dataset, weighting will be consistent with the quarterly-person LFS data from October to December 2023 onwards but will not be consistent with historic quarterly-household datasets prior to October to December 2023.

For longitudinal data, flows between October to December 2023 and January to March 2024 will similarly mark the start of a new time series. This will be consistent with LFS weighting from equivalent person-quarterly datasets, but will not be consistent with historic longitudinal data prior to this period.

One of the key challenges regarding the current weighting of the Labour Force Survey is the availability and timing of the required subnational population projections. The key statistical inputs required for LFS weighting are the official Subnational Population Projections (SNPPs). These SNPPs typically follow on from, and are consistent with, National Population Projections (NPPs), which themselves follow on from mid-year population estimates.

Previous experience has highlighted that this is a resource-intensive process, which reflects the breadth and depth of LFS datasets that must be reweighted so that these are all available on a consistent basis. This is particularly the case once a census has been carried out, as there can be larger revisions to our assumptions about the makeup of the population. For example, after the 2011 censuses across England and Wales, Northern Ireland, and Scotland, it took until October 2014 before the relevant SNPPs were reflected in published labour market statistics via updated LFS weights.

The context following on from the latest set of censuses across the UK is even more complex. The coronavirus (COVID-19) pandemic saw a one-year delay to the census in Scotland, meaning that consistent UK-wide population estimates and projections have been more difficult to produce to a similar timetable as was possible 10 years earlier.

Prior to February 2024, the population weights used to produce the latest LFS estimates assumed that the demographics of the population had not changed since June 2021. We were aware that updating for more recent demographic changes would affect estimates of the rates of employment, unemployment, and economic inactivity. Following stakeholder engagement, we undertook a partial LFS reweighting exercise in February 2024.

At present, LFS data for July to September 2022 onwards is weighted to populations that are consistent with the 2022 mid-year population estimates published in November 2023. However, we note that these are projected forward from this point using factors derived from older projections, with assumptions for Scotland being relatively more dated, given the delay to Scotland's Census. Prior to this point in their time series, the vast majority of LFS-based statistics have a structural break, with only a limited number of headline indicators having a scaling adjustment applied to allow for longer-term comparisons.

Furthermore, users may well be aware that a set of "interim" NPPs were published in January 2024. These were specifically produced in response to our stakeholders to reflect latest assumptions on international migration. However, these have not been accompanied by equivalent subnational projections, which would typically be used for weighting the LFS.

Looking ahead, we expect that a full set of NPPs will be published in autumn 2024, with consistent SNPPs following in spring 2025. These will both incorporate information from the 2022 census of Scotland, as well as taking on the latest assumptions around international migration, births and deaths.

Given this context, we are therefore currently working with key users to better understand their priorities in this area of producing consistent and comparable estimates over time, which reflect the availability of the latest complete information on the UK population.  

As a first step in establishing a way forward on LFS reweighting, we recently held a workshop with critical users to better understand their priorities. We will be using the feedback from this workshop to inform our future plans, which will be shared with all users as soon as is practicable.

Office for Statistics Regulation review of LFS-derived labour market statistics

In March 2024, the Office for Statistics Regulation (OSR) published a review of the reintroduced LFS-based labour market statistics, setting out a number of requirements for the Office for National Statistics (ONS) to address on LFS and the Transformed Labour Force Survey. The ONS welcomes this review and is already working to address OSR's requirements, with this report being a key first step in bringing together our engagement on LFS and TLFS. Further detail on our responses to the OSR requirements can be found in Annex A.

Work to prepare for transition to our Transformed Labour Force Survey (TLFS) has continued since our last update. This section provides a brief update on activities completed in recent months, before focusing on upcoming activities and milestones.

Latest developments

Elements of survey performance continue to be encouraging. The Wave 1 response rate for the TLFS is currently 39%. Evaluation of the adaptive survey design and targeted knock-to-nudge visits has shown it to be effective in reducing variability in response across regions, Indices of Multiple Deprivation, and Output Area Classifications. Quarterly person datasets include 76,680 individuals aged 16 to 64 years (GB only, December 23 to February 24) compared with 28,902 individuals on equivalent LFS datasets.

Since our last update in November, one of our key priorities has been to assess coherence and comparability of the TLFS data against the LFS and appropriate administrative data sources. This analysis allows us to understand any differences introduced in the TLFS and the reasons for them, i.e., expected changes where conceptual or definitional improvements have been introduced on the TLFS, or unexpected differences that could be introduced through changes to wording or routing, that we may determine require a further survey change to correct for. 

An example of this has been around ensuring the TLFS is appropriately capturing self-employed people. TLFS being an "online-first" survey, rather than interviewer led like the LFS, introduces some mode effect challenges. Prior analysis on the earlier TLFS results indicated some respondents were being classified as economically inactive when they were self-employed. It was found in some cases respondents did not select the answer we intended owing to misinterpretation of the question wording and their understanding of the definition of self-employment. Following an amendment to the survey through introduction of an additional question and some changes to the routing, the subsequent analysis has evidenced material improvements to the quality of results, aligning more closely with the LFS and administrative sources such as HM Revenue and Customs (HMRC) self-assessment data. 

Ongoing analysis of the TLFS data will continue to be a priority as we prepare for a transition.

Following these recent developments, as part of our ongoing expert peer review process, we will be conducting our next external data share in May 2024. This process has provided valuable feedback and further strengthened our quality assurance and analysis of the TLFS. The peer review process is ongoing, and we will continue to share data as we progress the transition from the LFS to the TLFS.

Key areas that we will be able to review using the May data share will include our first stable quarter-on-quarter change since the implementation of improvements to the survey around better capturing self-employed people.

In addition to the May data share, a methodological stocktake of TLFS survey design and response patterns is currently underway with Professor Ray Chambers and Professor James Brown, to give an independent expert view of our position. We are also creating a new stakeholder panel to ensure user needs continue to drive our decisions on labour market statistics.

Principles and quality criteria for transition

In our Labour Market Transformation Update from May 2023, we set out quality criteria that would be used to determine readiness for transition to the Transformed Labour Force Survey (TLFS). We recognise the importance of these criteria as part of any transformation of economic statistics, particularly in being able to explain the implications of any survey improvement.

Since then, further work has been done to adjust, refine and expand upon these criteria with a view to assuring the quality of TLFS ahead of transition, which we currently plan to make in September 2024, subject to these criteria being met, and in conjunction with stakeholders. These adjustments have been made in part as our knowledge and understanding of the TLFS has developed, but also reflecting recent experience on the Labour Force Survey (LFS). Our expectation is that future communications around the impact of the TLFS will in part be based on these criteria.

A helpful way to frame these criteria is through the five European Statistical System dimensions of quality, which in turn are formally reflected in the UK Code of Practice for official statistics. These comprise:

• Timeliness and Punctuality

• Accessibility and Clarity

• Relevance

• Accuracy and Reliability

• Comparability and Coherence

The existing measures described in May 2023 will be retained, with some refinements. They are largely focused under Relevance (meeting international standards) and Accuracy and Reliability (bias, attrition, response rates and methodology).

Where additions are likely to be made is under comparability and coherence where it is clear from our users that it will be necessary to provide our best understanding of why TLFS may differ from existing data sources in the story it tells about the UK labour market.

Stakeholder panel

We are looking to learn from best practice established in the Office for National Statistics's (ONS') work on Prices Transformation. Specifically, we are setting up an expert stakeholder panel, whose membership will consist of representatives from a range of users including academia and think tanks. This panel will help to inform our principles and criteria around transition across these dimensions. It will also provide crucial external feedback to our ongoing judgement on our readiness to transition to TLFS. Furthermore, this panel will consider not only our work on LFS and TLFS, but also wider work by the ONS across the labour market statistics landscape. We will publish the Terms of Reference for this stakeholder panel when it is established.

We expect the panel to meet for the first time in June 2024, reviewing analysis done to date with a focus on key areas that we will be able to assess following on from the May data share, as described previously in this publication. 

Next steps

We will provide our next update in due course, where we will bring together the key insights from the May data share and early feedback from the first meeting of our stakeholder panel.

As referenced earlier in the report, in March 2024, the Office for Statistics Regulation (OSR) published a review of the reinstated Labour Force Survey (LFS)-based labour market statistics (PDF, 200KB). Included in this report were a number of requirements. We set out these requirements alongside our latest response to each of them.

1. Requirement: to help users to appropriately use the available data and make plans for future analysis, the Office for National Statistics (ONS) should clearly communicate to users its plans on:

a. further reweighting of the LFS, including making it easier for users to understand what population data have been used in the reweighting and explain to users the impacts of the lack of Scotland Census 2022 data.

b. reintroduction of the currently suspended longitudinal and household LFS data; if these data are not planned to be reinstated, the ONS should set out when it expects these datasets to be available based on Transformed Labour Force Survey (TLFS) data.

1a: This article provides further information on the population data currently available and how this has been used in the current weighting approach.

We are aware that there are a range of datasets and weights currently available on different bases. A further annex in this article sets out some of the considerations that users should make when using different data from the LFS ecosystem.

As explained in the article, on 22 April 2024, ONS held a workshop with users to understand their priorities regarding reweighting. We will use the feedback from this meeting to make further plans for reweighting, which we will share in due course.

1b: Subject to quality assurance, we plan to reintroduce the publication of longitudinal and household data as part of the labour market statistics published in May 2024. This will provide data back to October to December 2023.

At the time of publication, these data will be consistent with data from the LFS, but not with data prior to October to December 2023.

The further availability of historic data for household and longitudinal datasets on a consistent basis will be considered as part of wider ongoing conversations around LFS reweighting.

2. Requirement: to help users understand and keep up to date with the full narrative around developments to the LFS, ONS should gather this information in one place and make it easily accessible to users.

We support this requirement and recognise that the audience for information on the LFS and TLFS is the same: labour market statistics users. Hence, this article covers both LFS improvements and information on transition to TLFS in one update.

3. Requirement: to enable users to more fully understand the uncertainty around the LFS data, ONS should be clearer in its communication around terms such as "volatility" and "caution". It should also link clearly to the data which demonstrates where this volatility and uncertainty exists to enable appropriate use of the data.

We are setting out in more detail how users should interpret LFS statistics in our bulletins, reference tables and in our supporting metadata, such as our quarterly Performance and quality monitoring reports and our estimates of sampling variability published in Table A11 every month. We will continue to keep this under review, and provide further detail as and when required, such as in Annex B to this report.

4. Requirement: to help users more fully understand how the reweighted data can be used at the sub-UK level, ONS should include more explanations about what the reweighting improvements mean for the use of this data.

Annex B to this report sets out considerations users should make when making use of different LFS and APS data and statistics, given the current range of weighting approaches being used.

5. Requirement: to help users interpret the data appropriately and gain the most value from them, ONS should take a consistent and clear approach to communicating data quality issues and improvements.

Across our bulletins and reference tables, we are regularly providing information on the quality of our labour market statistics, both at a national and subnational level. Any quality concerns are reflected in our communication of these statistics on our website and in the media. This is reinforced by the publication of our quarterly LFS performance and quality monitoring reports.

6. Requirement: to support users' confidence during the transition to the TLFS, ONS should publish updated information setting out the principles and quality criteria it will consider in making further LFS improvements and the transition to the TLFS.

The same quality considerations are important to both the LFS and TLFS. As this article describes, we are using the European Statistical System (ESS) dimensions of quality to steer our judgement on quality across both surveys.

7. Requirement: to further support users during the transition to the TLFS, ONS should publish its plans for further improvements to the LFS as soon possible, making it easy for users to find on the website. These should include plans, priorities, and progress and how its human, financial and technological resources are being used to deliver labour market statistics that serve the public good using LFS and, in the future, TLFS data during this transition period.

This article is part of our ongoing commitment to keep users up to date on the impact of the actions that the ONS has taken to improve the quality of the LFS estimates and our progress to transitioning to our TLFS. We will provide future updates to communicate progress on the LFS and TLFS as part of keeping users informed of key decisions during this period, which will impact upon the quality of our labour market statistics. We will also reference these updates in our monthly LFS publications so that the latest information is readily available to our users.

We are aware that the lower number of responses to the Labour Force Survey (LFS) in late 2023 led to there being more volatility in the single month and three-month movements in our labour market estimates. We will continue to monitor how the sampling variability in the LFS is evolving over time for our headline estimates of employment, unemployment, and economic inactivity. We will communicate to users if there is any additional uncertainty for these or other high-profile LFS indicators that we publish. Users should continue to be aware that more detailed breakdowns of LFS data will be more susceptible to smaller sample sizes and therefore higher uncertainty.

We also recognise that there are, presently, a range of different weighting populations being used across the LFS ecosystem. This ecosystem currently includes the quarterly-person LFS datasets and the annual-person Annual Population Survey (APS) datasets. This is likely to bring some communication challenges in respect to the coherence of these LFS datasets.

From May, we will be looking to also reintroduce the quarterly household dataset as well as the two-quarter longitudinal dataset. This annex sets out the different populations used within these versions of LFS and APS data at present and from May (for household and longitudinal data), highlighting important considerations that users should make when interpreting labour market statistics across these datasets.

We explain how some of these LFS datasets have currently been produced, as part of improving user understanding around their coherence.

LFS person-quarterly

England, Wales, and Northern Ireland use mid-2022 population estimates, projected forward using scaling factors from 2020-based projections, whereas Scotland uses mid-2021 population estimates, initially projected forward using scaling factors from 2018-based projections, then from 2022 projected forward using scaling factors from 2020-based projections.

Annual Population Survey (person)

Data here use older population totals, aligned with the LFS approach which had been used prior to the partial reweighting first published in February 2024. It therefore does not incorporate population estimates updated since the 2021 censuses of England and Wales and Northern Ireland. As such, headline labour market statistics are not directly comparable with LFS data which is either weighted to a different population total (July to September 2022 onwards) or has had scaling adjustments applied (pre-July to September 2022).

LFS household-quarterly

It is our intention to reinstate these statistics from our May 2024 publication of Working and workless households in the UK. This will include data back to October to December 2023 and will be weighted to totals consistent with LFS person-quarterly datasets for the same periods. However, there will be no reweighting of datasets prior to October to December 2023, so there will effectively be a structural break in statistics published from this dataset.

LFS two-quarter longitudinal

Similarly, we are looking to reinstate estimates of labour market flows derived from the two-quarter longitudinal LFS dataset from the May 2024 publication of labour market statistics. These will initially contain one new data period, showing labour market flows between October to December 2023 and January to March 2024. As with the Household dataset, these will effectively be consistent with the LFS person-quarterly weighting populations, but not with historic longitudinal datasets. This will mean that there will be a structural break between the latest data point published in May 2024, and previously publis