Transformed Labour Force Survey TLFS technical design review: April 2025

• We have successfully tested an experimental shortened Transformed Labour Force Survey (TLFS) questionnaire and undertaken a period of quantitative and qualitative research and methodological development, working closely with external experts and in partnership with our key users.

• Testing has indicated that a shorter, labour market (LM) focused survey will significantly reduce missingness and bias for LM data, with remaining missingness and bias to be addressed through collection approaches and methods.

• A new design will therefore be introduced that prioritises the sustainability and quality of labour market statistics but also provides an approach for meeting other household, socio-economic and local requirements.

• Respondent needs are the focus of the design and we have tested approaches to support online self-completion.

• The “two survey” design will be implemented with a longitudinal “Core” survey (90,000 households per quarter with 45,000 followed up for four additional waves) and cross-sectional “Plus” survey (90,000 households per quarter in single wave) design.

• The Core content will be similar to the short survey test with the Plus content initially being based on the current TLFS; a content and prioritisation framework will be used to protect the length of the Core and Plus surveys and minimise respondent burden.

• The combined sample size for Core and Plus surveys will be larger than the current TLFS and, owing to the shorter questionnaire length, the overall respondent burden will be lower.

• Key users and our academic experts Professor Ray Chambers and Professor James Brown have assured and endorsed the improved design.

It has long been recognised that the Labour Force Survey (LFS) in its current form is not sustainable. The survey length and complexity has grown significantly over time and now covers a wide range of topics rather than being a targeted labour market survey. It is also becoming harder to get face-to-face responses, which has affected response rates.

The Transformed Labour Force Survey (TLFS) is our long-term solution to falling response rates and quality challenges with the LFS. It aims to provide a more adaptive and responsive survey to meet user needs, enhance respondent experience and improve the quality of labour market (LM) statistics.

In summer 2024, we conducted an assessment of the TLFS, including feedback from expert peer reviewers, an independent academic review, and a new external expert Stakeholder Advisory Panel for LM statistics.

The assessment provided evidence that we can have confidence in the original concept of the TLFS of a short online survey collecting data on core LM variables, with the capacity for designed modular additions to capture additional information related to the LM. However, the survey had become too long in an attempt to meet all the requirements from the current LFS, with quality impacts.

While the TLFS provides a larger achieved sample and more stable weighted outputs than the LFS, the current survey length and its self-completion (online) nature contributed to higher levels of individual drop-off and within-household missingness. This led to complex biases in the estimates. Collection of complex variables, such as industry and occupation, online without an interviewer was also problematic. As a result, a decision was made to extend the parallel running of the TLFS and LFS and to start a period of redesign.

This technical paper summarises the evidence from recent work on the TLFS and outlines the future design. The design prioritises delivery of good quality headline LM data alongside design solutions for collecting wider information including that not directly related to the LM (referred to as “non-LM data”).

TLFS test design

In autumn 2024, a series of tests known as Labour Market eXperimental (LMX) were conducted to address individual and household drop-off and improve data quality for industry and inactivity. These tests were separate from the live Transformed Labour Force Survey (TLFS) operation. The purpose of the test was not to assess the impact of the changes on estimates – it would have needed a much larger sample to do this, which would have taken longer, cost much more and put at risk fieldwork for “live” surveys.

The test consisted of three treatment groups, with 30,000 households across England invited to take part in total. The test assessed the impact on response, drop-off and quality of a shorter labour market (LM) focused questionnaire; the inclusion of earnings questions; the inclusion of employer name and address (and the quality of the onward link to administrative sources); and different question response ordering for reasons for inactivity.

A QR code was added to the invitation letters to help people access the survey more easily. This is likely to have helped the overall level of response as well as capturing more younger respondents. Full details on the test and results can be found in Section 6: Additional detail on TLFS test.

LMX test results

Overall response, and levels of household and individual partial response

Household completion time reduced (from around 30 to 19 minutes) although remained above the recommended maximum length of 15 minutes (based on average TLFS time for partial completions) so further reductions would be beneficial. Introducing data rotation (only asking respondents to answer questions where responses could have changed at waves 2 to 5, rather than asking the whole wave 1 questionnaire at every wave) will also help reduce survey length and respondent burden in future waves.

Compared with the TLFS, the LMX saw improvements in:

• household response (5 percentage point increase in overall response rate and 28% reduction in the proportion of partial responses)

• proportions of larger households (3 plus people; increase from 21.4% of responding households to 25.2%)

• proportion of younger people (increase from 5.7% to 7.1% of respondents aged 16 to 24 years) and students (15.9% to 21% of inactive respondents)

• individual drop-off (8.9% to 8.1%)

• response for all index of multiple deprivation (IMD) deciles and an improved ratio for the highest to lowest performing decile

Although individual drop-off rate did reduce significantly, people who were employed were still more likely to drop-off of the survey, meaning that the missingness to questions later on in the survey (disability, education) may still be biased by employment status.

Inclusion of employer name and address

Industry data collection through an online mode is challenging as respondents do not always provide sufficient detail in the open textbox to enable accurate coding to Standard Industrial Classification (SIC).

If collected and of sufficient quality, employer name and address could enable linkage to the Inter-Departmental Business Register (IDBR) and, as a result, a SIC code.

When questions about employer name and address were added to the survey, we found there was a high level of drop-off at these questions (10% of households). Of those that continued, only three-quarters provided a response, with only a quarter of these matchable to the IDBR. A sample review of the small number of linked responses concluded that the quality of the matched SIC was not sufficient, partly owing to the complexity of SIC for large organisations.

Asking employer name and address is therefore considered to have a detrimental impact on overall data quality as it leads to a high proportion of households leaving the survey at that point and is unlikely to provide any benefit for the quality of industry data.

This leaves a challenge for the collection of industry and for other complex variables. We have been exploring two main alternatives.

Generative AI integrated with a respondent-centred frame

An AI solution that asks further questions depending on what the respondent provides. Early indications from the prototype are promising, but significant work is required to refine and implement.

Search As You Type integrated with a respondent-centred lookup

Work is needed to assess the feasibility of a respondent-centred lookup and its integration with Search As You Type.

Alongside these two solutions, we also continue to develop Office for National Statistics (ONS) post-collection coding solutions, but these are limited by the quality of the input data we receive from respondents.

Inclusion of earnings

Earnings is a topic that leads to the highest overall level of drop-off, so we wanted to test the impact of the additional questions on employer name and address without the earnings questions. Including earnings did lead to an increased drop-off, however, not to the same extent as the inclusion of business name and address. As there is currently no alternative source we can readily use for earnings to meet user needs, we will retain it on the survey, with a view to removing it at a later date if user needs can be met with it being on only the Plus survey, or if Pay As You Earn (PAYE) Real Time Information (RTI) data is proven to be a viable alternative.

Quality of reasons for inactivity data

The LFS and TLFS are currently reporting different levels of long-term sickness as a reason for economic inactivity. There are a range of potential reasons for this, including mode impacts and changes in question design. We tested whether the wording and order of response options could be contributing to the differences but found they did not have a significant impact on any of the reasons for inactivity.

This remains a key area users want to understand more. We will undertake further research, using linked administrative data or a cross-over survey, where the same respondents answer both surveys, in order to understand these differences further.

Wider benefits and trade-offs of a shorter survey

As well as some improvements in missingness demonstrated by the LMX pilot, a shorter survey should bring wider benefits, including:

• reduced respondent burden and fatigue – the vast majority of respondent feedback relates to current survey being too long

• potential for reduced attrition between waves, thus improving response and reducing bias – evidence from previous shorter TLFS questionnaires shows that this is likely (although we do not have evidence from the LMX test about what the level of improvement will be at this stage, and we will need to consider the impact of any residual attrition bias on estimates as the data are collected and analysed on the new basis, owing to increasing W2+ relative to W1 on the Core)

• more operationally sustainable – being easier to convert knock-to-nudge interviewer visits to face-to-face completion and support on the doorstep

• faster implementation of further changes as user needs evolve – the current survey can require more than 800 testing scenarios for a single change

• better future-proofing – given achieving response to longer surveys is likely to get harder over time.

• aligns with international best practice – the USA, Canada, Australia all have much shorter online labour force surveys

There are some trade-offs that will be managed and reviewed as part of our ongoing engagement with stakeholders. A shorter survey would mean:

• a reduction in information collected from the same person or household with a separation of labour market core data from the labour market-adjacent data, such as job quality, benefits or absences from work – this would affect, for example, the amount of cross-tabulations possible, the number of explanatory variables available for regression analysis and our ability to integrate administrative data or monitor some aspects of bias in our core labour market data

• non-labour market content currently collected on the live TLFS survey will only be available on Plus (covered further in next section) with a reduction in sample sizes and loss of longitudinal components

Two survey model

The Labour Force Survey (LFS) and Annual Population Survey (APS) (and Transformed Labour Force Survey (TLFS) as currently designed) aim to meet a wide range of labour market (LM) and non-LM user needs. The move to a much shorter Core survey to prioritise the quality of headline labour market estimates results in clearer distinction between these user needs. However, the overall survey design, across both the Core and Plus surveys, will still need to meet both sets of user needs. Four key designs were considered – see Section 7: Design options considered.

To enable the TLFS to meet the priority objective of improving the quality of headline labour market estimates, while also meeting users’ non-labour market needs, we plan to introduce the option that separates out the two surveys, ensuring that the LM data benefit from the reduced level of partial response expected in the Core survey (Option 3).

To further reduce non-response bias, we also propose introducing targeted face-to-face completion at wave 1, where non-responding households currently receiving knock-to-nudge visits to encourage online or telephone completion will be offered the opportunity to complete the survey with an interviewer there and then.

As knock-to-nudge visits are currently only offered to non-responding households in high priority, under-represented (low propensity to respond) areas following the Adaptive Survey Design (ASD), the same principle is likely to apply to face-to-face or assisted completion, with an expectation that up to 10% of knock-to-nudge visits would result in a face-to-face interview.

The introduction of face-to-face functionality requires additional development but is only expected to be slightly more costly than knock-to-nudge activity, given the costly aspect is getting to the house. It only requires a little more interviewer time once there to convert to an interview and that approach may save further knock-to-nudge visits. Consideration will be given to whether to offer face-to-face completion at further waves following the evaluation of its implementation of the shorter questionnaire at wave 1.

• This involves two separate surveys – Transformed Labour Force Survey (TLFS) Core survey and TLFS Plus survey.
• TLFS Core survey reaches 90,000 households a quarter with 45,000 selected for wave 2 onwards.
• TFLS Plus survey is a single wave of 90,000 households a quarter.
• Headline labour market outputs are produced from TLFS Core survey only.
• The Plus survey is required to meet non-core labour market and other household and socio-economic requirements.

Changes to Core survey

• Much shorter Core survey focusing on headline labour market requirements to boost response.
• Decrease in wave 1 sample size (140,000 to 90,000) but achieved sample size maintained through increased response and reduced attrition.
• Addition of targeted face-to-face completion for non-responding households in under-represented areas.
• Increase in longitudinal sample from 40,000 to 45,000 a quarter to improve flow dataset sizes.
• Longitudinal sample drawn only from TLFS Core sample at wave 1 (previously drawn from across TLFS Core and Plus samples).
• Data rotation implemented to ensure respondents are not asked repeated questions unnecessarily at waves 2 to 5, increasing response and reducing attrition.

Changes to Plus survey

• Increase from 70,000 to 90,000 enables requirement for data previously met on TLFS Core to be met on TFLS Plus.
• Quarterly datasets available but focus likely to be annual datasets to maximise dataset size.
• Potential for additional follow-up modules.
• Addition of targeted face-to-face completion for non-responding households in under-represented areas.
• Future developments likely to include modularisation with certain topics only asked on certain quarters to reduce respondent burden.
• Plus survey estimates calibrated to Core for headline measures.

TLFS Core sample size

The Core wave 1 issued sample size will be 90,000 households per quarter. This compares with the 140,000 combined Core and Plus sample used to produce headline labour market estimates currently. Although this is a reduction in the issued wave 1 sample size, a larger wave 2 to 5 sample, the expected improvements in response and attrition from the shorter survey, and targeted face-to-face are expected to provide an overall Core survey sample size that will better meet user needs for a given three-month reference period. This means we do not expect a reduction in the quarterly dataset sizes or reduced precision of the estimates.

The impact of the reduced sample has been assessed based on the expected improvements in response and attrition from the new design, informed by Labour Market eXperimental (LMX) test activity or previous attrition rates when the TLFS was shorter. Sensitivity analysis was conducted, including a worst-case outcome if the improvements to the response rate or attrition were not achieved and this showed an achieved sample size well above the current and historic Labour Force Survey (LFS) levels. However, the reduced sample in the design for some labour market (LM) and non-LM estimates will be smaller compared with the TLFS and therefore there will be more uncertainty around detailed data breakdowns, such as local area and single month data.

Alternative samples sizes of 80,000 and 100,000 households were considered but 90,000 provided the best balance between user needs and operational feasibility at this time. The overall sample size can be explored as part of ongoing continuous development research using evidence from the implemented survey.

International best practice is for a wave design, although there are different models. Some adopt four or eight waves, while South Korea adopt a 36-monthly wave design. A five-quarterly wave design is planned for the Core as it strikes the best balance between user need (for two and five quarter flows), data quality (added sample stability makes it easier to identify significant changes over time) and respondent burden.

One of the concerns with the longitudinal data on the current TLFS is that the sample sizes for the two-quarter and five-quarter datasets are too small and levels of overlap between waves are too low to enable estimates of change to be of sufficient quality for users. The new design therefore increases the longitudinal sample from 40,000 households to 45,000 households, now 50% of the wave 1 sample for the Core survey. Improved sample sizes are also expected from the two-quarter longitudinal dataset, improving the quality of estimates of labour market flows.

TLFS Plus sample size

As the content on the TLFS Core reduces to enable a shorter questionnaire, more users will rely on the TLFS Plus data to meet their needs for the topics not included on the TLFS Core. To establish the minimum sample size required to meet the TLFS Plus user requirements, case studies were developed around requirements for geographic and characteristic granularity. These were then compared with pre- and post-pandemic equivalent sample sizes achieved from the Annual Population Survey (APS). Coefficients of Variation (CV) were also calculated to enable a relative assessment of quality against the data from recent APS datasets.

The case studies included smoking, well-being, Welsh language, sexual orientation, families and households and human capital. For each case study, an issued household sample size of 70,000 (current TLFS Plus sample size), 80,000 and 90,000 a quarter was evaluated. At a local authority (LA) level, an issued sample size of 90,000 is expected to deliver sample sizes larger than the 2023 January to December APS dataset for at least 80% of LAs for each of the topics where LA-level estimates are required.

Overall, the sample size of 90,000 was deemed sufficient to meet minimum requirements for all the case studies, but a sample size of 70,000 or 80,000 would not. Noting that the January to December 2023 APS dataset had some quality concerns, further analysis suggested that an issued sample of 110,000 households would be needed to ensure pre-pandemic APS achieved sample sizes for most topics, but this would be operationally challenging to implement.

As for the Core, the overall sample size for the Plus can be explored as part of ongoing continuous development research, using evidence from the implemented survey and through engagement with users. The exact datasets to come from the Plus survey would need to be agreed but would likely be a mix of quarterly and rolling annual datasets with different weights.

Methods

Under the design of having two separate surveys there are several options for estimation. The simplest framework would be to treat the two surveys as separate entities, each calibrated and adjusted with standard weighting, non-response and attrition methods as applicable. This relies on each smaller survey being self-sufficient in meeting user needs and users accepting different estimates produced from Core and Plus.

A more complex approach is to use the employment questions in both the Core and Plus to produce the labour market estimates. This would help sample size and coherence, but it is much more complex and time consuming to implement. While we do not recommend using the more complex approach (at least from the outset), estimation from the Plus survey data is likely to benefit from using information from the Core, as the Core will provide benchmark estimates for key labour market outputs. Further work is needed to evaluate whether there is a need for combined estimation, and if so, the most appropriate methods to achieve this with assurance from our academic experts, Professors Ray Chambers and James Brown.

Regardless of the estimation framework chosen, while the changes to design may help missingness on the Core survey, it will not remove the problem entirely. Therefore, both Core and Plus will need several weights developed to address the remaining partial responses. The methods can only be fully developed, refined and assured once we know the impact of implementing the new design and are collecting data on the new basis.

Content

Core and Plus content

The TLFS Core content has been developed based on the data required to produce headline labour market estimates and productivity outputs, and has been assured by our key user groups (read more in the subsection on “Stakeholder engagement and assurance”).

The content will be similar to the LMX, with a few reductions, notably:

• removing national identity and marital status from the socio-demographics block for Core respondents
• restricting the individual block to over 16s only
• reducing the number of questions asked about tenure

In the new Core survey, as for the LMX, questions about trade union membership, night-time economy, number of employees and supervisory status in the employment block will not be asked and these will only be available on the Plus survey.

Summary of content by topics on Core and Plus surveys

Household block

• Name, age and sex (Core and Plus)
• Household structure (Core and Plus)
• Eligibility (Core and Plus)
• Relationship grid (Core and Plus)
• Tenure (Core and Plus, however content reduced compared with current TLFS provision)

Individual block (16 years and over only)

Socio-demographics

• Country of birth (Core and Plus)
• Passports (Core and Plus)
• Welsh language (Plus only)
• Ethnicity (Plus only)
• Religion (Core and Plus)
• National identity (Plus only)
• Marital status (Plus only)
• Sexual orientation (Plus only)

Solutions are being investigated to enable collection of data for children aged under 16 years for the country or birth and Welsh language topics.

Core labour market

• Unemployment (Core and Plus)
• Employment (Core and Plus, however Core content reduced compared with current core provision)
• Earnings (Core and Plus)

Labour market adjacent

• Benefits (Plus only)
• Sickness (Plus only)
• Job quality (Plus only)
• Social mobility (Plus only)

Non-labour market topics

• Education (Core and Plus, however content reduced compared with current TLFS provision)
• Veterans (Plus only)
• Health and disability (Core and Plus)
• Accidents at work (Plus only)
• Work-related illness (Plus only)
• Smoking (Plus only)
• Well-being (Plus only)
• Travel to work (Plus only)

The priority has been to develop the content of the Core survey and as such, the Plus survey will initially be similar to the current TLFS. This means the Plus survey is likely to suffer from some of the same challenges of missingness and bias as the current TLFS, but the Core could be used for calibration to help mitigate the impact on quality. The introduction of targeted face-to-face completion and a reduction in some of the household and education content common to the Core and Plus surveys may also help mitigate some of the impact.

Following the LM content development, focus will move to refining and developing the Plus content in collaboration with users – reducing content if suitable data are available elsewhere, if questions can be included on other surveys or can be asked with a lower frequency in a modularised design. To help steer this work and ensure the Plus survey meets user needs and gets sufficient focus, we have set up a Technical Group with non-LM users (Household, Socio-economic and Local Group).

Until this development is undertaken, a decision is required as to whether non-LM needs are met from the continued running of the LFS, from the TLFS, or whether non-LM data production should be paused. To inform this decision, a quality assessment of non-LM data from the existing TLFS is taking place to establish if it meets user needs.

Framework for prioritisation of content on Core and Plus surveys

For the Core survey to focus just on collecting data required to produce priority labour market outputs, we have developed a framework for deciding what content should be considered for inclusion on Core and Plus surveys. Read more in Section 8: TLFS content and prioritisation framework.

Although all new content requirements will be dealt with on a case-by-case basis, the framework aims to assist in ensuring that the Core survey will not quickly become bloated again, having previously reduced it. It also ensures that we minimise respondent burden to drive up data quality.

For all content meeting the eligibility criteria to be considered for inclusion on the TLFS Core or Plus, a prioritisation criterion will then be used. This is to ensure the surveys deliver the highest priority requirements and allow TLFS to remain relevant in a controlled way.

The content framework and prioritisation criteria have been supported by key LM users (Stakeholder Advisory Panel and LM Technical User Group).

Stakeholder engagement and assurance

There is a vast and complex landscape of LFS and Annual Population Survey (APS) users with varying needs that have been considered throughout this programme of transformation. While the new design proposals have prioritised the production of high-quality labour market statistics, we have also considered how to best meet other household, socio-economic and local requirements. However, trade-offs will be necessary across all groups of users.

Engagement with key stakeholders has been integral in the development of the proposed re-design for the TLFS, with proposals being shared with the LM Senior Steering Group, Labour Market Technical Group, the (T)LFS/APS User Group and the Stakeholder Advisory Panel.

We have also presented the designs to non-labour market users at the Household, Socio-economic, and Local Technical Group. They have also been reviewed by the independent academic experts, Professors Ray Chambers and James Brown, who are supporting the assurance of the TLFS design.

Our key stakeholders have been broadly supportive of the design and approach, agreeing with the principles of the shorter survey and the framework for content inclusion.

Respondent needs

Respondents have been considered at the heart of the TLFS design which is particularly important for the longitudinal aspect, to reduce attrition between waves. A respondent-centred design approach has been used to enable high-quality data collection without the benefit of highly trained and dedicated interviewers to engage and support respondents in answering the questions. This approach brings together best practice from the social research field of survey design, and user experience design from the computer technology field. The intention is for any future changes and improvements to continue to be guided by this approach.

The design should help respondents as, while the overall sample size for the integrated design is larger, half of the sample receive a much shorter questionnaire for the full five waves. While those on the Plus survey receive the fuller questionnaire only once, so the respondent burden in total is much lower. Respondent burden for Plus survey respondents is also expected to further reduce as we progress development for a more modular design.

TLFS test design

The test consisted of three treatment groups, with 30,000 households invited to take part in total. 10,000 households were assigned to each treatment group using a systematic random sample and split across two cohorts (5,000 households per treatment, per cohort).

The first treatment was a test control treatment where households were asked to complete a shorter labour-market focused questionnaire with no other changes. The second treatment included questions on employer name and address, and also included changes to the response options and order for reasons for inactivity. The third treatment group replicated the second treatment but earnings questions were excluded.

Collection took place over 23 days from the 21 October 2024 to the 12 November 2024, with a 16-day collection period for each cohort. The test was online-only, with no telephone response, or knock-to-nudge visits that are usually part of the TLFS data collection design. Households were sampled from England only (usual TLFS operations cover Great Britain). When comparing the TLFS and the LMX, only the first 14 days are used as knock-to-nudge starts on day 15, and telephone responses are excluded from the TLFS data.

TLFS test results

Overall response, and levels of household and individual partial response

A driver behind the high levels of individual drop-off and within-household missingness is believed to be due to the survey length. On average, it takes a household around 30 minutes to complete, but this varies significantly based on household and individual characteristics (for example, it takes nearly twice as long for employed people compared with economically inactive respondents). This contributes to missingness and bias in the data. We wanted to test the impact of a shorter survey on individual and household levels of partial response.

Compared with TLFS, the LMX found:

• a reduced household completion time (from around 30 minutes to 19 minutes) although slightly above the recommended maximum length for online surveys of 15 minutes
• a significantly increased response rate for households (and a significant reduction in partial responses for everyone in their household (Figure 1)
• a significantly increased proportion of larger households (three or more people) completing the survey helping representation (Figure 2)
• a significant reduction in individual drop-off, other than when employer name and address was included, which increased drop-off significantly (Table 1)
• the reduction in individual drop-off was not as large as we had hoped and people who were employed were still more likely to drop-off of the survey, meaning that the missingness to questions later on in the survey (disability, education) may still be biased by employment status (Table 2)
• a higher overall level of response for all index of multiple deprivation (IMD) deciles, compared with the TLFS and an improved ratio for the highest to lowest performing decile (from 2.29 on TLFS to 1.94)
• a significantly higher proportion of younger people (aged 16 to 24 years) and students in the LMX responding sample, and a significantly lower proportion of those aged 55 years and over and retired people helping representation (Figure 3). We did not test the impact of the QR code separately from the messaging telling people it would only take 15 minutes to complete, so we cannot definitively say whether it is the shorter survey or the QR code leading to greater levels of response from younger people and students. While the proportion of 16- to 24-year-olds is significantly higher in the LMX (7.1% compared to 5.7%), this group is still under-represented compared to general population levels (10.6% from 2021 Census).
• no significant difference in ethnic group make-up between the TLFS and the LMX
• a significant increase in the proportion of respondents in employment (from 55.9% on the TLFS to 57.3%), however, when the data were weighted for age and sex, no significant difference was found, suggesting the increase in employed persons taking part is a result of the proportional increase in younger responders

Figure 1: Household response rates for the LMX and TLFS

Online response only over a two week collection period, with no follow-up of non-responding households, 21 October 2024 to 12 November 2024

Notes:

1. Asterisks (*) show statistically significant difference to TLFS (p<0.05).

Figure 2: Proportion of responses by household size for the LMX

Notes:

1. Asterisks (*) show statistically significant difference to TLFS (p<0.05).

Figure 3: Distribution of individuals responding to the TLFS and the LMX, by age bands

Notes:

1. Asterisks (*) show statistically significant difference to TLFS (p<0.05).

Inclusion of employer name and address

When questions about employer name and address were added to the survey, 10% of households dropped out of the survey. Of the 3,345 people asked to provide employer name and address, 2,608 (77%) of those provided a name and postcode but only 691 (26%) could actually be linked to the Inter-Departmental Business Register (IDBR), due to the quality of the data supplied and differences with this information on the IDBR. Of those linked responses, 431 (68%) did not match the code assigned based on the respondent’s answers to the industry question on the survey.

When a sample of the non-matched responses were analysed, the survey assigned industry code was considered to be a better fit than the IDBR-assigned code in 70% of cases. As a result, asking employer name and address is considered to have a detrimental impact on overall data quality, as it leads to a high proportion of households leaving the survey at that point, and is unlikely to provide any benefit for the quality of industry data.

The following four design options were considered.

Option 1: Transformed Labour Force Survey (TLFS) as-is

TLFS Plus data and TLFS Core data (both long surveys) are combined together to produce headline labour market estimates.

Pros

Maximises sample sizes available.

Reduces timeline for transition as:

• no major development needed to design and implement a shorter survey and face-to-face collection
• no further time needed to deliver stable series and evaluate data quality

Cons

• Data rotation development still likely to be needed.
• Current data quality does not meet user needs.
• Unlikely to be sustainable due to the length and complexity of the existing questionnaire.

Option 2: Two linked surveys

Core and Plus design: Implementing the short survey on the Core component, combine both Core and Plus to produce headline labour market (LM) estimates

Pros

• Maximises sample sizes available.
• Reduces reliance on data affected by attrition bias as a higher proportion of the sample is made up of wave 1 data compared with Option 3.

Cons

• Requires development of novel, complex methodological processes to account for the different biases in shorter and longer surveys (upfront cost implication and may become unsustainable overtime).
• Reduces flexibility on Plus due to need to harmonise key questions.
• Added complexity for users if multiple different weights are required.
• Work will be required to identify what changes will need to be made to the processing pipeline and the time that may take to account for the linking of the surveys.

Option 3: Two separate surveys

Core and Plus design: Implementing the short survey on the core component, only Core used to produce headline LM estimates.

Pros

• Enables prioritisation of labour market content.
• Optimises design for each survey – Core survey can focus on a short survey at every wave, while the Plus survey may be able to reduce some labour market content and develop a modular design.
• Uses established weighting methods.
• Increased overlap in sample across quarters should improve precision around measures of change.
• Easier to manage the operations (for example, fieldwork, administration, incidents) of two separate independent surveys than trying to manage two inter-linked surveys.

Cons

• Increased reliance on data from waves 2 to 5 may lead to an increased impact of attrition bias on estimates
• Reduction in sample size (compared with existing TLFS, but still at least as good as Labour Force Survey (LFS)) for content moving from Core to Plus.
• Work will be required to identify what changes will need to be made to the processing pipeline to account for separate surveys, for example, weights and the time that work may take.

Option 4: Labour market Core only

Core content only, smaller sample with all households invited to all five waves, and all non-responding households followed up with knock-to-nudge visits and offered a face-to-face interview.

Pros

• Higher overall level of response.
• Greater overlap between quarters.
• Simpler survey operations, and likely cheaper as it is smaller and easier to manage.

Cons

• Provides core labour market survey only, with no collection of non-core data, which would not meet user requirements.
• As waves 2 to 5 make up a greater proportion of data collected, there is a greater risk of attrition bias affecting estimates.
• Greater reliance on field interviewers impacts sustainability as recruitment and retention of interviewers securing interviews becomes more challenging.
• Risk that design would not produce larger sample sizes of the current TLFS design, which enable more granular analysis to meet user needs.

Prioritisation criteria for content on the Transformed Labour Force Survey (TLFS)

Labour market prioritisation

• Low priority: No impact on labour market (LM) statistics.
• Medium priority: LM adjacent topics or topics required for LM analysis.
• High priority: Essential for production of headline LM tables.

Legal or statutory requirement to deliver data

• Low priority: No requirement.
• Medium priority: Not applicable.
• High priority: Yes.

Impact on respondent burden

• Low priority: Leads to or likely to lead to a high level of drop-off.
• Medium priority: Some impact.
• High priority: Minimal impact or may help to improve respondent experience by, for example, improving the flow of the questionnaire.

Required for Accredited Official Statistics

• Low priority: No.
• Medium priority: Contributes to, but not integral to accredited outputs.
• High priority: Essential component of accredited outputs.

Funded topic

• Low priority: No.
• Medium priority: Yes.
• High priority: Not applicable.