The introduction of tenure weighting causes a large downward revision of around 1.0 percentage points in the employment rate.
The unemployment rate is revised upwards by around 0.2 percentage points.
The economic inactivity rate is revised upwards by around 0.8 percentage points.
In March 2020, as a result of the coronavirus (COVID-19) pandemic, the Labour Force Survey (LFS) had to change the way it contacted people for initial interviews, from face-to-face interviewing to telephone-based. This change in method of initial contact has had an impact on both the level of response and the non-response bias of the survey, and consequently the survey estimates.
The change in non-response bias was significantly evident in a change to the housing tenure of the Household Reference Person, with a lower proportion of rented addresses being included and an increase in the proportion of those owned outright by the occupier.
To mitigate the impact of this non-response bias, we have looked at introducing housing tenure into the LFS weighting methodology. While not providing a perfect solution, this has redressed some of the issues that had previously been noted in the survey results.
As a result of this investigative work, from the labour market release published today, we have moved to using the new tenure weights for the headline labour market measures for periods from January to March 2020 onwards.
Further work will look at how other datasets derived from the survey can incorporate tenure weighting to allow other outputs to be produced on a consistent basis.Back to table of contents
The Labour Force Survey (LFS) is a survey of households living in private addresses in the UK. It is the largest regular social survey in the UK. Its purpose is to provide information on the UK labour market that can then be used to develop, manage, evaluate and report on labour market policies. The sample consists of around 35,000 households every quarter, representing around 0.13% of the population.
The LFS uses a rotational sample design, whereby a household, once initially selected for interview, is retained in the sample for a total of five consecutive quarters, or waves, with interviews taking place exactly 13 weeks apart so that the fifth interview takes place one year on from the first.
Before restrictions were imposed as a result of the coronavirus (COVID-19) pandemic, most households were interviewed face-to-face for their first interview, then by telephone, if possible, at quarterly interviews thereafter. During their first interview, respondents were encouraged to provide a telephone number and agree to subsequent waves via the telephone. The exception was the small proportion of households sampled from north of the Caledonian Canal in Scotland who were approached by telephone only.
Impact of the coronavirus on survey operations
Timeline of events for the Labour Force Survey (LFS):
9 March 2020: The public's heightened awareness of the coronavirus started to affect participation in the LFS.
17 March 2020: Wave 1 face-to-face data collection was suspended, in line with government guidelines, while systems were developed to allow interviewers to conduct telephone interviewing from their homes.
Telematching, which uses lookup information on telephone numbers associated with addresses (already used for respondents north of the Caledonian Canal) was extended to the rest of Great Britain to obtain additional telephone numbers for addresses in Wave 1.
LFS Waves 2 to 5 continued to be conducted via telephone where possible.
23 March 2020: Commencement of official UK lockdown measures.
Face-to-face interviewers restarted interviewing intermittently using telephone mode. Additional advance materials were prepared to allow respondents to contact interviewers.
30 March 2020: Telephone interviewing was rolled out fully to face-to-face interviewers.
20 April 2020: A new online portal was put in place to allow improved collection of respondent telephone details.
1 July 2020: Wave 1 sample size was doubled.
Impact on achieved sample size
Our initial concerns for the LFS were around the achieved sample size.
Before the impact of the pandemic, the LFS was achieving a sample size of around 6,500 interviews per week, with around 1,600 of those interviews coming from Wave 1.
In the weeks leading up to lockdown, this had dropped to around 5,300 interviews, with a further drop to fewer than 4,000 interviews relating to the third week of March 2020, with around 420 of those interviews coming from Wave 1.
Once telephone interviewing was fully rolled out, with the use of telematching to obtain numbers for Wave 1 interviews, this recovered to around 5,250 interviews, with around 520 of those coming from Wave 1. An online portal was subsequently introduced to facilitate the collection of telephone numbers; this helped to increase the number of achieved interviews by a further 500.
As we approached late June 2020, the low numbers achieved across the waves in the first weeks of lockdown became the basis for the next waves of interviews, seeing another drop to fewer than 4,500 interviews, with around 870 of those coming from Wave 1.
Since the introduction of the double-size Wave 1 sample from the start of July 2020, we now have over 5,600 interviews a week, with around 1,550 coming from Wave 1.
The relatively small achieved sample for Wave 1 from the end of March to the end of June 2020 will remain within the LFS sample throughout the five interviews; the increased Wave 1 sample from July to September should partially compensate for this reduced wave with a slightly larger one.
The weighting for the LFS normally assumes that response from each week of the survey is roughly the same size, with data pooled to a single set of data representing the 13-week period. However, with the changing sample levels, each week of response would make up a different proportion of the 13-week sample, with the last few weeks of March making up the least proportion.
These weeks were also going to be the most different from the other weeks in the period with regard to the labour market. To allow for this, in May 2020 we introduced new weighting methodology for the January to March 2020 estimates, which meant that the data for each week had roughly equal weight within the quarterly estimate.
Potential sample bias
Although the initial concern was around the achieved sample size for the LFS, there was a secondary concern around the potential bias introduced into the survey because of the change in method of initial contact necessitated by the pandemic.
The monthly results produced from the samples achieved through March to June 2020 showed a different trend from HMRC's experimental Real Time Information (RTI) figures. However, some of this difference could be explained through differences in coverage and definitions. While some of the results were not entirely in line with predictions, it was not known what effect lockdown, government support schemes (such as furloughing), and the subsequent loosening of restrictions would have on the labour market.
Each month, results for a limited range of LFS measures are produced and published, with more extensive tables produced once a quarter. The first quarterly results following the changes to the LFS - for the April to June 2020 quarter published in August - produced results for specific groups that were unexpected and prompted us to carry out further investigations into the pattern of the achieved interviews as a result of the change.
The resultant investigations showed a different distribution of certain personal characteristics than we normally see within LFS samples. Some of these, particularly age, sex and location, were not unexpected and we were already aware of them. These are factors that generally influence the likelihood of someone to respond. However, since population totals broken down by these factors are built into the weighting process, an imbalance in response among these characteristics is already allowed for.
In addition to these, we noticed differences in proportions responding with other characteristics, such as ethnicity, disability status, nationality, country of birth and type of housing. Achieving different proportions of respondents in different types of housing was of particular concern, because this split is one of the least changeable over extended periods of time. While factors around the labour market, migration, or health awareness will cause significant change over time in the other variables, the proportions of the housing stock that are rental, or owner-occupied are far more stable.
|Wave 1||Mar 19||37.2%||28.6%||0.5%||32.7%||0.9%||0.0%|
|Waves 2 to 5||Mar 19||37.3%||31.2%||0.7%||30.0%||0.8%||0.0%|
|All waves||All waves||All waves||All waves||All waves||All waves||All waves||All waves|
Download this table Table 1: Housing Tenure of Household Reference Person.xls .csv
Although we could see that the change in initial contact method resulted in a change in the characteristics of the achieved sample, it has not been possible to identify a causal characteristic that could be directly correlated to the effect seen. What we have been able to establish is that a survey method that relies on obtaining telephone numbers in order to carry out Wave 1 interviews does not give us the same achieved sample as visiting people at home.
Although type of housing cannot be identified to be directly correlated to a household's likelihood to reply, we have focused on this as a variable because it has been so significantly affected, and because it is less likely than other variables to have large changes in a short or medium space of time. This relative constancy makes it a suitable variable to be used in mitigating actions because it should be relatively predictable from one period to the next.
Based on housing type, we looked to see what other characteristics are correlated with it. Our investigations showed that the distribution of other variables that we were concerned about were linked to housing type. For example, people with a non-UK country of birth are more prevalent in rented accommodation than in owner-occupied, relative to their UK-born counterparts.
We considered the possibility that the drop in rented tenure might be driven by students returning from rented accommodation to their out-of-term addresses as the country went into lockdown, however, we found that the change was distributed across all age bands.
Based on the impact of the initial method of contact change, and the relationship between tenure and those characteristics most affected, we investigated the use of housing tenure within the survey weighting.
The use of tenure weighting is not an ideal solution for a number of reasons. Firstly, while it is a strongly correlated symptom of the bias, it is not necessarily the actual cause. This may come down to factors such as the length of time that someone has lived at an address, or whether or not they have a landline. Any of these could be the underlying cause of an imbalance in housing tenure in the achieved sample.
Secondly, it is a variable from within the survey so in effect we are benchmarking the survey against itself. As such, the balance between housing tenure is only as good as the survey's ability to reflect that split in the first place. While this is not ideal for weighting purposes, it would maintain consistency by ensuring that the resultant splits are equally as imperfect as the survey was before the impact of the coronavirus pandemic.
Finally, for the duration of time that we would use tenure weighting, we would be fixing the tenure split against a point in time. Although tenure is relatively stable over a short period of time, the tenure split within the housing stock would change over time, but the survey would not reflect that change.
Therefore, the use of tenure from a specific point in time and from within the survey as a weighting variable should be considered a temporary mitigation action, rather than a long-term solution.
Potentially the LFS could use external data on tenure or some other measure of the socio-economic type of households within the survey weighting, but this is a possibility for longer term consideration.
After consideration of a number of options, it was decided to use the average LFS tenure across the four quarters of 2019 as a basis for weighting. This means we are using weights from directly before the impact of the pandemic, while the averaging means that they do not relate to a specific seasonal quarter to reduce the potential impact of sampling variability on a single period's estimate.
Datasets have been reweighted from January to March 2020, to the latest period, June to August 2020.
Impact of tenure weighting
Using housing tenure within the weighting would generally reduce the weight given to each "owned outright" case and increase the weight given to each "rented" case. There would also be smaller impacts on other types of housing tenure included within the weighting calculation. Although we have additionally introduced housing tenure weighting, it should be remembered that the datasets are still subject to weighting on the grounds of age, sex and geography.
The general effect is that those characteristics that are more present in "owned outright" housing will be reduced in the weighted estimates, while those characteristics more prevalent in "rented" housing will increase. This relates not just to personal characteristics but also employment status and type of employment.
Figures 1 to 7 compare series produced using the original weights with series produced using the new tenure weights. The charts are based on tenure-weighted datasets for the periods January to March 2020, April to June 2020 and June to August 2020. Any other datapoints have been interpolated on a straight-line basis between those periods. All figures are presented not seasonally adjusted.
Along with the personal characteristics previously identified, people living in owner-occupied accommodation are more likely to be employed than those in rented accommodation. Consequently, the introduction of tenure weighting has reduced the total employment level and therefore the employment rate.
In the latest period, the not seasonally adjusted employment rate is around 1.0 percentage points lower than under the original weights, with the level nearly 400,000 people lower.
Although we are seeing a very large downward revision in employment, this translates to a smaller upward revision of around 0.2 percentage points (50,000 people) in the unemployment rate and level. The remainder results in a large upward revision in the numbers of economically inactive.
The use of tenure weighting reduced the total numbers in employment and affected the different categories of people in employment. In general, the categories that were previously reported as being significantly affected continue to show big impacts.
However, some of the categories that were previously reported to be relatively unaffected are now showing more significant impacts. So, for example, we had previously reported that the self-employed, part-time workers and younger workers saw larger reductions in employment. This continues to be the case but the estimates for categories such as employees, full-time workers and workers in their 30s and 40s are not as strong as previously reported.
Previously we stated that we had seen differences in the populations of people with certain characteristics that we would not expect to see. For example, for April to June 2020 our original weighting resulted in a significant drop in the number of people meeting the harmonised definition of disability. The tenure-weighted figures increase the number of people in this group to much nearer the long-term trend.
Similarly, when considering country of birth, the increase in the numbers of people born in the UK has revised to be much nearer the long-term trend than the large increase previously reported. In contrast we are still seeing a decrease in the number of people born outside the UK, although the fall is not as big as previously reported.
While the use of tenure weighting redresses the imbalance in some of these characteristics, the labour market impact may not be even across those characteristics. For example, when looking at ethnicity, despite the large downwards revision in total employment, the total revision to the employment level for people in the Ethnic minorities group has been slightly upwards, with all of the downward revision appearing in the series for people in the White group.
The attached datasets include comparative series for the headline measures, age bands, type of employment, disability, ethnicity, country of birth, nationality and occupation.
It is not possible to say that the figures achieved from using housing tenure within the weighting are correct, or even to be sure that they measure the labour market more accurately than the previous weighting.
However, the estimates we have produced across the range of variables that we have looked at appear to be more consistent and credible with other external information and expectations, than those produced without the use of tenure within the weighting. For example, while we would not expect to be consistent with HMRC RTI information, because of coverage and definitional differences, the reweighted figures close much of the unexplained gap between the two. Consequently, we currently believe that the introduction of tenure weighting, as a short-term mitigation to deal with the effects of the change, is giving an improved picture of the labour market.
Although we believe the use of tenure weighting offers improvement across the range of variables that we have investigated, it is possible that as further outputs are produced, some analysis may be adversely affected by their introduction.Back to table of contents
At this stage we have only reweighted main Labour Force Survey (LFS) datasets and been able to rework a limited number of series as part of the October 2020 labour market release.
By November we will aim to have reworked all of the series that we publish as part of the monthly release from those main datasets, including the single month series, and the breakdowns that we only produce quarterly.
We will also look at whether the methodology can be suitably applied to produce longitudinal datasets. These are datasets that are used to produce estimates of flows between labour market statuses due for publication as part of the November 2020 release.
Other datasets, such as the weekly LFS, household datasets (used in the Working and workless households release) and Annual Population Survey datasets will not be considered until after these main outputs. Until we can reweight these datasets, caution should be used when considering outputs from these datasets, and analysis should be considered in the context of the potential impact of the change in initial contact method for Wave 1 described in this article.
There may also be further refinements to the methodology, such as constraining the tenure weighting within region, that lead to further improvements and revisions in the future. We will continue investigating these options and implement them if they are of sufficiently significant benefit.
The Office for National Statistics (ONS) is investigating other options to improve social survey response - including the option of collecting contact details on the doorstep while complying with social distancing requirements - to facilitate subsequent telephone interviewing. In the meantime, we will continue to monitor the public health situation and public attitudes, along with national and local government guidelines, to inform ONS strategy towards a return to face-to-face interviewing.Back to table of contents
Contact details for this Article
Telephone: +44 (0)1633 455070