1. Disclaimer

These Research Outputs are not official statistics relating to the labour market. Rather, they are published as outputs from research into an alternative prototype survey instrument (the Labour Market Survey (LMS)) to that currently used in the production of labour market statistics (the Labour Force Survey (LFS)).

It is important that the information and research presented here is read alongside the accompanying technical report to aid interpretation and to avoid misunderstanding. These Research Outputs must not be reproduced without this disclaimer and warning note.

Back to table of contents

2. Overview

Between October 2018 and April 2019, the Office for National Statistics (ONS) conducted a large-scale, mixed-mode test of the prototype Labour Market Survey (LMS). This test formed an important part of an on-going research programme that is being conducted as part of the ONS Census and Data Collection Transformation Programme (CDCTP). For a comprehensive summary of the context and design for this test, please see the accompanying technical report.

The purpose of this report is to provide a summary of the socio-demographic characteristics of the responding sample to the LMS Statistical Test, one of the main objectives of the test. Comparisons are made with the responding sample for the Labour Force Survey (LFS) over a comparable data collection period. Owing to differences in the survey designs and modes of collection, no statistical tests have been carried out on the data compared. The results presented will require further consideration and will form the basis of further evaluation. Along with the initial evidence provided in the other reports, this evaluation will inform the future design and testing of the LMS.

This report should be considered alongside the results and evidence presented in the technical report and the comparative estimates report. They document how this research represents a single point of comparison between the LMS and LFS. The differences in the socio-demographic characteristics of the responding sample are indicative of potential differences based on the two survey designs; they are not reflective of the final design of the LMS.

Back to table of contents

3. Datasets used for comparisons

Three datasets have been used in this report for the purpose of comparing the characteristics of responding individuals. All figures for the Labour Market Survey (LMS) and Labour Force Survey (LFS) presented in this report are produced from unweighted data. This allows for the report to highlight any areas of notable difference from these raw unweighted counts and percentages and to suggest avenues for further research and consideration regarding the survey design.

Labour Market Survey dataset

The full unweighted dataset from the LMS Statistical Test, covering the period of October 2018 to April 2019, is used for all LMS analysis presented in this report. This dataset contains 18,673 responding individuals (14,351 adult responses) from across Great Britain, including both individual and proxy responses. As a mixed-mode test, the LMS dataset contains both online and face-to-face responses.

Though the sample design used for this test of the LMS has many similarities with the LFS sample, there are several exclusions and differences that were required. More detailed information on the nature of these differences can be found in the technical report.

The LMS dataset for analysis of the socio-demographic characteristics differs from the 13-week subset dataset produced for the LMS comparative estimates report and used for the purpose of comparing statistical estimates to an equivalent LFS dataset. More information on that dataset can be found in the LMS and LFS comparative datasets section of the LMS comparative estimates report.

Labour Force Survey comparison dataset

A bespoke LFS comparison dataset was used for all LFS analysis presented in this report. This dataset is the same dataset used for producing the comparable statistical estimates presented in the comparative estimates report. It includes data collected for a 13-week period between November 2018 and January 2019. The dataset contains 18,256 responding individuals, including both proxy and individual responses.

This LFS comparative dataset differs from the full LFS dataset used to produce published estimates in that it only includes Wave 1 responses and excludes all individual responses achieved for the longitudinal component of the full quarterly LFS dataset (Waves 2 to 5). The comparative dataset also excludes all responses achieved in Northern Ireland and north of the Caledonian Canal, to correspond to the sample area used for the LMS Statistical Test. Wave 1 of the LFS is predominantly administered via face-to-face interviews.

The mid-year population estimates 2018

Mid-year estimates (MYEs) are the official source of population sizes in-between censuses, covering populations of local authorities, counties, regions and countries of the UK by age and sex. The estimates use the census definition of people who are “usually resident” in the UK for 12 months, excluding short-term migrants, and counting students at their term-time addresses. The estimates roll forward the population found by the previous census one year at a time by accounting for births, deaths, international migration and internal migration. To accomplish this multiple registration, survey and administrative data sources are used including the General Register Office (GRO), the International Passenger Survey (IPS), the Higher Education Statistics Agency (HESA), the National Health Service Central Register (NHSCR) and the Ministry of Justice (MoJ). Comparable estimates for Scotland and Northern Ireland are produced by National Records of Scotland (NRS) and the Northern Ireland Statistics and Research Agency (NISRA) respectively.

The 2018 population projections for Great Britain are used in this report; it was not possible to exclude north of the Caledonian Canal from these projections, but the impact of the inclusion is likely to be negligible as it comprises a very small proportion of the overall population projection.

Back to table of contents

4. Sex and age bands

Sex

Table 1 presents the proportion of the responding sample by sex for the Labour Market Survey (LMS) and Labour Force Survey (LFS) and provides a comparison against the mid-year estimates (MYEs). For both the LMS and LFS, 48.4% of the responding sample were male and 51.6% were female. Compared to the MYEs, the LMS and LFS under-represented males by 1.0% and over-represented females by 1.0%.

Within the LMS, 482 cases did not provide an answer to the sex question, representing 2.6% of the data supplied.

The collection of a respondent’s sex is a mandatory response variable on the LFS, with interviewers required to ask or record this for each individual.1 There are therefore no missing values in the LFS data. For the purposes of this LMS Statistical Test, no variables were mandatory for completion and it was possible for respondents to bypass questions or to leave their answers blank. Possible reasons for LMS respondents not providing their sex may be that they believe this information is too personal or that there is not a suitable response option, for example, in the case of a transgender respondent. Further research is required in this area before any firm conclusions can be made. More analysis will be conducted on other data provided by respondents who did not provide a response for sex to identify any trends or similarities among this cohort. For LMS respondents who did provide a response, the distribution of sex across the responding samples is very comparable with that of the comparative LFS dataset.

Age

Table 2 presents the proportional distribution by age band for the responding sample of the LMS and LFS, comparing these to the distributions from the MYEs. Compared to the MYEs, respondents aged 55 years and over are over-represented in both the LMS and LFS and respondents aged 16 to 34 years are under-represented. Respondents aged 15 years and under are under-represented in the LMS and over-represented in the LFS. The largest responding group for both the LMS and LFS was individuals aged 65 years and over, with 21.2% and 20.8% of the individuals in the achieved sample falling in this group respectively. The second largest group was those aged 15 years and under; these responses would have been gathered by proxy. The group with the lowest proportion of responders was those aged 16 to 24 years, with the LMS sample having a slightly higher proportion than the LFS.

Age is another variable that is mandatory for completion by interviewers on the LFS, resulting in no instances of missing data. On the LMS, 7.2% of respondents did not provide their age; this includes respondents who did not progress past the initial stages of the interview and “dropped out”. Again, there are various possible reasons for missing values being provided such as respondents having concerns about confidentiality or being unable to recall this information. As both age and sex are important variables used in the weighting process, methods of reducing the proportion of missing values will need to be considered for the future design of the LMS.

Age by sex

Table 3 presents the responding sample for the LMS and LFS, categorised by age band and sex. Both the LMS and LFS have lower proportions of 16- to 24-year-olds and 25- to 34-year-olds for both males and females, when compared to the MYEs. The proportion of 45- to 54-year-old males on the LMS was 13.6%, which is almost the same as the MYEs (13.8%); the LMS proportion was lower, at 12.6%. The LMS and LFS contain higher proportions of 55 to 64 year-olds, for both males and females, than the MYEs. Male and female respondents aged 65 years or over were also over-represented on both surveys, with the LMS proportion of males being higher by almost 4 percentage points (20.9% for the LMS and 17.0% for the MYEs) and the LFS by 3.4 percentage points (20.4%). The underrepresentation of younger age groups for both sexes is a common factor across voluntary social surveys, and these results suggest that further investigations into how to engage with this population need to be explored.

Notes for: Sex and age bands

  1. No set question text is provided to interviewers for the LFS, although they are required to ask respondents if their sex is not spontaneously provided. See Labour Force Survey User Guide: Vol. 2 – LFS Questionnaire 2019 (2.44MB).
Back to table of contents

5. Country of birth, ethnicity and nationality

Country of birth

Table 4 presents the responding sample by country of birth. The majority of the responding samples for both the Labour Market Survey (LMS) and Labour Force Survey (LFS) stated they were born in the UK (LMS at 86.2% and LFS at 87.1%). The proportion of non-UK-born respondents was therefore slightly higher on the LMS than on the LFS by 0.9 percentage points. Owing to small sample sizes, the non-UK countries of birth have been aggregated into the “other” category for both the LMS (9.4% of the responding sample) and LFS (8.8%). The LMS Statistical Test did not include a coding frame for country of birth nor for any other question where a free text “other” option was available. On the LFS, interviewers code a respondent’s country of birth during the interview, based on a coding frame. By not using a coding frame, it was possible to examine the quality of the free text entry by online respondents and to code these responses to specific countries during the data cleaning process. There were 1,700 free text entries for country of birth on the LMS and only 14 of these could not be coded to a specific country.

Ethnicity

Table 5 presents the proportion of the responding samples for the LMS and LFS by their stated ethnicity. The majority of respondents on the LMS (80.4%) and LFS (81.9%) identified as “White British”. The proportion of respondents from minority ethnic groups was marginally higher in the LMS than in the LFS.

Nationality

Table 6 presents the responding sample by their stated nationality. Both the LMS and LFS allow respondents to select a single nationality. The vast majority of respondents stated their nationality as “United Kingdom” or “British” (91.1% LMS, 91.6% LFS). The proportion of respondents who had a stated nationality representing one of the EU14 group (see footnote on Table 6 for a definition) was 2.5% on the LMS, slightly higher than the 2.0% on the LFS.

Back to table of contents

6. Household size

The number of eligible people in each responding household for the Labour Market Survey (LMS) and Labour Force Survey (LFS) is shown in Table 7. The eligibility of individuals for inclusion in the household for both the LMS and LFS was defined as any adults, children or babies who define the sampled household as being their main residence, even if they were currently away for a continuous period of up to six months. The LFS permits up to 16 people to be added to the household grid, whereas the LMS allowed a maximum of 10. This was a tactical decision for the purposes of the test to avoid potential performance issues with the online data collection instrument.

There was little variation between the results from the LMS and LFS, which demonstrates that for this particular household characteristic, there is consistency between the two surveys despite the differences in the survey designs (detailed in the technical report). This is also evident in Table 7, which presents the mean number of eligible individuals in a household, which was 3.0 people for both the LMS and the LFS.

Back to table of contents

7. Index of Multiple Deprivation

The Index of Multiple Deprivation (IMD) is a relative measure of deprivation across small areas based on seven domains of deprivation: income, employment, education, health and disability, crime, barriers to housing, and services and living environment. Various indicators are used to measure deprivation in each domain, using information such as school performance or access to local hospitals. Each devolved government produces their own IMD, updating it every four years. The IMDs used in this analysis are the English indices of deprivation 2019, the Welsh Index of Multiple Deprivation 2019 and the Scottish Index of Multiple Deprivation 2016. This report presents sample and response data categorised by IMD deciles, with decile “1” representing the most deprived areas and decile “10” representing the least deprived.

Table 8 presents the proportion of sampled addresses for the LMS by their IMD and demonstrates that the proportions ranged from 8.6% in decile 10 to 11.3% in decile 2. The IMD of a property was not factored into the sampling process, and therefore the sampled distribution of addresses by IMD is a by-product of the process. For further information on the sampling process, see the technical report.

Table 9 presents the distribution of responding households to the LMS and LFS by IMD deciles. The LMS has a relatively even distribution of response across all IMD deciles, ranging from 9.4% (deciles 3 and 5) to 10.9% (decile 6). The LFS has a more varied response distribution, ranging from 6.9% (decile 3) to 13.6% (decile 8).

If the IMD deciles are categorised as areas of higher deprivation (deciles 1 to 5) and areas of lower deprivation (deciles 6 to 10), the LFS has a response distribution of 39.3% in areas of high deprivation and 60.7% in areas of lower deprivation. The distribution of response for the LMS is more evenly split, with 49.2% in areas of higher deprivation and 50.8% in areas of lower deprivation. These results imply that for this test, the LMS achieved a more representative responding sample by deprivation indicator than the LFS.

The engagement rate by IMD decile for the LMS is shown in Table 10. The “engagement rate” is defined as the proportion of households that either visited the survey website and entered their unique access code (UAC) to access the survey or provided any survey information to a face-to-face interviewer. More information on this measure can be found in the LMS technical report.

Areas with an IMD of 10 (least deprived) had an engagement rate of 65.4% compared with areas with an IMD of 1 to 3 (most deprived) having an engagement rate of between 51.1% and 52.6%. There is a trend that the higher the IMD, the higher the engagement rate.

Table 11 presents the split of engagement by mode for the LMS, by IMD decile. In areas with an IMD of 1, for households that engaged with the survey, 29.2% utilised the online mode and 70.8% engaged with the face-to-face mode. Inversely, 65.0% of those that engaged with the survey in areas with an IMD of 10 chose to access using the online mode and 35.0% used the face-to-face mode. The higher the IMD (indicating a less deprived area), the higher the use of the online survey; the lower the IMD (indicating a more deprived area), the higher the use of the face-to-face survey. In the future, the IMD could be used to target respondents in different areas with bespoke mode invitations or for planning field worker distribution in areas where there is likely to be a higher demand for face-to-face interviews.

Back to table of contents

8. Conclusion

The characteristic profile of the Labour Market Survey (LMS) responding sample was broadly similar to that of the Labour Force Survey (LFS) across many of the demographic variables such as sex, age, country of birth, ethnicity, nationality and household size. Both the LMS and LFS responding samples slightly under-represent males and over-represent females when compared with the mid-year estimates (MYEs). Regarding the age distribution of the responding sample, respondents aged 55 years and over are over-represented in both the LMS and LFS compared with the MYEs.

Some interesting results were identified when analysing the responding sample using the Index of Multiple Deprivation (IMD). The LMS had a more even distribution of responding addresses across all IMD deciles compared with the LFS, which had a more varied distribution. The IMD also influenced the selection of the mode with which a household engaged with the LMS; the higher the IMD (indicating a less deprived area), the higher the online uptake, whereas the lower the IMD (indicating a more deprived area), the higher the face-to-face survey uptake.

These findings highlight several areas for exploration in future research and testing cycles, with the aim to further the understanding of any identified differences and feed into future re-design of the LMS, where appropriate. To gather a more complete picture of the characteristic profile of the LMS responding sample, it would also be beneficial to undertake a longitudinal test in order to explore the effects of attrition on the characteristic profile of the responding sample.

Back to table of contents

Contact details for this Methodology

Colin Beavan-Seymour
lms.transformation@ons.gov.uk
Telephone: +44 (0)1633 455536