1. Output information

 Survey name   Opinions and Lifestyle Survey (OPN)
 Frequency   Fortnightly
 How compiled   Cross-sectional sample survey
 Geographic coverage   Great Britain
 Sample size   Currently approximately 3,000 to 3,500 individuals per fortnightly period achieved
 Last revised   30 September 2021


Back to table of contents

2. About this Quality and Methodology Information report

This quality and methodology information report contains information on the quality characteristics of the data (including the European Statistical System’s five dimensions of quality) as well as the methods used to create it.

The information in this report will help you to:

  • understand the strengths and limitations of the data
  • learn about existing uses and users of the data
  • understand the methods used to create the data
  • help you to decide suitable uses for the data
  • reduce the risk of misusing data
Back to table of contents

3. Important points

  • From 20 March 2020 the Opinions and Lifestyle Survey (OPN) became weekly, to understand how the coronavirus (COVID-19) pandemic is affecting life in Great Britain. From 25 August 2021, as we have moved to a period where COVID-19 restrictions have been lifted across Great Britain, the OPN covers roughly fortnightly periods with the sample size reduced to around 5,000 households in each period to help ensure the survey remains sustainable.

  • Questions on the OPN cover topics primarily related to how the coronavirus pandemic is affecting households and individuals in Great Britain. These can be sponsored by other government departments.

  • Around 5,000 adults are contacted in every period, with the achieved sample for the OPN currently approximately 3,000 to 3,500 individuals per week, an average response rate of 69%, based on data collected since 25 August 2021.

  • Data collection is conducted by an online self-completion questionnaire. Telephone interviews are available if requested by a respondent, however the predominant mode of collection is online.

  • Estimates (including associated confidence intervals) from the OPN measuring the impact of the coronavirus pandemic on people, households and communities in Great Britain, are currently published by the ONS on a fortnightly basis. This includes breakdowns by age, sex, region and country.

  • OPN anonymised datasets are made available via the ONS’ Secure Research Service (SRS). Anonymised data is also made available to sponsors of survey questions on the OPN. The data are available, without charge, to registered accredited researchers.

Back to table of contents

4. Quality summary

Overview

This report relates to the Opinions and Lifestyle Survey (OPN), which collects data from individuals (aged 16 years or older) in Great Britain.

Uses and users

From March 2020, the OPN was adapted to become a weekly survey used to collect data on the impact of the coronavirus pandemic on day-to-day life in Great Britain. As we have moved to a period where restrictions have been lifted across Great Britain, the OPN has moved to a fortnightly data collection from 25 August 2021 onwards.

The questionnaire collects timely data for research and policy analysis evaluation into the impact that the coronavirus pandemic has had on individuals and households in Great Britain. OPN anonymised datasets are made available via the ONS’ Secure Research Service (SRS). The data are available, without charge, to registered accredited researchers.

Data and estimates are published on the ONS website, with estimates also shared with Government Departments.

ONS produce a fortnightly bulletin to monitor the social impacts of coronavirus. Data from the OPN also form part of regularly updated ONS publications including:

Data and estimates are also shared with the Cabinet Office (CO) as well as other government departments. This is to provide indicators to inform policy, and to evaluate existing policies and their effectiveness at driving social and behavioural change.

Strengths and limitations

The main strengths of the OPN include:

  • it allows for a very quick turnaround of data: the OPN currently collects data fortnightly, over a twelve-day period, with estimates and reference tables published on the ONS website within five days of survey completion.

  • it meets data needs: the questionnaire is developed with customer consultation and design expertise is applied in the development stages

  • it is flexible and responsive, allowing new questions to be included at a fast pace

  • it meets users’ sampling needs: questions can be run for multiple weeks, with the data combined to increase the sample size for examining small sub-groups of the population

  • its questions are straightforward and directed at the majority of the population, however it is also possible to include questions only relevant for sub-samples

  • robust methods are adopted for the survey’s sampling and weighting strategies to limit the impact of bias

  • it is accurate and reliable; the questionnaire is rigorously tested and the data is quality assured

The main limitations of the OPN include:

  • in-depth probing of topic modules is not possible due to the length of the questionnaire

  • Data for Scotland and Wales cannot currently be analysed with the same level of granularity as data for English regions (from 21 October 2020 the size of the sample frame was boosted for England). We are currently investigating options to increase the sample size for Wales and Scotland.

Recent updates

To understand how the coronavirus (COVID-19) pandemic is affecting life in Great Britain, the ONS adapted the monthly OPN survey to a weekly survey. The weekly survey was created to collect timely information on people’s experiences and opinions related to the pandemic. Each week, some of the survey questions change to reflect changing circumstances and priorities during the pandemic.

From 25 August onwards, as we have moved to a period where COVID-19 restrictions have been lifted across Great Britain, the OPN covers roughly fortnightly periods with the sample size reduced to around 5,000 households in each period to help ensure the survey remains sustainable.

The survey continues to cover residents of Great Britain who are aged 16 years and over.

Like the OPN survey prior to the pandemic, the OPN has continued to be primarily collected through a self-completion online questionnaire. However, some responses (around 1% on average) have been collected with the help of telephone interviewers, for example when respondents indicate they are unable to complete the online survey themselves.

When weekly data collection started on 20 March 2020, the achieved sample was approximately 1,500 respondents per week. To allow for more granular analysis, the sample in England was boosted from 21 October 2020, achieving an overall response of approximately 4,000 to 4,500 per week. From 25 August onwards, the OPN covers roughly fortnightly periods with the sample size reduced to around 5,000 households in each period, achieving an overall response of approximately 3,000 to 3,500 per period.

Prior to the changes to the OPN survey during the coronavirus pandemic, there had been on-going improvements to the OPN. In recent years, work has been undertaken to change the design of the OPN from a face-to-face survey to a mixed mode design (online first with telephone follow-up). Mixed mode collection allows respondents to complete the survey more flexibly and provides a more cost-effective service for customers.

The first phase of transformation was completed in April 2018 when the OPN moved to telephone data collection. This phase of transformation included a questionnaire redesign. Specialist research, design and testing has been conducted to inform the transformation of the survey. Three pilot tests were conducted to understand the impact of changing mode and sample frame on data quality, sample representativeness and bias. Findings demonstrated minimal impact on most of the data. Where some differences were found, these were likely due to the redesigned question wording, which was optimised for telephone collection. For more information on the pilots and findings, please see the Opinions and Lifestyle Survey: mixed mode pilot analysis article.

The second phase of transformation was completed in November 2019 with the move from telephone only to mixed mode collection. From November 2019, OPN data are collected using an online self-completion questionnaire. Alternatively, if required, the interview can be conducted by telephone.

Back to table of contents

5. Quality characteristics of the Opinions and Lifestyle Survey (OPN)

This section provides a range of information that describes the quality and characteristics of the survey.

Relevance

(The degree to which the survey meets users’ needs.)

The survey is currently a fortnightly survey with a focus on collecting information on how the coronavirus pandemic is affecting people and households in Great Britain. The questions are primarily requested by government departments, as well as by universities and charities.

The Opinions and Lifestyle Survey (OPN) provides rapid answers to questions of immediate policy interest, helping to measure public awareness of new policies.

Topics that have been requested include physical and mental health measures; well-being; loneliness; public understanding of government publicity and information campaigns related to the coronavirus pandemic; whether people are adhering to the latest guidance relating to the coronavirus pandemic; attitudes to vaccines and mass testing, as well as how the pandemic has affected people’s work and education.

Accuracy and reliability

(The degree of closeness between an estimate and the true value.)

The total error in a survey estimate is made up of two types: sampling error and non-sampling error.

Sampling error

The OPN is a sample survey, so estimates are subject to sampling variability. Sampling variability is dependent on several factors, including:

  • the size of the sample
  • the effects of the sampling method
  • the effects of weighting

Non-sampling error

The main sources of non-sampling error are:

  • frame under-coverage
  • non-response
  • response errors (such as misleading questions or interviewer bias)
  • errors when imputing or processing data

To minimise the effects of non-sampling errors, the questionnaire is carefully designed and tested, several attempts are made to contact respondents and extensive quality control procedures are used throughout. Weighting is also used to compensate for non-response and frame under-coverage.

Coherence and comparability

(Coherence is the degree to which data that are derived from different sources or methods, but refer to the same topic, are similar. Comparability is the degree to which data can be compared over time and domain, for example, geographic level.)

The demographic data collected and used on the OPN meet harmonised standards (across Government Statistical Service (GSS) surveys). The questions used on the survey follow the available GSS harmonisation guidance for collecting data about the impact of the coronavirus (COVID-19).

In November 2019, the OPN moved to mixed mode collection. This means that OPN data is collected using an online self-completion questionnaire. Alternatively, if required, the interview is conducted by telephone. Three pilot tests were conducted to understand the impact of changing mode and sample frame on data quality, sample representativeness and bias. For more information on the pilots and findings, please see the Opinions and Lifestyle Survey: mixed mode pilot analysis article.

During the coronavirus (COVID-19) pandemic, the Office for National Statistics has published estimates of personal well-being using both the Annual Population Survey (APS) and the OPN. A technical paper considers the impact that the pandemic has had on data collection, to what extent this has influenced estimates of personal well-being and reviews the comparability of these estimates.

Accessibility and clarity

(Accessibility is the ease with which users can access the data, also reflecting the format in which the data are available and the availability of supporting information. Clarity refers to the quality and sufficiency of the release details, illustrations and accompanying advice.)

Anonymised data are sent to ONS’ Secure Research Service (SRS). Data are added on a fortnightly basis, with an approximate two-week lag between the fortnightly publication and the data available on the SRS. The data are available, without charge, to registered accredited researchers.

Timeliness and punctuality

(Timeliness refers to the lapse of time between data collection and data delivery. Punctuality refers to the gap between planned and actual data delivery dates.)

The OPN currently collects data fortnightly, excluding certain national holidays, such as Christmas. The on-line data collection period is twelve days, with questionnaire content agreed approximately one week before data collection commencing. After the data collection period, the data are cleaned, weighted, and analysed. Anonymised data is sent to government departments who have an agreement to access the survey data, usually approximately five days after the data collection period. Data is also currently published on the ONS website, in the form of aggregated published tables.

For more details on related releases, the GOV.UK release calendar is available online and provides advance notice of release dates.

Concepts and definitions (including list of changes to definitions)

(Concepts and definitions describe the legislation governing the output, and a description of the classifications used in the output.)

National Statistics Socio-economic Classification (NS-SEC)

We use the National Statistics Socio-economic Classification (NS-SEC). The NS-SEC aims to differentiate positions within labour markets and production units in terms of their typical “employment relations”.

Proxy response

The OPN is different from other ONS social surveys in that it does not, generally, collect proxy responses (responses from another adult on behalf of the selected adult). This is because a large proportion of questions asked on the survey are opinions questions. These are unsuitable for proxy collection due to their subjective nature, which if collected would reduce the accuracy and quality of the data through adding missing or inaccurate responses.

There are some exceptions to this where it is appropriate to collect a proxy response. These questions generally reflect the experiences of the household rather than the individual. For example, questions on household income.

Why you can trust our data

The ONS is the UK’s largest independent producer of statistics and its National Statistics Institute. The Data Policies and Information Charter, available on the ONS website, details how data are collected, secured and used in the publication of statistics. We treat the data that we hold with respect, keeping it secure and confidential, and we use statistical methods that are professional, ethical and transparent. More information about our data policies is available.

The OPN is carefully designed and tested, and extensive quality control procedures are used throughout.

Back to table of contents

6. Methods used to produce the Opinions and Lifestyle Survey (OPN) data

Data collection for the OPN is primarily collected through a self-completion online questionnaire. Alternatively, if required, the interview is conducted by telephone, with approximately 1% of responses being collected with the help of telephone interviewers. The population of interest is residents of Great Britain aged 16 years and older.

Sampling frame

From March 2020, respondents who have taken part in previous ONS household surveys (such as the Labour Force Survey and The Living Costs and Food Survey) have been included in the sample frame for the OPN.

From September 2020 onwards, the OPN sample has been drawn from respondents to the Labour Market Survey (LMS).

The sampling frame used for the LMS is the Royal Mail’s Postcode Address File (PAF) of small users. The PAF is the most comprehensive address database in the UK. It is updated daily and contains approximately 30 million addresses. The sample for the LMS covers the whole of Great Britain.

Sampling

A two-stage approach is applied to sampling. In the first stage, a sample of households is drawn from the sample frame; in the second stage, one individual from each sampled household is selected. To overcome under-representation of younger age groups in the sample taken from the LMS, younger and older people are oversampled.

Each week, a random selection of adults are asked to take part in the OPN. Since August 2021, a sample of approximately 5,000 adults have been sampled each week to take part in the OPN (comprising of a boosted sample in England to provide estimates at local levels, with additional sample in Wales and Scotland). This sample also included the OPN Legacy, which is a cohort of 1500 respondents who have previously complete the OPN approximately a year before. The selected respondent (aged 16 and older) is the only household member who is eligible to participate in the survey. Proxy interviews or responses are not permitted.

Using the LMS introduces a potential source of bias as it only includes those who have completed the LMS and not objected to re-contact. However, data collected in the previous LMS are used in the weighting process to reduce this potential bias. This is achieved by including population controls based on variables collected in the LMS. Previous data are also used to adjust for non-response bias before weighting the adjusted sampling weights to population estimates.

The use of this previous data in weighting also improves precision. This is because sampling from the LMS and the oversampling of younger age groups, makes the sampling weights in the new design more variable.

For more information please email the ONS Omnibus Service.

How we process the data

The data is validated and cleaned, variables are derived, and weights are applied. As the OPN collects information on a sample of the population, the data is weighted to enable us to make inferences from this sample to the entire population. Weighting the OPN includes calibrating by factors including region, qualification, tenure, employment, sex, and age. ONS produces and publishes estimates from the OPN together with confidence intervals for questions with sample base sizes greater than 10 cases. Where sample base for questions allows, published analysis also includes estimates broken down by various demographics (for example, age and sex).

How we quality assure and validate the data

There are quality assurance processes from the drawing of the sample and development of the questionnaire to publication of the data and estimates. These processes include:

  • the accuracy of contact information in the sample is validated
  • the questionnaire is tested, both by the research team and telephone operations through test scenarios
  • post collection, a series of checks are conducted on the data to identify inconsistencies and invalid responses
  • inconsistent or invalid responses are individually reviewed by a researcher before a decision is made about how to deal with them
  • estimates are dual run and checked at multiple stages between analysis and final reference tables and publication
  • the application of statistical disclosure control - suppressing estimates to avoid any disclosure of personal information.

How we disseminate the data

ONS currently produces a fortnightly bulletin based on OPN data to monitor the social impacts of coronavirus on adults in Great Britain. This includes a set of reference tables providing further data than given in the bulletin.

Data from the OPN have also been used in other ONS publications examining the social impacts of coronavirus, including:

Anonymised data from the OPN are provided to the Cabinet Office (CO) as well as other government departments on a fortnightly basis. This is to provide early indicators to inform policy, and to evaluate existing policies and their effectiveness at driving social and behavioural change.

Additionally, anonymised data are sent to ONS’ Secure Research Service (SRS). Data are added on a fortnightly basis, with an approximate two-week lag between the fortnightly publication and the data available on the SRS. The data are available, without charge, to registered accredited researchers.

Back to table of contents

Contact details for this Methodology

You might also be interested in: