1. Methodology background


 National Statistic   
 Survey name  Annual Population Survey (APS)
 Frequency  Quarterly
 How compiled  Sample based survey
 Geographic coverage  UK
 Sample size  150,000 (approximately)
 Last revised  17 May 2018

Back to table of contents

2. Important points

  • Personal well-being is assessed through four measures: Life satisfaction, feeling the things done in life are Worthwhile, Happiness, and Anxiety.

  • To collect this data, Office for National Statistics (ONS) asks people in the UK to rate their well-being on an 11-point scale.

  • Data for personal well-being estimates are sourced from the Annual Population Survey (APS), which is the UK’s largest household survey containing the ONS personal well-being questions.

  • Personal well-being data are presented as both average means and thresholds (very low/low, medium, high/very high); the mean averages provide an overall estimate of personal well-being and the thresholds allow us to look at the distribution of the scores.

  • Personal well-being estimates and accompanying datasets are published quarterly, based on the most recent year’s worth of data from the APS.

  • If using local authority data, the most appropriate comparisons to make are progress over time within the same local authority, or across local authorities that share a similar demographic composition to one another; simply ranking local authorities by their numerical scores can be misleading due to several reasons including sample sizes and mode effects.

Back to table of contents

3. Overview

This report relates to the personal well-being statistics produced from the Annual Population Survey (APS). These statistics were designated as National Statistics from the April 2013 to March 2014 dataset onwards (April 2014); before this, they were designated as experimental.

The term “personal well-being” replaced “subjective well-being” after user consultation found it was easier to understand. Personal well-being is assessed through four measures, often referred to as the ONS4 (please see Table 1).

The ONS4 personal well-being questions are asked on several surveys, both internal and external to Office for National Statistics (ONS). Primarily, analysis of personal well-being from ONS originates from the APS.

Personal well-being data from the APS are available on both one-year and three-year datasets:

  • the one-year datasets are published quarterly based on the previous year’s worth of data; the first one-year dataset covered the financial year ending 2012

  • the three-year datasets were originally produced on a financial year basis (April 2011 to March 2014 and April 2012 to March 2015), however, these are now published on a calendar year basis, the first one being January 2013 to December 2015

Since July 2016, personal well-being datasets are no longer included in a separate Personal well-being APS dataset but are now included in the main APS dataset release allowing for analysis by more domains.

Headline estimates of how people view their well-being in the UK are provided in these releases alongside estimates for:

  • countries within the UK and lower geographic areas, such as local authority level

  • main demographic characteristics (such as age, sex, ethnic group)

  • other characteristics that previous research has found to be important determinants of well-being (for example, personal relationships, health and work situation)

Caution should be taken when using local authority data to make comparisons. The most appropriate comparisons are progress over time within the same local authority, or across local authorities that share a similar demographic composition to one another. Simply ranking local authorities by their numerical scores can be misleading due to several reasons including sample sizes and mode effects.

The APS is a continuous household survey, covering the UK, with the aim of providing estimates between censuses of important social and labour market variables at a local area level. The APS is not a stand-alone survey but uses data combined from two waves of the main Labour Force Survey (LFS) with data collected on a local sample boost. For more information on the APS, please see the Annual Population Survey Quality and Methodology Information report.

Personal well-being is presented as both average means and thresholds. Thresholds present the proportion responding in defined response categories as outlined in Table 2. Cognitive testing was undertaken to understand how respondents chose their score on the 11-point scale and what they considered to be “high”, “low”, and “average” ratings of personal well-being. These discussions are outlined in Overview of ONS phase three cognitive testing of subjective well-being questions (PDF, 328KB).

Supporting information on other methodological aspects can be found in the Personal well-being user guidance. This report contains the following sections:

  • Output quality

  • About the output

  • How the output is created

  • Validation and quality assurance

  • Concepts and definitions

  • Other information, relating to quality trade-offs and user needs

  • Sources for further information or advice

Back to table of contents

4. Output quality

This report provides a range of information that describes the quality of the data and details any points that should be noted when using the output.

We have developed Guidelines for measuring statistical quality; these are based upon the five European Statistical System (ESS) Quality Dimensions. This report addresses these quality dimensions and other important quality characteristics, which are:

  • relevance

  • timeliness and punctuality

  • coherence and comparability

  • accuracy

  • output quality trade-off

  • assessment of user needs and perceptions

  • accessibility and clarity

More information is provided about these quality dimensions in the following sections.

Back to table of contents

5. About the output

Relevance

(The degree to which statistical outputs meet users’ needs.)

One of the main benefits of collecting information on personal well-being is that it is based on people’s views of their own individual well-being. In the past, assumptions were made about how objective conditions, such as people’s health and income, might influence their individual well-being. Personal well-being measures, on the other hand, take account of what matters to people by allowing them to decide what is important when they respond to questions.

The uses of personal well-being data are varied, but four main uses have been identified:

  • overall monitoring of national well-being

  • use in the policy-making process

  • international comparisons

  • allowing individuals to make informed decisions about their lives

There is demand for personal well-being information to inform the policy-making process, both in central government and local government. Personal well-being data can be used in a number of ways. The large sample sizes of the Annual Population Survey (APS) datasets allow for comparison between different sub-groups of the population (for example, different age groups or different ethnic groups) and between different areas within the UK (for example, countries and regions). This can help policy-makers target policies at the groups or areas with the highest need in terms of personal well-being.

Analysis can also be carried out to look at how different objective domains relate to personal well-being and which have the biggest effect on personal well-being. This can help in identifying which policy measures could improve personal well-being most effectively.

Another use is in cost-benefit analysis in the policy appraisal process, which could help inform decisions around which forms of spending will lead to the largest increases in personal well-being (PDF, 97KB) (Dolan and others, 2011). The Green Book is HM Treasury's guide for government departments on the appraisal of the costs and benefits of projects through social cost-benefit analysis. A Green Book discussion paper, produced jointly by HM Treasury and the Department for Work and Pensions (DWP), looks at the potential uses of personal well-being measures in social cost-benefit analysis. It looks specifically at the life satisfaction approach, which involves estimating the impact of certain outcomes or non-market goods on life satisfaction from surveys such as the APS.

Moreover, looking at policies through a “well-being lens” and using data to inform not only the formulation of policy but also how policy could be better implemented with people’s well-being in mind is an important use. Evaluation and monitoring of policy could also potentially benefit from using personal well-being information.

Measuring personal well-being is not a new approach. Various international surveys have collected this type of information around the world over the last few decades. What is new, is that there is increasing international recognition that this should be included in official data collection and also that the European Statistical System (ESS) would benefit from the inclusion of personal well-being measures.

The Organisation for Economic Co-operation and Development (OECD) has developed guidance on the measurement of subjective well-being to improve the harmonisation of data collection for national statistics offices around the world. The ONS Measuring National Well-being programme is aligned with this international agenda for measuring subjective well-being, enabling comparisons to be drawn between the personal well-being of people in the UK and in other countries.

APS datasets, including the ONS4, are deposited quarterly at the UK Data Service where they can be accessed by academic institutions and members of the public under end-user licence or the approved researcher route.

Back to table of contents

6. How the output is created

The analysis presented on personal well-being discussed in this report is from the Annual Population Survey (APS). For more information on the APS, please see the APS Quality and Methodology Information report. The APS surveys the population of the UK but excludes those in communal establishments.

The achieved sample size of the APS is approximately 122,000 households (or around 320,000 respondents) on each annual APS dataset. However, of these 320,000 respondents only around 150,000 provide valid personal well-being responses. This difference is due to personal well-being questions having to be answered in person and respondents of the personal well-being questions having to be over the age of 16 years. Non-response is also a factor.

The APS has the largest coverage of any household survey in the UK and enables the production of statistics for small geographical areas. Sampling errors are smaller compared with other social surveys, because the APS has a single stage sample of addresses.

Interviews in all waves are carried out either on a face-to-face basis with the help of laptops, known as computer-assisted personal interviews (CAPI) or on the telephone, known as computer-assisted telephone interviews (CATI). Mode of interview has been previously known to affect personal well-being responses.

The APS datasets are weighted to reflect the size and composition of the general population, by using the most up-to-date official population data. Weighting factors take account of the design of the survey (which does not include communal establishments) and the composition of the local population by age and sex. The APS datasets are reweighted historically to use more up-to-date mid-year population estimates and subnational projection estimates.

Sample attrition is the term applied to respondents who begin the survey but subsequently drop out before completing all the survey waves. It is likely that these respondents have different characteristics to those who remain in all waves of the survey and can, therefore, result in sample attrition bias. The large sample size of the datasets along with the weighting applied reduce both attrition and non-response bias.

The non-proxy adult weight is provided for analysis of the personal well-being questions. On the APS, the well-being questions are only asked of persons aged 16 years and over who gave a personal interview, as proxy answers are not accepted. Therefore, the well-being weight is calculated for each individual and is zero for respondents who were under 16 years of age or who were not present in person for the interview. Cases with weights of zero will not count towards the analysis of results.

The weights for non-proxy adults tend to be higher than the corresponding weights on the APS dataset to account for those adults with similar age, sex and geographical demographics for which proxy responses were obtained, who are effectively non-responders. Applying the well-being weight to the APS dataset will cause the total number of cases in the dataset to be grossed up to the estimated population of adults (aged 16 years and over) within the UK as at mid-point of the time period analysed. Further details on the weights included in the well-being dataset can be found in the Personal well-being user guidance.

Both the one-year and three-year releases of personal well-being include two measures of personal well-being. These are mean averages and thresholds (the proportion of people reporting defined responses on the 0 to 10 scale). The purpose of presenting both these measures is to provide both a summary measure of well-being through the mean and an indication of inequality through the dispersion of the results. The aim of these releases is to present statistics on personal well-being and to inform the debate on what matters most to the population of the UK.

To ensure that the output meets quality and accuracy standards the APS data are quality assured at an early stage by running frequencies and cross-tabs to identify whether there are any discontinuities. Running these checks at an early stage allows for any concerns to be investigated thoroughly, identifying if it could be caused by a questionnaire change, processing error or real-world change. When calculating the personal well-being estimates themselves, the team dual-runs them independently to make sure errors have not been brought in during their calculation. These estimates are compared against one another to ensure that they are the same, prior to comparing with previous estimates for a validity check. The estimates are also peer-reviewed with experts in personal well-being to ensure that the data are as we would expect.

As previously mentioned, all personal well-being estimates produced are weighted to account for them being produced from a survey. Sample sizes are also provided as unweighted counts of valid responses to each of the well-being questions. It is possible for each of the four questions (life satisfaction, worthwhile, happiness and anxiety) to have different sample sizes; this is because respondents are able to choose which, if any, of the personal well-being questions they answer.

From the April 2015 to March 2016 dataset onwards, personal well-being data and estimates have been reweighted on a biannual basis. Re-weighting involves recalibrating the weights so that they sum to the most recent population estimates. The effect of this on personal well-being estimates is minimal. However, it does provide a more accurate representation of the results at that time.

As a consequence of the reweighting and the transition of the personal well-being questions to the APS main data release, the personal well-being data series has been revised. Sensitivity analysis has been carried out to assess the impact to the personal well-being series as a consequence of this change and it has shown minimal impact. For more information on the sensitivity analysis, please see the Impact of transition to Annual Population Survey dataset in personal well-being in the UK: 2015 to 2016.

Timeliness and punctuality

(Timeliness refers to the lapse of time between publication and the period to which the data refer. Punctuality refers to the gap between planned and actual publication dates.)

Personal well-being estimates produced from one-year datasets are published on a quarterly basis, whereas, the three-year datasets are published on an annual basis. Personal well-being estimates are published within five months from the end of the reporting period for the quarterly releases. The personal well-being three-year dataset and associated results have been previously published with a time lag of 12 months.

For more details on related releases, the release calendar provides advance notice of release dates. In the unlikely event of a change to the pre-announced release schedule, public attention will be drawn to the change within the 28 days prior to publication, and the reasons for the change will be explained fully at the same time, as set out in the Code of Practice for Statistics.

Back to table of contents

7. Validation and quality assurance

Accuracy

(The degree of closeness between an estimate and the true value.)

As the Annual Population Survey (APS) – which is made up of the Labour Force Survey (LFS) and a local sample boost – is a sample survey, it provides estimates of population characteristics rather than exact measures. In principle, many random samples could be drawn and each would give different results, because each sample would be made up of different people, who would give different answers to the questions asked. The spread of these results is the sampling variability.

Confidence intervals are used to present the sampling variability. For example, with a 95% confidence interval, it is expected that in 95% of survey samples, the resulting confidence interval will contain the true value that would be obtained by surveying the whole population. Confidence intervals and sample sizes are presented in the quality datasets accompanying each personal well-being release.

Personal well-being estimates follow Government Statistical Service (GSS) Statistical Disclosure Control and Communicating Uncertainty and Change guidance. As such, estimates of personal well-being are not published if the sample size from which they are based is fewer than 50, their numerator is five or fewer, or their coefficient of variation (CV) is greater than 20%. These rules are for both quality reasons and for disclosure control.

The CV is the ratio between the standard error of the estimate and the estimate itself and it gives an indication of the variability and accuracy of the estimate. Table 3 presents the bounds used to suppress or colour code the accuracy of the estimates.

Coherence and comparability

(Coherence is the degree to which data that are derived from different sources or methods, but refer to the same topic, are similar. Comparability is the degree to which data can be compared over time and domain, for example, geographic level.)

The first personal well-being Annual Population Survey (APS) dataset was published for the period April 2011 to March 2012. The personal well-being questions have remained the same since they were first introduced to the APS in April 2011. They gained National Statistics status in 2014.

The questions have been tested extensively both on the Opinions and Lifestyle Survey (OPN) and using cognitive testing methods. Results from OPN survey testing are available in the report Investigation of subjective well-being data from the ONS OPN.

Placement of the questions on the APS

Responses to evaluative questions can be determined in part by the context of the interview. For example, placing after questions relating to health or the labour market may affect the answers that respondents make.

Prior to April 2011, we carried out small-scale cognitive testing of the placement of the personal well-being questions in the Integrated Household Survey (IHS). As a result, it was decided that the placement of the overall monitoring questions would be fairly early on in the questionnaire, directly after the core questions on household and individual demographics.

This allows time for rapport to be built up between the interviewer and the respondent by the time the personal well-being questions are asked without allowing later questions, such as those on employment, to influence response to the personal well-being questions.

Question order

The importance of the ordering of the personal well-being questions themselves also needs to be taken into consideration when designing personal well-being question modules. Any changes year-to-year could affect the ability to examine changes in personal well-being over time, as some of the observed change could be due to ordering effect.

Currently, within the APS, the question on life satisfaction is followed by the worthwhile question, which is then followed by the happiness yesterday and the anxious yesterday questions respectively. This order has not changed on the APS since the questions were introduced in April 2011.

Question wording

It is also known that for any survey question the wording can have an effect on the answer that is received from respondents. Personal well-being questions are no exception. It is not only response scales that differ from survey to survey but also question wording, including the use of time frames.

We have carried out testing on the effect of the change in question wording including the inclusion of “even if yesterday was not a typical day” for happiness and the effect of moving from “nowadays” to “these days” for the question on life satisfaction. For more information on this testing, please see the Summary of results from testing of experimental subjective personal well-being questions – December 2012 (PDF, 519KB).

Question scales

We decided that an 11-point scale from 0 to 10, where 0 is “not at all” and 10 is an absolute value such as “completely”, should be used for all the APS personal well-being questions. The reason for this decision was to ensure that the scales between the questions are consistent to help respondents answer the questions more easily and also to aid analysis across the separate questions.

Further to this, 11-point scales of this nature are commonly used across other surveys of interest, particularly internationally and using the same type of scale aids comparisons with these estimates. An 11-point scale also allows for finer distinctions than a smaller (7-point) scale, but not so many points that would make it difficult for respondents to select one value over another.

Mode of interview

Testing has also shown that people respond more positively to the personal well-being questions when interviewed by telephone rather than face-to-face. As people are interviewed using both methods on the APS, this will have some effect on the personal well-being results. The regression analysis Measuring national well-being - what matters most to personal well-being? explores this in more detail. Table 4 shows the split between face-to-face and telephone interviewing in the APS over the period we have captured personal well-being.

The ONS4 personal well-being questions are asked across several surveys that are internal and external to the organisation. Although the same questions are asked, the results of each of the surveys may not be comparable due to the sampling frame, sampling method and placement. For a full list of surveys that use the ONS4, please see Surveys using the four ONS personal well-being questions.

Back to table of contents

8. Concepts and definitions

(Concepts and definitions describe the legislation governing the output and a description of the classifications used in the output.)

The personal well-being questions on the Annual Population Survey (APS) do not currently come under any specific regulations or legislation. However, consideration is being given to including personal well-being in European regulations within coming years. Additionally, the EU Statistics on Income and Living Conditions (EU-SILC) Survey ran an ad hoc module on personal well-being during 2013 and will be doing so again this year. The EU-SILC is covered by EU regulations.

The personal well-being statistics gained National Statistics accreditation in 2014 and the ONS4 personal well-being questions are included in the harmonised standard classifications.

Domain

Individual well-being is one of several domains that make up a framework for measuring national well-being. This “framework” consists of a set of 10 areas or “domains”, with other domains covering a range of objective and subjective measures such as health, economy, our relationships and what we do.

Eudemonic

The “eudemonic” approach is sometimes referred to as the psychological or functioning and flourishing approach, which draws on self-determination theory and tends to measure such things as people’s sense of meaning and purpose in life, connections with family and friends, a sense of control and whether they feel part of something bigger than themselves. “Overall, to what extent do you feel the things you do in your life are worthwhile?” is the eudemonic question included on the APS.

Evaluative

The “evaluative” approach asks individuals to step back and reflect on their life and make a cognitive assessment of how their life is going overall, or on certain aspects of their life. “Overall, how satisfied are you with your life nowadays?” is the evaluative question included on the APS.

Experience

The “experience” approach seeks to measure people’s positive and negative experiences (or effect) over a short time-frame to capture people’s personal well-being on a day-to-day basis. The APS includes both positive (“Overall, how happy did you feel yesterday?”) and negative (“Overall, how anxious did you feel yesterday?”) effect questions.

Individual well-being

Personal well-being is based on people's views of their own individual well-being. Personal well-being measures are grounded in individuals’ preferences and take account of what matters to people by allowing them to decide what is important when they respond to questions.

Thresholds

Thresholds are used in the report to present dispersion in the data. These show the proportion of responses that fall between certain ratings on the 0 to 10 scale. The thresholds are divided into four categories.

Back to table of contents

9. Other information

Output quality trade-offs

(Trade-offs are the extent to which different dimensions of quality are balanced against each other.)

The mixed mode of data collection is likely to have an effect on the personal well-being results. By using a mixed mode approach, a larger sample size is achieved allowing more detailed analysis of sub-groups, including ethnic minority groups.

However, whilst a larger sample size enables us to drill down to look at personal well-being at lower levels, for example, ethnic minority groups, the differences that exist between groups may not always reflect true differences of personal well-being but could instead reflect cultural and/or linguistic differences.

Assessment of user needs and perceptions

(The processes for finding out about users and uses, and their views on the statistical products.) We have been using various groups and ways to engage with users of personal well-being data. These include:

  • Technical Advisory Group, which includes experts from a variety of sectors including academia and other government departments

  • Measuring National Well-being Advisory Forum

  • Social Impacts Task Force

  • National Statistician’s Advisory Group

  • Well-being Policy Steering Group

  • focus groups to determine citizen users’ level of understanding of the presentation of personal well-being estimates and to determine the clearest and most easily understood methods of presenting results

  • consultations and user feedback on content, format and presentation of personal well-being results following releases

We run public consultations on the major decisions affecting users. This has included:

  • the national well-being debate

  • consultation on domains and measures of national well-being

We consult known users on issues that we consider do not warrant full public consultation. Previously, guidance was also sought from a Technical Advisory Group.

The Measuring National Well-being Programme keeps a log of ad hoc user engagement and monitors this to ensure that users’ needs are being addressed.

Additionally, we have carried out three stages of cognitive testing and a series of quantitative testing on the Opinions and Lifestyle Survey (OPN), as part of the programme of testing of the experimental personal well-being questions, to understand better how people interpret and answer the questions.

Back to table of contents

10. Sources for further information or advice

Accessibility and clarity

(Accessibility is the ease with which users are able to access the data, also reflecting the format in which the data are available and the availability of supporting information. Clarity refers to the quality and sufficiency of the release details, illustrations and accompanying advice.)

The UK Data Service, at Essex University and Manchester University, provides free access to the various Annual Population Survey (APS) datasets and can be contacted via the UK Data Service website.

The ONS Social Survey Data Access and Response Team provide APS data for a fee and can be contacted by telephone on +44 (0)1633 455678. Tables using APS data can also be requested by emailing the ONS data service at socialsurveys@ons.gov.uk.

For queries regarding personal well-being and information for how to access the data, please contact the Quality of Life team via email at QualityOfLife@ons.gov.uk or by telephone on +44 (0)1633 455871.

Our recommended format for accessible content is a combination of HTML web pages for narrative, charts and graphs, with data being provided in usable formats such as CSV and Excel. We also offer users the option to download the narrative in PDF format. In some instances, other software may be used, or may be available on request.

Available formats for content published on our website but not produced by us, or referenced on our website but stored elsewhere, may vary. For further information please contact the Quality of Life team via email at QualityOfLife@ons.gov.uk or by telephone on +44 (0)1633 455871. For information regarding conditions of access to data, please refer to the following links:

Back to table of contents

Contact details for this Methodology

QualityOfLife@ons.gov.uk
Telephone: +44 (0) 1633 455871