1. Methodology background

 National Statistic   
 Survey name  Annual Population Survey (APS)
 Frequency  Annual
 How compiled  Sample based survey
 Geographic coverage  UK
 Sample size  158,000
 Last revised  8 July 2016

Back to table of contents

2. Executive summary

This report relates to the personal well-being statistics produced from the Annual Population Survey (APS). These statistics were designated National Statistics from the April 2013 to March 2014 dataset onwards (April 2014); prior to which they were designated as experimental.

The term personal well-being replaced subjective well-being after user consultation found it was easier to understand.

Personal well-being is assessed through four measures, often referred to as the ONS4 (please see Table 1).

The ONS4 personal well-being questions are asked on a number of surveys, both internal and external to Office for National Statistics (ONS). Primarily, analysis of personal well-being from ONS originates from the APS. A list of all surveys using the four ONS personal well-being questions is available.

Personal well-being data from the APS are available on both annual and 3-year datasets. The first annual dataset covers the financial year ending 2012. The first 3-year dataset is available for the period April 2011 to March 2014.

From July 2016, personal well-being datasets will no longer be included in a separate “Personal Well-being Annual Population Survey” dataset but will be included in the main APS dataset release.

Headline estimates of how people view their well-being are provided in these releases alongside estimates for: main demographic characteristics (such as age, sex, ethnic group); different geographic areas and countries within the UK; and, other characteristics that previous research has found to be important determinants of well-being (for example, personal relationships, health and work situation).

The APS is a continuous household survey, covering the UK, with the aim of providing estimates between censuses of important social and labour market variables at a local area level. The APS is not a stand-alone survey, but uses data combined from two waves of the main Labour Force Survey (LFS) with data collected on a local sample boost. For more information on the APS, please see the Annual Population Survey Quality Methodology Information report.

Supporting information on methodological aspects can be found in the Personal Well-being Survey User Guide and in Supplementary information to accompany the 3-year Annual Population Survey and personal well-being dataset.

Personal well-being is presented as both average means and thresholds. Thresholds present the proportion responding in defined response categories as outlined in Table 2.

This report contains the following sections:

  • Output quality
  • About the output
  • How the output is created
  • Validation and quality assurance
  • Concepts and definitions
  • Other information, relating to quality trade-offs and user needs
  • Sources for further information or advice
Back to table of contents

3. Output quality

This report provides a range of information that describes the quality of the data and details any points that should be noted when using the output.

We have developed Guidelines for measuring statistical quality; these are based upon the five European Statistical System (ESS) quality dimensions. This report addresses these quality dimensions and other important quality characteristics, which are:

  • relevance
  • timeliness and punctuality
  • coherence and comparability
  • accuracy
  • output quality trade-off
  • assessment of user needs and perceptions
  • accessibility and clarity

More information is provided about these quality dimensions in the following sections.

Back to table of contents

4. About the output


(The degree to which statistical outputs meet users’ needs.)

One of the main benefits of collecting information on personal well- being is that it is based on people’s views of their own individual well-being. In the past, assumptions were made about how objective conditions, such as people’s health and income, might influence their individual well-being. Personal well-being measures, on the other hand, take account of what matters to people by allowing them to decide what is important when they respond to questions.

The uses of personal well-being data are varied, but four main uses have been identified:

  • overall monitoring of national well-being
  • use in the policy-making process
  • international comparisons
  • allowing individuals to make informed decisions about their lives

There is demand for personal well-being information to inform the policy-making process, both in central government and local government. Personal well-being data can be used in a number of ways. The large sample sizes of the personal well-being datasets allow for comparison between different sub-groups of the population (for example, different age groups or different ethnic groups) and between different areas within the UK (for example, countries and regions). This can help policy-makers target policies at the groups or areas with the highest need in terms of personal well- being.

Analysis can also be carried out to look at how different objective domains relate to personal well- being and which have the biggest effect on personal well-being. This can help in identifying which policy measures could improve personal well-being most effectively.

Another use is in cost-benefit analysis in the policy appraisal process, which could help inform decisions around which forms of spending will lead to the largest increases in personal well-being (Dolan et al. 2011). The Green Book is HM Treasury's guide for government departments on the appraisal of the costs and benefits of projects through social cost-benefit analysis. A Green Book discussion paper, produced jointly by HM Treasury and the Department for Work and Pensions (DWP), looks at the potential uses of personal well-being measures in social cost-benefit analysis. It looks specifically at the life satisfaction approach, which involves estimating the impact of certain outcomes or non-market goods on life satisfaction from surveys such as the Annual Population Survey (APS).

Added to this, looking at policies through a “well-being lens” and using data to inform not only the formulation of policy but also how policy could be better implemented with people’s well-being in mind is an important use. Evaluation and monitoring of policy could also potentially benefit from using personal well-being information.

Measuring personal well-being is not a new approach. Various international surveys have collected this type of information around the world over the last few decades. What is new, is that there is increasing international recognition that this should be included in official data collection and also that the European Statistical System (ESS) would benefit from the inclusion of personal well-being measures.

The Organisation for Economic Co-operation and Development (OECD) has developed guidance on the measurement of subjective well-being to improve the harmonisation of data collection for national statistics offices around the world. The ONS Measuring National Well-being programme is aligned with this international agenda for measuring subjective well-being, enabling comparisons to be drawn between the subjective well-being of people in the UK and in other countries.

APS datasets, including the ONS4, are deposited annually at the UK Data Service where they can be accessed by academic institutions and members of the public under end user licence or the approved researcher route.

Back to table of contents

5. How the output is created

The analysis presented on personal well-being discussed in this report is from the APS. For more information on the Annual Population Survey (APS), please see the APS Quality and Methodology Information report. The APS surveys the population of the UK but excludes those in communal establishments and those under the age of 16 years old.

The achieved sample size of the APS is approximately 122,000 households (or 320,000 respondents) on each annual APS dataset. However, of these 320,000 respondents over 150,000 provide valid personal well-being responses. This difference is in part due to personal well-being questions having to be answered in personal and in part due to non-response.

The APS has the largest coverage of any household survey in the UK and enables the production of statistics for small geographical areas. Sampling errors are smaller compared with other social surveys, because the APS has a single stage sample of addresses.

Interviews in all waves are carried out either on a face-to-face basis with the help of laptops, known as computer assisted personal interviews (CAPI) or on the telephone, known as computer assisted telephone interviews (CATI). Mode of interview has been previously known to effect personal well-being responses. This is something that should be accounted for in personal well-being analysis.

The APS datasets are weighted to reflect the size and composition of the general population, by using the most up to date official population data. Weighting factors take account of the design of the survey (which does not include communal establishments) and the composition of the local population by age and gender. The weights for other sample members are then adjusted to compensate for this.

Sample attrition is the term applied to respondents who begin the survey but subsequently drop out before completing all the survey waves. It has been known for some time that these respondents tend to have different characteristics to those who remain in all waves of the survey, and can, therefore, result in sample attrition bias. The large sample size of the datasets along with the weighting applied remove both attrition and non-response bias.

The APS datasets are reweighted back for 2 years, to use more up-to-date mid-year population estimates and are reweighted back for 10 years using the latest census population estimates.

The non-proxy adult weight is provided for analysis of the personal well-being questions. On the APS, the well-being questions are only asked of persons aged 16 and over who gave a personal interview as proxy answers are not accepted. Therefore, the well-being weight is calculated for each individual and is zero for respondents who were under 16 years of age or who were not present in person for the interview. Cases with weights of zero will not count towards analysis of results.

The weights for non-proxy adults tend to be higher than the corresponding weights on the APS dataset because these responding adults receive a higher weight to account for those adults with similar age, sex and geographical demographics for which proxy responses were obtained, who are effectively non-responders. Applying the well-being weight to the APS dataset will cause the total number of cases in the dataset to be grossed up to the estimated population of adults (aged 16 and over) within the UK as at mid-point of the time period analysed. Further details on the weights included in the well-being dataset can be found in the Personal Well-being Survey User Guide (12-month dataset).

Both the annual and 3-year releases of personal well-being include two measures of personal well- being. These are mean averages and thresholds (the proportion of people reporting defined responses on the 0 to 10 scale). The purpose of presenting both these measures is to provide both a summary measure of well-being through the mean and an indication of inequality through the dispersion of the results. The aim of these releases is to present statistics on personal or subjective well-being, to inform the debate on what matters most to the population of the UK.

To ensure that the output meets quality and accuracy standards the data is quality assured at an early stage by running frequencies and cross-tabs to identify whether there are any discontinuities. Running these checks at an early stage allows for any concerns to be investigated thoroughly to identify if it could be caused by a questionnaire change, processing error or real world change. When calculating the estimates themselves the team dual run’s them independently to make sure errors have not been brought in during their calculation. These estimates are compared against one another to ensure that they are the same, prior to comparing with previous estimates for a validity check. The estimates are also peer reviewed with experts in personal well-being to ensure that the data is as we would expect.

As previously mentioned, all personal well-being estimates produced are weighted to account for them being produced from a survey.

Sample sizes are also provided as unweighted counts of valid responses to each of the well-being questions. It is possible for each of the four questions (life satisfaction, worthwhile, happiness and anxiety) to have different sample sizes; this is because respondents are able to choose which, if any, of the personal well-being questions they answer.

From the April 2015 to March 2016 dataset onwards, personal well-being data and estimates will be reweighted on an annual basis. Re-weighting involves recalibrating the weights so that they sum to the most recent population estimates. The effect of this on personal well-being estimates is minimal. However, it does provide a more accurate representation of the results at that time. As a consequence of the reweighting and the transition of the personal well-being questions to the APS main data release, the personal well-being data series has been revised. Sensitivity analysis has been carried out to assess the impact to the Personal Well-being series as a consequence of this change and it has shown minimal impact. For more information on the sensitivity analysis, please see the Impact of transition to Annual Population Survey dataset in personal well-being in the UK: 2015 to 2016.

Timeliness and punctuality

(Timeliness refers to the lapse of time between publication and the period to which the data refer. Punctuality refers to the gap between planned and actual publication dates.)

Personal well-being estimates produced from both the annual and 3-year datasets are produced on an annual basis.

Personal well-being estimates have traditionally been published 5 months from the end of the reporting period for the annual release. For the time period April 2015 to March 2016 onwards, this time lag will reduce to 3 months.

The personal well-being 3-year dataset and associated results have previously been published with a time lag of 12 months and 10 months respectively. This is something we plan to address with the aim of reducing further.

For more details on related releases, the release calendar is available online and provides 12 months’ advance notice of release dates. In the unlikely event of a change to the pre-announced release schedule, public attention will be drawn to the change and the reasons for the change will be explained fully at the same time, as set out in the Code of Practice for Official Statistics.

Back to table of contents

6. Validation and quality assurance


(The degree of closeness between an estimate and the true value.)

As the Annual Passenger Survey (APS) (which is made up of the Labour Force Survey (LFS) and a local sample boost) is a sample survey, it provides estimates of population characteristics rather than exact measures. In principle, many random samples could be drawn and each would give different results, because each sample would be made up of different people, who would give different answers to the questions asked. The spread of these results is the sampling variability. Confidence intervals are used to present the sampling variability. For example, with a 95% confidence interval, it is expected that in 95% of survey samples, the resulting confidence interval will contain the true value that would be obtained by surveying the whole population.

Confidence intervals and sample sizes are presented in the datasets accompanying each personal well-being release.

Personal well-being estimates follow Government Statistical Service (GSS) Statistical Disclosure Control and Communicating Uncertainty and Change guidance. As such, estimates of personal well-being are not published if the sample size from which they are based is less than 50, their numerator is 5 or less, or their co-efficient of variation (CV) is greater than 20%. These rules are for both quality reasons and for disclosure control.

The CV is the ratio between the standard error of the estimate and the estimate itself and it gives an indication of the variability and accuracy of the estimate.

Table 3 presents the bounds used to suppress or colour code the accuracy of the estimates.

Personal well-being threshold estimates are also secondary suppressed. This means that if one threshold has been suppressed for a particular breakdown, then all thresholds will be so that the estimate cannot be derived back.

Coherence and comparability

(Coherence is the degree to which data that are derived from different sources or methods, but refer to the same topic, are similar. Comparability is the degree to which data can be compared over time and domain, for example, geographic level.)

The first subjective well-being Annual Population Survey (APS) dataset was published for the period April 2011 to March 2012.

The subjective or personal well-being questions have remained the same since they were first introduced to the APS in April 2011.They gained National Statistics status in 2014.

The questions have been tested extensively both on the Opinions and Life Style Survey (OPN) and using cognitive testing methods. Results from OPN survey testing are available in the report Investigation of subjective well-being data from the ONS Opinions Survey.

Placement of the questions on the APS

Responses to evaluative questions can be determined in part by the context of the interview. For example, placing after questions relating to health or the labour market may affect the answers that respondents make.

Prior to April 2011, we carried out small-scale cognitive testing of the placement of the subjective well-being questions in the Integrated Household Survey (IHS). As a result, it was decided that the placement of the overall monitoring questions would be fairly early on in the questionnaire, directly after the core questions on household and individual demographics.

This allows time for rapport to be built up between the interviewer and the respondent by the time the subjective well-being questions are asked without allowing later questions, such as those on employment, to influence response to the subjective well-being questions.

Question order

The importance of the ordering of the personal well-being questions themselves also needs to be taken into consideration when designing personal well-being question modules. Any changes year to year could affect the ability to examine changes in personal well-being over time, as some of the observed change could be due to ordering effect.

Currently, within the APS, the question on life satisfaction is followed by the worthwhile question, which is then followed by the happiness yesterday and the anxious yesterday questions respectively. This order has not changed on the APS since the questions were introduced in April 2011.

Question wording

It is also known that for any survey question the wording can have an effect on the answer that is received from respondents. Personal well-being questions are no exception. It is not only response scales that differ from survey to survey but also question wording, including the use of time frames.

We have carried out testing on the effect of the change in question wording including the inclusion of “even if yesterday was not a typical day” for happiness, and the effect of moving from “nowadays” to “these days” for the question on life satisfaction. For more information on this testing, please see the Summary of results from testing of experimental subjective well-being questions – December 2012.

Question Scales

We decided that an 11-point scale from 0 to 10, where 0 is “not at all” and 10 is an absolute value such as “completely”, should be used for all the APS subjective well-being questions. The reason for this decision was to ensure that the scales between the questions are consistent to help respondents answer the questions more easily and also to aid analysis across the separate questions.

Further to this, 11-point scales of this nature are commonly used across other surveys of interest, particularly internationally, and using the same type of scale aids comparisons with these estimates. An 11-point scale also allows for finer distinctions than a smaller (7-point) scale, but not so many points that would make it difficult for respondents to select one value over another.

Mode of interview

Testing has also shown that people respond more positively to the subjective well-being questions when interviewed by telephone rather than face-to-face. As people are interviewed using both methods on the APS, this will have some effect on the subjective well-being results. Further work is planned to look at the nature and scale of any effect of the mode of interview.

Table 4 shows the split between face-to-face and telephone interviewing in the APS over the period we have captured personal well-being.

The ONS4 personal well-being questions are asked across a number of surveys that are internal and external to the organisation. Although the same questions are asked, the results of each of the surveys may not be comparable due to the sampling frame, sampling method and placement. For a full list of surveys that use the ONS4, please see Surveys using the four ONS personal well-being questions.

Back to table of contents

7. Concepts and definitions

(Concepts and definitions describe the legislation governing the output and a description of the classifications used in the output.)

The personal well-being questions on the Annual Passenger Survey (APS) do not currently come under any specific regulations or legislation. However, consideration is being given to including personal well-being in European regulations within coming years. Additionally, the EU statistics on income and living conditions (EU-SILC) survey ran an ad hoc module on subjective well-being during 2013, and will be doing so again in 2017 and 2018. The EU-SILC is covered by EU regulations.

The personal well-being statistics gained National Statistics accreditation in 2014 and work is ongoing for the ONS4 personal well-being questions to be included in the harmonised standard classifications.


See “Experience”.


Individual well-being is one of a number of domains that makes up a framework for measuring national well-being. This “framework” consists of a set of 10 areas or “domains”, with other domains covering a range of objective and subjective measures such as health, economy, our relationships and what we do.


The “eudemonic” approach is sometimes referred to as the psychological or functioning and flourishing approach, which draws on self-determination theory and tends to measure such things as people’s sense of meaning and purpose in life, connections with family and friends, a sense of control and whether they feel part of something bigger than themselves. “Overall, to what extent do you feel the things you do in your life are worthwhile?” is the eudemonic question included on the APS.


The “evaluative” approach asks individuals to step back and reflect on their life and make a cognitive assessment of how their life is going overall, or on certain aspects of their life. “Overall, how satisfied are you with your life nowadays?” is the evaluative question included on the APS.


The “experience” approach seeks to measure people’s positive and negative experiences (or affect) over a short time frame to capture people’s subjective well-being on a day-to-day basis. The APS includes both positive (“Overall, how happy did you feel yesterday?”) and negative (“Overall, how anxious did you feel yesterday?”) affect questions.

Individual well-being

Subjective or personal well-being is based on people's views of their own individual well-being. Personal well-being measures are grounded in individuals’ preferences and take account of what matters to people by allowing them to decide what is important when they respond to questions.


Thresholds are used in the report to present dispersion in the data. These show the proportion of responses that fall between certain ratings on the 0 to 10 scale. The thresholds are divided into four categories or two for the smaller geographic areas so that estimates are based on larger sample sizes.

Back to table of contents

8. Other information

Output quality trade-offs

(Trade-offs are the extent to which different dimensions of quality are balanced against each other.)

The mixed mode of data collection is likely to have an effect on the personal well-being results (see further details in the “Coherence and comparability” section). By using a mixed mode approach, a larger sample size is achieved allowing more detailed analysis of sub-groups, including ethnic minority groups.

However, whilst a larger sample size enables us to drill down to look at personal well-being of ethnic minority groups and the differences that exist between these groups, it is not clear whether these are true differences of subjective well-being or reflect cultural and/or linguistic differences.

Assessment of user needs and perceptions

(The processes for finding out about users and uses, and their views on the statistical products.)

We use and have used a large number of groups to engage with users of personal well-being data. These include:

  • Technical Advisory Group, which includes experts from a variety of sectors including academia and other government departments
  • Measuring National Well-being Advisory Forum
  • Social Impacts Task Force
  • National Statisticians Advisory Group
  • Well-being Policy Steering Group
  • focus groups to determine citizen users’ level of understanding of the presentation of subjective well-being estimates and to determine the clearest and most easily understood methods of presenting results
  • consultations on format and presentation of subjective well-being results following releases

We run public consultations on the major decisions affecting users. This has included:

  • the national well-being debate
  • consultation on domains and measures of national well-being

We consult known users on issues that we consider do not warrant full public consultation. Previously, guidance was also sought from a Technical Advisory Group.

The Measuring National Well-being Programme keeps a log of ad hoc user engagement and monitors this to ensure that users’ needs are being addressed.

Additionally, we have carried out three stages of cognitive testing and a series of quantitative testing on the Opinions and Life Style Survey (OPN), as part of the on-going programme of testing of the experimental subjective well-being questions, to understand better how people interpret and answer the questions.

Back to table of contents

9. Sources for further information or advice

Accessibility and clarity

(Accessibility is the ease with which users are able to access the data, also reflecting the format in which the data are available and the availability of supporting information. Clarity refers to the quality and sufficiency of the release details, illustrations and accompanying advice.)

The UK Data Service, at Essex University and Manchester University, provides free access to the various Annual Population Survey (APS) datasets and can be contacted via the UK Data Service website.

The ONS Social Survey Data Access and Response Team provide APS data for a fee and can be contacted by telephone on +44 (0)1633 455678. Tables using APS data can also be requested by emailing the ONS data service at socialsurveys@ons.gov.uk.

For queries regarding personal well-being and information for how to access the data, please contact the Quality of Life team via email at QualityOfLife@ons.gsi.gov.uk or by telephone on +44 (0) 1633 455713.

Our recommended format for accessible content is a combination of HTML web pages for narrative, charts and graphs, with data being provided in usable formats such as CSV and Excel. We also offer users the option to download the narrative in PDF format. In some instances other software may be used, or may be available on request. Available formats for content published on our website but not produced by us, or referenced on our website but stored elsewhere, may vary. For further information please contact the Quality of Life team via email at QualityOfLife@ons.gsi.gov.uk or by telephone on +44 (0) 1633 455713.

For information regarding conditions of access to data, please refer to the following links:

Back to table of contents

Contact details for this Methodology

Telephone: +44 (0) 845 6041858