1. Introduction

This guide outlines the different data sources and outputs that feed into the analysis of income and earnings within the UK. It explains important information for each data source, including what data are available and the sources’ main uses, strengths and limitations. Contact details are also provided for each data source if you require additional information.

This guide brings together information on data sources from a number of government departments, including the Office for National Statistics (ONS), Department for Work and Pensions (DWP) and HM Revenue and Customs (HMRC).

The guide begins with an overview of the different types of income and earnings and how they are all related. Sections 3, 4, 5 and 6 look at the different sources of data relating to wages and earnings, labour costs, household income and other income, respectively. Finally, Section 7 provides information on upcoming publications related to income and earnings; a link to this timetable can also be found in Related downloads.

If you are looking to use or understand income or earnings data, this guide can be particularly useful for comparing the different data sources and identifying the most appropriate data source for your needs. The Annex tables in Related downloads summarise the strengths and limitations of each of the data sources, to aid comparison. The information for each source also contains links to relevant datasets and other articles for further reading.

There are, however, some limitations of this guide. It is not necessarily suited to answering specific user questions about income and earnings statistics as it is structured around the different sources. Explaining income and earnings: important questions answered may be more useful for users looking for specific answers.

Back to table of contents

2. Income and earnings overview

It is useful to first outline how income and earnings are related, and to explain how the various components are defined and calculated.

The components of income and earnings are summarised in Figure 1.


Income and earnings, which is earnings before any deductions are made from taxes (including National Insurance contributions (NICs)) and additions from benefits.

From gross earnings, we can also look at two similar measures. Firstly, take-home pay refers to the earnings received by an employee after they have paid tax on their earned income (typically referred to as “Income Tax”) and after their employee NICs have been deducted. Take-home pay is typically what is paid into an employee’s bank account.

Measuring take-home pay can be particularly challenging and so some of the estimates and measures outlined in this guide may not adhere precisely to this definition. For instance, the deduction of student loan repayments may or may not be included depending on which source is being considered.

Alternatively, from the point of view of the employer, we can also consider labour costs. These typically refer to the gross earnings paid by a business (wage costs), plus a number of non-wage related costs such as employer NICs, pension contributions and benefits in kind paid by the employer.

Returning to the starting point of gross earnings, we can consider how earnings feed into income.


Combining employee earnings with those of the self-employed, along with private pensions and other sources such as income from investments, gives original income. Then adding cash benefits to original income, such as the State Pension, Child Benefit or Jobseekers’ Allowance, gives gross income.

From gross income, we then consider deductions. Firstly, subtracting direct taxes (for example, Income Tax), employee NICs and Council Tax or Northern Ireland Rates, leaves disposable or net income. Then further deductions of indirect taxes (where the tax is typically levied on one entity but paid by another) such as Value Added Tax (VAT) and duties on alcohol or tobacco, result in what is referred to as post-tax income.

Finally, we add in benefits in kind paid by the state such as health and education, which are allocated on the basis of household characteristics. This leaves us with final income.

Income and earnings data are often collected and analysed at different levels. Typically, earnings data concern individuals. This is because it is relatively easy to attribute earnings from employment to each employee.

On the other hand, income data are often considered at both the individual and the household level. This is because many forms of government intervention, particularly cash benefits, are often assessed at the benefit unit1 level and are therefore more difficult to attribute to individual household members.

Income statistics that are considered at the household level are also often adjusted or “equivalised” to account for the fact that different-sized households require and generate different levels of income. Equivalisation then allows for comparisons to be made across households with differing compositions (for example, comparing a single-person household with a household containing a family of four).

Notes for: Introduction

A household consists of either one person living alone, or a group of people (not necessarily related) living at the same address who share cooking facilities and share a living room, sitting room or dining area. A household will consist of one or more families or benefit units.

Back to table of contents

3. Wages and earnings

The sources in this section collect and/or publish data on gross earnings, which is that paid to an employee in return for employment before any deductions are made. This section also covers sources of information on take-home pay. It should be noted that measures of take-home pay often differ conceptually between different data sources.

Average weekly earnings

Average weekly earnings can be found in the monthly labour market statistical bulletin, published by Office for National Statistics (ONS) under Section 8 and also in the datasets EARN01 to EARN03.

Average weekly earnings microdata are not publicly available. For more information, contact earnings@ons.gov.uk or call +44 (0)1633 45 6120.

What exactly does it measure?

The average weekly earnings (AWE) measure is our lead indicator of short-term changes in earnings. It replaced the Average Earnings Index (AEI) in 2010.

AWE is calculated from returns to the Monthly Wages and Salaries Survey (MWSS), a survey of 9,000 businesses covering 13.8 million employees. The MWSS captures information about each company’s total wage bill and the number of people paid in the reference period. Having been weighted to the Great Britain level, the total wage bill is then divided by the number of employees to give average weekly earnings.

AWE also reflects changes to the industrial composition of the workforce. For instance, all other things being equal, an increase in the relative number of employees in highly paid industries will cause average earnings to rise. This compositional effect, known as the employment contribution, was not captured by AWE’s predecessor, the AEI. We publish separate estimates of the wage and employment contributions to AWE growth in supplementary tables called the AWE decomposition.

What sort of information is usually published and what can we extract from the data?

AWE is published on a monthly basis as part of the labour market statistical bulletin. The section on AWE typically covers:

  • the levels of average weekly earnings broken down into: regular pay (excluding bonuses), total pay (including bonuses), and bonus pay average weekly earnings by sector (public and private)

  • average weekly earnings by industry

  • real average weekly earnings by sector (AWE adjusted for inflation)

  • wage and employment contributions

Average weekly earnings by sector are available both before and after seasonal adjustment. Seasonal adjustment is widely used in official statistics as a technique to interpret time series data, which would otherwise be distorted by seasonal factors that do not reflect underlying trends.

Additional analysis is published alongside the main results; this analysis includes real average weekly earnings (AWE adjusted for inflation using the Consumer Prices Index including owner occupiers’ housing costs (CPIH) as a deflator) and information on wage and employment contributions to nominal earnings growth. The employment contribution changes if the relative proportion of employment in the 24 industrial headings changes, but will not necessarily change if total employment increases.

What are the main uses and strengths?

A main strength of using AWE is its frequency and timeliness with results being produced on a monthly basis, usually with a six- to seven-week gap between the end of the reference period and the publication date. This timeliness and frequency is why the AWE is our lead indicator of short-term changes in earnings. It is the AWE measure of average earnings that is most often compared with measures of inflation to calculate changes in real earnings.

Another strength of AWE is its ability to capture bonus payments. With the MWSS being conducted monthly it captures bonus payments in every month of the year, with March often being the main month in which bonuses are paid. An article Average Weekly Earnings – Bonus Payments in Great Britain is published on our website once a year.

Given its strength in capturing bonus payments, AWE is also used to supplement ASHE data in our estimates of public and private sector pay differentials. ASHE data are adjusted at an industry level, in line with the AWE measures of bonuses. This is crucial given that bonus payments are one of the main differences in pay between the public and private sectors.

What are the main limitations?

The MWSS is a survey of employers and as such does not cover the self-employed. Nor does it cover HM Armed Forces or government-supported trainees. Furthermore, it does not collect any information on individual employees’ characteristics and as such does not allow analysis beyond sector and industry.

The MWSS also excludes businesses with fewer than 20 employees to limit costs and respondent burden. Figures for these businesses are estimated: employment figures are taken from the Inter-Departmental Business Register (IDBR) while earnings are estimated using a factor from the Annual Survey of Hours and Earnings (ASHE), both of which do cover small businesses.

The composition effect captured by the AWE refers only to changes between industries. Therefore, it does not capture compositional changes between occupation, age or changes within the same industry. It should also be highlighted that AWE does not differentiate between full-time and part-time workers, and so a relative increase in the prevalence of part-time working would indicate that average weekly pay was falling, whereas average hourly pay may remain the same.

Annual Survey of Hours and Earnings

The latest Annual Survey of Hours and Earnings (ASHE) publication (now known as Employee earnings in the UK) is available.

Due to the sensitive nature of ASHE microdata they are not available for download from the UK Data Service website but may be accessed through Secure Access arrangements. Access requires accreditation by the UK Statistics Authority as an Approved Researcher, completion of face-to-face training, and agreement to the Secure Access's User Agreement and Breaches Penalties Policy. For more information on access to ASHE microdata, please see the UK Data Service.

For more information on ASHE, please contact us at earnings@ons.gov.uk or on +44 (0)1633 45 6120.

What exactly does it measure?

The Annual Survey of Hours and Earnings (ASHE) provides information about the levels, distribution and make-up of earnings and hours paid for employees in all industries and occupations across the UK. ASHE and the New Earnings Survey (NES), which preceded it, have been collected every year since 1970 allowing for comparison over time. It should be noted that weighted data are available from 1997 onwards, prior to 1997 data are unweighted. Furthermore, data prior to 1997 refer only to Great Britain; data from 1997 onwards cover the UK as a whole.

Businesses are surveyed in April of each year and are asked to provide information on employees who fall within a 1% sample of the HM Revenue and Customs (HMRC) Pay As You Earn (PAYE) register taken in January of the same year (with follow-up surveys to capture those employees who change jobs or join the labour market between January and April). The final ASHE dataset typically covers around 180,000 jobs from around 60,000 responding businesses.

What sort of information is usually published and what can we extract from the data?

The main ASHE publication is, as its name suggests, annual, with the survey relating to April and publication taking place in the following October.

From October 2018, the ASHE publication was split into three separate bulletins: Employee earnings in the UK, Low and high pay in the UK and Gender pay gap in the UK.

The main Employee earnings publication focuses on gross weekly earnings for full-time employees on adult rates of pay whose pay in the reference period was unaffected by absence. It covers:

  • weekly and annual earnings

  • earnings trends (that is, changes over time)

  • public sector compared with private sector

  • earnings by region, age, occupation and industry

  • the components of earnings (such as overtime and incentive pay)

  • paid hours worked

The Low and high pay bulletin focuses on the distribution of earnings of high- and low-paid jobs and jobs paid below the National Minimum Wage, and the Gender pay gap bulletin focuses on the differences in pay between men and women.

ASHE publishes a collection of data tables alongside the bulletins and a Guide to tables is published to help users identify which dataset is most relevant to their needs. While the main release focuses on full-time employees, the data cover part-time employees too, such that they can be analysed separately or incorporated with full-time employees. Furthermore, where appropriate, analysis can also include those whose pay was affected by absence and/or those who were not on adult rates of pay.

Because the sample upon which ASHE is based does not change from year to year (except for new entries or retirees), ASHE datasets are combined to create a panel dataset. This dataset observes the same employees in each year that they are in employment and their employer responded to the survey.

The panel dataset covers most of the same variables as the annual datasets, allowing for the analysis of hours and earnings for different cohorts of employees. For instance, we made use of the panel dataset in its analysis of Earnings in the UK over the last four decades, by tracking those who were aged 21 years in 1975, 1985 and 1995 up to 2013 to compare the first 18 years of the careers of each cohort (from the age of 21 years).

What are the main uses and strengths?

The main use of ASHE is to analyse the distribution of earnings in the UK. While this is also possible using the Labour Force Survey (LFS), ASHE is thought to be a more accurate source of information on earnings as the information is provided by employers rather than being self-reported by employees.

ASHE also benefits from a large sample size which, coupled with the array of individual and geographic characteristics that are also gathered, allows for more detailed analysis than other sources of earnings data.

ASHE is the principal source of data used in our estimates of the public and private sector pay differential and the gender pay gap. ASHE is also the principal data source used for estimating the number of employee jobs being paid below the National Minimum Wage and National Living Wage.

ASHE also benefits from sampling the same employees over time, which gives it a longitudinal aspect that is not present to the same extent on other surveys of earnings.

What are the main limitations?

The most obvious limitation of ASHE is that it is not as timely as other measures of earnings. Whereas AWE has a six- to seven-week lag, the lag from the ASHE reference period to publication is usually six to seven months.

Like AWE, ASHE only covers employees and therefore excludes the self-employed. The selection of personal characteristics is also more limited than other sources, such as the Labour Force Survey.

A further limitation is that there is no information on what individuals are doing when they are not present on the survey. Potentially they could be unemployed; have switched to self-employment; have exited the labour market for a period or their employer may not have responded to the survey.

There are known coverage issues with data on bonus and incentive payments relating to the reference period. Primarily, this is because the information is not available to respondents at the time when they are required to provide the information to us. Data on annual bonus payments are thought to be better in this respect, though some respondents still do not have access to the necessary information at the time when they complete the questionnaires.

Since 1970, there have also been several changes to coverage, methodology, and classification conventions for the survey. These result in a series of discontinuities where the data may not strictly be comparable between any given pair of years where a discontinuity exists. However, it is thought that when comparing over a longer time series that the discontinuities have a minimal impact on longer-term trends. Since 2000, discontinuities in the series exist between 2003 to 2004, 2005 to 2006 and 2010 to 2011. For each of these discontinuities we have produced two versions of results, enabling valid comparisons with series on either side of the discontinuity.

Labour Force Survey

Labour Force Survey (LFS) earnings information is published in the monthly labour market statistical bulletin on the ONS website in datasets EARN04 to EARN08. These datasets are updated on a quarterly basis.

LFS microdata can be accessed through the UK Data Service website under a standard End User Licence. More detailed microdata are available via the Secure Research Service.

For more information about the LFS, please contact us at social.surveys@ons.gov.uk or on +44 (0)1633 455678

What exactly does it measure?

The Labour Force Survey (LFS) is a rolling survey of households, which has taken place on a quarterly basis since 1992. It is a main input into the monthly publication of labour market statistics, forming the basis for the rate or level of both employment and unemployment. Each sample is made up of approximately 40,000 responding households. Each household selected remains in the survey for five quarters with information on earnings requested only in quarters 1 and 5, thus enabling year-on-year comparisons. The earnings questions were not part of the initial variable set in the LFS, with earnings questions in respect of weekly pay being added in winter 1992 and hourly pay in autumn 1993. Until 1997, earnings questions were asked only in quarter 5; the change to asking in quarters 1 and 5 reduced sampling errors by approximately 30%. The current range of earnings questions dates from 1999.

The primary purpose of the LFS is “the prompt publication of main aggregate, whole economy, indicators, for the integrated assessment of labour market conditions”. The “labour market” covers all aspects of people’s work, including the education and training needed to equip them for work, the jobs themselves, job-search for those out of work, and income from work and benefits.

As such the LFS collects information on a wide range of characteristics including both gross earnings (hourly and weekly) and take-home pay after deductions.

What sort of information is usually published and what can we extract from the data?

Earnings estimates from the LFS are published quarterly and typically cover:

  • gross weekly earnings of full-time employees by region, occupation and industry

  • the distribution of gross hourly earnings of employees

LFS microdata also include information on usual pay and why this may have differed from pay in the reference week. The LFS also captures respondents’ take-home pay after “all deductions”. It is important to note that this differs conceptually to the measure of take-home pay that can be estimated from ASHE (see Section 2) as it may include deductions such as pension contributions and student loan repayments.

The microdata also cover a wide range of individual characteristics including but not limited to:

  • age

  • sex

  • qualifications

  • employment status: full-time or part-time, public or private, and permanent or temporary

  • ethnicity

  • disability

The full catalogue of LFS variables can be found in the LFS user guides – Details of LFS variables

What are the main uses and strengths?

The principal strength of using the LFS measures of earnings is the rich selection of classificatory variables that are gathered on the survey. While ASHE is often favoured for its accuracy, there are many individual characteristics that are only gathered on the LFS. For instance, information on education, ethnicity, nationality, disability and many other variables are captured on the LFS but not on ASHE. Analysing how earnings vary with any of these characteristics is typically done using LFS data.

The LFS also has a longitudinal element to its data as individuals’ earnings data are captured in both waves 1 and 5 of the survey. This allows analysis of changes to individuals’ earnings over the course of the year.

The LFS is also used widely in academia due to the ease of access offered by the UK Data Archive. By contrast, ASHE microdata are only accessible via secure microdata labs.

What are the main limitations?

The data on individuals’ earnings captured by the LFS are thought to be of a lower quality than ASHE as LFS information is self-reported through computer-assisted or telephone interview. ASHE and AWE, however, gather information from the employer that is thought to be more accurate, as employers can consult payroll records. Individuals may not have such records to hand and their responses may therefore be subject to higher levels of recall error.

Furthermore, LFS responses can be given by proxy (by other individuals in the same household) when an individual is unavailable for interview. This gives further scope for recall error from respondents. Due to this recall error, our estimates of earnings based on the LFS typically exclude those who earn more than £100 per hour as a quality assurance measure. These factors combined mean that gross weekly and hourly pay are known to be underestimated on the LFS.

Earnings information on the LFS is only captured for employees and so, like AWE and ASHE, it excludes the self-employed (though other information is captured for the self-employed).

The LFS has also suffered from falling response rates over recent years. In 1999 the Wave 1 (that is, the first quarter that a household is included in the LFS sample) response rates were around 80%, whereas a steady decline since then has meant that by 2018 they were less than 55%.

In addition to the falling response rates there is a large percentage of non-response to the specific earnings questions. Typically, only around 10% of respondents answer these questions. This impacts on the robustness of the data, especially when applying additional industry or occupation filters to the data.

A further limitation of the LFS from an analytical perspective is the repeated changes to some classifications (for example, ethnicity) and year-specific variable names. Extensive recoding may be required where comparing some of these variables or characteristics over time.

Back to table of contents

4. Labour costs

Labour costs typically refer to the gross earnings paid by a business (wage costs), plus several non-wage related costs such as employer National Insurance contributions, employer pension contributions and benefits in kind provided by the employer.

Index of Labour Costs per Hour

Index of Labour Costs per Hour (ILCH) is an experimental statistic published on a quarterly basis. ILCH microdata are not publicly available.

For more information on ILCH, please contact us at earnings@ons.gov.uk or on 44 (0)1633 45 6120.

What exactly does it measure?

The Index of Labour Costs per Hour (ILCH) is a measure of the cost of having an employee for an hour of work. ILCH was first published in 2005 and is published quarterly. It reflects changes in wages and salaries, non-wage costs and the quantity of hours worked over the quarter and is important for monitoring inflationary pressures in the labour market.

ILCH is primarily based on the Monthly Wages and Salaries Survey (MWSS) and the Labour Force Survey (LFS). The MWSS sample is drawn from the Inter-Departmental Business Register (IDBR), which is also used to weight the data. The LFS provides estimates of total hours worked based on average total hours worked in first and second jobs by employees. The Annual Survey of Hours and Earnings (ASHE) provides estimates of pay for employees of small businesses and the data are also used to produce factors to estimate Northern Ireland’s wages and salaries. The Labour Costs Survey (LCS) and the Annual Business Survey (ABS) are also inputs to ILCH, providing estimates of non-wage labour costs.

The ILCH index goes beyond other earnings indicators to include non-wage costs (sickness, maternity and paternity costs, pensions contributions, benefits in kind and National Insurance contributions), as well as the wages and salaries component. ILCH is currently published as an Experimental Statistic. Further development work is required before ILCH is submitted for assessment as a National Statistic by the UK Statistics Authority.

What sort of information is usually published and what can we extract from the data?

ILCH is published on a quarterly basis from 2000. The main publication of ILCH provides both index numbers and growth rates and covers:

  • the most recent labour costs data broken down by: wage costs, non-wage costs, and labour costs excluding bonuses and arrears

  • ILCH by industry (Standard Industrial Classification 2007: SIC 2007 A to S)

  • ILCH by sector (including public and private)

  • ILCH before and after seasonal adjustment

What are the main uses and strengths?

The production of ILCH is driven by European legislation, which requires that a harmonised index of labour costs be produced by all member states. It is the only short-term earnings per hour indicator in the UK and can be used as an early gauge of economic performance. This is because a business will typically alter the number of regular or overtime hours its employees work before considering changing the number of people it employs.

What are the main limitations?

The main limitation of ILCH is that only index numbers are published quarterly with no monetary values, though Eurostat derive annual data on hourly labour costs, which are published on the Eurostat website. This then also means that ILCH cannot be used to investigate the distribution of labour costs.

Revisions to the data series are frequent; this is due to the number of data sources used as an input in ILCH and the need to revise these when new data become available.

Furthermore, ILCH can be volatile when observing low-level industry aggregates where individual firms can have a larger impact on the data.

Given that the MWSS is a survey of employers, ILCH refers only to employees and therefore excludes the self-employed.

Unit labour costs

Unit labour costs are published as part of the quarterly Labour productivity statistical bulletin.

For more information about unit labour costs, please contact us at productivity@ons.gov.uk or on +44 (0)1633 455086.

What exactly does it measure?

Unit labour costs (ULCs) measure the cost of labour input per unit of real (inflation-adjusted) economic output. This is calculated by dividing total labour costs by output. With gross value added (GVA) used as the measure for real output, total labour costs can be broken down into the compensation of employees (CoE) and the labour share of mixed income. The CoE measures the aggregate labour costs of employees and mixed income measures the income of the self-employed, which conceptually includes returns to labour and to capital employed.

As such, ULCs reflect the full labour costs, including social security and employers’ pension contributions, incurred in the production of a unit of economic output. This methodology is consistent with our other data on labour inputs and the returns of labour.

What sort of information is usually published and what can we extract from the data?

Estimates of ULCs for the whole economy are published quarterly in the labour productivity bulletin.

Within this dataset are sectional ULCs, broken down by industry, in both index levels and percentage changes on the previous quarter and on the previous year. The ULCs are provided in both seasonally adjusted and non-seasonally adjusted terms. The seasonally adjusted figures are presented for the whole economy, market sector, manufacturing, production and total services industry classifications. In addition to these categories, the non-seasonally adjusted data are disaggregated further into:

  • agriculture, forestry and fishing

  • mining and quarrying

  • utilities

  • construction

  • wholesale and retail trade

  • transportation and storage

  • accommodation and food

  • information and communication

  • financial and insurance industries

  • real estate activities

  • professional, scientific and technical activities

  • administrative and support service activities

  • public administration and defence

  • education, health and social work

  • arts and entertainment

  • other services

What are the main uses and strengths?

Since labour costs account for around two-thirds of the cost of production of UK economic output, unit labour costs provide an indication of inflationary pressures in the economy. ULCs are used by the Bank of England among others when analysing the extent of spare capacity in the labour market; as they provide a comprehensive indicator of the inflationary pressure in the supply side of the economy. This is especially true when non-wage labour costs move differently to wages.

Although not a direct measure of productivity, an inverse relationship between unit costs and productivity tends to be observed. Movements in ULCs can be decomposed into movements in costs per unit of labour (which can be approximated by an index of earnings) minus the movement in labour productivity. Thus, other things equal, increases in labour productivity will tend to reduce ULCs.

ULCs also have the benefit of being fully consistent with both the output and income presentations of the national accounts.

What are the main limitations?

ULC measures deal exclusively with the cost of labour, which though important, should also be considered in relation to changes in the cost of capital. This consideration is more relevant in capital-intensive economies.

One of the difficulties with compiling an accurate measure of the costs associated with labour input within the UK is that there is no direct measure of labour costs in the self-employed sector. However, by using a factor share of mixed income derived from national accounts estimates of mixed income, we obtain an implied return to self-employed labour, which is conceptually preferable to the measure used previously as it incorporates information on the income of the self-employed.

One of the main limitations previously was that estimates were not available below the whole economy level, so manufacturing unit wage costs (UWCs) were used. This is no longer the case, as manufacturing ULCs, alongside other sectional ULCs, are available as part of the labour productivity bulletin. UWCs are limited in two ways.

Firstly, wage costs are a narrower measure than compensation of employees, which includes non-wage labour costs, including social security and employers’ pension contributions. Secondly, the UWC methodology assumes that wage costs of the self-employed were equal to those of employees. This was despite evidence from the national accounts that such an assumption is inconsistent with the income attributed to the self-employed.

Back to table of contents

5. Household income

The surveys and publications in this section are the principal sources of information on household income. They cover some or all of the stages identified in the introduction between original and final income, though they may differ slightly in how certain deductions and additions are made at the different stages of income redistribution.

Family Resources Survey

The Family Resources Survey annual report is published on GOV.UK alongside information for respondents and additional papers on measuring uncertainty and grossing methodology.

Family Resources Survey microdata can be accessed under a standard End User Licence on the UK Data Service website. More detailed microdata are available as a Secure Access File to be used in the Safe Room at the University of Essex – additional approval is needed for this level of access.

For more information on the Family Resources Survey, please contact the Department for Work and Pensions at team.frs@dwp.gov.uk or on +44 (0)20 7449 7074.

What exactly does it measure?

The Family Resources Survey (FRS) collects information on the incomes and circumstances of private households in the UK. It is designated as a National Statistic.

The FRS has been running since 1992 and is an annual cross-sectional survey. Until the financial year ending 2003, the survey covered Great Britain; it was then extended to cover all of the UK.

The target sample size for the FRS since April 2011 is 20,000 households; this was reduced from 25,000 households. The achieved sample size for that latest year was around 19,000 households.

What sort of information is usually published and what can we extract from the data?

The FRS covers:

  • income and state support receipt
  • tenure
  • disability
  • carers
  • pension participation

Information can be derived on an individual, benefit unit or household level depending on which is the most appropriate.

The FRS is the primary data source for households below average income and the pensioners’ incomes series. It is also a main source of the Department for Work and Pensions’ (DWP’s) estimates of take-up of income-related benefits and the policy simulation model, which is used extensively by DWP analysts for policy evaluation and costing of policy options.

The FRS is also used by other government departments including HM Revenue and Customs and HM Treasury and is also made available for external users via the UK Data Service.

What are the main uses and strengths?

The FRS is considered to be the best source for looking at cash benefit and tax credit receipt by characteristics not captured on administrative sources, and for looking at total benefit receipt on a benefit unit or household basis.

It is also good at capturing information on incomes: it captures more detail on different income sources compared with most other household surveys.

It collects a lot of contextual information on the household and individual circumstances, such as employment, education level and impairment. The FRS is therefore a comprehensive data source allowing for a wide variety of detailed analysis.

The FRS began in 1992 and so allows for comparisons over time.

What are the main limitations?

In common with all income surveys, the FRS results suffer from several issues of non-sampling error. These include the following:

  • measurement error – in particular, the FRS is known to under-report benefit receipt

  • non-response error – the FRS response rate in the recent years has tended to be between 50% and 55%; to correct for differential response rates, estimates are weighted using population totals

  • survey coverage – the FRS covers private households in the UK and so individuals in nursing or retirement homes, for example, will not be included

  • survey design – the FRS uses a stratified clustered sample designed to produce robust estimates at region level; the FRS is therefore not suitable for analysis below region level

  • sample size – although the FRS has a relatively large sample size for a household survey, small sample sizes may require several years of data to be combined; estimates by ethnic group are published using three-year averages

Households below average income

The Households below average income (HBAI) annual report is published on GOV.UK.

HBAI microdata can be accessed under a standard End User Licence on the UK Data Service website. More detailed microdata are available as a Secure Access File to be used in the Safe Room at the University of Essex – additional approval is needed for this level of access.

For more information on HBAI, please contact the Department for Work and Pensions at team.hbai@dwp.gov.uk or on +44 (0) 114 358 6070.

What exactly does it measure?

The households below average income (HBAI) uses data from the Family Resources Survey (FRS) to represent information on living standards in the UK using disposable household income. The data are also equivalised, which considers the size and composition of households to make the income figures comparable1.

The HBAI statistics incorporate widely-used, international standard measures of low income and inequality. There are a range of measures of low income, income inequality and material deprivation, which capture different aspects of changes to living standards. HBAI is generally held to be the foremost source of UK data and information about household net income and poverty.

Annual estimates for several statistics related to the number and percentage of people living in low-income households are provided against several important client groups including children, working-age adults, pensioners, and individuals living in a family where someone is disabled. Results are available for the UK from the period financial year ending 2003, with earlier results for Great Britain from the financial year ending 1995.

Household income is defined as the following.

Income before housing costs (BHC) takes income from all household members including dependants and includes the following main components:

  • usual net earnings from employment

  • profit or loss from self-employment (losses are treated as a negative income)

  • all Social Security benefits and tax credits

  • income from occupational and private pensions

  • investment income

  • maintenance payments, if a person receives them directly

  • income from educational grants and scholarships (including, for students, top-up loans and parental contributions)

  • the cash value of certain forms of income in kind (free school meals, Healthy Start vouchers and free school milk, and free TV licence for those aged 75 years and over)

Income is net of the following items:

  • Income Tax payments

  • National Insurance contributions

  • domestic rates or Council Tax

  • contributions to occupational pension schemes

  • all maintenance and child support payments, which are deducted from the income of the person making the payment

  • parental contributions to students living away from home

  • student loan repayments

Income after housing costs (AHC) is derived by deducting a measure of housing costs from the previously detailed income measure.

Housing costs include the following main components:

  • rent (gross of housing benefit)

  • water rates, community water charges and council water charges

  • mortgage interest payments

  • structural insurance premiums (for owner-occupiers)

  • ground rent and service charges

What sort of information is usually published and what can we extract from the data?

HBAI covers:

  • relative low income

  • absolute low income

  • income inequality, for example, Gini coefficient2

  • household income distributions and averages before and after housing costs

  • income components including income from wages, self-employment and benefits

What are the main uses and strengths?

HBAI is a main source of data and information about household income, income poverty and inequality and is used for the analysis of low income by researchers and the government. Main uses include:

  • informing policy development and monitoring of low income across the population

  • international comparisons within EU and Organisation for Economic Co-operation and Development (OECD) countries (though EU Statistics on Income and Living Conditions (SILC) is considered the principal source for these comparisons)

  • Parliamentary, academic, voluntary sector and lobby group analysis; examples include using the HBAI data to examine income inequality, poverty, the distributional impacts of fiscal policies and understanding the income profiles of vulnerable groups

What are the main limitations?

In common with all income surveys, the HBAI results suffer from several issues of non-sampling error. These include:

  • measurement error – in particular, the FRS (on which HBAI is based) is known to under-report benefit receipt; that said, the FRS is considered to be the best source for looking at benefit and tax credit receipt by characteristics not captured on administrative sources, and for looking at total benefit receipt on a benefit unit or household basis

  • non-response error – the FRS response rate in the recent years has tended to be between 50% and 55%; to correct for differential response rates, estimates are weighted using population totals

  • coverage error – the FRS cover private households in the UK – therefore, individuals in nursing or retirement homes, for example, will not be included

Income dynamics

Income Dynamics are official statistics. They are published annually by the Department for Work and Pensions (DWP) on the GOV.UK website.

Income Dynamics microdata are not publicly available. For more information on Income Dynamics, please email the DWP at teamincome.dynamics@dwp.gov.uk.

What it measures

Income Dynamics are official statistics that supplement the households below average income (HBAI) series, providing information on persistent low income and changes in income over time. These estimates are based on the Understanding Society (USoc) survey; this is a longitudinal survey carried out over a two-calendar-year period, with each individual being interviewed on a yearly basis.

Income Dynamics focuses on the persistence of relative low incomes; an individual is described as being in persistent low income if they are in relative low income in at least three out of four consecutive years. The report also looks at mobility across the income distribution. These statistics have been produced since 2017 in response to the DWP’s statutory obligation to publish a measure of persistent low income for children, under Section 4 of the Welfare Reform and Work Act 2016.

What information is published and what we can extract from the data

The Income Dynamics annual report contains information on persistent low-income trends and the characteristics of individuals in persistent low income, including children, working-age adults and pensioners, both before housing costs (BHC) and after housing costs (AHC).

Data are also published for low-income entry and exit rates, movements between quintiles over time, and where in the income distribution individuals spend the most time.

Supporting tables are published alongside the annual report. These provide further breakdowns on persistent low income and movements within the income distribution.

The income measure used in Income Dynamics is weekly net (disposable) equivalised household income. This comprises total income from all sources of all household members including dependents. Income is adjusted for household size and composition by means of equivalisation scales. Incomes are also adjusted for inflation, corresponding to the middle January of the latest USoc wave. Like the HBAI series, Income Dynamics uses variants of the Consumer Prices Index (CPI) to adjust for inflation and to look at how incomes are changing over time in real terms.

Main uses and strengths

Income Dynamics statistics provide a longitudinal perspective on low incomes and how the duration of low incomes differs for different groups and characteristics. They fill an important gap that emerges if low-income statistics are only considered cross-sectionally, as in the HBAI series.

By looking longitudinally at low incomes, the durations of low incomes for different groups and what characteristics generate persistent low income can be analysed. Longitudinal analysis also allows the calculation of rates of entry into and exits from persistent low income as well as analysis of movements within the income distribution overall.

Users of Income Dynamics statistics include policy and analytical teams within the DWP, the devolved administrations and other government departments, local authorities, Parliament, academics, think tanks, journalists, and the voluntary sector.

Main limitations

As with most longitudinal surveys, attrition reduces the USoc sample size over time. Individuals who do not respond to the survey for all four relevant consecutive waves are excluded from the analysis.

As well as attrition reducing the sample size, there are also missing data for many variables used in the analysis. Records with missing data are excluded from the analysis when the variable with missing values is relevant to the analysis. The most common reason for not having income information is having individual responses but not complete household response information. Other issues include small sample sizes for some groups (for example, some ethnicity categories) and missing interview dates (which means incomes cannot be deflated).

USoc produces weighting factors rather than grossing factors, so estimates of the numbers of individuals in low income are not produced. Instead, percentages of the population are provided.

USoc covers private households only; therefore, individuals not living in private accommodation, such as nursing or retirement homes, are not included. This means that figures relating to pensioners in particular may not be representative of the UK population.

There are known issues with the income information in the first USoc wave covering 2009 to 2010. See Dr Paul Fisher’s paper, Does repeated measurement improve income data quality?, for details of why income data on the first wave of USoc are not comparable with subsequent waves and are likely to be of lower quality. The first wave has been excluded from analysis presented in the latest annual publication for this reason.

Effects of taxes and benefits on household income and Living Costs and Food Survey

Headline estimates from the effects of taxes and benefits on household income (ETB) are published in the new, annual household disposable income and inequality bulletin, designed to provide more timely figures of main indicators. The main effects of taxes and benefits (ETB) statistical bulletin is published several months later, providing further detail on the full impacts of taxes and benefits on the distribution of income. Alongside the statistical bulletins, a report is also published covering further analysis of the results and information on how the data are derived.

Additionally, new experimental “nowcast” estimates are published a few months after the end of the income year, providing an earlier insight into how incomes have changed. The Living Costs and Food Survey (LCF) annual publication Family spending is also available.

ETB and LCF microdata are available at various levels of detail under different user agreements. For more information on accessing these data, please see the UK Data Service website.

For more information about the ETB publication, please contact us at hie@ons.gov.uk or on +44 (0)1633 45 6082.For more information on the LCF, please contact us at social.surveys@ons.gov.uk or on +44 (0)1633 45 5914.

What exactly does it measure?

The effects of taxes and benefits on household income (ETB) is one of our longest standing outputs, having been produced each year since 1961. It is an annual analysis looking at how taxes and benefits affect the income of households in the UK. It provides estimates of household incomes, including the average amount of taxes that households pay and also the value of benefits that they receive.

ETB data are from the Living Costs and Food Survey (LCF), formally known as the Expenditure and Food Survey (EFS), which is a voluntary sample survey of around 5,500 private households in the UK, covering both income and expenditure. Each individual aged 16 years and over in the household visited is asked to keep diary records of daily expenditure for two weeks. Information about regular expenditure, such as rent and mortgage payments, is obtained from a household interview along with retrospective information on certain large, infrequent expenditures such as those on vehicles. Detailed information on income (including cash benefits received from the state) of each adult member of the household is collected through the interview. In addition, personal information such as age, sex and marital status is recorded for each household member. Children aged 7 to 15 years are asked to keep a simplified version of the diary.

One of the unique contributions of ETB is the detailed breakdown of household income it provides, including estimates of both direct and indirect taxes, and both cash benefits and “in kind” benefits provided by the state. The ETB analysis uses five main measures of household income:

  • the starting point of the analysis is original income: this is the annualised income in cash of all members of the household before the deduction of taxes or the addition of any state benefits; it includes income from employment, self-employment, investment income, private pensions and annuities, which include all workplace pensions, individual personal pensions and annuities

  • the next stage of the analysis is to add cash benefits and tax credits to original income to obtain gross income

  • Income Tax, Council Tax and Northern Ireland rates, and employee’s and self-employed National Insurance contributions are then deducted to give disposable income

  • the next step is to deduct indirect taxes (such as Value Added Tax (VAT), and fuel and alcohol duties) to give post-tax income

Finally, the analysis adds benefits that are provided “in kind” to households by government for which there is a reasonable basis for allocation to households to obtain final income; these “in kind” benefits include the provision of education, health services and subsidised travel and housing. The measure of disposable income used in ETB is designed to be consistent with the international standards set out in the Canberra Group Handbook on Household Income Statistics (UNECE, 2011). This results in a small number of differences between ETB disposable income and the before housing costs (BHC) measure in households below average income (HBAI). For example:

  • ETB includes benefits in kind provided by employers (for example, company cars) within income, but these are not included within HBAI

  • HBAI includes certain benefits in kind provided by the state (such as free school meals and Healthy Start vouchers) within BHC income; in ETB, these are included with other benefits in kind as part of final income

What sort of information is usually published and what can we extract from the data?

ETB data are published in both an annual statistical bulletin and a supplementary analysis and methodology paper. The bulletin provides analysis of each stage of the redistribution process from original income through to final income, looking at the impact of taxes and benefits on income inequality. The bulletin and tables also look at:

  • long-term trends in household income for income quintiles deciles, with detailed breakdowns by income component (including individual taxes and benefits)

  • income for quintiles or deciles of retired and non-retired households, again with detailed breakdowns by income component

  • long-term trends in income inequality, measured through the Gini coefficient2 and S80/S203 and P90/P104 ratios

  • average incomes, taxes and benefits by household type, tenure status and region

  • taxes paid (direct and indirect) as a proportion of income and expenditure

  • households receiving more in benefits than paid in taxes

  • characteristics of income quintile or decile groups (including number of adults or children, household type, tenure, age or employment status of chief economic supporter)

The ETB data are also used to produce a range of ad hoc pieces of analysis by ONS and others. Examples of analysis using the ETB data include:

The main LCF publication, Family spending, is primarily concerned with assessing household expenditure. The publication does, however, include a chapter on income, which considers how expenditure varies across the income distribution.

What are the main uses and strengths?

These statistics allow for analysis of the distributional impact of government policy on taxes and benefits. They are the only statistics available that are able to give such a complete picture of the distribution of income including indirect taxes and benefits in kind. The 2009 report by the Commission on the Measurement of Economic Performance and Social Progress by Stiglitz, Sen and Fitoussi identified income distribution, in addition to measures of average income, as an important factor in the measurement of well-being, giving a more complete picture of the standards of living experienced by individuals.

These characteristics of the ETB data also mean that they are the best source for any analysis of household income that includes a breakdown by source or individual taxes or benefits.

ETB data are the primary source used by HM Treasury in their Intra-Governmental Tax and Benefit Microsimulation Model (IGOTM). This is used to model possible tax and benefit changes before policy changes are decided and announced.

As ETB data come from the LCF survey, which is the primary source of household expenditure data in the UK, it is also possible to use these data to carry out joint analysis of income and expenditure, something that is strongly recommended by the Organisation for Economic Co-operation and Development (2013) and many others for better understanding people’s economic well-being in terms of their material living standards.

From 2016, in response to requirements to improve the timeliness of some of the main income series published within the effects of taxes and benefits release, we started publishing a new short bulletin that provides users with main income statistics, along with longer-term trends (going back to 1977), which are an important aspect of the effects of taxes and benefits series. The first edition of this new bulletin, household disposable income and inequality (HDII), was published in February 2016, covering household incomes for the financial year (April to March) ending 2015. The statistics in this release include measures such as median disposable income, inequality measures such as the Gini coefficient2 and breakdowns of the components of disposable income by quintile and decile groups.

Additionally, we are continuing to develop new experimental “nowcast” estimates of the ETB data, with the aim of providing provisional estimates of main indicators far earlier than has previously been possible. An initial methodological paper was published in July 2015, followed by the release of nowcast estimates for the financial year ending 2015 in October that year.

Together, HDII and the experimental nowcast estimates provide the earliest analysis of the household income distribution each year, allowing people insight into the evolution of living standards as early as possible.

Notes for: Household income

  1. The Department for Work and Pensions has published an infographic summarising the process behind estimates of low income in HBAI.

  2. The Gini coefficient is the most widely used summary measure of inequality in the distribution of household income. The lower its value, the more equally household income is distributed.

  3. The income quintile share ratio or the S80/S20 ratio is a measure of the inequality of income distribution. It is calculated as the ratio of total income received by the 20% of the population with the highest income (the top quintile) to that received by the 20% of the population with the lowest income (the bottom quintile).

  4. The income decile ratio or the P90/P10 ratio is a measure of the inequality of income distribution. It is calculated as the ratio of the level of income at the 90th percentile to the level of income at the 10th percentile.

Back to table of contents

6. Other income

The sources covered in this section again refer to income. However, unlike the sources listed in the previous section, income may not be the sole focus of the source or the source may only cover a very specific measure of income.

European Union Statistics on Income and Living Conditions

Indicators based on EU-Statistics on Income and Living Conditions (EU-SILC) for the UK and other EU countries are published on Eurostat’s website, along with more detailed articles and analysis.

EU-SILC microdata are also available to researchers through Eurostat’s User Database (UDB).

For more information, please contact Office for National Statistics (ONS) at hie@ons.gov.uk or on +44 (0)1633 45 6082, or contact the Department for Work and Pensions at team.frs@dwp.gov.uk or on +44 (0)20 7449 7341.

What exactly does it measure?

EU-Statistics on Income and Living Conditions (EU-SILC) is the EU reference source for comparative statistics on income, poverty, social exclusion and living conditions at the European level.

It provides two types of annual data for the 28 European Union countries as well as Iceland, Norway, Switzerland and Turkey:

  • cross-sectional data pertaining to a given time or a certain time period with variables on income, poverty, social exclusion and other living conditions

  • longitudinal data pertaining to individual-level changes over time, observed periodically over a four-year period

EU-SILC is co-ordinated by Eurostat (the European Commission’s statistical agency) under European Commission regulation. In the UK, EU-SILC is collected jointly by the Department for Work and Pensions (DWP) and Office for National Statistics (ONS), while information for other countries is mainly collected by other national statistical institutes (NSIs).

The main topics covered by the EU-SILC microdata are:

  • income
  • poverty
  • material deprivation
  • housing
  • labour
  • education
  • health

What sort of information is usually published and what can we extract from the data?

Indicators and reports based on EU-SILC data are made available through the Eurostat website. Published indicators cover the following areas:

  • relative low income
  • material deprivation
  • low work intensity
  • income inequality (including Gini and S80/20 ratios)
  • housing deprivation, overcrowding and housing cost overburden
  • persistent at-risk-of-poverty (relative low income in current year and at least of two preceding years) labour market and pay transitions

Many of these indicators are broken down by age, sex, employment status, level of education, housing tenure, country of birth, and citizenship.

More detailed analysis of EU-SILC is frequently published by the European Commission in publications such as Employment and Social Developments in Europe.

EU-SILC also includes annual ad hoc modules on important topics of policy interest, allowing more detailed analysis in these areas. Recent ad hoc modules include:

  • subjective well-being
  • housing conditions
  • intergenerational transmission of disadvantages
  • intra-household sharing of resources
  • over-indebtedness and financial exclusion
  • social participation

There are many publications and papers, based on these modules, produced by the European Commission, national statistical offices and academia. For example, the Intergenerational Transmission of disadvantage in the UK and EU, 2014 and Persistent Poverty in the UK and EU, 2014.

What are the main uses and strengths?

A main strength of EU-SILC is the consistency of the variables and concepts across countries, which allows for meaningful comparative analysis across the EU. For this reason, EU-SILC is the main source of data used for policy monitoring and development at a European level. At a national level, this comparability is helpful in allowing researchers to examine what is happening in the UK in comparison with other countries with different welfare regimes and different policy interventions.

EU-SILC is used to measure the European Commission’s Europe 2020 target on the number of people at risk of poverty or social exclusion. This measure combines a number of different dimensions of poverty and social exclusion into a single indicator. According to this definition, people are considered at risk of poverty or social exclusion if they are experiencing at least one of three conditions – having a household income below the poverty threshold, being severely materially deprived, or living in a household with low work intensity. Probably the most important strength of EU-SILC over other official sources of income data is its longitudinal component, which provides crucial information for policy development. In particular, it can help us understand how, when and for how long people move into and out of poverty.

The longitudinal component of EU-SILC allows the measurement of persistent relative low income. It is widely agreed that the impact of long-term poverty on individuals is worse than when poverty is experienced only for a short time (see, for example, Atkinson and others, 2002; Dickerson and Popli, 2011; Jenkins and Van Kerm, 20132). Whilst short episodes can sometimes be dealt with through use of savings, borrowing or reduced consumption, long-lasting poverty is a lot more likely to damage life chances. For this reason, persistent poverty is particularly interesting to policymakers.

Additionally, researchers within and outside government have used EU-SILC to research topics including:

  • the relationship between single year at-risk-of-poverty rates and persistent risk-of-poverty
  • the evolution over time of poverty entry and exit rates
  • the relationship between entering the labour market and moving out of poverty
  • which individuals are most at risk of persistent poverty
  • dynamics of material deprivation
  • flexible employment and poverty
  • disability and low income persistence

What are the main limitations?

As a survey-based source, EU-SILC shares the same limitations as households below average income (HBAI) and effects of taxes and benefits (ETB).

There are also some limitations relating to the international dimension of EU-SILC. EU-SILC is harmonised in terms of variables and concepts but the means of data collection are more loosely specified. Some countries collect data via household surveys, whilst others collect information via administrative sources. Additionally, there may be variation between countries due to translation issues with the questionnaire, or culturally-based differences. The need to specify a framework for classifying income that is consistent across countries means that the benefit income variables made available to researchers are grouped into streams such as “old-age benefits”, rather than classified as the individual benefits themselves1.

Pensioners' incomes series

The Pensioners’ incomes series annual report is published on the GOV.UK website.

Anonymised microdata are made available on the UK Data Service website.

For more information about the pensioners’ incomes series, please contact the Department for Work and Pensions at pensioners-incomes@dwp.gov.uk or on +44 (0)191 2168 950.

What exactly does it measure?

Pensioners’ incomes series (PI) contains estimates of the levels, sources and distribution of pensioners’ incomes. It also examines the position of pensioners within the income distribution of the population as a whole. The series includes information about gross and net income, both before and after housing costs.

The publication is based on two household surveys: the Family Expenditure Survey (FES) and the Family Resources Survey (FRS). The latest information comes solely from the Family Resources Survey.

What sort of information is usually published and what can we extract from the data?

The annual Pensioners’ incomes publication covers the following:

  • income for different groups of pensioners by age, for singles and couples, and broken down by region
  • detailed look at various sources of income, including the proportion of pensioners who receive income from these different sources
  • the distribution of pensioners’ incomes, both within the pensioner population and within the household population overall
  • additional analysis, including couples where one member is above State Pension age and the other below, married and cohabiting couples and results for ethnic groups

The datasets are also deposited with the UK Data Archive post-publication, to allow researchers to conduct their own analysis. These datasets have adjustments made to ensure anonymisation of the data, such as rounding all incomes to the nearest pound and “top-coding” ages at 80 years.

What are the main uses and strengths?

One of the main strengths of pensioners’ incomes is the relatively long time series available for assessing trends going back to the financial year ending 1995 on the FRS – and further with the FES.

What are the main limitations?

As pensioners’ incomes is from the FRS, many of the caveats that apply for the FRS also apply for pensioners’ incomes. For example, the levels of receipt of a number of benefits are under-reported on the FRS, and this will feed into pensioners’ incomes.

Pensioners’ incomes is based on survey data, hence is subject to sampling variation and other forms of error associated with a sample survey. As such, it is often difficult to draw conclusions about significant changes in incomes from one year to the next.

The surveys on which the series are based are household surveys and so people living in institutions – such as nursing homes and communal establishments – are not covered.

When carrying out analysis of pensioners’ incomes it is worth bearing in mind the definitions that apply to the data, and how this affects results and interpretation. For example, analysis is at a household level, and incomes are not equivalised (unlike in HBAI).

Analysis can be carried out on gross or net income, on all pensioner units or by splitting out singles and couples, on a “Before” or “After” housing costs basis, and may use means or medians to summarise results. We advise some thought to ensure analysts are using the most appropriate measures for any work.

Some analysis of pensioners’ incomes looking at small subgroups may need to pool years of data to produce large enough samples from which to draw robust results. Some analysis in the main pensioners’ incomes publication uses three years’ worth of data to achieve this (for example, analysis by region, ethnicity, some quintile analysis).

Survey of Personal Incomes

The Personal incomes statistics release is published on the GOV.UK website.

Survey of Personal Incomes (SPI) Public Use Tape (PUT) microdata are available on the UK Data Service website.

For more information about the Survey of Personal Incomes, please contact HM Revenue and Customs (HMRC) at spi.enquiries@hmrc.gov.uk or on +44 (0)3000 586132.

What exactly does it measure?

The SPI is an administrative dataset based on information held by HMRC on individuals who could be liable to UK tax. It is carried out annually by HMRC and covers income assessable to tax for each tax year. Not all individuals sampled are taxpayers because the operation of personal reliefs and allowances may remove them from liability. Where income exceeds the threshold for operation of Pay As You Earn (PAYE), the survey provides the most comprehensive and accurate official source of data on personal incomes.

The SPI sample is drawn from three separate systems:

  • the National Insurance and PAYE Service (NPS) system covers all employees and occupational pension recipients with a PAYE record; NPS replaced the Computerisation of PAYE (COP) system

  • the Computerised Environment for Self Assessment (CESA) system covers people with self-employment, rental or untaxed investment income and it also covers directors, those subject to higher rate tax and other people with complex tax affairs; where people have both NPS and CESA records, their CESA record is selected because it provides a more complete picture of their taxable income

  • the Claims system covers people without NPS or CESA records who have had too much tax deducted at source and claim a repayment

There were approximately 733,000 valid cases on the tax year 2017 to 2018 SPI dataset.

What sort of information is usually published and what can we extract from the data?

The dataset contains a range of variables about personal incomes arising from employment, self-employment, pension, taxable benefits, property, savings, investments and other income sources. The dataset also contains variables about tax allowances, deductions and reliefs, which people might be due. There is also a regional code variable on the dataset and a trade code for cases that are self-employed.

An anonymised version of the SPI called the PUT is also published and available to the public for download; the UK Data Service currently holds the PUT data for 1985 to 1986 and 1995 to 1996 onwards (data for 2008 to 2009 are currently unavailable). For further details of sampling and coverage criteria, see the accompanying documentation on the UK Data Service website.

The SPI is also used to create HMRC’s personal income statistics publications, which contain a variety of charts and tables illustrating the findings from that year’s SPI including breakdowns of income information by, for example, age, geographical region, sex and self-employment.

What are the main uses and strengths?

The SPI is compiled to provide a quantified evidence base from which to cost proposed changes to tax rates, personal allowances and other tax reliefs for Treasury Ministers. It is used to inform policy decisions within HMRC and HM Treasury, as well as for tax modelling and forecasting purposes. In addition, it is used to provide summary information for the national accounts that are prepared by Office for National Statistics. Finally, it is used to provide information to Members of Parliament, other government departments, companies, organisations and individuals.

The SPI is also used to create HMRC’s aforementioned personal income National Statistics releases. These tables would be of interest to policymakers in government, academics, journalists, “think-tanks” and other research bodies. They would be of use to individuals or organisations interested in the distributions of numbers and amounts of personal incomes, for example, by taxpayer marginal rate or income band.

What are the main limitations?

The SPI is an individual-level survey based on administrative records; as HMRC does not require information on household earnings or hours worked to administer Income Tax, this information is not available from the SPI.

As mentioned in the supporting documentation, not all the individuals in the SPI sample are taxpayers. For tax year 2017 to 2018, about 25% have no Income Tax liability because deductions and reliefs and personal allowances exceed their total income. Where income exceeds the threshold for the operation of PAYE (that is, £11,500 for 2017 to 2018), the SPI provides the most comprehensive and accurate official source of data on personal incomes.

However, as HMRC does not hold information for all people with personal incomes below this level, the SPI is not a representative data source for this part of the population and no attempt has been made to estimate the number of cases below the tax threshold or the amount of their incomes.

Therefore, National Statistics published from the SPI – with the exception of Tables 3.9 and 3.10 – only cover individuals liable to UK Income Tax (taxpayers) and their incomes.

With regards to benefits, the SPI covers only taxable benefits (or benefits in kind). This means that it does not capture benefits like Jobseeker’s Allowance or housing benefits. It therefore cannot provide a complete picture of income.

Some other elements of the SPI have to be imputed or estimated based on other available information and, as with all sample surveys, estimates from the SPI have a sampling error attached to them. These issues are covered in more detail in the supporting documentation published on GOV.UK.

Wealth and Assets Survey

Wealth in Great Britain Wave 5: 2014 to 2016 is the most recent publication of the Wealth and Assets Survey (WAS) analysis, which focuses on income measures.

WAS datasets are accessible only via an End User Licence (EUL) sent to the UKDA and a less anonymised version via the Secure Research Service (formerly the Virtual Microdata Library), which is onsite access to ONS data. This contains data at a very low level, so specific permission has to be granted for access. The EUL dataset is much easier to access as all geographical data have been removed, except region – age has been top coded only for those over the age of 85 years and some detail of non-wealth related variables has been removed.

For more information about the Wealth and Assets Survey, please contact us at wealth.and.assets.survey@ons.gov.uk or on +44 (0)1633 45 6295.

What exactly does it measure?

The Wealth and Assets Survey (WAS) commenced in 2006 with the aim of measuring the economic well-being of households and individuals in Great Britain. It is a biennial, longitudinal survey conducted by Office for National Statistics and funded by a consortium of government departments.

WAS data are fundamental to understanding the distribution of assets, debts, and liabilities in Great Britain and fill gaps in official statistics for which evidence was lacking from either other survey sources or administrative data. It collects information concerning all forms of personal assets held by individuals within private households, including their financial wealth, property wealth, physical wealth and private pension wealth.

During the period 2014 to 2016 – the fifth wave of WAS – over 42,000 individuals were interviewed in more than 18,000 households across Great Britain.

The design of the survey recognises the fact that wealth is highly skewed, with a small proportion of households owning a large share of the wealth. The efficiency of the sample is therefore improved by over-sampling addresses likely to be in the wealthiest 10% of households (established using data from HM Revenue and Customs (HMRC)).

Studies investigating a household’s economic resources have often focused primarily on income. Such estimates of income are frequently used as a main indicator on which governments are held to account, and by which countries are compared. Nevertheless, income provides just one estimate of material well-being. The 2009 report by the Commission on the Measurement of Economic Performance and Social Progress3 highlighted the need to consider wealth in conjunction with income to better assess living standards. This need to look beyond income was an important motive for the introduction of the WAS.

Income data collected by WAS has been extensively quality assured and compares well with data from sources that are focused on income, such as the Family Resource Survey. Whilst the primary focus of WAS is the estimation of household and individual wealth, data on a household’s total income are available for the period 2010 to 20164. The income data are all self-reported, regular income is available as both net and gross annual measures and covers:

  • earned income from employees and self-employed (main and second or other jobs)
  • income from benefits (including State Pensions)
  • income from private pensions (including occupational and personal pensions)
  • income from investments
  • other regular income

Whilst the coverage of income is good, as the data are self-reported, they are not expected to be as accurate as some other sources (for example, employer-based earnings data). It is therefore recommended that income be used as a classificatory variable only (for example, analysed by deciles or quintiles) alongside wealth estimates. In addition, proxy information is taken for some respondents who cannot be contacted, and any missing data are statistically imputed5.

Unlike some other sources the definition of net measures cannot be specifically defined. Respondents are asked, for example, for their net earnings “after all deductions (excluding Tax Credits, bonuses and commissions not received each pay period)”. For some, this will be entirely net of Income Tax and National Insurance, but for many will also be net of any pension contributions, student loan or other employer loans.

In addition, the survey also collects net irregular income such as inheritances and lottery wins.

What sort of information is usually published and what can we extract from the data?

To date, only limited income estimates have been published from WAS – concentrating on the relationship between household wealth and income. However, the scope for analysis is huge.

The main publication for WAS data is Wealth in Great Britain. The main report for Wave 3 did not contain income data, but this was published subsequently, as it was the first time total household income was published from WAS.

Estimates of means are no longer published from WAS. As the distribution of wealth is so highly skewed towards the wealthiest, the median value gives a much better measure of average than the mean (and unlike other financial surveys WAS does not top code any measures of wealth or income).

WAS datasets are accessible only via an End User Licence (EUL) sent to the UKDA and a less anonymised version via the Secure Research Service (formerly the Virtual Microdata Library), which is onsite access to ONS data. This contains data at a very low level, so specific permission has to be granted for access. The EUL dataset is much easier to access as all geographical data have been removed, except region – age has been top coded only for those over the age of 85 years and some detail of non-wealth related variables has been removed.

What are the main uses and strengths?

Income is only one measure of material well-being and alone, might not accurately reflect the full picture. For example, some older people may have relatively low incomes, but have, over their lifetime accumulated a good degree of wealth – for example, property wealth and financial investments. Conversely, some young people may have high incomes but have not had time to accumulate much wealth – for example, have large student loans and mortgages.

WAS allows consideration of factors other than income when considering an individual’s or a household’s economic well-being. This is of growing importance in many areas of government policy that are focusing on wealth rather than income or are concerned with saving for retirement.

In addition to this, the longitudinal design of the survey means that the impact of specific life events (for example, from working life to retirement) can be analysed, as can general life cycle effects.

What are the main limitations?

The focus of WAS is the estimation of household and individual wealth. Income measures are only designed to provide classificatory variables, for example, used to divide the population into income groups to analyse wealth by income grouping.

National accounts estimates of gross disposable household income

National accounts estimates of gross disposable household income are available. For more information, please contact us at saving.ratio@ons.gov.uk or on +44 (0)1633 45 5612 our website.

What exactly does it measure?

Household sector accounts are published by Office for National Statistics (ONS) on quarterly and annual bases: quarterly from Quarter 1 (Jan to Mar) 1995 and annually from 1948. Estimates in the household sector accounts are comparable across time and across countries, as they are produced in line with international standards set out in the European System of Accounts: ESA 2010 and the UN System of National Accounts 2008: SNA 2008. However, the entire back series is not always comparable because of changes in international national accounts standards over this period. As with other components of the national accounts, household sector accounts are compiled using information from a multitude of different sources, including household and business surveys as well as administrative records, to ensure that estimates are as coherent and integrated as possible.

What sort of information is usually published and what can we extract from the data?

Household sector accounts use the following process to derive gross disposable income:

  • total household income equals gross operating surplus plus mixed income plus compensation of employees plus property income plus pension income plus Social Security benefits received (other than pensions) plus miscellaneous transfers and insurance claims received
  • total uses equals taxes paid plus social contributions paid plus property expenditures plus miscellaneous transfers and insurance premiums paid
  • gross disposable household income (GDHI) equals total household income minus total uses

What are the main uses and strengths?

The main use of national accounts estimates of gross disposable household income is within the calculation of gross national income (GNI) and gross domestic product (GDP). Furthermore, they are used to inform decisions made by policymakers in both central and local government as well as frequently being used in academia.

One of the main strengths of using the national accounts estimate of gross disposable income is that it includes the income of institutional households that are often excluded from other survey sources of income.

The national accounts also offer a long time series, with data going back to 1948 (1955 on a quarterly basis). Furthermore, the more recent data are also widely comparable on an international basis as it follows the ESA 2010 and SNA 2008 regulations on national accounts.

What are the main limitations?

Due to compliance with national accounts concepts, such as the inclusion of imputed rental of owner-occupiers as income, estimates may not match specific user requirements.

Regional accounts estimates of gross disposable household income

Regional gross disposable household income (GDHI) data are published annually in the spring. Data are available in Excel datasets or via NOMIS.

For more information about GDHI, please contact us at regionalaccounts@ons.gov.uk or on +44 (0)1633 45 6878.

What exactly does it measure?

Gross disposable household income (GDHI) is the amount of money that all of the individuals in the households sector have available for spending or saving after they have paid direct and indirect taxes and received any direct benefits. GDHI is a concept that is seen to reflect the “material welfare” of the households sector.

The households sector comprises all individuals in an economy, that is, people living in traditional households as well as those living in institutions such as retirement homes and prisons. It should be noted that regional GDHI relates to all individuals within the households sector for a region and not to an actual average household or family unit. The households sector also includes sole trader enterprises (the self-employed) but now excludes non-profit institutions serving households (NPISH), for example, charities and most universities.

GDHI estimates in this bulletin are available at four geographical levels, in accordance with the Nomenclature of Units for Territorial Statistics (NUTS) classification, which came into force on 1 January 2018. NUTS provides a single uniform breakdown for the production of regional statistics for the EU:

  • NUTS1: Wales, Scotland, Northern Ireland and the nine English regions, can be collectively referred to as regions
  • NUTS2: 41 regions – mainly groups of counties and unitary authorities; can be referred to as subregions
  • NUTS3: 179 regions – principally individual counties and unitary authorities; also known as local areas
  • LAU1 – 400 local authority or local council areas

A “top-down” approach is used for the production of regional GDHI estimates, whereby the national aggregate, consistent with the UK National Accounts (the Blue Book), is allocated to regions using a regional indicator dataset. Numerous regional indicator datasets are used in the production of regional GDHI, comprising survey and administrative data, which conform as far as possible to Eurostat standards.

Local authority data for the UK are available alongside the NUTS1, NUTS2 and NUTS3 estimates of GDHI. These local authority estimates can be aggregated to form other geographic breakdowns such as combined authorities and local enterprise partnerships (LEPs), which are available to download. These data are also available on the Nomis website.

More information can be found on the regional accounts methodology web page.

What sort of information is usually published and what can we extract from the data?

For domestic publication, regional GDHI estimates are published annually for the period 1997 to T-2 years (T being the year of publication) and are consistent with the previous year’s UK National Accounts (the Blue Book).

Total GDHI and the components of income are published in pounds million, pounds per head and per head index at all four geographical levels. Estimates are produced at current prices, which means the effect of inflation has not been removed.

The components of income are as follows.

Allocation of primary income account:

  • resources: operating surplus, mixed income, compensation of employees and property income received
  • uses: property income paid

Balance of primary income equals primary resources minus primary uses.

Secondary distribution of income account:

  • resources: social benefits and other current transfers received
  • uses: current taxes on income and wealth, social contributions and other current transfers paid

Balance of secondary income equals secondary resources minus secondary uses.

The balancing items of these two accounts then allow us to calculate GDHI as:

  • GDHI equals balance of primary income plus balance of secondary income

The production of regional GDHI at the NUTS3 level is a legal requirement under European Commission (EC) legislation. The methods and data used are consistent with the guidance set out in the European System of Accounts 2010: ESA 2010.

What are the main uses and strengths?

As with the national estimates, one of the main strengths of using a national accounts estimate of GDHI is that it includes the income of institutional households.

Previous estimates of GDHI were produced for the combined households and NPISH sectors. In the UK National Accounts, estimates for the households and NPISH sectors were presented separately for the first time in Blue Book 2017. In line with Blue Book 2017, our regional estimates for GDHI have been produced for the households sector only.

Unlike the national estimates, regional GDHI is available at a detailed geographic breakdown. Estimates are published at NUTS1, NUTS2, NUTS3 and local authority levels allowing for comparison across UK and EU areas. Regional GDHI is considered valuable as a measure of relative wealth between regions.

Regional GDHI estimates provide an overview of economic diversity and social welfare at regional, subregional and local area levels. These estimates are used by the UK government and the devolved administrations of Northern Ireland, Scotland and Wales to formulate and monitor economic policy and allocate resources.

The European Union (EU) uses these estimates to inform regional policy and analysis, and to identify disparities in regional welfare across the member states.

A consistent time series is available back to 1997.

What are the main limitations?

Regional GDHI estimates are produced in current prices (which include the effects of inflation). Constant price estimates of regional GDHI cannot be produced due to the lack of appropriate regional deflators.

Regional GDHI does not take account of mortgage capital repayments or payments of rent. In national accounts terms, the former is counted in the capital account, while the latter is considered final consumption expenditure. Regional GDHI gives no information on the spending patterns of households. However, regional accounts are currently looking to develop a measure of household final consumption expenditure for regions.

Small area income estimates

The latest Small area income estimates are available. For more information about these estimates, please contact hie@ons.gov.uk.

What exactly does it measure?

The estimates provide the average household income for small areas within England and Wales. It is the income a household receives from wages and salaries, self-employment, benefits, pensions, plus any other source of income. These estimates are produced at the Middle layer Super Output Area (MSOA).

The technique used to produce small area income estimates is a multiple linear regression modelling technique with synthetic estimation. This technique is used when survey data alone are insufficient to produce accurate estimates of income in smaller geographical areas. At the level of these areas, sample survey sources often do not have sufficient sample size and it is therefore necessary to combine survey data with other data sources known as auxiliary data or covariates. The auxiliary data, which can be obtained from an administrative system or a previous census, can provide data on a small area basis for all areas within the target population.

To produce small area income estimates, it is necessary to model the area-level relationship between the survey variables and auxiliary variables. The survey data can be used to estimate the mean level of income for a particular type of individual within the larger population, while other data sources can be used to estimate the proportion of a particular type of individual living within the small area, for example, the proportion of the population claiming Income Support. The relationship between the two can then be fitted to derive an estimate of weekly household income within a small area. The fitted model produces an estimate that describes the relationship between the area-level values of the target survey variable and the auxiliary variable.

This modelling methodology enables survey data from the Family Resources Survey to be combined with census and administrative data to improve the quality of estimates at the small area level. As the estimates are model-based they are different to standard direct estimates.

What sort of information is usually published and what can we extract from the data?

The small area income estimates are produced for four different income types:

  • average weekly household total income (unequivalised)
  • average weekly household net income (unequivalised)
  • average weekly household net income before housing costs (equivalised)
  • average weekly household net income after housing costs (equivalised)

What are the main uses and strengths?

The main strength of small area income estimates is the low level of geography that the data are provided at. There is currently no other source of income data that can produce estimates at such a low geographical level. As such, the estimates are often used to answer questions from the public, academics and Parliament that require detailed information on income at low levels of geography.

What are the main limitations?

A limitation of the small area income estimates is the measure of central tendency produced by the models, which is the mean household income. This measure does not provide an indication of the distribution of income within or between areas, which is a limitation because some areas have a more skewed distribution (with a small number of households having very high income) than others. We are working on a new method of modelling MSOA level income, which can produce an estimate of the median household income and therefore an understanding of the distribution of income within MSOAs.

Estimates have been produced for four different types of income. In some areas, slight inconsistencies may occur between the income types for a particular MSOA, for example, a MSOA may have a larger estimate for net income when compared with total income. Although there may be some such inconsistencies, the models selected are the best possible to describe the general pattern of income over all MSOAs. This reinforces the need to look at the confidence interval for income estimates, not just the point estimate, since the confidence intervals summarise the variability in the estimates caused by the modelling process.

These estimates do not enable direct comparisons of income over time. Each time a new set of estimates is produced, a different model is used, which means any two sets of estimates do not form a time series as such. However, the 95% confidence intervals can be used to make indicative comparisons of income for an area over time. If the 95% confidence intervals for a given MSOA do not overlap for two different years’ estimates, then there is reasonable certainty that household income in that MSOA has changed.

The MSOA level estimates can be aggregated to provide income estimates for larger geographical areas such as local authorities or regions. However, this method is approximate and so it is not possible to assess the precision of the aggregated estimates.

Notes for: Other income

  1. Though in most countries, including the UK, the data are collected at the level of the individual benefit.

  2. For more information, see Atkinson T, Cantillon B, Marlier E and Nolan N (2002), ‘Social Indicators: The EU and Social Inclusion’, Oxford: Oxford University Press; Dickerson A and Popli G (2011), ‘Persistent poverty and children’s cognitive development: Evidence from the Millennium Cohort Study’, Working Paper, Department for Economics, University of Sheffield; and Jenkins SP and Van Kerm P (2013), ‘The relationship between EU indicators of persistent and current poverty’, CASEpapers, CASE/169, London School of Economics.

  3. For more information, see the report by the Commission on the Measurement of Economic Performance and Social Progress, by Professors Joseph E Stiglitz, Amartya Sen and Jean-Paul Fitoussi, published on 14 September 2009. In particular, WAS supports two of the recommendations: Recommendation 3: Consider income and consumption jointly with wealth; and Recommendation 4: Give more prominence to the distribution of income, consumption and wealth. Both recommendations underpin how UK, EU and OECD are shaping up their requirements on the wider measurement of economic performance, social progress, the environment and sustainability.

  4. In the first two waves of the survey, the only income data that were successfully collected related to earned income – so excluded income from benefits, private pensions and other income such as that from investments and property rentals.

  5. For more information, see the Technical chapter of the Wealth in Great Britain Report.

Back to table of contents

7. Upcoming income and earnings publications

A spreadsheet of upcoming publications relating to income and earnings statistics (xlsx, 23.9 kB), from a number of government departments, is available. The publications are from:

  • the ONS
  • HMRC
  • Department for Work and Pensions (DWP)
  • Welsh Government
  • Scottish Government
  • Northern Ireland Statistics and Research Agency (NISRA)

This document is updated on a regular basis.

Back to table of contents

Contact details for this Methodology

Katie Healey
Telephone: +44 (0) 1633 456120