This guide aims to outline the different data sources and outputs that feed into the analysis of earnings and income within the UK.
It would be pertinent first to outline the different stages of earnings and income and to explain how these various stages can be calculated and how they fit together.
The best place to start is with earnings. In this article earnings refer to that remuneration received by employees in return for employment. Most analyses of earnings consider only gross earnings, which is earnings before any deductions are made in light of taxes (including National Insurance Contributions (NICs)) and benefits.
From gross earnings we can also look at 2 similar measures. Firstly, take-home pay refers to the earnings received by an employee after they have paid tax on their earned income, typically referred to as “Income Tax” and after their employee National Insurance contributions have been deducted. This is typically what is paid into an employee’s bank account. Measuring take-home pay can be particularly challenging and so some of the estimates and measures outlined in this guide may not adhere precisely to the definition above. For instance, the deduction of student loan repayments may or may not be included depending on which source is being considered.
Alternatively, from the point of view of the employer, we can also consider labour costs. These typically refer to the gross earnings paid by a business (wage costs), plus a number of non-wage related costs such as employer National Insurance contributions, pension contributions and benefits in kind paid by the employer.
Returning back to the starting point of gross earnings we can then consider how earnings feed into income.
Combining employee earnings with those of the self-employed, along with private pensions and other sources such as income from investments, gives original income. Then adding cash benefits to original income, such as the state pension, child benefit or Jobseekers’ Allowance, gives gross income.
From gross income we then consider deductions. Firstly subtracting direct taxes (for example, Income Tax), employee National Insurance contributions and Council Tax or Northern Ireland Rates, leaves disposable or net income. Then further deductions of indirect taxes (where the tax is typically levied on one entity but paid by another) like VAT and duties on alcohol or tobacco. result in what is referred to as post-tax income.
Finally we add in benefits in kind paid by the state such as health and education which are allocated on the basis of household characteristics. This leaves us with final income.
Earnings and income data are often collected and analysed at different levels. Typically earnings data concern individuals. This is because it is relatively easy to attribute earnings from employment to each employee. On the other hand, income data is often considered at both the individual and the household level. This is because many forms of government intervention, particularly cash benefits, are often assessed at the benefit unit1 level and are therefore more difficult to attribute to individual household members. Income statistics that are considered at the household level are also often adjusted or “equivalised” to account for the fact that different sized households require and generate different levels of income. Equivalisation then allows for comparisons to be made across households with differing compositions (for example, comparing a single-person household with a household containing a family of 4).
The stages of earnings and income are summarised in Figure 1.
The Office for National Statistics (ONS) and a number of government departments are responsible for collecting data and publishing analysis on all these stages of earnings and income, often with more than one survey or publication covering any given stage. This guide will go on to outline many of the different sources available at the different stages, pointing out what they measure, their weaknesses and their strengths. If you use earnings and income data this guide makes it easier to select the most appropriate data source for your analytical needs.
Notes for Introduction:
- A household may consist of one person living alone or a group of people (not necessarily related) who live at the same address that share cooking facilities and share a living room, sitting room, or dining area. A household will consist of one or more benefit units/families. A family or benefit unit consists of a single adult or a couple living as married and any dependent children
The sources in this section collect and/or publish data on gross earnings which is that paid to an employee in return for employment before any deductions are made. This section also covers sources of information on take-home pay. It should be noted that measures of take-home pay often differ conceptually between different data sources.
2.1 Average weekly earnings
Average weekly earnings can be found in our monthly labour market statistical bulletin, under section 9 and also in the datasets EARN01 to EARN03.
Average weekly earnings microdata are not publically available.
+44 (0)1633 456724
2.1.1 What exactly does it measure?
The Average Weekly Earnings (AWE) measure is our lead indicator of short-term changes in earnings. It replaced the Average Earnings Index (AEI) in 2010.
AWE is calculated from returns to the Monthly Wages and Salaries Survey (MWSS), a survey of 9,000 businesses covering 13.8 million employees. The MWSS captures information about each company’s total wage bill and the number of people paid in the reference period. Having been weighted to the Great Britain level, the total wage bill is then divided by the number of employees to give average weekly earnings.
AWE also reflects changes to the industrial composition of the workforce. For instance, all other things being equal, an increase in the relative number of employees in highly paid industries will cause average earnings to rise. This compositional effect, known as the employment contribution, was not captured by AWE’s predecessor, the AEI. We publish separate estimates of the wage and employment contributions to AWE growth in supplementary tables called the AWE decomposition.
2.1.2 What sort of information is usually published and what can we extract from the data?
AWE is published on a monthly basis as part of the labour market statistical bulletin. The section on AWE typically covers:
- the levels of average weekly earnings broken down into:
- regular pay (excluding bonuses)
- total pay (including bonuses)
- bonus pay
- average weekly earnings by sector (public and private)
- average weekly earnings by industry
- real average weekly earnings by sector (AWE adjusted for inflation)
- wage and employment contributions
Average weekly earnings by sector are available both before and after seasonal adjustment. Seasonal adjustment is widely used in official statistics as a technique to interpret time series data which would otherwise be distorted by seasonal factors that don’t reflect underlying trends.
Additionally, supplementary analysis is published alongside the main results. The supplementary analyses includes real average weekly earnings (AWE adjusted for inflation using the Consumer Prices Index (CPI) as a deflator) and information on wage and employment contributions. The employment contribution changes if the relative proportion of employment in the 24 industrial headings changes, but will not necessarily change if total employment increases.
2.1.3 What are the main limitations?
The MWSS is a survey of employers and as such does not cover the self-employed. Nor does it cover HM Armed Forces or government-supported trainees. Furthermore it does not collect any information on individual employees’ characteristics and as such does not allow analysis beyond sector and industry.
The MWSS also excludes businesses with fewer than 20 employees in order to limit costs and respondent burden. Employment figures for these businesses are taken from the Inter-Departmental Business Register (IDBR) while earnings are estimated using a factor from the Annual Survey of Hours and Earnings (ASHE), which does cover small businesses.
The composition effect captured by the AWE refers only to changes between industries. Therefore it does not capture compositional changes between occupation, age or changes within the same industry.
It should also be highlighted that AWE does not differentiate between full-time and part-time workers, and so a relative increase in the prevalence of part-time working would indicate that average weekly pay was falling whereas average hourly pay may remain the same.
2.1.4. What are the main uses and strengths?
A main strength of using AWE is its frequency and timeliness with results being produced on a monthly basis, usually with a 6 to 7 week gap between the end of the reference period and the publication date. This timeliness and frequency is why the AWE is our lead indicator of short-term changes in earnings. It is the AWE measure of average earnings that is most often compared with measures of inflation to calculate changes in real earnings.
Another strength of AWE is its ability to capture bonus payments. With the MWSS being conducted monthly it captures bonus payments in every month of the year, with March often being the main month in which bonuses are paid. An article “Average Weekly Earnings – Bonus Payments in Great Britain” is published on our once a year.
Given its strength in capturing bonus payments, AWE is also used to supplement ASHE data in our estimates of public and private sector pay differentials. ASHE data is adjusted at an industry level, in line with the AWE measures of bonuses. This is crucial given bonus payments are one of the main differences in remuneration between the public and private sectors.
2.2 Annual Survey of Hours and Earnings
The latest Annual Survey of Hours and Earnings publication can be found on our website.
Due to the sensitive nature of ASHE microdata they are not available for download from the UK Data Service website but may be accessed through Secure Access arrangements. Access requires accreditation by the UK Statistics Authority as an Approved Researcher, completion of face-to-face training and agreement to the Secure Access's User Agreement and Breaches Penalties Policy. For more information on access to ASHE microdata please see the UK Data Service website.
+44 (0)1633 456724
2.2.1 What exactly does it measure?
The Annual Survey of Hours and Earnings (ASHE) provides information about the levels, distribution and make-up of earnings and hours paid for employees in all industries and occupations across the UK. ASHE and the New Earnings Survey (NES), which preceded it, have been collected every year since 1970 allowing for comparison over time. It should be noted that weighted data are available from 1997 onwards, prior to 1997 data are unweighted. Furthermore data prior to 1997 refers only to Great Britain; data from 1997 onwards covers the UK as a whole.
Businesses are surveyed in April of each year and are asked to provide information on employees who fall within a 1% sample of the HM Revenue and Customs (HMRC) Pay-As-You-Earn (PAYE) Register taken in January of the same year (with follow-up surveys to capture those employees who change jobs or join the labour market between January and April). The final ASHE dataset typically covers around 180,000 jobs from around 60,000 responding businesses.
2.2.2 What sort of information is usually published and what can we extract from the data?
The main ASHE publication is, as its name suggests, annual, with the survey relating to April and publication taking place in the following October or November.
The main publication focuses on full-time employees on adult rates of pay whose pay in the reference period was unaffected by absence. It covers:
- hourly, weekly and annual earnings
- distribution of earnings
- earnings trends (that is, changes over time)
- pay differences between sexes
- public sector versus private sector
- earnings by:
- the components of earnings (overtime, incentive pay)
- estimates of the number of low paid workers
While the main release focuses on full-time employees, the data covers part-time employees too, such that they can be analysed separately or incorporated with full-time employees. Furthermore, where appropriate, analysis can also include those whose pay was affected by absence and/or those who were not on adult rates of pay.
Because the sample upon which ASHE is based doesn’t change from year to year (except for new entries or retirees) ASHE datasets are combined to create a panel dataset. This dataset observes the same employees in each year that they are in employment and their employer responded to the survey.
The panel dataset covers most of the same variables as the annual datasets, allowing for the analysis of hours and earnings for different cohorts of employees. For instance, we made use of the panel dataset in its analysis of Earnings in the UK Over the Last Four Decades, by tracking those who were 21 in 1975, 1985 and 1995 up to 2013 to compare the first 18 years of the careers of each cohort (from the age of 21).
2.2.3 What are the main limitations?
The most obvious limitation of ASHE is that it is not as timely as other measures of earnings. Whereas AWE has a 6 to 7 week lag, the lag from the ASHE reference period to publication is usually 6 to 7 months.
Like AWE, ASHE only covers employees and therefore excludes the self-employed. The selection of personal characteristics is also more limited than other sources, such as the Labour Force Survey.
A further limitation is that there is no information on what individuals are doing when they are not present on the survey. Potentially they could be unemployed; have switched to self-employment; have exited the labour market for a period or their employer may not have responded to the survey.
There are known coverage issues with data on bonus and incentive payments relating to the reference period. Primarily, this is because the information is not available to respondents at the time when they are required to provide the information to us. Data on annual bonus payments are thought to be better in this respect, though some respondents still do not have access to the necessary information at the time when they complete the questionnaires.
Since 1970, there have also been a number of changes to coverage, methodology, and classification conventions for the survey. These result in a series of discontinuities where the data may not strictly be comparable between any given pair of years where a discontinuity exists. However, it is thought that when comparing over a longer time series that the discontinuities have a minimal impact on longer-term trends. Since 2000, discontinuities in the series exist between 2003 to 2004, 2005 to 2006 and 2010 to 2011. For each of these discontinuities we have produced 2 versions of results, enabling valid comparisons with series on either side of the discontinuity.
2.2.4 What are the main uses and strengths?
The main use of ASHE is to analyse the distribution of earnings in the UK. While this is also possible using the Labour Force Survey, ASHE is thought to be a more accurate source of information on earnings as the information is provided by employers rather than being self-reported by employees.
ASHE also benefits from a large sample size which, coupled with the array of individual and geographic characteristics that are also gathered, allows for more detailed analysis than other sources of earnings data.
ASHE is the principal source of data used in our estimates of the public and private sector pay differential and the gender pay gap. ASHE is also the principal data source used for estimating the number of employee jobs being paid below the National Minimum Wage and National Living Wage.
ASHE also benefits from sampling the same employees over time which gives it a longitudinal aspect which isn’t present to the same extent on other surveys of earnings.
2.2.5 Estimated take-home pay and labour costs data
+44 (0)2072 153945
It is possible to make estimates of individual “take-home pay” and the “labour costs“ associated with employees at different points of the wage distribution using gross wage information from the Annual Survey of Hours and Earnings.
This analysis defines “take-home pay” as gross wages with employees’ National Insurance Contributions (NICs) and Income Tax deducted, and “labour costs” as gross wages plus employer NICs. This is consistent with the Organisation for Economic Co-operation and Development (OECD)’s methodology employed in its “Taxing Wages” publications. To create these variables, Income Tax and NICs (employee and employer) liabilities for earnings at the average or at certain points of the wage distribution are calculated based on the relevant tax rates and thresholds.
This form of analysis does not typically take into account pensions contributions, or any other salary deductions, such as student loan repayments. As such, the analysis assumes that NICs are calculated based on being contracted into the second state pension. However, once ASHE pension data are made available (typically in February or March), a variable is available which indicates whether an individual is contracted into NICs allowing NICs rates to be modelled explicitly. In addition, the analysis does not include the self-employed, nor does it include income from the benefits system.
Showing the impact of Income Tax and employee NICs on individual take-home pay is a better reflection of the individual gains to work than earnings alone. Furthermore, impacts of all government tax and benefit policies depend on each household’s composition and situation and so it is difficult to relate this to the earnings of individual employees.
As well as in the OECD’s “Taxing Wages” publications, analysis of this nature has also been published in the past by the Department for Business, Innovation and Skills1. It is also considered as part of the government’s evidence to the Low Pay Commission.
2.3 Labour Force Survey
Labour Force Survey (LFS) earnings information is published in the monthly labour market statistical bulletin on our website in datasets EARN04 to EARN08. These datasets are updated on a quarterly basis.
LFS microdata can be accessed through the UK Data Service website under a standard End User Licence. More detailed microdata are available under a Special Licence, though users must be Approved Researchers to attain this special licence.
+44 (0)1633 455277
2.3.1 What exactly does it measure?
The Labour Force Survey (LFS) is a rolling survey of households which has taken place on a quarterly basis since 1992. It is a main input into the monthly publication of labour market statistics, forming the basis for the rate or level of both employment and unemployment. Each sample is made up of approximately 40,000 responding households. Each household selected remains in the survey for 5 quarters with information on earnings requested only in Quarters 1 and 5, thus enabling year-on year comparisons. The earnings questions were not part of the initial variable set in the LFS, with earnings questions in respect of weekly pay being added in winter 1992 and hourly pay in autumn 1993. Until 1997, earnings questions were asked only in Quarter 5; the change to asking in Quarters 1 and 5 reduced sampling errors by approximately 30%. The current range of earnings questions dates from 1999.
The primary purpose of the LFS is “the prompt publication of main aggregate, whole economy, indicators, for the integrated assessment of labour market conditions”. The “labour market” covers all aspects of people’s work, including the education and training needed to equip them for work, the jobs themselves, job-search for those out of work, and income from work and benefits.
As such the LFS collects information on a wide range of characteristics including both gross earnings (hourly and weekly) and take-home pay after deductions.
2.3.2 What sort of information is usually published and what can we extract from the data?
Earnings data from the LFS are published quarterly and typically cover:
- gross weekly earnings of full-time employees by:
- the distribution of gross hourly earnings of employees
LFS microdata also include information on usual pay and why this may have differed from pay in the reference week. The LFS also captures respondents’ take-home pay after “all deductions”. It is important to note that this differs conceptually to the measure of take-home pay that can be estimated from ASHE (see Section 2.2.5) as it may include deductions such as pension contributions and student loan repayments.
The microdata also cover a wide range of individual characteristics including but not limited to:
- employment status
- full-time or part-time
- public or private
- permanent or temporary
The full catalogue of LFS variables can be found in the LFS User Guides – Details of LFS Variables.
2.3.3 What are the main limitations?
The data on individuals’ earnings captured by the LFS are thought to be of a lower quality than ASHE as LFS information is self-reported through computer-assisted or telephone interview. ASHE and AWE, however, gather information from the employer which is thought to be more accurate as employers can consult payroll records. Individuals may not have such records to hand and their responses may therefore be subject to higher levels of recall error. Furthermore, LFS responses can be given by proxy (by other individuals in the same household) when an individual is unavailable for interview. This gives further scope for recall error from respondents. Due to this recall error, our estimates of earnings based on the LFS typically exclude those who earn more than £100 per hour as a quality assurance measure. These factors combined mean that gross weekly and hourly pay are known to be underestimated on the LFS.
Earnings information on the LFS is only captured for employees and so, like AWE and ASHE, it excludes the self-employed (though other information is captured for the self-employed).
The LFS has also suffered from falling response rates over recent years. In 1999 the Wave 1 (that is, the first quarter that a household is included in the LFS sample) response rates were around 80%, whereas a steady decline over the intervening period meant that in 2014 they were approximately 60%.
In addition to the falling response rates there is a large percentage of non-response to the specific earnings questions. Typically only around 10% of respondents answer these questions. This impacts on the robustness of the data, especially when applying additional industry or occupation filters to the data.
A further limitation of the LFS from an analytical perspective is the repeated changes to some classifications (for example, ethnicity) and year-specific variable names. Extensive recoding may be required where comparing some of these variables or characteristics over time.
2.3.4 What are the main uses and strengths?
The principal strength of using the LFS measures of earnings is the rich selection of classificatory variables that are gathered on the survey. While ASHE is often favoured for its accuracy, there are many individual characteristics that are only gathered on the LFS. For instance, information on education, ethnicity, nationality, disability and many other variables are captured on the LFS but not on ASHE. Analysing how earnings vary with any of these characteristics is typically done using LFS data.
The LFS also has a longitudinal element to its data as individuals’ earnings data are captured in both Waves 1 and 5 of the survey. This allows analysis of changes to individuals’ earnings over the course of the year.
The LFS is also used widely in academia due to the ease of access offered by the UK Data Archive. By contrast, ASHE microdata are only accessible via secure microdata labs.
Notes for Wages and earnings:
- See for instance, BIS ad hoc analysis on ‘take-home pay’ increases between 2012 and 2013.
Labour costs typically refer to the gross earnings paid by a business (wage costs), plus a number of non-wage-related costs such as employer national insurance contributions, employer pension contributions and benefits in kind provided by the employer.
3.1 Index of Labour Costs per Hour
Index of Labour Costs per Hour (ILCH) is published on a quarterly basis and can be found on our website.
ILCH microdata are not publically available.
firstname.lastname@example.org +44 (0)1633 456724
3.1.1 What exactly does it measure?
The Index of Labour Costs per Hour (ILCH) is a measure of the cost of having an employee for an hour of work. ILCH was first published in 2005 and is published quarterly. It reflects changes in wages and salaries, non-wage costs and the quantity of hours worked over the quarter and is important for monitoring inflationary pressures in the labour market.
ILCH is primarily based on the Monthly Wages and Salaries Survey (MWSS) and the Labour Force Survey (LFS). The MWSS sample is drawn from the Inter-Departmental Business Register (IDBR) which is also used to weight the data. The LFS provides estimates of total hours worked based on average total hours worked in first and second jobs by employees. The Annual Survey of Hours and Earnings (ASHE) provides estimates of pay for employees of small businesses and the data are also used to produce factors to estimate Northern Ireland’s wages and salaries. The Labour Costs Survey (LCS) and the Annual Business Survey (ABS) are also inputs to ILCH, providing estimates of non-wage labour costs.
The ILCH index goes beyond other earnings indicators to include non-wage costs (sickness, maternity and paternity costs, pensions contributions, benefits in kind and National Insurance contributions), as well as the wages and salaries component. ILCH is currently published as an experimental statistic. Further development work is required before ILCH is submitted for assessment as a National Statistic by the UK Statistics Authority.
3.1.2 What sort of information is usually published and what can we extract from the data?
The Index of Labour Costs per Hour is published on a quarterly basis. The main publication of ILCH provides both index numbers and growth rates and covers:
- the most recent labour costs data broken down by:
- wage costs
- non-wage costs
- labour costs excluding bonuses and arrears
- ILCH by industry (Standard Industry Classification 2007 (SIC07) A to S)
- ILCH by sector (including public and private)
- ILCH before and after seasonal adjustment
3.1.3 What are the main limitations?
The main limitation of ILCH is that only index numbers are published quarterly with no monetary values, though Eurostat derive annual data on hourly labour costs, which are published on the Eurostat website . This then also means that ILCH cannot be used to investigate the distribution of labour costs.
Furthermore ILCH can be volatile when observing low-level industry aggregates where individual firms can have a larger impact on the data.
Given that the MWSS is a survey of employers ILCH refers only to employees and therefore excludes the self-employed.
3.1.4 What are the main uses and strengths?
The production of ILCH is driven by European legislation which requires that a harmonised index of labour costs be produced by all member states. It is the only short-term earnings per hour indicator in the UK and can be used as an early gauge of economic performance. This is because a business will typically alter the number of regular or overtime hours its employees work before considering changing the number of people it employs.
3.2 Unit labour costs
Unit labour costs are published as part of the quarterly labour productivity statistical bulletin on our website.
+44 (0)1633 455981
What exactly does it measure?
Unit labour costs (ULCs) measure the cost of labour input per unit of real (inflation-adjusted) economic output. This is calculated by dividing total labour costs by output. With gross value added (GVA) used as the measure for real output, total labour costs can be broken into the compensation of employees (CoE) and the labour share of mixed income. The CoE measures the aggregate labour costs of employees and mixed income measures the income of the self-employed, which conceptually includes returns to labour and to capital employed.
As such, ULCs reflect the full labour costs, including social security and employers’ pension contributions, incurred in the production of a unit of economic output. This methodology is consistent with our other data on labour inputs and the returns of labour.
3.2.2 What sort of information is usually published and what can we extract from the data?
Estimates of ULCs for the whole economy are published quarterly in the labour productivity bulletin.
Within this dataset are sectional ULCs, broken down by industry, in both index levels and percentage changes on the previous quarter and on the previous year. The ULCs are provided in both seasonally adjusted and non- seasonally adjusted terms. The seasonally adjusted figures are presented for the whole economy, market sector, manufacturing, production and total services industry classifications. In addition to these categories, the non-seasonally adjusted data are disaggregated further into:
- agriculture, forestry and fishing
- mining and quarrying
- wholesale and retail trade
- transportation and storage
- accommodation and food
- information and communication
- financial and insurance industries
- real estate activities
- professional, scientific and technical activities
- administrative and support service activities
- public administration and defence
- education, health and social work
- arts and entertainment
- other services
3.2.3 What are the main limitations?
ULC measures deal exclusively with the cost of labour, which though important, should also be considered in relation to changes in the cost of capital. This consideration is more relevant in capital-intensive economies.
One of the difficulties with compiling an accurate measure of the costs associated with labour input within the UK is that there is no direct measure of labour costs in the self-employed sector. However, by using a factor share of mixed income derived from national accounts estimates of mixed income, we obtain an implied return to self-employed labour, which is conceptually preferable to the measure used previously as it incorporates information on the income of the self-employed.
One of the main limitations previously was that estimates were not available below the whole economy level, thus manufacturing unit wage costs (UWCs) were used. This is no longer the case, as manufacturing ULCs, alongside other sectional ULCs, are available as part of the labour productivity bulletin. UWCs are limited in 2 ways. Firstly, wage costs are a narrower measure than CoE, which includes non-wage labour costs, including social security and employers’ pension contributions. Secondly, the UWC methodology assumes that wage costs of the self-employed were equal to those of employees. This was despite evidence from the national accounts that such an assumption is inconsistent with the income attributed to the self-employed.
3.2.4 What are the main uses and strengths?
Since labour costs account for around two-thirds of the cost of production of UK economic output, unit labour costs provide an indication of inflationary pressures in the economy. ULCs are used by the Bank of England among others when analysing the extent of spare capacity in the labour market; as they provide a comprehensive indicator of the inflationary pressure in the supply side of the economy. This is especially true when non-wage labour costs move differently to wages.
Although not a direct measure of productivity, an inverse relationship between unit costs and productivity tends to be observed. Movements in ULCs can be decomposed into movements in costs per unit of labour (which can be approximated by an index of earnings) minus the movement in labour productivity. Thus other things equal, increases in labour productivity will tend to reduce ULCs.
ULCs also have the benefit of being fully consistent with both the output and income presentations of the national accounts.Back to table of contents
The surveys and publications in this section are the principal sources of information on household income. They cover some or all of the stages identified in the introduction between original and final income, though they may differ slightly in how certain deductions and additions are made at the different stages of income redistribution.
4.1 Family Resources Survey
The Family Resources Survey annual report is published on the GOV.UK website alongside information for respondents and additional papers on measuring uncertainty and grossing methodology.
Family Resources Survey microdata can be accessed under a standard End User Licence on the UK Data Service website. More detailed microdata are available as a Secure Access File to be used in the Safe Room at the University of Essex – additional approval is needed for this level of access.
Department for Work and Pensions
+44 (0)20 74497341
4.1.1 What exactly does it measure?
The Family Resources Survey (FRS) collects information on the incomes and circumstances of private households in the UK. It is designated as a National Statistic.
The FRS has been running since 1992 and is an annual cross-sectional survey. Until the financial year (April to March) ending 2003 the survey covered Great Britain; it was then extended to cover all of the UK.
The target sample size for the financial year ending 2015 survey was 20,000 households; this was reduced from 25,000 households from April 2011. The achieved sample size for the financial year ending 2015 was 19,538 households.
From April 2012, the FRS is also being used as the survey source for the cross-sectional element of the European Union Statistics on Income and Living Conditions (EU-SILC), on page 29. The first EU-SILC findings using the FRS as a source were released in December 2013.
4.1.2 What sort of information is usually published and what can we extract from the data?
The FRS covers:
- income and state support receipt
- pension participation
Information can be derived on an individual, benefit unit or household level depending on which is the most appropriate.
The FRS is the primary data source for households below average income and the pensioners’ incomes series. It is also a main source of the Department for Work and Pensions (DWP)’s estimates of take-up of income related-benefits and the policy simulation model which is used extensively by DWP analysts for policy evaluation and costing of policy options.
The FRS is also used by other government departments including HM Revenue and Customs and HM Treasury and is also made available for external users via the UK Data Service.
4.1.3 What are the main limitations?
In common with all income surveys, the FRS results suffer from several issues of non-sampling error. These include the following.
Measurement error – in particular, the FRS is known to under-report benefit receipt.
Non-response error – the FRS response rate in the financial year ending 2015 was 58%. To correct for differential response rates, estimates are weighted using population totals.
Survey coverage – the FRS covers private households in the UK. Therefore individuals in nursing or retirement homes, for example, will not be included.
Survey design – the FRS uses a stratified clustered sample designed to produce robust estimates at region level. The FRS is therefore not suitable for analysis below region level.
Sample size – although the FRS has a relatively large sample size for a household survey, small sample sizes may require several years of data to be combined. Estimates by ethnic group are published using 3-year averages.
4.1.4 What are the main uses and strengths?
Specific strengths of the FRS include the following.
The FRS is considered to be the best source for looking at cash benefit and tax credit receipt by characteristics not captured on administrative sources and for looking at total benefit receipt on a benefit unit or household basis.
Capturing information on incomes: it captures more detail on different income sources compared with most other household surveys.
It collects a lot of contextual information on the household and individual circumstances, such as employment, education level and impairment. The FRS is therefore a comprehensive data source allowing for a wide variety of detailed analysis.
The FRS began in 1992 and so allows for comparisons over time.
4.2 Households below average income
The HBAI annual report is published on the GOV.UK website.
HBAI microdata can be accessed under a standard End User Licence on the UK Data Service website. More detailed microdata are available as a Secure Access File to be used in the Safe Room at the University of Essex – additional approval is needed for this level of access.
Department for Work and Pensions
+44 (0) 114 2825365
4.2.1 What exactly does it measure?
The households below average income (HBAI) uses data from the Family Resources Survey (FRS) to represent information on living standards in the UK using disposable household income. The data are also equivalised which takes into account the size and composition of households to make the income figures comparable1.
The HBAI statistics incorporate widely-used, international standard measures of low income and inequality. There are a range of measures of low income, income inequality and material deprivation which capture different aspects of changes to living standards. HBAI is generally held to be the foremost source of UK data and information about household net income and poverty.
Annual estimates for several statistics related to the number and percentage of people living in low-income households are provided against a number of main client groups including children, working-age adults, pensioners and individuals living in a family where someone is disabled. Results are available for the UK from the financial year (April to March) ending 2003 through to the financial year ending 2015, with earlier results for Great Britain from the financial year ending 1995.
Household income is defined as the following.
Income Before Housing Costs (BHC) takes income from all household members including dependants and includes the following main components:
- usual net earnings from employment
- profit or loss from self-employment (losses are treated as a negative income)
- all Social Security benefits and tax credits
- income from occupational and private pensions
- investment income
- maintenance payments, if a person receives them directly
- income from educational grants and scholarships (including, for students, top-up loans and parental contributions)
- the cash value of certain forms of income in kind (free school meals, Healthy Start vouchers and free school milk and free TV licence for those aged 75 and over)
Income is net of the following items:
- Income Tax payments
- National Insurance contributions
- domestic rates or Council Tax
- contributions to occupational pension schemes
- all maintenance and child support payments, which are deducted from the income of the person making the payment
- parental contributions to students living away from home
- student loan repayments
Income After Housing Costs (AHC) is derived by deducting a measure of housing costs from the above income measure.
Housing costs include the following main components:
- rent (gross of housing benefit)
- water rates, community water charges and council water charges
- mortgage interest payments
- structural insurance premiums (for owner occupiers)
- ground rent and service charges
4.2.2 What sort of information is usually published and what can we extract from the data?
- relative low income
- absolute low income
- income inequality, for example, Gini coefficient2
- household income distributions and averages before and after housing costs
- income components including income from wages, self-employment and benefits
4.2.3 What are the main limitations?
In common with all income surveys, the HBAI results suffer from several issues of non-sampling error. These include the following.
Measurement error – in particular, the FRS (on which HBAI is based) is known to under-report benefit receipt. That said, the FRS is considered to be the best source for looking at benefit and tax credit receipt by characteristics not captured on administrative sources and for looking at total benefit receipt on a benefit unit or household basis.
Non-response error – the FRS response rate in the financial year ending 2015 was 58%. To correct for differential response rates, estimates are weighted using population totals.
Coverage error – the FRS covers private households in the UK – therefore individuals in nursing or retirement homes, for example, will not be included.
4.2.4 What are the main uses and strengths?
HBAI is a main source of data and information about household income, income poverty and inequality and is used for the analysis of low income by researchers and the government. Main uses include:
- informing policy development and monitoring of low income across the population
- international comparisons within EU and Organisation for Economic Co-operation and Development (OECD) countries (though EU Statistics on Income and Living Conditions (SILC) is considered the principal source for these comparisons)
- Parliamentary, academic, voluntary sector and lobby group analysis; examples include using the HBAI data to examine income inequality, poverty, the distributional impacts of fiscal policies and understanding the income profiles of vulnerable groups
4.3 Effects of taxes and benefits on household income / Living Costs and Food Survey
Headline estimates from the effects of taxes and benefits on household income (ETB) are published in the new, annual household disposable income and inequality bulletin, designed to provide more timely figures of main indicators. The main effects of taxes and benefits (ETB) statistical bulletin is published several months later, providing further detail on the full impacts of taxes and benefits on the distribution of income. Alongside the statistical bulletins, a report is also published covering further analysis of the results and information on how the data are derived.
Additionally, new experimental “nowcast” estimates are published a few months after the end of the income year, providing an earlier insight into how incomes have changed. The Living Costs and Food Survey (LCF) annual publication “Family Spending” is also available on our website.
ETB - ONS contact
+44 (0)1633 456082
LCF - ONS contact
+44 (0)1633 455914
4.3.1 What exactly does it measure?
The effects of taxes and benefits on household income (ETB) is one of our longest standing outputs, having been produced each year since 1961. It is an annual analysis looking at how taxes and benefits affect the income of households in the UK. It provides estimates of household incomes, including the average amount of taxes that households pay and also the value of benefits that they receive.
ETB data are from the Living Costs and Food Survey (LCF), formally known as the Expenditure and Food Survey (EFS) which is a voluntary sample survey of around 5,500 private households in the UK, covering both income and expenditure. Each individual aged 16 and over in the household visited is asked to keep diary records of daily expenditure for 2 weeks. Information about regular expenditure, such as rent and mortgage payments, is obtained from a household interview along with retrospective information on certain large, infrequent expenditures such as those on vehicles. Detailed information on income (including cash benefits received from the state) of each adult member of the household is collected through the interview. In addition, personal information such as age, sex and marital status is recorded for each household member. Children aged 7 to 15 are asked to keep a simplified version of the diary.
One of the unique contributions of ETB is the detailed breakdown of household income it provides, including estimates of both direct and indirect taxes, and both cash benefits and “in kind” benefits provided by the state. The ETB analysis uses 5 main measures of household income:
the starting point of the analysis is original income: this is the annualised income in cash of all members of the household before the deduction of taxes or the addition of any state benefits; it includes income from employment, self-employment, investment income, private pensions and annuities which include all workplace pensions, individual personal pensions and annuities
the next stage of the analysis is to add cash benefits and tax credits to original income to obtain gross income
Income Tax, Council Tax and Northern Ireland rates, and employee’s and self-employed National Insurance contributions are then deducted to give disposable income
the next step is to deduct indirect taxes (such as VAT, and fuel and alcohol duties) to give post-tax income
Finally the analysis adds benefits which are provided “in kind” to households by government for which there is a reasonable basis for allocation to households to obtain final income; these “in kind” benefits include the provision of education, health services and subsidised travel and housingThe measure of disposable income used in ETB is designed to be consistent with the international standards set out in the Canberra Group Handbook on Household Income Statistics (UNECE, 2011). This results in a small number of differences between ETB disposable income and the before housing costs (BHC) measure in households below average income (HBAI). For example:
ETB includes benefits in kind provided by employers (for example, company cars) within income, but these are not included within HBAI
HBAI includes certain benefits in kind provided by the state (such as free school meals and Healthy Start vouchers) within BHC income; in ETB, these are included with other benefits in kind as part of final income
4.3.2 What sort of information is usually published and what can we extract from the data?
ETB data are published in both an annual statistical bulletin and a supplementary analysis and methodology paper. The bulletin provides analysis of each stage of the redistribution process from original income through to final income, looking at the impact of taxes and benefits on income inequality. The bulletin and tables also look at:
long-term trends in household income for income quintiles deciles, with detailed breakdowns by income component (including individual taxes and benefits)
income for quintiles or deciles of retired and non-retired households, again with detailed breakdowns by income component
- long-term trends in income inequality, measured through the Gini coefficient2 and S80/S203 and P90/P104 ratios
- average incomes, taxes and benefits by household type, tenure status and region
- taxes paid (direct and indirect) as a proportion of income and expenditure
- households receiving more in benefits than paid in taxes
- characteristics of income quintile or decile groups (including number of adults or children, household type, tenure, age or employment status of chief economic supporter)
The ETB data are also used to produce a range of ad hoc pieces of analysis by ONS and others. Examples of analysis using the ETB data include:
An expenditure-based analysis of the redistribution of household income
The main LCF publication, Family Spending, is primarily concerned with assessing household expenditure. The publication does, however, include a chapter on income which considers how expenditure varies across the income distribution.
4.3.3 What are the main limitations?
As a survey-based source, ETB shares many of the same limitations as HBAI, such as the under-reporting of benefit receipt.
An additional limitation of the ETB data compared with HBAI is the sample size on which the figures are based. All survey-based estimates are subject to a level of uncertainty. As ETB is based on a smaller sample of households than HBAI, the level of precision is lower. This means that for income- based analysis which does not require examination of individual taxes and benefits (especially indirect taxes or benefits in kind) or expenditure, it may be preferable to use HBAI as a source in order to benefit from the larger sample size. This is particularly the case when looking at smaller subgroups of the population.
4.3.4 What are the main uses and strengths?
These statistics allow for analysis of the distributional impact of government policy on taxes and benefits. They are the only statistics available that are able to give such a complete picture of the distribution of income including indirect taxes and benefits in kind. The 2009 report by the Commission on the Measurement of Economic Performance and Social Progress by Stiglitz, Sen and Fitoussi identified income distribution, in addition to measures of average income, as an important factor in the measurement of well-being, giving a more complete picture of the standards of living experienced by individuals.
These characteristics of the ETB data also mean that they are also the best source for any analysis of household income that includes a breakdown by source or individual taxes or benefits.
ETB data are the primary source used by HM Treasury in their Intra-Governmental Tax and Benefit Microsimulation Model (IGOTM). This is used to model possible tax and benefit changes before policy changes are decided and announced.
As ETB data come from the LCF survey, which is the primary source of household expenditure data in the UK, it is also possible to use this data to carry out joint analysis of income and expenditure, something that is strongly recommended by the Organisation for Economic Co-operation and Development (2013) and many others for better understanding people’s economic well-being in terms of their material living standards.
From 2016, in response to requirements to improve the timeliness of some of the main income series published within the effects of taxes and benefits release, we started publishing a new short bulletin that provides users with main income statistics, along with longer-term trends (going back to 1977), which are an important aspect of the effects of taxes and benefits series. The first edition of this new bulletin, household disposable income and inequality (HDII), was published in February 2016, covering household incomes for the financial year (Apr to Mar) ending 2015. The statistics in this release include measures such as median disposable income, inequality measures such as the Gini coefficient2 and breakdowns of the components of disposable income by quintile and decile groups.
Additionally, we are continuing to develop new experimental “nowcast” estimates of the ETB data, with the aim of providing provisional estimates of main indicators far earlier than has previously been possible. An initial methodological paper was published in July 2015, followed by the release of nowcast estimates for the financial year ending 2015 in October that year. It is anticipated that nowcasts for the financial year ending 2016 will be published in summer 2016.
Together, HDII and the experimental nowcast estimates provide the earliest analysis of the household income distribution each year, allowing people insight into the evolution of living standards as early as possible.
Notes for Household income:
- The Department for Work and Pensions has published an infographic summarising the process behind estimates of low income in HBAI.
- The Gini coefficient is the most widely used summary measure of inequality in the distribution of household income. The lower its value, the more equally household income is distributed.
- The income quintile share ratio or the S80/S20 ratio is a measure of the inequality of income distribution. It is calculated as the ratio of total income received by the 20% of the population with the highest income (the top quintile) to that received by the 20% of the population with the lowest income (the bottom quintile).
- The income decile ratio or the P90/P10 ratio is a measure of the inequality of income distribution. It is calculated as the ratio of the level of income at the 90th percentile to the level of income at the 10th percentile.
Other income measures
The sources covered in this section again refer to income. However, unlike the sources listed in the previous section, income may not be the sole focus of the source or the source may only cover a very specific measure of income.
5.1 European Union-Statistics on Income and Living Conditions
Indicators based on EU-Statistics on Income and Living Conditions (EU-SILC) for the UK and other EU countries are published on Eurostat’s website, along with more detailed articles and analysis.
EU-SILC microdata are also available to researchers through Eurostat’s User Database (UDB).
+44 (0)1633 456082
+44 (0)2074 497341
5.1.1 What exactly does it measure?
EU-Statistics on Income and Living Conditions (EU-SILC) is the EU reference source for comparative statistics on income, poverty, social exclusion and living conditions at the European level.
It provides 2 types of annual data for the 28 European Union countries as well as Iceland, Norway, Switzerland and Turkey:
cross-sectional data pertaining to a given time or a certain time period with variables on income, poverty, social exclusion and other living conditions
longitudinal data pertaining to individual-level changes over time, observed periodically over a 4 year period
EU-SILC is co-ordinated by Eurostat (the European Commission’s statistical agency) under European Commission regulation. In the UK, EU-SILC is collected jointly by the Department for Work and Pensions (DWP) and the Office for National Statistics (ONS), while information for other countries is mainly collected by other national statistical institutes (NSIs).
The UK cross-sectional component is made up of a sample of around 10,000 households, which is taken from the first 6 months of Family Resources Survey (FRS) interviews in any given year. A sub-sample of these households are then tracked longitudinally through a follow-up survey to the FRS called the Survey on Living Conditions (SLC). The overall achieved longitudinal sample is approximately 7,500 households each year.
The main topics covered by the EU-SILC microdata are:
- material deprivation
5.1.2 What sort of information is usually published and what can we extract from the data?
Indicators and reports based on EU-SILC data are made available through the Eurostat website.
Published indicators cover the following areas:
- relative low income
- material deprivation
- low work intensity
- income inequality (including Gini and S80/20 ratios)
- housing deprivation, overcrowding and housing cost overburden
- persistent at-risk-of-poverty (relative low income in current year and at least of 2 preceding years)
- labour market and pay transitions
Many of these indicators are broken down by age, sex, employment status, level of education, housing tenure, country of birth, and citizenship.
More detailed analysis of EU-SILC is frequently published by the European Commission in publications such as Employment and Social Developments in Europe.
EU-SILC also includes annual ad hoc modules on important topics of policy interest, allowing more detailed analysis in these areas. Recent ad hoc modules include:
- subjective well-being
- housing conditions
- intergenerational transmission of disadvantages
- intra-household sharing of resources
- over-indebtedness and financial exclusion
- social participation
There are many publications and papers, based on these modules, produced by the European Commission, national statistical offices and academia. For example, the Intergenerational Transmission of Disadvantage in the UK and EU, 2014 and Persistent Poverty in the UK and EU, 2014.
5.1.3 What are the main limitations?
As a survey-based source, EU-SILC shares the same limitations as households below average income (HBAI) and effects of taxes and benefits (ETB).
There are also some limitations relating to the international dimension of EU-SILC. EU-SILC is harmonised in terms of variables and concepts but the means of data collection are more loosely specified. Some countries collect data via household surveys, whilst others collect information via administrative sources. Additionally there may be variation between countries due to translation issues with the questionnaire, or culturally-based differences. The need to specify a framework for classifying income that is consistent across countries means that the benefit income variables made available to researchers are grouped into streams such as “old-age benefits”, rather than classified as the individual benefits themselves1.
5.1.4 What are the main uses and strengths?
A main strength of EU-SILC is the consistency of the variables and concepts across countries, which allows for meaningful comparative analysis across the EU. For this reason, EU-SILC is the main source of data used for policy monitoring and development at a European level. At a national level, this comparability is helpful in allowing researchers to examine what is happening in the UK in comparison with other countries with different welfare regimes and different policy interventions.
EU-SILC is used to measure the European Commission’s Europe 2020 target on the number of people at risk of poverty or social exclusion. This measure combines a number of different dimensions of poverty and social exclusion into a single indicator. According to this definition, people are considered at risk of poverty or social exclusion if they are experiencing at least 1 of 3 conditions – having a household income below the poverty threshold, being severely materially deprived, or living in a household with low work intensity.
Probably the most important strength of EU-SILC over other official sources of income data is its longitudinal component, which provides crucial information for policy development. In particular it can help us understand how, when and for how long people move into and out of poverty.
The longitudinal component of EU-SILC allows the measurement of persistent relative low income. It is widely agreed that the impact of long-term poverty on individuals is worse than when poverty is experienced only for a short time (see for example, Atkinson et al., 2002; Dickerson and Popli, 2011; Jenkins and Van Kerm, 20132). Whilst short episodes can sometimes be dealt with through use of savings, borrowing or reduced consumption, long-lasting poverty is a lot more likely to damage life chances. For this reason, persistent poverty is particularly interesting to policy makers.
Additionally, researchers within and outside government have used EU-SILC to research topics including:
- the relationship between single year at-risk-of-poverty rates and persistent risk-of-poverty
- the evolution over time of poverty entry and exit rates
- the relationship between entering the labour market and moving out of poverty
- which individuals are most at risk of persistent poverty
- dynamics of material deprivation
- flexible employment and poverty
- disability and low income persistence
5.2 Pensioners’ incomes series
The pensioners’ income series annual report is published on the GOV.UK website.
Anonymised microdata are made available on the UK Data Service website as part of the Family Resources Survey (FRS) project.
Department for Work and Pensions
+44 (0)2074 495636
5.2.1 What exactly does it measure?
Pensioners’ incomes series (PI) contains estimates of the levels, sources and distribution of pensioners’ incomes. It also examines the position of pensioners within the income distribution of the population as a whole. The series includes information about gross and net income, both before and after housing costs.
The publication is based on 2 household surveys: the Family Expenditure Survey (FES) and the Family Resources Survey (FRS). The latest information, for the financial years (Apr to Mar) ending 1995 to ending 2015, comes solely from the Family Resources Survey.
5.2.2 What sort of information is usually published and what can we extract from the data?
The annual PI publication covers the following:
- income for different groups of pensioners by age, for singles and couples, and broken down by region
- detailed look at various sources of income, including the proportion of pensioners who receive income from these different sources
- the distribution of pensioners’ incomes, both within the pensioner population and within the household population overall
- additional analysis, including couples where one member is above State Pension age and the other below, married and cohabiting couples and results for ethnic groups
The datasets are also deposited with the UK Data Archive post-publication (as part of the FRS project), to allow researchers to conduct their own analysis. These datasets have adjustments made to ensure anonymisation of the data, such as rounding all incomes to the nearest pound and “top-coding” ages at 80.
5.2.3 What are the main limitations?
As PI is from the Family Resources Survey, many of the caveats that apply for the FRS also apply for PI. For example, the levels of receipt of a number of benefits are under-reported on the FRS and this will feed into PI.
PI is based on survey data, hence is subject to sampling variation and other forms of error associated with a sample survey. As such, it is often difficult to draw conclusions about significant changes in incomes from one year to the next.
The surveys on which the series are based are household surveys and so people living in institutions – such as nursing homes and communal establishments – are not covered.
When carrying out analysis of PI it is worth bearing in mind the definitions that apply to the data, and how this affects the results and interpretation. For example, analysis is at a household level, and incomes are not equivalised (unlike in HBAI).
Analysis can be carried out on gross or net income, on all pensioner units or by splitting out singles and couples, on a “Before” or “After” housing costs basis, and may use means or medians to summarise results. We advise some thought to ensure analysts are using the most appropriate measures for any work.
Some analysis of PI looking at small sub-groups may need to pool years of data to produce large enough samples from which to draw robust results. Some analysis in the main PI publication uses 3 years of data to achieve this (for example, analysis by region, ethnicity, some quintile analysis).
5.2.4 What are the main uses and strengths?
The data can be used to understand the levels, sources and distribution of pensioners’ incomes – as covered in Sections 5.2.1 and 5.2.2.
One of the main strengths of PI is the relatively long time series available for assessing trends going back to the financial year ending 1995 on the FRS – and further with the FES.
5.3 Survey of Personal Incomes
Personal Income National Statistics is published on the GOV.UK website.
SPI Public Use Tape (PUT) microdata are available on the UK Data Service website.
+44 (0)300 528 243
5.3.1 What exactly does it measure?
The Survey of Personal Incomes (SPI) is an administrative dataset based on information held by HM Revenue and Customs (HMRC) tax offices on individuals who could be liable to UK tax. It is carried out annually by HMRC and covers income assessable to tax for each tax year. Not all individuals sampled are taxpayers because the operation of personal reliefs and allowances may remove them from liability. Where income exceeds the threshold for operation of Pay-As-You-Earn (PAYE), the survey provides the most comprehensive and accurate official source of data on personal incomes.
The SPI sample is drawn from 3 separate systems:
- the National Insurance and PAYE Service (NPS) system covers all employees and occupational pension recipients with a Pay-As-You-Earn (PAYE) record; NPS replaced the Computerisation of PAYE (COP) system
- the Computerised Environment for Self Assessment (CESA) system covers people with self-employment, rental or untaxed investment income; it also covers directors, those subject to higher rate tax and other people with complex tax affairs – where people have both NPS and CESA records, their CESA record is selected because it provides a more complete picture of their taxable income
- the Claims system covers people without NPS or CESA records who have had too much tax deducted at source and claim a repayment
There were approximately 728,000 valid cases on the final tax year ending 2014 SPI dataset.
5.3.2 What sort of information is usually published and what can we extract from the data?
The dataset contains a range of variables about personal incomes arising from employment, self-employment, pension, taxable benefits, property, savings, investments and other income sources. The dataset also contains variables about tax allowances, deductions and reliefs, which people might be due. There is also a regional code variable on the dataset and a trade code for cases which are self-employed.
An anonymised version of the SPI called the Public Use Tape (PUT) is also published and available to the public for download; the UK Data Archive (UKDA) currently holds the PUT data for 1985 to 1986 and 1995 to 1996 onwards (data for 2008 to 2009 is currently unavailable). For further details of sampling and coverage criteria, see the accompanying documentation on the UKDA website.
The SPI is also used to create HMRC’s personal income statistics publications, which contain a variety of charts and tables illustrating the findings from that year’s SPI including breakdowns of income information by, for example: age, geographical region, sex and self employment.
5.3.3 What are the main limitations?
The SPI is an individual level survey based on administrative records; as HMRC does not require information on household earnings or hours worked to administer Income Tax, this information is not available from the SPI.
As mentioned in Section 5.3.1, not all of the individuals in the SPI sample are taxpayers. For tax year ending 2014 about 23% have no Income Tax liability because deductions and reliefs and personal allowances exceed their total income. Where income exceeds the threshold for the operation of Pay-As-You-Earn (PAYE) (£9,440 for 2013 to 2014), the SPI provides the most comprehensive and accurate official source of data on personal incomes.
However, as HMRC does not hold information for all people with personal incomes below this level, the SPI is not a representative data source for this part of the population and no attempt has been made to estimate the numbers of cases below the tax threshold or the amount of their incomes.
Therefore National Statistics published from the SPI – with the exception of Tables 3.9 and 3.10 – only cover individuals liable to UK Income Tax (taxpayers) and their incomes.
With regards to benefits, the SPI covers only taxable benefits (or benefits in kind). This means that it doesn’t capture benefits like Jobseekers Allowance or housing benefits. It therefore cannot provide a complete picture of income.
Some other elements of the SPI have to be imputed or estimated based on other available information and, as with all sample surveys, estimates from the SPI have a sampling error attached to them. These issues are covered in more detail in Annex B of our personal income statistics publication on GOV.UK.
5.3.4 What are the main uses and strengths?
The SPI is compiled to provide a quantified evidence base from which to cost proposed changes to tax rates, personal allowances and other tax reliefs for Treasury Ministers. It is used to inform policy decisions within HMRC and HM Treasury, as well as for tax modelling and forecasting purposes. In addition, it is used to provide summary information for the national accounts that are prepared by the Office for National Statistics. Finally, it is used to provide information to Members of Parliament, other government departments, companies, organisations and individuals.
The SPI is also used to create HMRC’s aforementioned personal income National Statistics releases. These tables would be of interest to policy makers in government, academics, journalists, “think-tanks” and other research bodies. They would be of use to individuals or organisations interested in the distributions of numbers and amounts of personal incomes, for example by taxpayer marginal rate or income band.
5.4 Wealth and Assets Survey
Wealth in Great Britain Wave 4: 2012 to 2014, is the most recent publication of Wealth and Assets Survey (WAS) analysis which focuses on income measures.
WAS datasets are available in 2 forms from the UK Data Service – a Special Licence dataset which, although anonymised, contains data at a very low level so specific permission has to be granted for access; and an End User Licence dataset – much easier to access as all geographical data has been removed, age has been top coded and some detail of non-wealth related variables has been removed.
+44 (0)1633 456295
5.4.1 What exactly does it measure?
The Wealth and Assets Survey (WAS) commenced in 2006 with the aim of measuring the economic well-being of households and individuals in Great Britain. It is a biennial, longitudinal survey conducted by the Office for National Statistics and funded by a consortium of government departments.
WAS data is fundamental to understanding the distribution of assets, debts, and liabilities in Great Britain and fills gaps in official statistics for which evidence was lacking from either other survey sources or administrative data. It collects information concerning all forms of personal assets held by individuals within private households, including their financial wealth, property wealth, physical wealth and private pension wealth.
During the period 2010 to 2012 – the third wave of WAS – over 40,000 individuals aged 16 or over were interviewed in more than 21,000 private households across Great Britain.
The design of the survey recognises the fact that wealth is highly skewed, with a small proportion of households owning a large share of the wealth. The efficiency of the sample is therefore improved by over-sampling addresses likely to be in the wealthiest 10% of households (established utilising data from HM Revenue and Customs (HMRC)).
Studies investigating a household’s economic resources have often focused primarily on income. Such estimates of income are frequently used as a main indicator on which governments are held to account, and by which countries are compared. Nevertheless, income provides just one estimate of material well-being. The 2009 report by the Commission on the Measurement of Economic Performance and Social Progress3 highlighted the need to consider wealth in conjunction with income to better assess living standards. This need to look beyond income was an important motive for the introduction of the WAS.
Income data collected by WAS has been extensively quality assured and compares well with data from sources that are focused on income, such as the Family Resource Survey, page 20. Whilst the primary focus of WAS is the estimation of household and individual wealth, data on a household’s total income are available for the period 2010 to 20124. The income data is all self-reported, regular income is available as both net and gross annual measures and covers:
- earned income from employees and self-employed (main and second or other jobs)
- income from benefits (including state pensions)
- income from private pensions (including occupational and personal pensions)
- income from investments
- other regular income
Whilst the coverage of income is good, as the data is self-reported, it is not expected to be as accurate as some other sources (for example, employer-based earnings data). It is therefore recommended that income be used as a classificatory variable only (for example, analysed by deciles or quintiles) alongside wealth estimates. In addition, proxy information is taken for some respondents who cannot be contacted, and any missing data is statistically imputed5.
Unlike some other sources the definition of net measures cannot be specifically defined. Respondents are asked, for example, for their net earnings “after all deductions (excluding Tax Credits, bonuses and commissions not received each pay period)”. For some this will be entirely net of Income Ttax and National Insurance, but for many will also be net of any pension contributions, student loan or other employer loans.
In addition the survey also collects net irregular income such as inheritances and lottery wins.
5.4.2 What sort of information is usually published and what can we extract from the data?
To date, only limited income estimates have been published from WAS – concentrating on the relationship between household wealth and income. However, the scope for analysis is huge.
The main publication for WAS data is Wealth in Great Britain. The main report for Wave 3 did not contain income data, but this was published subsequently, as it was the first time total household income was published from WAS.
Estimates of means are no longer published from WAS, as the distribution of wealth is so highly skewed towards the wealthiest, the median value gives a much better measure of average than the mean (and unlike other financial surveys) WAS does not top code any measures of wealth or income.
The datasets are available in 2 forms from the UK Data Archive – a Special Licence dataset which, although anonymised, contains data at a very low level so specific permission has to be granted for access; and an End User Licence dataset – this is much easier to access as all geographical data have been removed, age has been top-coded and some detail of non-wealth related variables have been removed.
5.4.3 What are the main limitations?
The focus of WAS is the estimation of household and individual wealth. Income measures are only designed to provide classificatory variables, for example, used to divide the population into income groups in order to analyse wealth by income grouping.
5.4.4 What are the main uses and strengths?
Income is only one measure of material well-being and alone, might not accurately reflect the full picture. For example, some older people may have relatively low incomes, but have, over their lifetime accumulated a good degree of wealth – for example, property wealth and financial investments. Conversely some young people may have high incomes but have not had time to accumulate much wealth – for example, have large student loans and mortgages.
The Wealth and Assets Survey allows consideration of factors other than income when considering an individual’s or household’s economic well-being. This is of growing importance in many areas of government policy which are focusing on wealth rather than income or are concerned with saving for retirement.
In addition to this, the longitudinal design of the survey means that the impact of specific life events (for example, from working life to retirement) can be analysed, as can general life cycle effects.
5.5 National accounts estimates of gross disposable household income
National accounts estimates of gross disposable household income are available on our website.
+44 (0)1633 455612
5.5.1 What exactly does it measure?
Household sector accounts are published by the Office for National Statistics (ONS) on quarterly and annual bases: quarterly from Quarter 1 (Jan to Mar) 1995 and annually from 1948. Estimates in the household sector accounts are comparable across time and across countries, as they are produced in line with international standards set out in the European System of Accounts (ESA10) and the United Nations System of National Accounts (SNA08). However, the entire back series is not always comparable because of changes in international national accounts standards over this period. As with other components of the national accounts, household sector accounts are compiled using information from a multitude of different sources, including household and business surveys as well as administrative records, to ensure that estimates are as coherent and integrated as possible.
5.5.2 What sort of information is usually published and what can we extract from the data?
Household sector accounts use the following process to derive gross disposable income:
a) total household income = gross operating surplus + mixed income + compensation of employees + property income + pension income + Social Security benefits received (other than pensions) + miscellaneous transfers and insurance claims received b) total uses = taxes paid + social contributions paid + property expenditures + miscellaneous transfers and insurance premiums paid c) gross disposable household income (GDHI) = total household income – total uses
5.5.3 What are the main limitations?
For the time being, we do not separately publish information on non-profit institutions serving households (NPISH), but include them alongside households as part of the household sector. The publication of separate household and NPISH accounts will be introduced from Blue Book 2017.
Due to compliance with national accounts concepts, such as the inclusion of imputed rental of owner occupiers as income, estimates may not match specific user requirements.
5.5.4 What are the main uses and strengths?
The main use of national accounts estimates of gross disposable household income is within the calculation of gross national income (GNI) and gross domestic product (GDP). Furthermore they are used to inform decisions made by policy makers in both central and local government as well as frequently being used in academia.
One of the main strengths of using the national accounts estimate of gross disposable income is that it includes the income of institutional households which are often excluded from other survey sources of income.
The national accounts also offer a long time series, with data going back to 1948 (1955 on a quarterly basis). Furthermore the more recent data is also widely comparable on an international basis as it follows the ESA10 and SNA08 regulations on national accounts.
5.6 Regional accounts estimates of gross disposable household income
Regional gross disposable household income (GDHI) data are published annually in the spring on our website. Data are available in Excel datasets or via the ONS Data Explorer tool.
+44 (0)1633 456878
5.6.1 What exactly does it measure?
Gross disposable household income (GDHI) is the amount of money that individuals in the household sector have available for spending or saving. This is money left over after expenditure associated with income, for example, taxes and social contributions, property ownership and provision for future pension income. It is calculated gross of any deductions for capital consumption, which is the decline in value of fixed assets due to normal physical deterioration and obsolescence.
The household sector comprises all individuals in an economy, that is, people living in traditional households as well as those living in institutions such as retirement homes and prisons. It should be noted that regional GDHI relates to all individuals within the household sector for a region and not to an actual average household or family unit. The household sector also includes sole trader enterprises (the self-employed) and non-profit institutions serving households (NPISH).
The Nomenclature of Units for Territorial Statistics (NUTS) is the geographic framework used to produce regional GDHI throughout the European Union. These estimates are compiled on a residence basis. Regional GDHI is published at 3 levels of geography:
- NUTS1: Wales, Scotland, Northern Ireland and the 9 English regions
- NUTS2: 40 sub-regions – mainly groups of counties and unitary authorities
- NUTS3: 173 local areas – principally individual counties and unitary authorities.
A “top-down” approach is used for the production of regional GDHI estimates, whereby the national aggregate, consistent with the UK National Accounts (the Blue Book), is allocated to regions using a regional indicator dataset. Numerous regional indicator datasets are used in the production of regional GDHI, comprising survey and administrative data, which conform as far as possible to Eurostat standards.
More information can be found on the regional accounts methodology webpage.
5.6.2 What sort of information is usually published and what can we extract from the data?
For domestic publication, regional GDHI estimates are published annually for the period 1997 to T-2 years (T being the year of publication) and are consistent with the previous year’s UK National Accounts (the Blue Book).
Total GDHI and the components of income are published in pounds million, pounds per head and per head index at all 3 NUTS levels.
The components of income are as follows.
Allocation of primary income account:
- resources: operating surplus, mixed income, compensation of employees and property income received
- uses: property income paid
Balance of primary income = primary resources – primary usesSecondary distribution of income account:
- resources: social benefits and other current transfers received
- uses: current taxes on income and wealth, social contributions and other current transfers paid
Balance of secondary income = secondary resources – secondary uses
The balancing items of these 2 accounts then allow us to calculate GDHI as:
GDHI = balance of primary income + balance of secondary income
The production of regional GDHI at the NUTS3 level is a legal requirement under European Commission (EC) legislation. The methods and data used are consistent with the guidance set out in the European System of Accounts (ESA).
5.6.3 What are the main limitations?
As with the national estimates, for the time being we do not separately publish information on NPISH, but includes them alongside households as part of the household sector.
The NUTS framework is not wholly consistent with standard UK geographies. For instance NUTS3 areas are generally groups of local authorities, which means that data at a local authority level are not available. The regional accounts team are currently working on a flexible geographies project which should help to address this limitation.
Regional GDHI estimates are produced in current prices (which include the effects of inflation). Constant price estimates of regional GDHI cannot be produced due to the lack of appropriate regional deflators.
Regional GDHI does not take account of mortgage capital repayments or payments of rent. In national accounts terms, the former is counted in the capital account, while the latter is considered final consumption expenditure. Regional GDHI gives no information on the spending patterns of households. However, regional accounts are currently looking to develop a measure of household final consumption expenditure for regions.
5.6.4 What are the main uses and strengths?
As with the national estimates, one of the main strengths of using a national accounts estimate of GDHI is that it includes the income of institutional households.
Unlike the national estimates, regional GDHI is available at a detailed geographic breakdown. Estimates are published at NUTS1, NUTS2 and NUTS3 levels allowing for comparison across UK and EU areas. Regional GDHI is considered valuable as a measure of relative wealth between regions.
Regional GDHI estimates provide an overview of economic diversity and social welfare at regional, sub-regional and local area levels. These estimates are used by the UK government and the devolved administrations of Northern Ireland, Scotland and Wales to formulate and monitor economic policy and allocate resources.
The European Union (EU) uses these estimates to inform regional policy and analysis and to identify disparities in regional welfare across the member states.
A consistent time series is available back to 1997.
5.7 Small area income estimates
Small area income estimates are available on our website.
+44 (0)1329 447934
5.7.1 What exactly does it measure?
The estimates provide the average household income for small areas within England and Wales. It is the income a household receives from wages and salaries, self-employment, benefits, pensions, plus any other source of income. These estimates are produced at the middle layer super output area (MSOA).
The technique used to produce small area income estimates is a multiple linear regression modelling technique with synthetic estimation. This technique is used when survey data alone are insufficient to produce accurate estimates of income in smaller geographical areas. At the level of these areas, sample survey sources often do not have sufficient sample size and it is therefore necessary to combine survey data with other data sources known as auxiliary data or covariates. The auxiliary data, which can be obtained from an administrative system or a previous census, can provide data on a small area basis for all areas within the target population.
To produce small area income estimates, it is necessary to model the area-level relationship between the survey variables and auxiliary variables. The survey data can be used to estimate the mean level of income for a particular type of individual within the larger population, while other data sources can be used to estimate the proportion of a particular type of individual living within the small area, for example, the proportion of the population claiming Income Support. The relationship between the two can then be fitted to derive an estimate of weekly household income within a small area. The fitted model produces an estimate that describes the relationship between the area-level values of the target survey variable and the auxiliary variable.
This modelling methodology enables survey data from the Family Resources Survey to be combined with census and administrative data to improve the quality of estimates at the small area level. As the estimates are model-based they are different to standard direct estimates.
5.7.2 What sort of information is usually published and what can we extract from the data?
The small area income estimates are produced for 4 different income types:
- average weekly household total income (unequivalised)
- average weekly household net income (unequivalised)
- average weekly household net income before housing costs (equivalised)
- average weekly household net income after housing costs (equivalised)
5.7.3 What are the main limitations?
A limitation of the small area income estimates is the measure of central tendency produced by the models, which is the mean household income. This measure does not provide an indication of the distribution of income within or between areas, which is a limitation because some areas have a more skewed distribution (with a small number of households having very high income) than others. We are working on a new method of modelling MSOA level income which can produce an estimate of the median household income and therefore an understanding of the distribution of income within MSOAs.
Estimates have been produced for 4 different types of income. In some areas slight inconsistencies may occur between the income types for a particular MSOA, for example, a MSOA may have a larger estimate for net income when compared with total income. Although there may be some such inconsistencies, the models selected are the best possible to describe the general pattern of income over all MSOAs. This reinforces the need to look at the confidence interval for income estimates, not just the point estimate, since the confidence intervals summarise the variability in the estimates caused by the modelling process.
These estimates do not enable direct comparisons of income over time. Each time a new set of estimates is produced, a different model is used which means any 2 sets of estimates do not form a time series as such. However, the 95% confidence intervals can be used to make indicative comparisons of income for an area over time. If the 95% confidence intervals for a given MSOA do not overlap for 2 different years estimates, then there is reasonable certainty that household income in that MSOA has changed.
The MSOA level estimates can be aggregated to provide income estimates for larger geographical areas such as local authorities or regions. However, this method is approximate and hence it is not possible to assess the precision of the aggregated estimates.
5.7.4 What are the main uses and strengths?
The main strength of small area income estimates is the low level of geography that the data are provided at. There is currently no other source of income data that can produce estimates at such a low geographical level. As such the estimates are often used to answer questions from the public, academics and Parliament that require detailed information on income at low levels of geography.
Notes on European Union-Statistics on Income and Living Conditions:
- Though in most countries, including the UK, the data is collected at the level of the individual benefit.
- Atkinson, T., Cantillon, B., Marlier, E. and Nolan, N. (2002) Social Indicators: The EU and Social Inclusion. Oxford: Oxford University Press Dickerson, A. and Popli, G. (2011). Persistent poverty and children’s cognitive development: Evidence from the Millennium Cohort Study. Working Paper. Department for Economics. University of Sheffield. Jenkins, S.P. and Van Kerm, P. (2013). The relationship between EU indicators of persistent and current poverty.CASEpapers, CASE/169, London School of Economics.
- Report by the Commission on the Measurement of Economic Performance and Social Progress, Professors Joseph E. Stiglitz, Amartya Sen and Jean-Paul Fitoussi, 14 September 2009. In particular, WAS supports 2 of the recommendations: Recommendation 3: Consider income and consumption jointly with wealth and Recommendation 4: Give more prominence to the distribution of income, consumption and wealth. Both recommendations underpin how UK, EU and OECD are shaping up their requirements on the wider measurement of economic performance, social progress, the environment and sustainability.
- In the first 2 waves of the survey, the only income data that were successfully collected related to earned income – so excluded income from benefits, private pensions and other income such as that from investments and property rentals.
- See the Technical chapter of the Wealth in Great Britain Report.