2007 Test Income evaluation
The 2007 Census Test included questions about the sources and level of each individual’s (usual resident’s) income. Around 100,000 households were asked to participate in the 2007 Test and around half of these received questionnaires that included the income questions. The remaining households received questionnaires with no income questions.
The evaluation of the Test included an assessment of the effect of the income questions on response rates, field operations, the public view of the Census and the quality of the data obtained from these questions. The key findings of the income question evaluation are summarised below, and the full report is available to download as a pdf file.
Inclusion of the income question - Key findings
Four key findings support the proposal to include an income level question in the 2011 Census:
-
Around 90.9 per cent of individuals who submitted valid responses to the 2007 Test also completed the income level question, even though it was the last question asked on the questionnaire
-
The inclusion of the income questions did not affect the item response rates to the other questions. This is consistent with findings from NISRAs 2007 Census Test in Northern Ireland, GROS’ 2006 Census Test in Scotland, and the 1997 Test taken in Great Britain
-
The inclusion of the income questions did not result in additional individuals contacting the Census Contact Centre regarding these questions
-
The inclusion of the income questions did not have a negative impact on the coverage of individuals within households
Exclusion of the income question – Key findings
On the other hand, five key findings support not including an income level question:
-
The overall response rate for questionnaires with no income questions was 53.3 per cent whereas the response rate for income questionnaires was 50.6 per cent - a statistically significant difference of 2.7 percentage points, which is consistent with findings from the 1997 Test. This indicates that more households would need to be followed-up for non-return of questionnaires if income questions were included in the 2011 Census
-
Individuals who were unemployed, over 65 years of age, less qualified, from certain ethnic minority backgrounds, living in Local Authorities identified as generally low income areas, or females tended to report lower income and had lower item response rates to the income level question than comparison groups
-
404 individuals who completed the 2007 Test income level question also took part in the Census Test Evaluation Survey (CTES), in which they were asked this question again. Responses matched in only 66.8 per cent of cases, indicating that the question was difficult to answer. This is consistent with findings from NISRAs 2007 Test
-
There was evidence that individuals who submitted valid 2007 Test responses had concerns about the income questions. More than half of those that did not answer the income level question did answer the ethnic identity or qualifications questions
-
Many newspapers took a negative stance in their reporting of the inclusion of income questions in the 2007 Test and proposals to include such questions in the 2011 Census
Conclusion
If income questions were included, to build on the substantial research already undertaken, further research would be needed to ascertain how to make the questions clearer, more acceptable to the public and provide more reliable data. The fact that the questions are difficult to answer and that some Census income data would be collected by proxy indicate that the Census may not be the best method to collect data on income. The Integrated Household Survey (IHS) or model-based income estimates produced by ONS should be able to meet at least some of Census users’ requirements for data on income. Additionally, these sources could provide information on a more frequent basis than the Census.
Taking full account of the limitations of generalising the findings from the 2007 Test to the population of England and Wales, the strength of Census users’ requirements for income questions in comparison to other questions, and the availability of alternative sources of income data, it is recommended that questions on income should not be included in the 2011 Census for England and Wales.
