Table of contents
1. Executive summary
The Index of Private Housing Rental Prices (IPHRP) is a monthly release that publishes private rental price indices for Great Britain, its component countries and English regions. The index is calculated using rental data collected by rent officers operating for the Valuation Office Agency, Scottish Government and Welsh Government.
Incorporating so many different data sources into any statistic involves a certain degree of risk. Administrative data, in particular, may be collected and compiled by third parties outside the Code of Practice for official statistics.
To further assure ourselves and users of the quality of our statistics, we have undertaken a thorough quality assessment of these data sources. This assessment is a continuous process and we will publish updates periodically.
We have followed the Quality Assurance of Administrative Data (QAAD) toolkit, as described by the Office for Statistics Regulation (OSR). Using the toolkit, we established the level of assurance we are seeking (or “benchmark”) for each source. The assurance levels are set as either “basic”, “enhanced” or “comprehensive”, depending on:
the risk of quality concerns for that source, based on various factors, such as the source’s weight in the headline index, the complexity of the data source, contractual and communication arrangements currently in place, and other important considerations
the public interest profile of the item which is being measured, and its contribution to the headline index
Through engagement with our suppliers, we have assessed the assurance level that we have currently achieved by considering:
the operational context of the data; why and how it is collected
the communication and agreements in place between ourselves and the supplier; the quality assurance procedures undertaken by the supplier
the quality assurance procedures undertaken by us
Table 1 summarises the quality assurance benchmarks that were set, and the assurance levels that we have assessed each source at during this assessment.
Table 1: Summary of administrative data sources used to produce the IPHRP and level of assurance applied to each source
|Administrative data source||Benchmark assurance level||Achieved Assurance level|
|Valuation Office Agency||Comprehensive||Enhanced|
|Ministry of Housing Communities and Local Government||Comprehensive||Enhanced|
|Source: Office for National Statistics|
Download this table Table 1: Summary of administrative data sources used to produce the IPHRP and level of assurance applied to each source.xls (36.4 kB)
As a result of this assessment, we have put in place an action plan to improve our quality assurance in some areas.
Data for the Valuation Office Agency (VOA) require a comprehensive level of assurance; however, we do not currently have access to the microdata, which limits our ability to quality assure the data; we have mitigated the risks involved by working with VOA to put in place aggregation methodologies, processes and quality assurance procedures and are requesting access to their data through powers of the Digital Economy Act.
Ministry of Housing Communities and Local Government (MHCLG) data require a comprehensive level of assurance; however, the data are constructed from a number of data sources, which have not necessarily been comprehensively quality assured. We are working with MHCLG to understand their departmental approach to the development of Quality Assurance of Administrative Data (QAAD) material.
We will continue to engage with our data suppliers to better understand any quality concerns that may arise and to raise their understanding of how their data are used in the construction of the Index of Private Housing Rental prices. We aim to publish an update to this QAAD at the beginning of 2019.Back to table of contents
Our assessment of data sources is carried out in accordance with the Office for Statistics Regulation’s Quality Assurance of Administrative Data (QAAD) toolkit. We are striving for a proportionate approach to assessing the required level of quality assurance for the varied data sources used in the compilation of the Index of Private Housing Rental Prices (IPHRP). We seek to highlight and address the shortcomings that we have identified and reassure users that the quality of the source data is monitored and fit for purpose.
In this paper, we set out the steps we have taken to quality assure our data and our assessment of each source. In section 3, we outline our approach to assessing our data sources. In section 4, we discuss the assurance levels we are seeking for each data source, and the resulting assessment. Finally, in section 5, we detail our next steps towards achieving full assurance.
This publication is part of an ongoing process of dialogue with our suppliers, to increase our understanding of any quality concerns in the source data, and to raise awareness of how it is utilised. Through this document, we aim to provide information and assurance to users that the sources used to construct the IPHRP is sufficient for the purposes for which it is used. We will, therefore, review this document every two years. For more information on our IPHRP measure please see our guidance document about the IPHRP.Back to table of contents
3. Approach to assessment
We have conducted our assessment of data sources used in the Index of Private Housing Rental Prices (IPHRP )using the Office for Statistics Regulation’s Quality Assurance of Administrative Data (QAAD) toolkit. We took the following steps for each data source:
establish the risk of quality concerns with the data
establish the level of public interest in the item that the data are being used to measure determine benchmark quality assurance levels, based on the risk and public interest
contact the suppliers of administrative data to understand their own practices and approach to quality assurance; generally, this consists of the following steps:
- send out questions to our data suppliers requesting information on their quality assurance procedures conduct follow-up meetings or discussions with our data suppliers to request further information and clarification
- maintain ongoing dialogues with data suppliers to develop a better understanding of any quality issues in the data, and raise awareness of how the source data are used
review our own quality assurance and validation procedures and processes
conduct an assessment of each data source using the four practice areas of the Quality Assurance of Administrative Data toolkit:
- operational context and data collection communication with data suppliers
- quality assurance procedures of the data supplier quality assurance procedures of producer
determine an overall quality assurance level based on our assessment
if this assurance level does not match the benchmark assurance level, then put steps in place to work towards meeting the required assurance level
review the quality assurance on an ongoing basis; we will publish a QAAD update every two years
3.1 Setting the benchmarks
In accordance with the QAAD toolkit, we have sought assurance for each data source based on the risk of quality concerns associated with that data source, and the public interest in the particular item being measured by that data source.
We considered a high, medium or low risk of data quality concerns based on:
the importance of the data source in calculating IPHRP, what would we do if we didn’t have this data
the complexity of the data source; for example, whether it is compiled from a number of different sources
the existing contractual and communication arrangements currently in place
other considerations, such as any existing published information on data collection, methodology or quality assurance, or mitigation of high-risk factors with the data
We considered a high, medium or low public interest profile based on:
the level of media or user interest in the IPHRP or its sub-components
the economic or political importance of the IPHRP
any additional scrutiny from commentators, based on particular concerns about the data
Together the risk of quality concerns and public interest profile are combined to set an overall assurance level that is required for a particular source. This assessment is based on the following matrix, as provided by UK Statistics Authority (Table 2).
Table 2: Quality assurance matrix
|Level of risk of quality concerns||Public interest profile: Lower||Public interest profile: Medium||Public interest profile: Higher|
|Low||Statistics of lower quality concern and lower public interest [A1]||Statistics of low-quality concern and medium public interest [A1/A2]||Statistics of low-quality concern and higher public interest [A1/A2]|
|Medium||Statistics of medium-quality concern and lower public interest [A1/A2]||Statistics of medium-quality concern and medium public interest [A2]||Statistics of medium-quality concern and higher public interest [A2/A3]|
|High||Statistics of higher quality concern and lower public interest [A1/A2/A3]||Statistics of higher quality concern and medium public interest [A3]||Statistics of higher quality concern and higher public interest [A3]|
|Source: UK Statistics Authority|
Download this table Table 2: Quality assurance matrix.xls (36.4 kB)
3.2 Quality Assurance of Administrative Data practice areas
We have aimed to assess the quality of each data source based on four broad practice areas. These relate to the quality assurance of official statistics and the administrative data used to produce them: our knowledge of the operational context in which the data are recorded, building good communication links with our data suppliers, an understanding of our suppliers’ quality processes and standards, and the quality processes and standards that we apply. This is in line with the Office for Statistics Regulations expectations for quality assurance of data sources. The full assessments for each data source can be found in Annex A. Table 3 provides a breakdown of these practice areas.
Table 3: Four practice areas associated with data quality
|Operational context and admin data collection||Communication with data partners||Quality Assurance principles, standards and checks by data suppliers||Producers’ Quality Assurance investigations and documentation|
|Environment and processes for compiling the administrative data||Collaborative relationships with data collectors, suppliers, IT specialists, policy and operational officials||Data assurance arrangements in data collection and supply||Quality assurance checks carried out by statistics producer|
|Factors which affect data quality and cause bias||Formal agreements detailing arrangements||Quality information about the data from suppliers||Quality indicators for input data and output statistics|
|Safeguards which minimise the risks||Regular engagement with collectors, suppliers, and users||Role of operational inspection and internal or external audit in data assurance processes||Strengths and limitations of the data in relation to use|
|Role of performance measurements and targets; potential for distortive effects||Explanation for users about the data quality and impact on the statistics|
|Source: UK Statistics Authority|
Download this table Table 3: Four practice areas associated with data quality.xls (28.2 kB)
4. Assurance level assessment
In this section we summarise our data sources, and consider the assurance level that we are seeking (or “benchmark”) for these. This is provided in Table 4:
Table 4: Benchmark assurance levels and assessment
|Risk or quality concern||Public interest profile||Benchmark Profile Assessment||Justification|
|MHCLG||Medium||High||Comprehensive||Some complexity in data due to various different sources being used in compilation.|
|Valuation Office Agency||High||High||Comprehensive||1) A relatively high weight in IPHRP. 2) No access to microdata.|
|Scottish government||Low||Medium||Enhanced||1) Access to microdata. 2) Lower weight.|
|Welsh government||Low||Medium||Enhanced||1) Access to microdata. 2) Lower weight.|
|Source: Office for National Statistics|
Download this table Table 4: Benchmark assurance levels and assessment.xls (37.4 kB)
The same rental data used in Index of Private Housing Rental Prices (IPHRP) is also used to construct the owner-occupiers housing component of Consumer Prices Index, including owner occupiers’ Housing costs (CPIH) under the rental equivalence approach. Within Annex A of the CPIH Quality assurance of administrative data can be found the detailed quality assurance practices applied to the Valuation Office Agency, Scottish Government, Welsh Government and Ministry of Housing Communities and Local Government data.Back to table of contents
5. Action plan
In the previous sections, we have considered quality assurance for all data sources in our Index of Private Housing Rental Prices. Of the data sources we investigated, there are some that need further work to reach the level of assurance we are seeking.
To address these shortcomings, we will carry out further steps to improve our quality assurance. All outstanding actions are summarised in Table 5 with details on what actions we intend to take to rectify them.
Table 5: Action plan
|MHCLG||Seek to understand MHCLG’s departmental approach to QAAD development, and where dwelling stock count data fits into this process.|
|Seek to establish a more robust delivery schedule, including annual supplier liaison meetings, and a Service Level Agreement for delivery.|
|VOA||We have drafted a business case to access the VOA microdata and are in discussions with the VOA to access the data through the Digital Economy Act.|
|Source: Office for National Statistics|
Download this table Table 5: Action plan.xls (27.1 kB)
This version of the Index of Private Housing Rental Prices (IPHRP) Quality Assurance of Administrative Data (QAAD) is intended to act as a progress update. Over the next few months, we intend to continue engaging with our data suppliers and, where appropriate, put in place firmer ongoing communications mechanisms and data delivery agreements. Importantly, this QAAD is not intended to serve as a final record of quality assurance. We view supplier engagement and feedback as an ongoing process, which we will continue to follow. We, therefore, intend to publish a review to this QAAD every two years.Back to table of contents
Contact details for this Methodology
Telephone: +44 (0)1633 456400