Transformation of consumer price statistics: July 2023

• We are undertaking a programme of transformation across our consumer price statistics, including identifying new data sources, improving methods, and developing systems to improve both the Consumer Prices Index including owner occupiers' housing costs (CPIH) and the Consumer Prices Index (CPI).

• We have now successfully implemented the first changes into headline statistics, starting with rail fares transaction data from Rail Delivery Group, included in our Consumer price statistics: February 2023 bulletin (published in March 2023).

• These developments are part of a continuous programme of improvement, updating our timelines and priorities for transformation; in 2024, we aim to include alternative, big data for second-hand cars and private rents, and in 2025 our focus will be on groceries.

• Our priority is the quality of consumer price statistics; our timelines are subject to continued systems development, research and impact analysis, and any decisions will be made through continuous engagement with our stakeholders and users.

• To assist the transformation of our headline statistics, CPIH and CPI, some changes may be proposed for Retail Prices Index (RPI); any proposed change to the RPI will go through the usual governance process (for more information, see Appendix 2 of our Consumer Prices Indices Technical Manual, 2019).

Accurate measures of inflation are, more than ever, playing a vital role in business, government, and everyday life. From rail fares to taxes and pensions, financial transactions and many other areas of our lives are regularly adjusted to reflect the change in prices over time. It is crucial to measure these changes in price as accurately as possible and continue to improve our statistics over time in line with latest research and responding to market trends.

We are currently undergoing transformation across many areas of our consumer price statistics, including identifying new data sources, improving methods, and developing statistical processing systems. This allows us to reflect our changing economy and produce more robust and granular inflation statistics for businesses, individuals, and government.

For the first time, in March 2023 we published headline consumer price statistics including alternative data sources, marking a major milestone in this transformation process. Using rail fares transaction data from Rail Delivery Group, we published six new indices based primarily on ticket type, improving the granularity of information we provide, as detailed in our Impact analysis on transformation of UK consumer price statistics article.

In our previous Transformation of consumer price statistics: April 2022 article, we had also planned to include web-provided data on second-hand cars in March 2023. We have been clear that our highest priorities are improving the quality and upholding the integrity of our statistics, and that our timelines are subject to continued research and quality-assurance. For second-hand cars, the quality assurance at the end of 2022 highlighted challenges that could have affected the reliability of the statistical production process. We are now addressing these throughout 2023. Therefore, we plan to incorporate the new second-hand cars data into headline consumer price statistics in 2024. For details on this decision, see our Impact analysis on transformation of UK consumer price statistics article.

Our timelines and plans are detailed in Section 6: Future developments.

In the future, our consumer price statistics will be a mix of traditionally collected data, existing administrative data, and the new data sources outlined in our previous Transformation of consumer price statistics: April 2022 article.

Traditionally collected price data and our existing administrative data sources will be used when they cannot be (or have not yet been) replaced by alternative data, such as for small independent shops who do not have a website.

Alternative data sources provide many benefits compared with our current sources, including improved product and geographic coverage, higher frequency of collection and additional information, such as expenditure per product. By adopting multilateral index number methods, we can make better use of these data sources and account for dynamic changes in the market and consumer spending habits, as outlined in our article, Introducing multilateral index methods into consumer price statistics.

Many types of data provide more detailed information so that we can produce additional, more granular indices, as we have done with rail fares, and will be able to do with second-hand cars. These additional indices are outlined in our Impact analysis on transformation of UK consumer price statistics article.

Through the richness of information that is increasingly available to us, we are not only improving our headline measures of inflation, but also better informing the narrative around the drivers of inflation.

These new data sources will result in hundreds of millions more price quotes being processed each month, with a richness and granularity of data that was not previously available. To maximise the benefit from these data, we will also be making changes in our methodologies and systems.

For any changes in methodologies we have undertaken, and will continue to undertake, extensive research and stakeholder engagement. We publish our Research and developments in the transformation of UK consumer price statistics articles series, as well as the papers and minutes of our discussions with the Advisory Panels for Consumer Prices (Stakeholder and Technical) on the UKSA website.

Systems changes are also required to integrate these data as part of our consumer price index production. We have used cloud infrastructure, in line with the Government Digital Service Cloud Strategy, and best practice in line with the Reproducible Analytical Pipelines (RAP) strategy defined by the UK Government Analysis Function (GAF). We have built our production systems ensuring flexibility so that future categories of data can be incorporated into the production with minimal additional development.

The Rail Delivery Group data for rail fares is already processed within this new system, and the indices produced are then incorporated into consumer prices production legacy systems.

By 2025, our system to process traditional data will also be updated to enable the introduction of grocery scanner data. The indices using new data and traditional data will need to be integrated at the lowest level to enable full market coverage.

Further details of our systems development can be found in on the United Nations Economic Commission for Europe (UNECE) website, under Meeting of the Group of Experts on Consumer Price Indices: Developing reproducible analytical pipelines for the transformation of consumer price statistics: rail fares.

The changes outlined are part of a continuous programme of improvement for consumer price statistics. Our ambition is to bring in new data sources for further categories and continue to improve consumer price statistics over the coming years. Our priority is the quality of consumer price statistics; our timelines are subject to continued systems development, research and impact analysis, and any decisions will be made through continuous engagement with our stakeholders and users.

Throughout each phase, we will be liaising regularly with our Advisory Panels for Consumer Price Statistics, our users, and the Office for Statistics Regulation. This is to ensure that our plans for consumer price inflation measurement are appropriate for improving the quality of our statistics and meeting our ongoing user requirements.

In Quarter 1 (Jan to Mar) 2024, we plan to include web-provided data for second-hand cars, as we aim to have resolved challenges with the production systems for second-hand cars (see Section 2: Overview of the transformation of consumer price statistics).

Also in Quarter 1 2024, we plan to include administrative microdata on private rents for Great Britain. Private rental prices statistics are used to inform the owner occupiers' housing (OOH) costs element of the Consumer Prices Index including owner occupiers' housing costs (CPIH), the Office for National Statistics' (ONS) lead measure of consumer prices inflation, as well as the "actual rentals for housing" aspect of the Consumer Prices Index (CPI) and CPIH. More information on the redevelopment plans for private rental prices statistics can be found in our Private rental prices development plan article.

These will be followed by groceries scanner data for food and non-alcoholic beverages, alcohol and tobacco in Quarter 1 2025. Our system to process traditional data will also be updated at this time to enable this introduction, as indices using new data and traditional data will need to be integrated at the lowest level to enable full market coverage. We also plan to include administrative microdata on private rents for Northern Ireland by March 2025.

Following feedback from stakeholders on our communications, we have agreed a forward plan for 2023 as follows:

• in July 2023, we will publish additional methodological articles regarding second-hand cars and groceries, as part of our Research and developments in the transformation of UK consumer price statistics articles series

• in November 2023, we will produce an impact analysis release, covering data for second-hand cars and private rents for Great Britain up to July 2022, published following feedback from the Advisory Panels for Consumer Prices in October

• in Quarter 1 2024, we aim for these new estimates for second-hand cars and private rents for Great Britain to be incorporated in our Consumer price inflation, UK statistical bulletins

We continue to gather feedback from users on this approach and our communications from users and ensure we are best meeting user and stakeholder needs.