Table of contents
- Comparing ONS rents data and private sector data
- Comparison of Countrywide and IPHRP
- Comparison of Index of Private Housing Rental Prices with Zoopla rental data
- Comparison of Index of Private Housing Rental Prices with VOA private rents data
- Annex A: How change in composition can impact on average rents
The Index of Private Housing Rental Prices (IPHRP) measures the change in price of renting residential property from private landlords. The same private rent price indices are also used for the rental equivalence measure of owner occupiers’ housing costs (OOH) in Consumer Prices Index including owner occupiers’ housing costs (CPIH).
With CPIH becoming the lead measure within the consumer prices statistical bulletin and CPIH being re- designated as a National Statistic, the quality of private rental data is paramount. This is because OOH are based on a rental equivalence (RE) approach; specifying that the cost of housing services is estimated by looking at the cost of renting an equivalent property to the one you own.
This RE method is conceptually strong and the approach used in Germany and the US, but requires a large amount of good quality data to be implemented. OOH in the UK is based on rental data drawn primarily from the Valuation Office Agency (VOA) and consists of around 500,000 separate rental estimates per year, giving approximately 40,000 prices per month.
VOA is one of a range of sources of rental data across the UK, but is the one that provides the highest quality and most robust data for measuring consumer inflation. The remainder of this article assesses the merits of the different data sources, highlighting why the VOA data best meets the requirements for a monthly Consumer Prices Index.Back to table of contents
Superficially there is a considerable difference between the Office for National Statistics’ (ONS’s) measures of rents, the Index of Private Housing Rental Prices (IPHRP) and owner occupiers’ housing costs (OOH), and comparable indicators from the private sector. However, once adjustments are made for differences in what the indices are measuring, the ONS measure of rents is more closely aligned with the other sources of rental prices.
Some of the differences can be explained by compositional changes and quality changes in the stock of rental properties. Both are specifically excluded from IPHRP to ensure that only pure price change is captured in the index. Put another way, the aim of the IPHRP is to compare “like with like”.
In addition, there are methodological differences in the way rental prices can be measured. One approach is to measure the stock of rents; that is, aiming to capture the price of all properties in the rental market. The second approach is to measure the flow of rents, in other words, to capture the price of new lettings made in the reference period. Both approaches yield different results.Back to table of contents
3. Comparing ONS rents data and private sector data
A number of private sector organisations – estate agents and property companies – produce estimates of changes in rental prices. These include Countrywide PLC, HomeLet and LSL Property Services. In addition, other companies such as Zoopla are a rich source of rental data. More details about these and the Office for National Statistics’ (ONS’s) measure, the Index of Private Housing Rental Prices (IPHRP), are given in this section.
Index of Private Housing Rental Prices
Index of Private Housing Rental Prices (IPHRP) measures the change in price that tenants face when renting residential property from private landlords, thereby allowing a comparison between the prices tenants are charged in the current month as opposed to the same month in the previous year. The index does not measure the change in newly advertised rental prices only, but reflects price changes for all private rental properties.
The IPHRP is constructed using administrative data. That is, the index makes use of data that are already collected for other purposes to estimate rental prices. The sources of rental prices are the Valuation Office Agency (VOA), Scottish Government and Welsh Government. All three organisations deploy rental officers to collect the price paid for privately-rented properties. The sources of expenditure weights are the Ministry of Housing, Communities and Local Government (MHCLG), Scottish Government, Welsh Government and the VOA.
Countrywide has over 1,300 offices nationwide operating under locally-recognised brands. Countrywide regularly publishes a monthly new lets average rental price, which is based on 4,000 to 5,000 properties let in the reference month by letting agencies under the Countrywide umbrella. The data are weighted and stratified by regions and number of bedrooms. The regional weights are derived from the English Housing Survey (EHS) and the “number of bedroom” weights are calculated using Countrywide data. Countrywide also manages a portfolio of 75,000 properties and publishes an occupied lets and a renewal average rental price based on this portfolio of properties.
HomeLet is a private company that provides tenant referencing and insurance for letting agents, landlords and tenants. Since January 2009, it has published a UK new lets monthly index called the HomeLet Rental Index, which is based on achieved private rental prices collected from its tenant referencing services (around 29,000 per month). In August 2016, HomeLet re-launched this index incorporating a methodology that stratifies by property type and geography and a historic series was also made available.
LSL Property Services
LSL Property Services is a provider of residential property services, including residential lettings. Its estate agency arm trades under a number of brands. LSL has commissioned a monthly private housing rental index since January 2011 called the Buy-to-Let Index, which is based on around 7,000 new lets in a month, collected from the mix of rental services that it provides across England and Wales. The sample is stratified and weighted by “government regions” and by property type. The weights for regions are derived from the EHS, whereas property type weights are derived from LSL’s database of privately-rented properties spanning a number of years.
Zoopla is a property website covering the residential property market, which focuses on providing users with the resources they need to make better-informed property decisions. Launched in 2008, the Zoopla website is a searchable directory of UK residential properties and is one of the leading online destinations for property consumers to search for homes and do their market research.
Tenancy deposit schemes
Landlords must put tenant deposits in a government-backed tenancy deposit scheme if the home is rented on an assured shorthold tenancy that started after April 2007. In England and Wales deposits can be registered with; Deposit Protection Service, MyDeposits and Tenancy Deposit Scheme.
In November 2017 the Deposit Protection Service launched their own rent index which they plan to publish quarterly. The series is constructed using the rental transactions it has collected over the last 10 years. A regression is used to control for changes in the mix of properties over time using; property type, number of bedrooms, furnished status, ACORN and parliamentary constituency as characteristics.
All of these sources provide an extensive range of rental data, but a closer look at the various rental measures published by them over recent years provides a diverse picture, as shown in Figure 1.
Figure 1: Index of Private Housing Rental Prices and private sector measures of rents percentage change over 12 months, January 2013 to March 2018, UK
Source: Homelet, LSL Property Services PLC, Countrywide plc and Index of Private Housing Rental Prices
The Countrywide new lets series is now calculated on a three-month rolling average rather than lets agreed in the last month.
LSL claim that the strong growth in their series for August 2016 was driven by student check-ins.
Download this chart Figure 1: Index of Private Housing Rental Prices and private sector measures of rents percentage change over 12 months, January 2013 to March 2018, UKImage .csv .xls
It is notable that historically the rent increases measured by the private sector have tended to be higher than those shown in IPHRP. Also, the private sector measures are more volatile, but this can be explained further in this section.
This higher level and volatility of the private sources can be mainly explained by what the different data are showing. Firstly, almost all the private sector measures primarily focus on newly let properties, whereas IPHRP picks up a mixture of newly let properties and existing lets (Countrywide have previously covered both, but Figure 1 just shows their new lets series).
This is important, as evidence obtained from VOA rental officers suggests that the greatest price rises occur when properties are newly let, compared with existing tenants, who tend to see smaller price increases.
In early 2016, the VOA placed a question in the BDRC Landlord Panel survey whose respondents comprise National Landlord Association members with a wide range of portfolio sizes and geographies. The question asked “What was the typical level of rent you were able to achieve?” broken down by “new tenants” and “renewals with existing tenants”, in 2015. It was reported that 23% of new lets were charged at the same rate as the previous letting, however, 46% of renewals saw no increase. For those whose rent did increase, the average increase for new lets was 5.0% while for renewals this was lower at 3.4%.
To summarise, landlords were less likely to increase rents for existing tenants, and when rents were increased, it was by a smaller percentage. This is thought to happen because landlords make a considered business decision, preferring a small increase from a known reliable tenant, rather than experiencing a void period with no income and the associated cost of re-letting. Consequently, larger price increases tend to occur when the existing tenant moves out and the property is advertised.
In addition, analysis of contract length statistics finds that rental contracts tend to last for around 12 months. During this contract period a tenant’s rent is unchanged. IPHRP, as a measure of the stock of rental properties, includes all new lets, those renewing their contract and those within their existing contract. This results in a smoother series than those that simply focus on new lets, where there is more volatility between each month.
So, differentiating between new and existing lets is crucial to analysing the differences. Unfortunately, not all the private sector providers separate out these, which makes it more difficult. However, analysis has been possible with one of the private sector data sources, namely by ONS on the Countrywide data.
Looking at the analysis of Countrywide data, Figure 2 shows that historically, newly let properties tend to have both higher and more volatile growth patterns compared with its “occupied lets” series, which captures the position for existing tenants. During 2017, growth in newly let properties is likely to have been lower than the “occupied lets” series due to a fall in rent prices in London.
Figure 2: Countrywide new lets and occupied lets percentage change over 12 months, January 2014 to February 2018, UK
Source: Countrywide PLC
The new lets series is now calculated on a three-month rolling average rather than lets agreed in the last month.
Countrywide are currently reviewing the methodology for their occupied lest series, as such, more recent estimates are not currently available.
Download this chart Figure 2: Countrywide new lets and occupied lets percentage change over 12 months, January 2014 to February 2018, UKImage .csv .xls
This leads onto a few important questions – what is the percentage split between new and occupied lets in the Countrywide data and more generally in the UK as a whole? And then how does this compare with the make-up of the IPHRP dataset?
Countrywide data broadly consists of a 90 to 10 split between occupied lets and new lets. The split within the VOA series is not currently available, although plans are in place to address this through transformation of their data collection system. We are currently discussing this with VOA and once the split is available, it will be incorporated in future analysis. In the meantime, as the raw data collected and used in IPHRP is representative of the full range of local market rents in each broad rental market area, it seems acceptable to assume that the IPHRP series consists of a broadly similar split.
So, on the basis of this assumption, there are three analyses to consider:
a growth comparison of the Countrywide occupied lets series with the all-IPHRP dataset; whilst this is strictly not comparing “like with like” (as the IPHRP also includes new lets), it is a reasonable assumption to start with, to assess overall quality comparisons between the two datasets
repeat the first point but expressing both series as an index
undertake a hedonic regression with all-Countrywide (both new and occupied lets) data, to enable a comparison with the all-IPHRP dataset; this will create more of a “like with like” basis for the comparison
The results of this are shown in Section 4.Back to table of contents
4. Comparison of Countrywide and IPHRP
Figure 3 compares the Countrywide occupied lets series with the all-Index of Private Housing Rental Prices (IPHRP) dataset (which is a mixture of new and occupied lets). Whilst they are on a different basis, the two datasets follow a very similar pattern. This is not that surprising, as occupied lets account for over 90% of the stock. Some other small disparities will also still remain, which are caused by other differences between the two sources, such as coverage and the methodology applied.
On coverage, Countrywide only covers properties that are let or managed by them, so whilst being a large data source, they don’t necessarily have data for all areas, for example, they might not have data for all local authorities. While Countrywide use data from the English Housing Survey (EHS) to weight their transactions at a regional level, an adjustment is not made below this level. This may cause a bias in the estimates for a region if the data collected does not appropriately represent that of rental properties for the region.
IPHRP is also weighted at the regional level, but biases below the regional level are minimised in IPHRP by the large sample size of the Valuation Office Agency (VOA) data. Rent officers aim to keep this sample representative within each broad rental market area by using various data sources (such as the census) and their local knowledge of the rental sector. Further information of the VOA data collection can be found within the article published in January 2015, Improvements to the measurement of owner occupiers' housing costs and private housing rental prices.
As for methods, IPHRP takes a sample at the start of the year, which is followed for a year to calculate a price index, whereas Countrywide follows the properties they manage and calculates an average price. There are also some differences in the weights applied, although these have a minimal impact, as essentially both are derived from Ministry of Housing, Communities and Local Government data (more details on the weighing can be found in Section 3).
Figure 3: Index of Private Housing Rental Prices and Countrywide occupied lets percentage change over 12 months, January 2013 to March 2018, UK
Source: Countrywide PLC and Index of Private Housing Rental Prices
- Countrywide are currently reviewing the methodology for their occupied lets series, as such, more recent estimates are not currently available.
Download this chart Figure 3: Index of Private Housing Rental Prices and Countrywide occupied lets percentage change over 12 months, January 2013 to March 2018, UKImage .csv .xls
Whilst the annual growth rates of both series are very similar for the data available, how do the levels of the series compare? The two series cannot be compared in monetary terms, as IPHRP is only published as an index; however, the Countrywide data can also be expressed as an index, so that a comparison can be made. Figure 4 again compares the Countrywide occupied lets series with IPHRP, with both series expressed as an index referenced to January 2013 equals 100.
Figure 4: Index of Private Housing Rental Prices and Countrywide occupied lets indices, January 2013 to March 2018, UK
Source: Countrywide PLC and Index of Private Housing Rental Prices
- Countrywide are currently reviewing the methodology for their occupied lets series, as such, more recent estimates are not currently available.
Download this chart Figure 4: Index of Private Housing Rental Prices and Countrywide occupied lets indices, January 2013 to March 2018, UKImage .csv .xls
The comparison shows minimal drift between the series and therefore Figures 3 and 4 indicate a positive outcome in showing that the datasets follow a similar levels trend. However, a better “like with like” comparison would be beneficial; to reinforce the conclusion that IPHRP is correctly measuring the stock of private rents.
This can be achieved by undertaking a hedonic regression on the full set of Countrywide data and comparing this with the IPHRP dataset (which covers all rents). For this, Countrywide transactions data have been used, which consists of three separate datasets:
new lets – a property let with a new tenant moving in (around 4,000 to 5,000 properties)
renewals – a property where the tenant’s contract is expiring and the tenant agrees a new contract (around 6,000 properties)
occupied lets – all the properties with active tenancies; these include historic new lets and all renewals (around 75,000)
A hedonic regression similar to that used in the production of the UK House Price Index has been developed by methodologists in Office for National Statistics (ONS). Further information on the method used can be found in Annex A of the article Comparing measures of private rental growth in the UK: January to March 2017, where this analysis was first presented. Given that the Countrywide occupied lets data are only available as a full year from 2014, this hedonic regression series can only currently be calculated for 2015. The resulting hedonic price index reflecting the full stock of rental properties (new and existing lets) is presented against IPHRP in Figure 5.
Figure 5: Index of Private Housing Rental Prices and Countrywide (new and existing lets, hedonic regression) indices, January 2015 to December 2015, UK
Source: Countrywide PLC (Office for National Statistics calculations) and Index of Private Housing Rental Prices
Download this chart Figure 5: Index of Private Housing Rental Prices and Countrywide (new and existing lets, hedonic regression) indices, January 2015 to December 2015, UKImage .csv .xls
Figure 5 shows that the two series, whilst based on different sources, produce an index with a similar trend pattern, albeit with slightly greater disparity in the summer months.
Therefore, when considering this latest chart (Figure 5) and the earlier comparisons with the Countrywide occupied lets data, the IPHRP shows very similar patterns – thus providing additional confidence in the IPHRP as a robust measure of the growth in the stock of rental properties.Back to table of contents
5. Comparison of Index of Private Housing Rental Prices with Zoopla rental data
Office for National Statistics (ONS) obtained rental data from Zoopla Property Group PLC through the Urban Big Data Centre. Focusing initially on flats within London, average rental prices in each London borough were calculated and aggregated together using the previous year’s volume of transactions as weights; this method is known as mix-adjustment. As the Index of Private Housing Rental Prices (IPHRP) is a stock measure and for comparability purposes, a 14-month moving average was also applied to the Zoopla data to reflect the average length of a contract. This also aided in removing some of the volatility in the series. Figure 6 presents the resulting 12-month growth rate (lagged by four months) against IPHRP.
Figure 6: London flats, Zoopla rental data (lagged by four-months) against Index of Private Housing Rental Prices, percentage change over 12 months, June 2014 to June 2017, UK
Source: Zoopla Property Group PLC (2016), Zoopla Historic Data (UK to 2017), Urban Big Data Centre, Index of Private Housing Rental Prices
A 4 month lag was chosen as it had the highest correlation
An earlier Zoopla series could not be constructed due to data limitations
Download this chart Figure 6: London flats, Zoopla rental data (lagged by four-months) against Index of Private Housing Rental Prices, percentage change over 12 months, June 2014 to June 2017, UKImage .csv .xls
Figure 6 shows that from around January 2015, trends in the two series are very similar. A divergence can be seen prior to this, which may be due to fewer transactions in the Zoopla sample data provided in earlier periods affecting the quality of the estimate in this period.
The constructed Zoopla series leads that of the IPHRP by around four months. There are a few possible reasons for this, which include:
advertised lets – Zoopla data are based on advertised lets, which is likely to be ahead of when the property is initially let
collection date – the IPHRP raw data represent when the data were collected, which may differ to when the property was initially let
While additional analysis is required to explore these areas further, this initial analysis contributes to the other evidence presented in this article in providing additional confidence to IPHRP being a robust measure of private rental growth. Further analysis and development will be conducted on this Zoopla data source. We are currently looking to acquire additional Zoopla data so that this time series can be updated to more recent periods and apply similar approaches for other geographies.Back to table of contents
6. Comparison of Index of Private Housing Rental Prices with VOA private rents data
In evaluating measures of rental price, some users have focused on the difference between the average rental prices published by the Valuation Office Agency (VOA), as part of their private rental market statistics (PRMS) publication, and the Index of Private Housing Rental Prices (IPHRP). Both are based on the same underlying private rents data collected by VOA rent officers for England.
Figure 7 presents the 12-month growth rates for the VOA PRMS series and IPHRP (using the England IPHRP series for comparability). Two additional series are also presented, which adjust for changes in the composition of the VOA PRMS sample over time.
Figure 7: Valuation Office Agency private rental market statistics adjusted for compositional change against Index of Private Housing Rental Prices, percentage change over 12 months
Source: Valuation Office Agency, Private Rental Market Statistic and Index of Private Housing Rental Prices
- Analysis of compositional change was conducted at the local authority level using published data from the VOA PRMS.
Download this chart Figure 7: Valuation Office Agency private rental market statistics adjusted for compositional change against Index of Private Housing Rental Prices, percentage change over 12 monthsImage .csv .xls
There are clearly some major differences between the VOA PRMS series and IPHRP. This raises the question; “How can we have confidence in IPHRP if it diverges so much from VOA private rents series?" Well, these differences can be explained.
IPHRP is a “price index”, in that it seeks to make pure price comparisons, whereas VOA’s PRMS is a simple “average of transactions” collected during the period – the latter is designed to provide a “snapshot” of the rental market over the previous 12 months.
Over the period presented, the average annual growth rate of the VOA PRMS is 3.6% compared with an average annual growth rate in IPHRP of 2.2%. The difference of 1.5 percentage points can be accounted for as follows:
geographical compositional effects (0.7 percentage points) – as shown by the lightest (yellow) line in Figure 7
compositional changes relating to the number of bedrooms of properties in the VOA PRMS sample (0.4 percentage points) – as shown by the darker (blue) line in Figure 7
coverage differences between PRMS and IPHRP (0.2 percentage points) – as shown in the article Explaining private rental growth
The remainder (around 0.2 percentage points) of the difference can be explained by:
the application of price index methodology to calculate price indices (a matched sample and mix adjustment); this is discussed further in the article Explaining private rental growth
changes in composition below the local authority level
a general improvement in the quality of the private rental sector – changes in quality would impact an average price measure but are intentionally removed from a price index; in recent times, the size of the privately-rented housing stock has more than doubled and some of this supply has come from the owner-occupied market, which is generally in better repair than the rental market, evidence from the English Housing Survey suggests that rented properties are now better maintained than they were a few years ago
We are therefore confident that we understand and can explain the differences between the tracks of the IPHRP and VOA PRMS, and are confident that IPHRP is a fair reflection of the increase in rental prices over time. Additional work will be taken forward to further break down and explain the remaining difference.
A simple example illustrating how compositional change impacts an average price measure is available in Annex A.Back to table of contents
The analyses in this article have shown the wide range of sources available for rental data and highlighted the extent of similarities, and differences, that exist between them. This has all pointed to the Index of Private Housing Rental Prices (IPHRP) being a high-quality and robust measure that is fit for purpose in measuring UK consumer price statistics.
However, there is still an ongoing requirement to ensure that IPHRP keeps abreast of any developments in the private rental market and that the statistics keep pace with the rapidly developing big data agenda. Section 7 of Comparing measures of private rental growth in the UK: January to March 2017 provides further details on some of the aspects of the work programme that provide the ongoing assurance and development of rental price statistics.Back to table of contents
8. Annex A: How change in composition can impact on average rents
Section 6 referred to change in the composition of the Valuation Office Agency (VOA) sample, but why does this matter? Well, this has a large impact on the “average rents” measure, in increasing it. However, there is not a similar impact on the index, as this does not happen with a price index measure – these effects are intentionally excluded as the index aims to compare “like with like”. A simple illustration of this is presented in Table 1.
Table 1: Example showing how compositional change can impact on average rents
|Local Authority 1||Local Authority 2|
|Rent||No of records||Rent||No of records||Simple average rent|
Download this table Table 1: Example showing how compositional change can impact on average rents.xls (26.6 kB)
Table 1 shows two local authorities (LAs) – LA1 has a monthly rent of £500 whereas LA2 has double this, at a monthly rent of £1,000.
The monthly rent is the same in both years 1 and 2 for each of the two LAs. However, the relative weights between the two years have changed, with both LAs accounting for 50% of the total market in year 1, but the split between LA1 and LA2 changes to 40% to 60% respectively in year 2.
The weighted average rent has therefore increased from £750 in year 1 to £800 in year 2 (a 6.7% increase), all due to a change in composition in the population. However, there has been no actual increase in the rental prices, so a rental price index would show no increase in prices between periods 1 and 2, whereas the average rents approach would show an increase.
The article Explaining private rental growth includes historic analysis on changes in the composition of the VOA private rental market statistics (PRMS) sample and its impact on the resulting series.Back to table of contents
Contact details for this Article
Telephone: +44 (0)1633 456400