This article presents some important methodological factors that can affect comparability between the Office for National Statistics (ONS) Retail Sales Index (RSI) and major external surveys on the retail industry. In this effort, the article also explores methodological differences between the ONS RSI and the BRC-KPMG UK Retail Sales Monitor (RSM), a prominent industry survey. An overview of discrepancies in survey outputs over time is also provided.
It is shown that differences between the two surveys arise in the following areas:
- sample size and coverage
- classification and conceptual differences
- weighting and estimation
Sample size and coverage
In drawing the ONS RSI sample, approximately 900 large and 4,100 small- to medium-sized retailers are selected using stratified random sampling. A sample of approximately 95 predominantly large businesses is taken for the BRC-KPMG RSM, based on BRC membership.
Classification and conceptual differences
ONS RSI is compiled and reported by retail industry in line with the UK Standard Industrial Classification 2007: SIC 2007 framework, while the BRC-KPMG RSM is classified by types of product sold. Using the SIC 2007 allows the ONS to separate retail sales from other services provided by retailers.
Weighting and estimation
In deriving total retail sales estimates, survey responses in the ONS RSI are weighted using data from the Inter-Departmental Business Register, such as annual registered turnover and business population counts. This allows ONS to aggregate for different types and sizes of retailer. In contrast, BRC-KPMG RSM is weighted by consumer spending on different types of products to derive the movement of total retail sales.
While selected outputs from the two surveys broadly track each other well, discrepancies persist over time. These are likely to arise from methodological differences. In addition, the headline figures from the two surveys are different: the ONS RSI figures are for quantities of goods on a seasonally adjusted basis, while the BRC-KPMG RSM headline is for value (price multiplied by quantity), non-seasonally adjusted.Back to table of contents
Following the EU referendum on 23 June 2016, an increased interest in the performance of the UK economy has led to greater interest in external surveys by economic commentators and the media. Looking at recent coverage of the retail industry, figures from external surveys are often reported alongside the official Office for National Statistics (ONS) Retail Sales Index (RSI) data.
When comparing the ONS RSI to external surveys on the retail industry it is important to assess any methodological differences that may impact comparability. Such differences may bear important implications for the interpretation of respective figures, as well as giving rise to discrepancies among the outputs concerned.
How are external surveys useful?
The RSI is an important economic indicator and is one of the most timely short-term measures of economic activity. It is used to estimate consumer spending on retail goods and the output of the retail industry, both of which are used in the compilation of the National Accounts.
Naturally, the RSI as well as external surveys relating to the retail industry are at the forefront of public and media interest in assessing the UK economy in the short term.
Two prominent private industry surveys that have been compared with the ONS RSI are the BRC-KPMG UK Retail Sales Monitor (RSM) and the CBI Distributive Trades Survey (DTS). External surveys can help supplement official ONS data for the following reasons discussed in this section.
The ONS RSI is usually published around 14 working days after the end of the month. In contrast, the BRC-KPMG RSM is usually published around seven working days after the end of the reference month, while the CBI DTS is published a few working days before the end of the month. Consequently, these private surveys can be valuable tools for providing an early indication of activity in the retailing industry.
The CBI DTS reports company expectations for variables such as sales and orders. These can be used to make inferences about the future prospects of the retailing industry. The ONS RSI does not include any forecasting or expectations.
External surveys measure a variety of variables that are not covered in ONS data. For example, the CBI DTS reports data on investment intentions for the year ahead, while the BRC-KPMG RSM provides detailed estimates on spending by product.Back to table of contents
Recent coverage of external surveys on the retail industry has highlighted discrepancies between recent Office for National Statistics (ONS) Retail Sales Index (RSI) estimates and the BRC-KPMG UK Retail Sales Monitor (RSM) in particular; this will be the focus for the rest of this article.
Following previous articles on the methodological differences among RSI, RSM and other external surveys1,2, this section provides an updated account of potential sources of discrepancy between the RSI and the RSM based on the latest available information.
Overview of differences in methodology
The main methodological differences between the RSI and RSM are listed in Table 1. A comprehensive list was also produced by Palmer (2007), which provides some additional detail on specific differences.
Table 1: Main methodological differences between ONS RSI and BRC-KPMG RSM
|Criterion||ONS RSI||BRC-KPMG RSM|
|Sample review||Sample updated each month, in accordance with the profile of the British retail industry||Sample related to BRC membership profile|
|Enforcement||Statutory survey||Voluntary survey (consistently high response).|
|Headline retail sales measures obtained||Values and volumes, seasonally adjusted and non-seasonally adjusted||Changes in total, like-for-like and online sales values compared with a year ago, no seasonal adjustment|
|Data Collection||Monthly sales||Weekly sales|
|Weighting method||Survey data weighted by sales for each type of retailer||Survey data covering thirteen product categories and ten online categories weighted by household expenditure on each type of product|
|Classification||Based on type of retailer, as defined by the Standard Industrial Classification (SIC). Turnover from the sale of services is not included.||Based on product categories. Can include sales from retail and catering services.|
|Revisions||Results continuously open to revision from late or amended data||No revisions. Annual growth rates derived from up-to-date comparison with the previous year reflecting any changes that affect the like-for-like figures|
|Automotive fuel sales||Included||Excluded|
|Source: Office for National Statistics, British Retail Consortium|
Download this table Table 1: Main methodological differences between ONS RSI and BRC-KPMG RSM.xls (37.4 kB)
Sample size and coverage
Retail sales data for the RSI are collected from a sample of approximately 5,000 retailers across Great Britain. Retailers in the RSI sample cover roughly 93% of retail turnover registered on the Inter-Departmental Business Register (IDBR). The IDBR is a comprehensive list of UK businesses used by government for statistical purposes.
Table 2 shows that, despite generally featuring higher response rates, the BRC-KPMG RSM has a smaller sample size, therefore representing fewer total responses than the official ONS data.
Table 2. Sample size and response rates for ONS RSI and BRC-KPMG RSM
|ONS RSI||93 per cent of total UK retail sales value (~5,000 companies)||87 per cent turnover response rate (61 per cent questionnaire response rate each month)|
|BRC-KPMG RSM||60 per cent of total UK retail sales value (~95 companies)||99 to 100 per cent response rate on a weekly basis across thirteen product categories|
|Source: Office for National Statistics, British Retail Consortium|
|1. Based on BRC membership list (accessible at https://brc.org.uk/join-brc/retailers )|
Download this table Table 2. Sample size and response rates for ONS RSI and BRC-KPMG RSM.xls (36.4 kB)
The BRC-KPMG RSM is a voluntary survey and its sample is formed by 95 predominantly large retailers, the majority of whom are included in BRC membership. As a result, despite representing a good proportion of total UK retail sales value, a more limited number of businesses are represented, particularly in small- and medium-sized business.
ONS is currently undertaking a programme of transformation to deliver improvements to the UK’s economic statistics. This programme aims to make use of administrative data and apply innovative approaches to enhance the quality of existing outputs. In the RSI, the use of turnover data collected by HM Revenue and Customs (HMRC) for the purposes of Value Added Tax (VAT) is explored as an alternative source in deriving turnover estimates for small- and medium-sized retail businesses.
Taken as a whole, VAT turnover data are a census of all businesses registered to pay VAT (at April 2017, VAT threshold is £85,000), therefore showing significant potential for enhancing the RSI. More detail can be found in the recently published Transforming short-term turnover statistics article.
Classification and conceptual differences
A further distinction between the RSI and the RSM is their reliance on different classification systems. The ONS RSI adheres to UK Standard Industrial Classification 2007: SIC 2007, which constitutes an agreed international standard for the comparability of official statistics. Under SIC 2007, a separation of the retail industry from the service sector is achieved. Particularly, under UK SIC 2007, retail trade is defined as “the resale (sale without transformation) of new and used goods mainly to the general public for personal or household consumption or utilisation, by shops, department stores, stalls, mail-order houses, door-to-door sales persons, hawkers and consumer co-operatives among others.”
For instance, businesses offering catering or retail support services are not consistent with the previous definition of retail sales and are therefore captured within the ONS Index of Services instead of the RSI. Companies of this type are present within BRC membership, therefore potentially contributing to some discrepancies between the RSM and the RSI official figures, though the contribution of these to RSM total value is expected to be small. Similarly, divergence may also arise from the fact that retailers in the ONS RSI sample are required to exclude non-retail turnover from provided figures in line with the previous definition.
Moreover, as RSI figures are classified by SIC 2007, they refer to retail sales by store type. This represents a notable conceptual difference to the categorisation of RSM figures by product type. Specifically, RSI collects total turnover data from sampled retailers, which are then aggregated for retailers of the same classification. For instance, the RSI will report on the overall performance of clothing retailers irrespective of their sales in individual commodities such as garments, accessories or footwear.
In contrast, the RSM reports on the sale of specific products as opposed to the overall performance of different store types. This has significant implications for comparability between the industry and commodity breakdowns of RSI and RSM respectively, as well as the weighting applied on the data in deriving aggregate estimates of total retail sales growth.
Weighting and estimation
The BRC-KPMG RSM uses figures from the ONS Living Costs and Food Survey published in the ONS Family spending in the UK publication, to provide the weighted aggregate movement of the different product categories covered. Weighting is performed with the aim to reflect the contribution of each product category to the growth of total retail sales, making aggregate figures representative of UK retail sales as a whole. Therefore, the weights applied on RSM pertain to the relative spending of households on different product categories.
In contrast, RSI estimates are derived by weighting survey responses using data from the IDBR such as registered turnover and business counts, aggregated by the size and type of retailer as defined by SIC 2007. More information about the estimation method employed can be found in the Retail Sales Index Quality and Methodology Information report.
Consequently, discrepancies in the movements shown by the two surveys may arise from the fact that the movement of total retail sales value in RSM will be affected by relative spending on specific products; whereas RSI will be based on population and turnover differences in the retail industries covered.
Notes for What are the differences between ONS RSI and BRC-KPMG RSM?
Palmer, N (2007), Comparing the ONS’s Retail Sales Index with the BRC’s Retail Sales Monitor, Office for National Statistics, Newport
For more information, please see the Retail Sales Index (RSI) and Other Surveys methodological note.
Large businesses are defined as having over 100 employees or a registered annual turnover of over £60 million and between 10 to 99 employees.
Small and medium businesses have 0 to 99 employees with an annual turnover of less than £60 million.
More information on estimation and sampling methods can be found in the Retail Sales Quality and Methodology Information
Given the methodological differences discussed in the previous sections, the most appropriate measure to compare with the BRC-KPMG UK Retail Sales Monitor (RSM) total sales series is the Retail Sales Index (RSI) retail sales value (current prices), non-seasonally adjusted and excluding automotive fuel. Figure 1 shows the year-on-year percentage change in the RSI current prices series (excluding fuel) and the RSM’s total sales growth series.
Despite some divergence, there is a clear relationship between the two series in the long term. Prior to 2010, growth in the RSI non-seasonally adjusted retail sales value estimate is shown to be generally lower than the RSM, while mostly remaining higher in subsequent periods. Looking at recent periods, the gap between the growth of the two outputs has shown some increase, however the scale of discrepancies is not unprecedented (Figure 2).
From January 2002 to December 2017, approximately 80% of differences in the year-on-year growth of the RSI for all retailers and RSM total value fell in the range of negative 3.0 to positive 2.9 percentage points (Figure 3).
Given the differences in sample coverage discussed previously, the RSI for large retailers (excluding automotive fuel) can be considered as a more appropriate series for comparison with the RSM total sales series than the RSI for all retailers. This comparison would be particularly relevant in removing the differing coverage of small- and medium-sized businesses in the two surveys.
As seen in Figure 4, growth in the RSI for large retailers generally tracks the RSM better that the RSI for all retailers. While recent and historical discrepancies between these two series tend to be smaller, some divergence persists over time (Figure 5).
Looking at the gap between year-on-year growth in the RSI for large retailers and the RSM from January 2002 to December 2017, roughly 80% of differences fall within the range of negative 1.0 to positive 2.9 percentage points (Figure 6).
Consequently, while accounting for the differing coverage of small- to medium-sized retailers reduces overall divergence, other methodological differences continue to drive discrepancies.
Looking at the RSM series, several spikes and troughs in year-on-year growth can be observed. This can also be seen in the RSI series for large retailers for certain periods. Measuring growth in a given month relative to the same month a year ago is a good indicator of the trend in retail sales, however, monthly comparisons can sometimes be erratic. In such cases, year-on-year growth on a three-month basis can provide a better indication of the underlying trend.
Figure 7 exhibits year-on-year growth on a three-month basis (three months leading to the indicated month compared with the same period a year ago) for the RSI large retailer series and RSM total value. It is also noted that while the monthly accounting periods used by the RSI and the RSM are typically the same (with a four-, four- and five-week pattern), there was some divergence in June 2005 and July 2005 that could have partly accounted for discrepancies in these months1.
The RSI for large retailers and the RSM track each other well when it comes to the underlying trend. While discrepancies persist over time, these tend to be less volatile than in the respective year-on-year growth series (Figure 8).
Looking at the distribution of differences in the year-on-year growth of the RSI against the RSM on a three-month basis between March 2002 and December 2017, it is shown that approximately 75% of differences fall in the range of negative 1.0 to positive 1.9 percentage points (Figure 9). Similarly, 65% of the differences between the respective monthly year-on-year growth series fall in the same range. While growth in the RSI has tended to be higher than the RSM in both cases, differences are generally distributed close to zero, indicating a broadly good fit.
The persistence of discrepancies, despite the use of series that account for the differing coverage of small businesses and volatility, further demonstrates the impact of differing coverage of large businesses and other methodological differences.
In an attempt to assess the effect of the differing coverage of large businesses, an RSI has been calculated for businesses included in BRC membership (Figure 10). This was achieved by matching retailers surveyed by the RSI with entries in the online BRC membership list and deriving an aggregate movement based on RSI responses. It should be stressed that, while it is broadly representative of BRC membership, the resultant series does not base on an exact match with the RSM survey panel due to the following reasons:
BRC membership includes businesses that are not classified as retailers (for example, service providers) and are therefore not included in the RSI; such businesses were not matched
several identified contributors are not necessarily separate reporting units on the RSI – this means that the matched RSI equivalents of these contributors may be inclusive of additional data that represent an overarching company group
the compiled series bases on annual and bi-annual updates of the BRC membership list up to 2016; as contributors to the RSM may have dropped off or joined the panel at a higher frequency or at later dates, this may account for some further discrepancy in the businesses represented
for a small number of businesses, an RSI match was not identified – this has been the case where businesses have been identified as small- to medium-sized retailers that have not been selected for the RSI
As seen in Figures 10 and 11, the movement of the compiled RSI for BRC contributors is closer to the RSM than the RSI series for all large retailers. Apart from any differences arising due to the compilation of the former series as discussed previously, discrepancies will be attributable to methodological differences relating to weighting and the reporting of service-related turnover.
Looking at the distribution of differences in year-on-year growth on a three-month basis between the RSM and the RSI for BRC contributors, it is shown that over 90% of differences fall in the range of negative 1.0 and positive 1.9 percentage points (Figure 12). More than 60% of differences fall within negative 1.0 and positive 0.9 percentage points.
The contracted discrepancies between the RSM and RSI for BRC contributors when compared with differences between RSM and the RSI for large retailers indicate the potential impact of the differing coverage of large retailers within the two surveys.
Figure 13 presents an overview of the distributions of differences between the RSI and RSM series that have been explored in this article.
Notes for The comparability of ONS official data and BRC-KPMG RSM
- RSI accounting periods for June and July 2005 covered 29 June 2005 to 02 July 2005 and 03 July 2005 to 30 July 2005 respectively. The respective RSM accounting periods covered 29 May 2005 to 25 June 2005 and 26 June 2005 to 30 July 2005.
The Office for National Statistics (ONS) Retail Sales Index (RSI) non-seasonally adjusted value series and the BRC-KPMG UK Retail Sales Monitor (RSM) total value series show some divergence but generally track each other well in the long term. Accounting for the differing coverage of small- and medium-sized businesses by comparing RSM to the RSI for large retailers moderately improves the overall fit between the two series. However, the continued presence of discrepancies indicates that the differing coverage of large businesses and other methodological differences in coverage, weighting and classification may still account for a good share of the divergence originally observed.
Considering year-on-year growth differences on a three-month basis, as opposed to a monthly basis, reduces volatility in the growth differences between RSI and RSM. A comparison between a constructed RSI that is broadly representative of BRC membership and RSM shows an exceptionally good fit between the two series. This may indicate that differences in coverage are significant drivers of the discrepancies observed between the RSM and the RSI for large retailers. Also, the divergence that persists between the constructed RSI for BRC contributors and the RSM may be indicative of differences in classification, weighting and reporting.
Overall, the ONS RSI and prominent external surveys on the retail industry, such as BRC-KPMG RSM and CBI Distributive Trades Survey (DTS), are all important and valuable indicators in their own right, serving a variety of user needs. Methodological differences among surveys are often driven by the addressed user needs. For example, the nature of the RSI is dictated by the need for consistency with other ONS business surveys and by international official statistics protocols and conventions. The nature of the RSM and DTS is driven by demands from within the retail industry and the objectives of the BRC and the CBI respectively.
A thorough understanding of methodological differences among outputs on the retail industry is of great importance in interpreting different measures of industry performance. This article has demonstrated some potential sources of divergence in the compilation methodology of the ONS RSI and external surveys.Back to table of contents
Contact details for this Methodology
Telephone: +44 (0)1633 455602