Skip to content

Micro-data Perspectives on the UK Productivity Conundrum

Released: 16 January 2013 Download PDF


This article uses firm-level micro-data to throw new light on UK productivity up to 2009. We find that recent productivity performance has generally been stronger among larger firms and those that are domestically-owned than among smaller firms or those that are foreign owned. In 2009, productivity was weaker among exporting firms in the manufacturing sector than non-exporters, while in services, exporters' productivity fell sharply but remained well above that of firms exposed only to the UK market. Productivity also fell sharply in 2009 among service sector firms with a track record of rapid growth in previous years. High levels of ICT adoption are associated with better productivity performance in all sectors. Analysis of the distribution of productivity across firms shows a widening in distributions in 2008 and 2009: high productivity firms became more productive, while low productivity firms became less productive (in some cases negatively productive). This suggests a weakening in competitive pressures to weed-out less productive firms, especially in the service sector.


We are grateful for financial support from the European Commission (Grant 50701.2010.001-2010.578), to the other 14 consortium members (listed in Appendix A), to Michail Skaliotis and Albrecht Wirthmann of Eurostat and to the academic advisers to the consortium: Eric Bartelsman of the Free University of Amsterdam and Patricia Kotnik of the University of Ljubljana. We thank our ONS colleagues Gaganan Awano for analytical input and Linda Allen and Tracy Blake for administrative support.


Recent ONS work on the productivity conundrum (ONS (2012a), ONS (2012b)) covers several perspectives, but not that from the micro-data perspective1. This article partly redresses the balance, using a unique dataset built from micro-data as part of the 15-country EU-funded ESSLIMIT project. The primary focus of the ESSLIMIT project is on the economic impact of information and communication technology (ICT). However, since one of the main impacts of ICT is on productivity, we can use this dataset to explore a number of lines of enquiry that cannot be addressed using conventional macro-level statistics2.

ESSLIMIT is an ESSnet3 project on LInking of Micro-data on ICT usage. The project ran over 2010-12 and follows on from an earlier project on micro-data linking co-ordinated by ONS and completed in 2008 (Eurostat, 2008). The underlying methodological approach is to use multiple micro-data sources to compile ‘micro-aggregated statistics’ designed to address areas of interest to policy-makers and researchers. Examples are where micro-data are stratified by two or more categories over time (such as industry and size class) and statistics on the distribution of the micro-data. This approach is also designed to comply with confidentiality and disclosure regimes governing the use of micro-data.

The layout of the article is as follows. We start with a brief description of the ESSLIMIT dataset and outline some of the issues that should be borne in mind in interpreting results. We then review productivity performance by size class and other firm-level characteristics identified in the ESSLIMIT dataset, looking also at related information on firm-level demographics. We then focus on the distribution of productivity across firms, again incorporating related demographic information. Lastly we briefly review comparisons between productivity in the UK and other countries within the ESSLIMIT consortium.

Notes for Introduction

  1. In this article, micro-data refers to data at the level of individual firms or enterprises, typically but not exclusively in the form of their responses to ONS surveys.
  2. In this article, 'macro' includes components within the national accounts framework, such as industry level measures of productivity.
  3. ESSnet projects are consortia of national statistical institutes (NSIs) and are aimed at providing results beneficial to the European Statistical System.

The ESSLIMIT Dataset: Sources and Methods

The primary data source for the ESSLIMIT dataset is the annual structural business survey, known as the Annual Business Survey (ABS) in the UK. This collects information on economic activity (turnover, value-added, employment and many other variables) from approximately 50 thousand firms. ABS is supplemented by firm-level information from a range of other surveys and sources including trade statistics, ICT usage by firms and estimates of firm-level capital stocks. The project also utilises a consistent register dataset containing basic information on the universe of firms of which the ABS is a sample. This register provides valuable information on firm demographics, and also enables derived statistics to be weighted to provide consistency over time.

The ABS provides a rich source of information, but requires a substantial investment in analytical coding to compile into a coherent framework for analysis. For example, ABS does not contain any direct information on productivity and in general contains no volumetric information on firm-level outputs. The analytical framework used here has been developed as part of the ESSLIMIT project (and its predecessor project, Eurostat, 2008). Although as its name implies the primary object of the ESSLIMIT project is to investigate the impact of firms’ use of ICT, much of the project infrastructure can be interpreted through a productivity lens. More information on the ESSLIMIT project is available on the project website.

The principal project outputs that we focus on in this article are metrics on productivity. In the ESSLIMIT project, LPV refers to labour productivity defined in terms of (deflated) value-added and LPQ is labour productivity in terms of (deflated) gross output. TFP refers to an index of total factor productivity (deflated value added divided by weighted labour and capital inputs) and MFP refers to an index of multi-factor productivity (deflated gross output divided by weighted labour, capital and intermediate inputs). Note that this usage differs from general ONS usage, where MFP refers to the value-added measure of output, as described in an article on multi-factor productivity.

Interpreting these statistics

There are a number of factors to be borne in mind in interpreting the ESSLIMIT statistics. Firstly, micro-aggregated statistics cannot be compared directly with published, official macro statistics, for example of labour productivity or MFP. In general, this is because macro statistics on output and labour and capital inputs are not derived from a single survey source, but rather are based on multiple sources, including non-survey sources, and are subject to balancing processes to produce a coherent description of the macro economy. Moreover, as noted above, ONS and other NSIs do not collect pricing information at the level of the individual firm. This means that, in practice, the processes of deflating nominal measures of output are likely to lead to differences between macro and micro-aggregated measures of real output. A further source of discrepancy is that the ABS does not cover the whole economy; for example, it does not cover unincorporated businesses.

Secondly, ABS micro-data are currently only available up to 2009, and the statistics reported in this article therefore end in that year. From a productivity perspective, 2009 is certainly of interest as the trough of the recession which began in 2008. In 2009 the macro statistics show the typical pattern of a sharp fall in labour productivity, as output fell more steeply than labour inputs. An important aspect of the productivity conundrum is why labour productivity has been so weak in the recovery period since 2009. To throw light on this from the micro-data perspective, ONS is a member of a follow-on EU-funded project which will extend the ESSLIMIT dataset to 2010 and perhaps to 2011.

Thirdly, there are two structural breaks in the micro-data in 2008 compared with 2007 and earlier years. In 2008, ABS and the Business Register Employment Survey replaced what were previously known as the Annual Business Inquiry (ABI) 1 and 2, and the sample frame was changed from SIC03 to SIC07. For reasons of comparability with other international data such as the EU-KLEMS growth accounting database, the ESSLIMIT dataset is organised in terms of the SIC03 (NACE1 in European parlance) industrial breakdown, whereas ONS national accounts and macro productivity statistics moved to a SIC07 (NACE2) industrial classification from Blue Book 2011.

Fourthly, one further issue to bear in mind in interpreting TFP statistics in this article is that TFP utilises firm-level estimates of capital stocks. There are no direct measures of capital at the firm level in the UK, and the estimates used here have been derived from firm level measures of capital investment. This process is computationally challenging and requires a number of assumptions to be made. Accordingly, we judge these estimates to be less robust than measures of firm-level labour inputs.

In this article we focus on statistics for three sectors1:

  • MexElec: Total manufacturing, excluding electrical machinery,

  • EleCom: ICT sector (Electrical machinery, Telecom services),

  • MServ:  Market services, excluding telecom services.

The weighted employment shares of these sectors in 2009 were 15%, 6% and 79% respectively. Basic information on these sectors is contained in Table 1. Although there are very many more small firms than large firms, employment is heavily weighted towards large firms. Exporting is more prevalent among manufacturing and the ICT sector than in market services, where the incidence of high growth enterprises is comparatively large.

Table 1: Summary Statistics

Weighted employment shares, 2009
Size class    Foreign-owned Exporter High Growth Enterprise
1 2 3 4
MexElec 18% 11% 25% 46% 40% 65% 33%
Elecom 17% 6% 12% 65% 30% 58% 22%
Mserv 26% 7% 14% 53% 29% 19% 39%

Table source: Office for National Statistics

Download table


Notes for The ESSLIMIT Dataset: Sources and Methods

  1. The ESSLIMIT project also generates micro-aggregated results at approximately 2-digit industry level. We focus on this higher level aggregation in this paper partly for reasons of brevity, and partly to minimise risk of disclosure.

Productivity by firm size

We start by looking at productivity movements over time by size class, where size is determined by number of employees, as follows:

  • Size class 1: emp<20,

  • Size class 2: 20<=emp<50,

  • Size class 3: 50<=emp<250,

  • Size class 4: 250<=emp.

Here and elsewhere in this article the underlying micro-data are weighted (by employment) to provide more consistency over time1.  Despite this, it is a feature of micro-aggregated data that we observe much more volatility than observed in comparable macro data.

Figures 1-6: Productivity by Size Class

1-3: Labout Productivity; 4-6: TFP
Source: Office for National Statistics

Download chart

Figures 1-3 show time series for labour productivity (value-added basis) for these 3 high-level industry categories and the 4 size classes. For manufacturing (excluding ICT) and the ICT sector there is clear evidence of a positive correlation between firm size (as measured by employment) and the level of value added per employee. However, this is not the case for market services, where the level of value-added per employee is higher for medium-sized firms than for either the smallest or largest firms. This may reflect fewer opportunities to exploit scale economies in services than in manufacturing.

In manufacturing (excluding ICT) labour productivity among smaller firms appears to have fallen sharply in 2009, whereas labour productivity among firms in the two larger size classes was broadly unchanged. Labour productivity of larger firms in the ICT sector was similarly broadly unchanged in 2009, but in this sector, productivity of firms in the smaller size classes grew sharply in 2009, after dipping in 2008. Labour productivity fell in 2009 among all size classes in market services (excluding ICT), with the largest falls being of those size classes with the highest levels of productivity.

The picture changes somewhat in terms of TFP (defined here as real value added per weighted inputs of labour and capital, where the weights are determined by the industry shares of returns to labour and capital respectively), shown in Figures 4-6. For manufacturing (excluding ICT), focusing on TFP narrows the differential between size classes compared with a focus on labour productivity (compare Figure 4 with Figure 1), and reveals a much more pronounced drop in productivity in 2009. For the ICT sector, the comparison between TFP and labour productivity (Figures 2 and 5) produces a dramatic change in the ordering, presumably because large firms in this sector (which includes telecommunications) tend to be very capital intensive, pushing up their labour productivity but not their TFP. For market services (excluding ICT), the main difference is that the drop in labour productivity in 2009 is less pronounced in terms of TFP among firms in size classes 1 and 3.

The ESSLIMIT dataset allows us to drill down into productivity movements using firm demographics, specifically to focus on changes of employment of continuing firms (that is, abstracting from the effects on employment of firm entry and exit). Some examples are shown in Figures 7-9, which show changes of employment for each size class, and separately according to whether firms have increased or decreased their payrolls2.

Figures 7-9: Employment changes by size class

Sums of positive and negative employment changes by industry and size class
Source: Office for National Statistics

Download chart

One immediate feature of Figures 7-9 is the demonstration that net changes of employment reflect large and partly offsetting hiring and firing by different firms, in turn reflecting heterogeneity between firms in the same industry. Another striking feature is that in the two largest sectors of market services and manufacturing (both excluding their ICT components) there is fairly clear evidence of an increase in net hiring in 2009 compared with earlier years. In both cases, the increase is most pronounced among large firms. 

Notes for Productivity by firm size

  1. The ESSLIMIT dataset contains statistics derived using several different weighting regimes. Generally speaking, the results are robust to different weighting regimes.
  2. This information is extracted from a time-consistent business register, known as the ‘ARD Register Panel’.

Productivity by other firm-level characteristics

One advantage of a micro-data based approach is that it allows structural business survey data to be analysed by firm-level attributes from other sources which can be matched at the firm-level using unique firm identifiers. In the following sub-sections we summarise productivity trends under a range of firm-level characteristics that are included in the ESSLIMIT database.

(i) Foreign versus domestically owned firms

For this analysis we use information from the business register on ownership to analyse productivity differences between foreign- and domestically-owned firms. Figures 10-12 show higher labour productivity among foreign-owned firms across all three of the broad sectors covered in this article. The discrepancy widens from 2007 for the ICT sector, before narrowing somewhat in 2009.

Figures 10 - 15: Productivity by Foreign and Domestic Ownership

Figs 10-12: Labour Productivity; Figs 1315: Total Factor Productivity
Source: Office for National Statistics

Download chart

By contrast, the discrepancy among firms in manufacturing (excluding ICT) disappears in 2009.

Turning to TFP, one interesting feature of Figures 13-15 is the closing of the gap for manufacturing (excluding ICT) for all years except 2009, when TFP fell sharply among foreign-owned firms. TFP also fell in 2009 among foreign-owned firms in the ICT sector, although in this case the productivity gap between foreign- and domestically-owned firms in terms of TFP is much more pronounced than in terms of labour productivity.

(ii) Exporters versus non-exporters

In this section we use an export flag to analyse productivity trends between exporters and non-exporters. Information on exports of goods is captured directly on the ABS. For exports of services, we use merged data from monthly trade surveys. Further analysis of this subject is available in the forthcoming ESSLIMIT project report (Hagsten et al, 2013).

Figures 16 - 21: Productivity by Exporters / Non-exporters

Figs 16-18: Labour Productivity; Figs 19-21: Total Labour Productivity
Source: Office for National Statistics

Download chart

Perhaps surprisingly, Figure 16 shows comparatively little difference in labour productivity between exporters and non-exporters in manufacturing (excluding ICT), and a pronounced increase in productivity among non-exporters in 2009. Also of note is that the average labour productivity of all firms is closer to that of exporters than non-exporters, consistent with information in Table 1 showing that more firms (in employment-weighted terms) in this sector are exporters than not.

Comparing Figure 16 with Figure 19, productivity performance in terms of TFP presents a different picture, and one more in line with the conventional wisdom that non-exporters lag behind in productivity performance when a more comprehensive measure of factor inputs is taken into account.

There is a similar contrast between labour productivity and TFP in the ICT sector (Figures 17 and 20). Figure 17 suggests that non-exporters are much more capital intensive than exporting firms in this sector, perhaps reflecting the influence of the telecoms sector, which is orientated towards domestic demand and highly capital intensive. There is much less divergence in terms of TFP. As in the MexElec sector, the average of all firms is closer to exporters than non-exporters.

This is not the case in market services (excluding telecoms), where the data suggest, firstly, that exporting firms are significantly more productive than non-exporters in terms of both labour productivity and TFP. And secondly, that productivity among exporting firms is much more volatile over time than among non-exporters, presumably reflecting the exposure of exporting firms to exchange rates and other sources of volatility from which non-exporters are insulated.

(iii) High-growth enterprises

High growth enterprises are those where employment has grown by at least 10 per cent per year for at least the previous 3 years.

Figures 22-27: Productivity by High-growth enterprises

Figs 22:24: Labour Productivity; Figs 25-27: TFP
Source: Office for National Statistics

Download chart

In 5 out of 6 cases in Figures 22-27, the level of productivity of high growth enterprises is higher than for other firms. The exception is labour productivity for the manufacturing sector (excluding ICT). This not the case for TFP, which displays the more intuitive ranking, and a sharp contrast in productivity movement in 2009. High growth enterprises are significantly more productive than other firms in terms of TFP in all three sectors. In both the ICT and market services (excluding ICT) sectors, labour productivity and TFP fell more sharply in 2009 among high growth enterprises than among other firms.

(iv) ICT maturity

In this section we look at productivity of firms categorised by their ICT maturity, measured here by the share of workers with access to a broadband internet connection. Note that this approach matches firm-level data from the E-Commerce survey (which captures information on business use of ICT) with the structural business survey. This means that the sample size of the overlapping (matched) sample is a fraction of the size of the ABS itself.

In Figures 28-33, Broadcat=1 represents the set of firms where fewer than 40% of workers have broadband access, and Broadcat=2 represents firms with more than this figure. In all cases the sample has been trimmed to take out firms with exceptionally low or high levels of broadband penetration1.

Figures 28-33: Productivity by ICT Maturity

Figs 28-30: Labour Productivity; Figs 31-33: Total Labour Productivity
Source: Office for National Statistics

Download chart

It is immediately apparent from Figures 28-33 that productivity tends to be higher in firms with higher levels of ICT maturity on this measure. A priori, one might imagine that the discrepancies would tend to diminish over time, as broadband penetration has generally increased over this time frame. However, only in the case of labour productivity in market services (excluding ICT) does this resonate with the data. For manufacturing (excluding ICT) and the ICT sector the differential in labour productivity is wider in 2009 than earlier in the decade. This may be suggestive of a diminishing rump of low-productivity firms resistant to adoption of ICT, while more dynamic firms have moved from Broadcat=1 to Broadcat=2.

Productivity differential are less pronounced in terms of TFP (though still significant, especially for market services), and the differential had closed completely in the ICT sector in 2009.

Notes for Productivity by other firm-level characteristics

  1. This measure of ICT maturity stems from econometric analysis carried out in the 2006-08 project.

Distribution of productivity across firms

One innovative feature of the ESSLIMIT dataset is the reporting of properties of the distribution of firms in terms of their productivity1. Figures 34 to 39 illustrate this. These figures illustrate, for each industry and year, the average productivity of firms in the lowest and highest quartiles of the labour productivity and TFP distributions2.

Figures 34 - 39: Productivity of highest and lowest quartiles of firms

Figs 34-36: Labour Productivity; Figs 37-39: Total Factor Productivity
Source: Office for National Statistics

Download chart

Figures 34 to 39 show a number of interesting features. Firstly, they are consistent with other micro-data based analysis of productivity in showing large and persistent productivity differentials between firms, in direct contrast to the common theoretical paradigm of homogeneity between firms. This is the case for both labour productivity and TFP; in both cases, productivity among the most productive quartile of firms is an order of magnitude higher than productivity among the least productive quartile. Second, there is clear evidence of widening productivity distributions in 2008 and 2009, particularly in manufacturing (excluding ICT) and the ICT sector. Third, this widening occurs at both ends of the productivity distributions – on average more productive firms became still more productive in 2008 (though not in 2009) while productivity fell at the bottom of the distribution. Fourthly, labour productivity turned negative among the least productive quartile of firms in the ICT sector in 2008 and became more negative for the same cohort in market services.

These findings may suggest that competitive pressures on low productivity firms to improve their performance or exit the market may have become less effective in 2008 and 2009, perhaps reflecting reductions in the cost of credit and increased forbearance by lenders.

Figures 40 - 42: Employment Growth Rates by Productivity Quartile

Growth rates of employment by quartile of the labour productivity distribution
Source: Office for National Statistics

Download chart

Again it is instructive to drill down in the ESSLIMIT dataset, in this case by looking at changes in employment by quartile of the (labour) productivity distribution3. Figures 40-42 show that for manufacturing (excluding ICT) and the ICT sector, employment changes are more or less consistent with the pattern from the productivity distributions above: low productivity firms in the ICT sector were shedding labour in 2009, while in manufacturing (excluding ICT) low productivity firms reduced their hiring rates, with job reductions weighted towards higher productivity firms.

The results for market services are rather perplexing. Although as we have seen, productivity performance among the least efficient quartile of firms was getting worse in 2009, with negative value added per employee on average for this quartile, labour hiring continued and at a faster rate than labour hiring by more productive firms.

Notes for Distribution of productivity across firms

  1. This section uses unweighted data drawn from the ABS.
  2. Note that outlier correction is used to remove firms in the 1% tails of the productivity distributions in all cases, and that firms may be in different quartiles of the distributions in terms of labour productivity and TFP.
  3. The approach here is different from the breakdown of employment changes by size class in figures 7-9 above. The information in figures 7-9 comes from the universe of firms, whereas here we are looking at the (unweighted) sample of firms, from the ABS.

International Comparisons

Figures 43 to 48 show UK productivity compared with weighted average productivity of the rest of the ESSLIMT consortium (See Appendix A for a list of consortium members). For ease of exposition we have indexed productivity estimates for each country to 2006=100. Weights are based on employment shares across the consortium, by sector.

Figures 43 - 48: Productivity Comparisons with other Project Countries

Figs 43-45: Labour Productivity; Figs 46-48: Total Factor Productivity
Source: Office for National Statistics

Download chart

For manufacturing (excluding ICT) and the ICT sector, UK labour productivity compares favourably with the average across the rest of the consortium since 2006. And UK labour productivity in market services grew more rapidly in the years before the recession. However, the drop in labour productivity in market services in 2009 was much steeper in the UK than elsewhere across Europe.

Background notes

  1. Copyright and reproduction

    © Crown copyright 2013

    Under the terms of the Open Government Licence and UK Government Licensing Framework, anyone wishing to use or re-use ONS material, whether commercially or privately, may do so freely without a specific application for a licence, subject to the conditions of the OGL and the Framework.

    For further information, contact the Office of Public Sector Information, Crown Copyright Licensing and Public Sector Information, Kew, Richmond, Surrey, TW9 4DU.

    Tel: +44 (0)20 8876 3444


  2. Follow ONS on Twitter or Facebook or view our podcasts on YouTube.

  3. Details of the policy governing the release of new data are available by visiting or from the Media Relations Office email:


  1. Eurostat (2008), 'ICT impact assessment by linking data from different sources - Final Report', Available at:
  2. Hagsten, Eva et al (2013), 'ESSnet on Linking of Microdata on ICT Usage - Final Report', Forthcoming
  3. ONS (2012a), 'The Productivity Conundrum, Interpreting the Recent Behaviour of the Economy', Available at:
  4. ONS (2012b), 'The Productivity Conundrum, Explanations and Preliminary Analysis', Available at: 

Appendix A: List of participants

  • UK Office for National Statistics,

  • Statistics Sweden,

  • Statistics Netherlands,

  • Statistics Norway,

  • National Statistical Institute of Italy,

  • Statistisches Bundesamt (Germany),

  • Institut National de la Statistique et des Études Économiques (France),

  • Statistics Denmark,

  • Central Statistics Office (Ireland),

  • Statistical Office of the Republic of Slovenia,

  • Statistics Austria,

  • Statistics Finland,

  • Service Central de la Statistique et des Etudes Economiques (Luxembourg),

  • Central Statistical Office of Poland,

  • National Institute of Statistics, Romania.

Content from the Office for National Statistics.
© Crown Copyright applies unless otherwise stated.