Funded occupational pension schemes in the UK QMI

• National Statistic: Yes

• Survey name: Financial Survey of Pension Schemes (FSPS)

• Data collection: Sample of funded occupational pension schemes

• Frequency: Quarterly

• How compiled: Sample-based survey

• Geographic coverage: UK

This quality and methodology report contains information on the quality characteristics of the data (including the European Statistical System five dimensions of quality (PDF, 3MB)) as well as the methods used to create it. The information in this report will help you to:

• understand the strengths and limitations of the data
• learn about existing uses and users of the data
• understand the methods used to create the data
• help you to decide suitable uses for the data
• reduce the risk of misusing data

This report aims to provide users of the FSPS estimates with information its usability and purpose.

• The FSPS was implemented as an online survey in April 2019.

• The FSPS sampling frame is updated at least every 18 months, and the data source for this is The Pensions Regulator (TPR) register.

• The quarterly survey covers the membership and financial activity (income and expenditure, transactions and assets and liabilities) of funded occupational pension schemes registered in the UK.

• The FSPS data are compiled primarily for inclusion in the Office for National Statistics (ONS) UK National Accounts and UK Balance of Payments. It also informs the estimate for gross domestic product (GDP).

• UK pension statistics are used for policy, analysis and negotiations by other government departments including the Department for Work and Pensions (DWP), the Organisation for Economic Co-operation and Development (OECD) and the International Monetary Fund (IMF).

Overview

The Financial Survey of Pension Schemes (FSPS) is designed to provide estimates of financial activity by funded occupational pensions schemes, which are registered in the UK.

It uses an online spreadsheet-based data collection instrument, replacing from 2019 the paper-based Quarterly Survey of Pension Funds: Income and Expenditure, Quarterly Survey of Pension Funds: Transactions and Balances and the Annual Survey of Pension Funds: Balance Sheet.

In addition, the survey provides a range of improved information to enable the release of more estimates split by benefit type: private and public sector (defined benefit and hybrid), and private sector defined contribution. A defined benefit (DB) pension is one in which the rules of the scheme specify the rate of benefits to be paid. A defined contribution (DC) pension is one in which the benefits are determined by the contributions paid, the investment return on those contributions (less charges) and the type of annuity purchased upon retirement, if any.

The ONS carries out the FSPS using a stratified random sample from The Pensions Regulator (TPR) register of UK-based pension schemes. Strata are defined based on the benefit type and membership size band of the scheme or scheme section. Data from the sample are then appropriately weighted, also accounting for non-response, to estimate UK funded occupational pension schemes. More information on weightings can be found in Section 6: Methods used to produce the FSPS data.

Estimates were first published in June 2020 in our UK pension surveys: redevelopment and 2019 results with further estimates being released quarterly via our Funded occupational pension schemes in the UK bulletin. For more information on upcoming releases, the UK National Statistics Publication Hub is available online and provides advance notice of release dates. Publications are announced at the beginning of the calendar year, as required by the European Statistical System code of practice.

Uses and users

The users of FSPS statistics include:

• Office for National Statistics (ONS) Balance of Payments and National Accounts
• other government departments such as the Department for Work and Pensions (DWP) or Her Majesty's Revenue and Customs (HMRC)
• Organisation for Economic Co-operation and Development (OECD)
• International Monetary Fund (IMF)
• Bank of England (BoE)

Trade associations, city analysts, institutional investors and fund managers use these data for modelling or forecasting purposes and to track asset allocation trends. Academics and journalists also use the data for research purposes, and subsequent media reporting.

There also is interest from stakeholders in the survey data, ranging from those involved in risk management of pension schemes, to recipients of private pensions. Other users who are interested in FSPS data include pension scheme trustees, administrators, and fund managers.

Strengths and limitations

Strengths

Some of the main strengths of the Financial Survey of Pensions Schemes (FSPS) are as follows.

Firstly, this is a comprehensive survey, covering detailed financial information on UK funded occupational pension schemes. The quarterly estimates are collected using a standard accounting format that is familiar to respondents; capturing the full balance sheet enhances analysis and data quality.

Secondly, the sample includes all funded public sector schemes, alongside all private sector defined benefit and hybrid (DBH) schemes with more than 10,000 members and private sector defined contribution (DC) schemes with more than 25,000 members.

Also, data are collected under the UK Statistics of Trade Act 1947, which means that its completion is mandatory; this has resulted in the survey having a consistently high response rate, averaging around 80% during the closedown period.

In addition, the FSPS has a "look-through" for pooled investment vehicles, through which we can see the asset allocation of schemes investing through such vehicles. This breakdown for investments in pooled vehicles by financial instrument type provides further insight into pension schemes' investment strategies, such as estimates on liability driven investment (LDI) pooled funds.

Furthermore, there is an expansion of overseas estimates being collected within the FSPS, in comparison to the previous pension fund surveys. There is now a wider geographical coverage, and these data are being collected on a quarterly basis. This includes a breakdown by country of the value of equities and debt securities issued by overseas governments or companies, and of pooled investment vehicles and structured products registered overseas.

There is also an improvement of the sampling frame by using data from The Pensions Regulator's (TPR's) register of all UK funded occupational pension schemes; TPR's register has a complete coverage of the population.

Limitations

Some of the main limitations of the FSPS are as follows.

Firstly, there is a lag between TPR getting information from pension schemes and the Office for National Statistics (ONS) drawing the sample. This results in some TPR membership figures, which are used to form the FSPS sample, being outdated. For example, a scheme may have experienced a significant transfer during this period. This will have an effect on how accurately the FSPS data are able to estimate for the whole of the UK; however, as this generally only affects a relatively small number of schemes within the sample, the impact is minimal. Additionally, methods have been applied to affected schemes to further minimise the impact of this limitation.

Secondly, the survey began in April 2019 so is still relatively new. As such, imputation methods for sections within the survey that have a low response rate, where data are not being reported in its entirety, or misreported, still require further development. One example of this is expenses, where it has so far proved problematic to capture the hidden costs relating to investment.

Also, there is a one-quarter lag in FSPS data collection when compared with other inputs into the National Accounts. This means that the latest quarter in the UK National Accounts will need to be forecasted, until FSPS data are supplied in the following quarter. However, this lag results in higher response rates from schemes, as they may have the data readily available. This means that there are less revisions made in subsequent quarters, with fewer "best estimates" provided by respondents, which contributes to improved data quality.

In addition, the original sample, TPR's register, contains executive pension plans (EPPs) and small self-administered schemes (SASSs), both of which are not in scope of the FSPS. Robust methods have been applied to the sampling process to remove such schemes from TPR's register, however it is possible that a small number of EPPs and SASSs remain in the final adjusted sample.

The FSPS is also unable to produce estimates of scheme numbers, partly because of the volatility caused by a small sample size for smaller schemes, and also based on whether schemes, scheme sections or sub sections and so on are defined consistently.

Relevance

The degree to which statistical outputs meet users' needs.

The Financial Survey of Pension Schemes (FSPS) was designed to meet the needs of the UK National Accounts (sector and financial accounts) and UK Balance of Payments. The aim was to do this through improving coverage, quality and granularity of data being fed in, in line with the latest requirements and definitions of the UK National Accounts. The resulting questionnaire was therefore balanced to ensure the receipt of high-quality and relevant data, but also to limit burden on respondents. As a result, certain adjustments need to be made to bring the estimates in line with the National Accounts requirements; for example, adjustments may be made to map investment categories to the National Accounts standard classifications.

Users were also consulted and contributed to the design of the survey. The Office for National Statistics (ONS) met with experts from Department for Work and Pensions (DWP), The Pensions Regulator (TPR), Financial Conduct Authority (FCA), Debt Management Office (DMO) and with pensions industry specialists including the Pensions and Lifetime Savings Association (PLSA). The questionnaire was then subjected to an extensive period of cognitive testing. This studied the ways in which individuals mentally processed and responded to survey questionnaires, including how they understood the questions and answered them. The interviewers also asked how the data would be obtained, who would be involved in completing the survey (including any external suppliers), and whether they would prefer the new survey to be an electronic questionnaire or a spreadsheet-based questionnaire. There was also an extensive period of cognitive testing of the questionnaire with potential responders, including schemes from the public and private sector, of varying (membership) sizes and different benefit types.

More information on the cognitive testing process can be found in Section 4: Survey redevelopment in UK pension surveys: redevelopment and 2019 results.

Accuracy and reliability

The degree of closeness between an estimate and the true value.

The FSPS is an online survey conducted using spreadsheets. Internal inconsistencies within the form are picked up and highlighted by error messages as the respondent completes the questionnaire. Returned data are also run through a series of validation tests to try to identify potential errors. The tests are designed to check whether all the data required have been provided and in the correct units. The tests also compare returned data with previous data for each respondent. Data that fail the tests are identified for further action. Where failures are judged potentially to have a large impact on survey results, they will be queried with respondents, with the aim of correcting or confirming the original data provided by the respondent.

Comprehensive guidance notes are provided within the survey return form, allowing for respondents to better understand the results requested in each survey area, improving accuracy and reliability of estimates. Guidance can also be found on our FSPS business page. You can make any further enquiries by emailing Financial.inquiries@ons.gov.uk or calling the helpline on 01633 456000. Please enter the three-digit survey ID 093.

To minimise non-response bias, a reminder to respond is issued via the data collection portal, followed by telephone or email response-chasing.

Quality assurance of data within one-quarter may also reveal issues with responses to previous returns. This may lead to revisions to previously published data.

Coherence and comparability

Coherence is the degree to which data that are derived from different sources or methods, but refer to the same topic, are similar. Comparability is the degree to which data can be compared over time and domain, for example, geographical level.

Over time

While there are some overlaps to estimates provided by the previous pension fund surveys, methodological and quality improvements through the FSPS have resulted in systematic changes to these areas. These include a continuation of key outputs identified in redevelopment of the pension survey, such as contributions and benefits.

Comparisons of FSPS estimates from different sample periods are subject to sampling variability. Sampling variability is dependent on several factors, including the size of the sample, the effects of the sampling method and the effects of weighting.

With alternative sources

The Annual Survey of Hours and Earnings (ASHE) collects information about earnings and pensions for a 1% sample of employee jobs from the Pay As You Earn (PAYE) system. The survey identifies which of these jobs has employee membership of the current employer's workplace pension scheme, which does not include preserved rights in any former employer's pension scheme or pensions paid by former employers. Within the overall category of workplace pensions, it separately identifies occupational pensions, group personal pensions and group stakeholder pensions, it does not cover individual personal or stakeholder pensions. The Results on pension participation and contribution rate bands are published annually, with breakdowns by sex, age, industry sector, earnings and firm size. There is minimal overlap with FSPS on the membership estimates but no direct overlap for other series - FSPS collects the value of contributions into pension schemes rather than the contribution rates of members.

The Pensions Regulator (TPR) defined contribution trust (DC) scheme return data 2022 to 2023 provides a snapshot of the landscape of occupational defined contribution (DC) trust-based pension provision in the UK, including information of the number of schemes, membership and total assets. Estimates in the TPR publication are not directly comparable to FSPS given the different treatment of "hybrid" arrangements (when members may receive both DB and DC benefits and/or the scheme offers both types of benefit). TPR's estimates also generally only refer to schemes with 12 or more members. The wider scope of the FSPS results contributes to the larger estimates compared to the TPR publication.

The Purple Book published by the Pension Protection Fund provides yearly data and analysis on the UK defined benefit (DB) pension landscape. Results here are not directly comparable with FSPS DB estimates given the differing purpose and therefore methods for calculating balances.

Timeliness and punctuality

Timeliness refers to the lapse of time between publication and the period to which the data refer. Punctuality refers to the gap between planned and actual publication dates.

Estimates are compiled on a quarterly basis. Our pensions statistics are currently published approximately two quarters after the end of the reference quarter the data relates to. A routine publication schedule has been embedded and the series of statistical bulletins have so far been published on the planned publication date. The ONS release calendar provides advance notice of release dates. In the unlikely event of a change to the pre-announced release schedule, there will be notification of the change and its reasons as set out in the Code of Practice for Statistics.

Please note, the frequency of publication of our Funded occupational pension schemes in the UK statistical bulletin and dataset has changed from quarterly to six monthly. Each statistical bulletin now publishes new estimates for two quarters. This decision gives us the opportunity to further develop the analysis of our data, so that we continue to provide users with high-quality, relevant data and analysis.

Accessibility and clarity

Accessibility is the ease with which users are able to access the data, also reflecting the format in which the data are available and the availability of supporting information. Clarity refers to the quality and sufficiency of the release details, illustrations and accompanying advice.

Our recommended format for accessible content is a combination of HTML web pages for narrative, charts and graphs, with data being provided in usable formats such as CSV and Excel files. Our website also offers users the option to download the narrative in PDF format. In some instances, other software may be used, or may be available on request.

You can make any further enquires by emailing Financial.Inquiries@ons.gov.uk. It may be possible to meet additional data requests, but these may be chargeable depending on the time required to produce the additional data requested. Metadata describing the limitations of additional data are provided with individual requests. User-requested data are also published on the ONS website.

Concepts and definitions

Concepts and definitions describe the legislation governing the output and a description of the classifications used in the output.

Survey data are collected under the statutory powers of the Statistics of Trade Act 1947. The surveys are conducted to meet the requirements specified in the European System of Accounts 2010 and international standard for the National Accounts.

The questionnaire includes definitions and guidance notes on how to complete - an example form can be downloaded from the FSPS business page.

Furthermore, a glossary of terms is provided in our Funded occupational pension schemes in the UK statistical bulletin along with a more comprehensive glossary in the 2019 FSPS redevelopment article.

Geography

Estimates are available at UK level and not further disaggregated by geography.

How we collect the data

Target population and sampling frame

The target population is all funded occupational schemes registered in the UK. This excludes small self-administered schemes (SSASs) and executive pension plans (EPPs).

The sampling frame has been provided by The Pension Regulator (TPR) register of occupational pension schemes.

Sample design

The sample was designed, in discussion with colleagues in the Office for National Statistics (ONS) methodology teams, as a stratified random sample with the strata defined by membership size and benefit type (defined benefit, including hybrid) and defined contribution (DC).

The Pensions Regulator (TPR) extract was restructured to create three lists of sampling units covering:

1. all schemes or sections of schemes that held defined benefit or hybrid (DBH) members

2. all schemes or sections of schemes that held DC members. This includes schemes identified by the TPR as master trusts.

3. all schemes classified (for the purposes of National Accounts) as "government managed" or "public pension funds", for example, local government pension schemes.

Schemes within list 3 are fully enumerated. For lists 1 and 2, stratum boundaries were created using Dalenius-Hodges "cumulative square root frequency method" and are given in Table 1.

For each size band within private sector DBH and DC schemes, Table 2a and Table 2b illustrate the following: the number of sampling units within the universe from the adjusted TPR register extract ("N"); the number sampled from this universe at the beginning of the current sample period ("n"); and, the percentage sampled. Each sampling unit consists of a part of a pension scheme determined by the benefit type and number of members. Pension schemes can be made up of one or more sections and the sections or scheme may have just DB, just DC, just hybrid members or a mixture of any of these. The sample is designed to optimise estimates of financial variables by benefit type - not to measure the numbers of schemes or scheme sections. This means that N and n should not be taken as an estimate of the number of funded occupational pension funds or schemes in the UK and are not comparable with estimates produced by The Pensions Regulator.

For DBH, units with greater than 10,000 members are fully enumerated (bands A and B); for DC, units with greater than 25,000 members (band A) are fully enumerated. Units within the remaining strata were sampled at random, using Neyman allocation to optimise the process.

The sample is fixed for a minimum period of 18 months (or six quarterly returns). Because of response burden in completing returns for small schemes (D strata), once selected these schemes would not be selected in the next sample period.

How we process, analyse and quality assure the data

Validation

To try to minimise non-response bias, a reminder letter is sent, followed by telephone or email response-chasing. There is also the possibility of using the legal powers of the Statistics of Trade Act 1947 to force response, though we prefer to work together with the respondent to get the necessary information.

Validation and quality assurance of the data are outlined in the of Section 5: Quality characteristics of the FSPS data.

Imputation and outlier treatment

Imputation methods, used where a responder has returned some data for the relevant quarter but not for all questions, are still being developed. Period-on-period movements will be used to impute returns for non-responders that have returned data for the previous period. If data does not exist for the previous period, it may be possible to use information from similar responders. This is also true for the approach for identifying and dealing with outliers. Currently outliers are, if necessary, being manually adjusted but the intention is to develop a more automated approach once more data are available. Manual treatment of outliers will require contacting respondents to investigate further any data queries.

Weighting and estimation

Each sampled unit effectively represents a number of similar units based on benefit type and membership size. Ratio estimation is applied to produce estimates for the entire population. As noted under "How we collect the data - Target population and sampling frame", adjustments have been made to the weights for outlier treatment.

The estimates are not subject to deflation or seasonal adjustment.

Statistical disclosure

The Government Statistical Service guidance on disclosure (252 KB, PDF) is followed. Data disclosure rules mean that data collected on the survey questionnaires cannot be published individually. Where data items are identified as being disclosive, higher level aggregates, combining the disclosive data with other data items, are created for publication purposes. Even though this is unavoidable, non-ONS users looking to analyse data at a more detailed level could consider this to be a limitation of the output.