This article provides an overview of our transformation of population and migration statistics to put administrative data at the core of what we do. You can also find an accompanying summary of recent updates on our statistics and research.
This article includes our plans and approach for delivering new estimates of migration based on administrative data with statistical modelling. These new estimates of migration will feed into our admin-based population estimates (ABPE). This article also provides information on our progress towards these estimates as well as an overview of our "administrative data first" approach to transforming data on population characteristics.
Your feedback on our latest update and plans would be valuable, please email email@example.com to tell us what you think.Back to table of contents
It is our mission to provide the best insights on population and migration, working with other government departments and using a range of new and existing data sources to meet the needs of our users. This is increasingly important in a rapidly changing policy and societal context, where we know our users need better evidence to support decision-making at both national and local levels.
As set out in our previous work on the Administrative Data Census project, the current population system is heavily reliant on the decennial census. While this provides granular data at the lowest levels of geography every 10 years, it delivers less detail throughout the interim years. Additionally, the quality of our population estimates declines as we move further away from the census year.
We have also long acknowledged that the International Passenger Survey (IPS), which previously underpinned our international migration estimates, has been stretched beyond its original purpose and that we need to consider all available sources to understand international migration.
Enabled by data-sharing powers in the Digital Economy Act 2017, guided by our data strategy and principles, and building on the ongoing collaboration and data-sharing under way with government departments, we are seizing the opportunity to make use of more data to give us a much richer understanding of how our population is changing.Back to table of contents
Population and migration statistics underpin a wide variety of other statistics (such as unemployment rates) to support a vast range of decisions and inform public debate. For example, the ability to forecast pensions, make decisions about local services (such as the number of school places or the provision of health services for an ageing population) and make decisions about where to site new businesses.
We know that our users are highly interested in how migration patterns are changing and what this means for society and the economy, especially since the coronavirus (COVID-19) pandemic. For example, users are interested in the contribution to and impact on the UK labour market and public services, such as education and healthcare, that migration has. This includes both the national picture and what is happening at a more detailed regional and local level.
It is therefore essential that our population and migration statistics are robust and timely and meet this broad range of user needs.Back to table of contents
In addition to the work set out in Section 4 that is being undertaken to understand migration statistics during the coronavirus (COVID-19) pandemic, we are continuing to progress our strategic plans to transform migration statistics. Using our data-sharing powers through the Digital Economy Act 2017, we have been progressing research into how we can bring a range of government data sources together to build an integrated system for measuring population and migration.
Our progress so far
In January 2019, we published a research engagement report that updated users, and we have since progressed a series of updates on our progress towards our ambition. However, there is still more to be done to embed administrative data in our population and migration statistics.
Current progress on our population statistics transformation
Our population statistics research has centred around creating population estimates from administrative data via our methodology of anonymously linking person records on administrative data sources. These admin-based population estimates (ABPEs) provide estimates of the usually resident population at both national and local level and will be used as a basis for producing admin-based household estimates and estimates of internal migration.
Previous iterations of our ABPEs (formerly known as Statistical Population Dataset) have used presence on multiple data sources as a main inclusion rule. Ongoing research consists of moving away from this approach to an activity-based approach using registration activity and other interactions with administrative sources as "signs of life" in our inclusion rules.
We published our ABPEs in June 2019 and then a further update in 2020. Our design aim was to reduce the over-coverage observed in our previous versions when compared with the official estimates. Our analysis concludes that moving to an activity-based approach has removed some of the over-coverage we observed in our previous ABPEs; however, over-coverage remains. This will need to be reduced further to enable the production of robust estimates using existing coverage adjustment methods.
A priority next step for us is developing our quality frameworks further and using the knowledge gained here to make the best use of the data. This is important for understanding how interactions with administrative sources change over time in both our existing and future data sources. Understanding the strengths and weaknesses of these administrative data sources over time will help us develop our ABPE methods and define the estimation problem we need to solve.
Given the current context of the coronavirus pandemic, we need these frameworks in place so we can understand any changes in the interactions with administrative sources, for example, students moving away from their university accommodation or deferring their courses. Now more than ever we see the need for timelier information about the size and structure of the population and population change.
Current progress on our migration statistics transformation
We have moved away from the International Passenger Survey (IPS) towards the use of administrative data with statistical modelling to deliver new measures of migration. In April 2021 we published our first in-depth look at the sources and methods being used in our admin-based migration estimates (ABMEs) research. International Migration: developing our approach for producing admin-based migration estimates presents research into a new methodology for measuring international migration. The estimates included in it are not official statistics on migration.
The methods used in the report are based on actual behaviours, rather than intentions which underpinned our previous estimates. This means waiting until a person has moved to the UK or left the UK for at least 12 months; there will be time lags before we can use these sources in isolation. We have also identified gaps that are harder to measure, such as students, children and UK citizens.
The development of ABMEs will be supported by statistical modelling. This will bring together our previous modelling work published in April and our ABMES research, with a greater focus on the use of RAPID data. This will help ensure our statistics are timelier and have greater coverage. ABMEs will also allow insightful analysis on subpopulations of migrants, including migrant workers, students and families and dependents.
We are making progress in the delivery of ABMEs supported by statistical modelling but recognise that there are still challenges to overcome. Therefore, throughout this period of development we are continuing to work closely with statisticians in other government departments to build our understanding of complex administrative data sources so that we can draw on the strengths of these together. We are collaborating with the Home Office to better understand how the new immigration system will inform our understanding of migration. Since January 2021, EEA nationals are included in Home Office travel and visa data, and together we are rapidly progressing work to understand the opportunities offered by these data.
We are building on this understanding to consider how best to make use of these sources to produce population estimates.
Our regular migration statistical reports will look different as we adapt our methods and go through a period of transitioning to administrative data-based estimates. They will continue to be badged as Experimental Statistics throughout the transition, to support users in understanding the changes. We will also highlight where we are unable to provide certain statistics or breakdowns of data while we build our new migration estimates and provide guidance on the quality and coverage to ensure the statistics can be interpreted appropriately. Our plans for forthcoming migration statistics publications are in the accompanying summary of recent updates.
We are also collaborating closely with National Records of Scotland (NRS) and the Northern Ireland Statistics and Research Agency (NISRA) to ensure that we can produce comprehensive UK migration statistics. This includes incorporating the Northern Ireland migration estimates produced separately by NISRA and ensuring alignment with the migration statistics for local areas in Scotland and Northern Ireland, produced by the NRS and NISRA respectively.
We will continue to regularly engage with users as we go through this transformation journey and gather feedback to inform our development work.
Population estimates for mid-2021
The mid-2021 population estimates for England, Wales and Northern Ireland will be based primarily on the results of the 2021 censuses adjusted for births, deaths and migration in period between census day and mid-year. Mid-2021 population estimates for Scotland will be rolled forward from 2020 as the census in Scotland is taking place in March 2022.
Following the publication of the 2021 Census results for England and Wales, we will conduct reconciliation work to understand how and why the estimates based on the 2021 Census differ from those based on the mid-year estimates rolled from 2011. The insights gained will be used to improve the production of future population estimates and will result in a revised back-series of population estimates for the period 2012 to 2020 which will increase the coherence between the census and mid-year estimates.
We have been consulting with stakeholders across central and local government and academia throughout this process and will continue to share our findings and consult with stakeholders as we progress.
Our plan for population and migration statistics
We will continue to ensure that the work to transform UK migration statistics aligns with our ongoing research to produce population statistics using administrative data. Migration statistics are an important component of estimating population change, and so we will ensure our research is integrated as we move towards a transformed population and migration statistics system in 2023.
A summary of the revised milestones on this transformation journey, by target date and what we aim to deliver, follows.
Summer 2021, an update on the weighting used in the Labour Force Survey (LFS) and Annual Population Survey (APS).
Autumn 2021, Population of the UK by country of birth and nationality: year ending December 2020. This is subject to progress made on reweighting the LFS and APS.
Autumn 2021, our research plans and progress, taking into account user feedback and data source strengths.
Autumn 2021, ABMEs supported by statistical modelling for Quarter 3 (July to Sept) and Quarter 4 (Oct to Dec) of 2020, building on methods used previously.
End 2021, an update on what we have learnt in producing population counts from administrative data (including timeseries analysis, the impact of the pandemic on our sources, how longitudinal data can be used, and how these take us towards a recommendation on the future of a population and migration statistics system).
Early 2022, ABMEs supported by statistical modelling for Quarter 1 (Jan to Mar) and Quarter 2 (Apr to June) of 2021. Further improvements to modelling methods including the use of more granular administrative information in RAPID and Home Office data.
Early 2022, a methods paper outlining our proposed approach to ABMEs supported by statistical modelling and how this feeds into a transformed population system. It will make use of the best sources available as well as initial proposals for a revisions policy.
From Spring 2022, Census 2021 statistics will be published.
We will publish population and migration statistics based on administrative data, including an understanding of how these are different when compared with census outputs and mid-year population estimates for England and Wales.
- End 2023, the National Statistician will make a recommendation to Government. This will outline what further change is needed to deliver a fully transformed population and social statistics system.
This transformation journey - alongside the work to transform surveys across the ONS - also supports our goal to produce census-type statistics in new ways using government-held administrative data and other data much more often and, importantly, that the outputs we publish are coherent and meet the needs of our users. This is essential for providing evidence to inform the final recommendation to the UK government in 2023 about the future of population and housing censuses in England and Wales. The ONS is also working with the devolved administrations on their respective programmes of work, including the NRS and NISRA, which are responsible for the census and producing population statistics in Scotland and Northern Ireland respectively. Ongoing collaboration will take place between the ONS, NRS and NISRA to consider harmonisation and ensure population statistics across the UK remain comparable.
It was announced in July 2020 that Scotland's census will be moved back by a year until March 2022. The census in Scotland is the responsibility of the NRS and Scottish Government ministers, and their decision is in response to the coronavirus pandemic. Many important activities have been delayed or prevented because of the coronavirus restrictions, including planning and testing the effectiveness and security of systems and processes as well as engagement with certain organisations. This announcement does not affect the England and Wales census, which was held as planned in March 2021. The three census offices of the UK (the NRS, NISRA, and, in England and Wales, the ONS) will work closely to minimise the impact on UK data coherence and to ensure that high-quality data are available to inform policy, investment decisions and the planning of services.
Decisions on the census after 2021 in Northern Ireland and after 2022 in Scotland will be made by the Northern Ireland Executive and Scottish Government ministers respectively.
We have taken the decision not to benchmark the administrative data census outputs with the outputs from Census 2021. Instead, we will ensure that we use the best available sources to produce the best possible outputs from the census - using our understanding of the differences between administrative data and the census and the strengths of all data sources.
We will iteratively develop our transformed population and migration statistics system, taking on board feedback from users and making the best use of new data and methods as they become available. We will rigorously quality assure new methods and share the impact of any changes made, demonstrating the benefits and improvements before implementing them.
Alongside supporting the decision on the future of the census, our transformation journey also plays an important role in supporting our progress in measuring the global Sustainable Development Goal (SDG) indicators in the UK. Migratory status is one of the standard disaggregations required by the UN to measure and report on the global SDGs, and our transformation work will help to improve the evidence base by providing more granular data in the future.Back to table of contents
Our users sit at the heart of our framework for transforming these statistics (Figure 1), which describes the important questions we need to answer to meet their needs.
To meet our users' needs, we need to provide coherent statistics on the size (or stock) of the population and how it changes over time (flows, both nationally and locally). We also need to tell a clear story about what is contributing to this change and show how different groups in the population impact on society and the economy, including on our workforce, communities and public services such as the NHS and schools. This needs to be recognised as the story that is being experienced by our users.
Our users have also told us that they want us to deliver these statistics frequently and in a timely manner to be able to make evidence-based decisions. Our statistics also need to be relevant in a rapidly changing society, and we need to be able to report on their quality.
In May 2021 the Office for Statistics Regulation (OSR) published a review of population estimates and projections. The review identified that we use internationally recognised methods and sources as the basis for population estimates and projections that are fit for purpose for national level estimates. However, the review also identified the need to keep these current and responsive especially at some lower levels where there is more variability and made some recommendations accordingly. We are already progressing work in this area as we consider how best to meet these recommendations, and we will report back to OSR about our plans while continuing to gather feedback on them more broadly.
Feedback gathered previously shows support for how we are reviewing current data sources and looking to maximise the value of administrative data in the future, while challenging us to provide more evidence to ensure we instil confidence in these statistics.
To make sure that our transformation journey continues to be as open and transparent as possible, we will:
regularly publish research and methods as we develop them
continue to present analysis showing the coherence between different sources of information
engage with our users and stakeholders, seeking regular feedback
use our research findings, the best available data and methods, and the feedback from users to make decisions about which improvements to make each year
implement changes to our statistics when and where appropriate, clearly communicating these changes to users in advance of making them
How to contact us
We welcome your feedback on this update and on our transformation journey. If you would like to contact us, please email us at firstname.lastname@example.org.
We have previously presented our research at the Royal Statistical Society Conferences, the British Society for Population Studies Conferences, the Migration Statistics User Forums, and internationally at the International Conference on Administrative Data Research and the International Forum on Migration Statistics.
We are also working with local authorities, experts from academia and other government departments through invited engagement groups to help shape and quality assure our work. Additionally, we are engaging with other National Statistical Institutes (NSIs) across the UK and internationally to share experiences and seek feedback on our plans.Back to table of contents
Contact details for this Article
Telephone: +44 (0)1329 444661