Admin-based long-term international migration estimates QMI

• National Statistic: Experimental Statistics

• Data collection: administrative data

• Frequency: bi-annually

• How compiled: administrative data supported by survey and statistical modelling

• Geographic coverage: UK

• Related publications: Long-term international migration, provisional

This QMI report contains information on the quality characteristics of the data (including the European Statistical System’s five dimensions of quality) as well as the methods used to create it.

Information on the historical methods and quality for the long-term international migration (LTIM) estimates based on the International Passenger Survey (IPS) can be found in our Long-Term International Migration QMI.

The information in this report will help you to:

• understand the strengths and limitations of the data
• learn about existing uses and users of the data
• understand the methods used to create the data
• help you to decide suitable uses for the data
• reduce the risk of misusing the data

• Long-term international migration (LTIM) statistics estimate the flow of migrants to and from the UK.
• We define a long-term international migrant (applying the United Nations definition) as someone who changes their country of usual residence for a period of twelve months or more, so that the country of destination effectively becomes the country of usual residence.
• We are currently transforming the way we produce population and migration statistics and have reclassified our migration statistics as “Experimental Statistics” in line with the Office for Statistics Regulation (OSR) guidance.
• Our estimates use different data sources and methods for each nationality grouping and we currently publish estimates on immigration, emigration and net migration for non-EU nationals, EU nationals and British nationals.
• While we are continuing to develop the methodology for our international migration estimates, the breakdown of data and levels of detail available are still somewhat reduced compared with those historically produced using the International Passenger Survey (IPS).

Overview

Long-term international migration (LTIM) statistics are the most comprehensive estimates of immigration, emigration and net migration flows to and from the UK.

Our estimates are produced using a method that relies less on International Passenger Survey (IPS) data, and makes greater use of administrative data. This is in line with our ambitions to move from intentions-based estimates from the IPS, to estimates based on actual observed activity in administrative data.

Uses and users

These statistics are used both nationally and internationally by government departments, academia, special interest groups, the media and the general public. International migration is also an important component of population change and is used in the production of mid-year population estimates and population projections. These are then used by central and local government and the health sector for planning and monitoring service delivery, resource allocation and managing the economy.

There are many sources of migration estimates available to our users and each of them may relate to slightly different populations which can cause some confusion as to which is the most appropriate dataset to use. We have been working across the Government Statistical Service (GSS) to pull together these estimates into a single dashboard. The aim is to allow users to access all datasets from a single source, alongside the relevant descriptions and other information. This will enable our users to understand the differences between the datasets and their appropriate uses. To find out more about this work please visit the GSS migration statistics publication strategy.

It is our mission to provide the best insights on population and migration using a range of new and existing data sources to meet the needs of our users. This is increasingly important in a rapidly changing policy context, where we know our users need better evidence to support decision-making at both national and local levels. We are therefore transforming our population and migration statistics to put administrative data at the core of what we do.

Strengths and limitations

The LTIM estimates demonstrate the potential of administrative data to better meet user needs and are based on multiple sources of data broken down by nationality groupings. This approach is to ensure that the best data source for each of the British, EU and non-EU estimates are used and are as representative as possible. Our methodology is continuously refined and we publish our estimates, with revisions as required, to ensure that our users are able to make appropriate use of the data. These improvements and revisions can be seen as both a strength and a limitation of our estimates. We understand that our users need regular, but consistent updates on LTIM estimates, but we also want to provide our best possible estimates with every release. Further information on our approach to revisions can be found in our Population and International Migration Statistics Revisions Policy.

The strengths of our current approach last updated in May 2023 are that:

• we provide timely, provisional estimates, these are subject to revision, as data are finalised and methods refined
• we include estimates that are increasingly sourced from administrative data that capture actual rather than intended migration patterns (as was measured by the IPS)
• we include improved methods, taking account of new data sources, better processing methods and the changing needs of our users, joining up with other Office for National Statistics (ONS) transformation work on population and migration statistics
• we have had greater involvement and collaboration of experts and other government departments in the process of producing these migration statistics; this leads to enhanced trust in, and an understanding of, these statistics, as well as more consistent methods across all government international migration data

The limitations are that:

• currently we can only produce headline figures for migration, by direction of flow and broad nationality groups (British, EU and non-EU) which does not meet all the needs of our users
• we include estimates that are currently provisional which means that there is a degree of uncertainty around them as we refine our methods and data sources
• the methodology for calculating uncertainty intervals using administrative data is still in development so we are unable to provide estimates with uncertainty intervals at this time
• we require users to adapt to the new practice of regular revisions of migration estimates because of the transition to estimates based on activity rather than stated intentions

Recent improvements

Delivering new measures of international migration using administrative data sources presents a substantial change in the way we measure it. Until 2020, estimates of international migration were based on the IPS, which interviews passengers entering and leaving the UK to record how long they were intending to remain in or out of the UK in the next 12 months. We now predominantly use administrative data, which are retrospective and tell us about actual activity that has already happened.

There have been several improvements seen in the LTIM estimates since the transformation from survey-based estimates to administrative data-based estimates. These include:

• the use of a State Space Model to generate migration estimates in the absence of the IPS because of its suspension during the coronavirus (COVID-19) pandemic for British nationals
• the use of Home Office Borders and Immigration data to provide international migration estimates of non-EU nationals
• the use of the Department for Work and Pensions (DWP) Registration and Population Interactions Database (RAPID) in the estimation of international migration for EU nationals
• the inclusion of people making an asylum application
• the re-introduction of reason for migration as part of the international migration estimates, based on visa type for non-EU nationals and using the IPS to inform estimates of EU and British nationals
• the use of Census 2021 outputs and other relevant sources to calibrate our results have helped us refine the rules around our definitions of LTIM based on activity within the UK

This section provides a range of information on the quality and characteristics of the data and identifies issues that should be noted when using the output.

We have developed guidelines for measuring statistical quality based on the five European Statistical System (ESS) quality dimensions. This report addresses the quality dimensions and other important quality characteristics, which are:

• relevance
• timeliness and punctuality
• coherence and comparability
• accuracy and reliability
• output quality
• accessibility and clarity

Relevance

(The degree to which statistical outputs meet users’ needs).

We have regularly shared our progress with users, including highlighting gaps in coverage and content to ensure the statistics are developed in line with their needs and remain relevant. Feedback received following previous releases of the Long-term international migration (LTIM) estimates and related bulletins have directly fed into the ongoing development of these statistics.

Our recently announced public consultation is seeking to understand views on our wider plans to transform our migration and population statistics. Responses to this consultation will also feed into the ongoing development of the LTIM estimates.

We acknowledge that at this stage of the transformation we are not yet able to provide the level of granularity (detail) and insight that is required by our users. We continue to work with our users to understand how to best meet their needs.

Timeliness and punctuality

(Timeliness refers to the lapse of time between publication and the period to which the data refer. Punctuality refers to the lag between planned and actual publication dates).

We currently publish our LTIM estimates twice yearly, in May and November. The data are published with a five-month period between the reference date and publishing date, so the data published in May will cover the period up to December of the previous year. The data published in November will cover up to the end of June of the same year.

Timeliness takes on a slightly different meaning when we refer to long-term international migration estimates sourced from administrative data compared with survey data. When using survey data, the results are based on intentions and therefore reference what the person intends to do in the next 12 months. However, administrative data are based on activity which has already occurred, meaning that a person is counted as a long-term migrant if there is evidence of residency over 12 months. This reduces the timeliness of our final estimates. We have developed methods for our EU and non-EU nationals estimates that try to compensate for this and provide provisional estimates for the latest 12 months.

There is also a delay related to the EU estimates created by using the Registration and Population Interactions Database (RAPID). This database is made available to the Office for National Statistics (ONS) on an annual basis in Quarter 3 (July to September) each year for the previous tax year. This means that for our year ending (YE) June estimates we have data up until March, so we are missing three months of administrative data for when we publish in November. For our YE December estimates (published in May each year), we are missing nine months of data for EU nationals. These data are estimated using signal data provided by the International Passenger Survey (IPS). Further details are provided in our Methods to produce provisional long-term international migration estimates.

Coherence and comparability

(Coherence is the degree to which data that are derived from different sources or methods, but refer to the same topic, are similar. Comparability is the degree to which data can be compared over time and domain, for example, geographic level).

The difference in methodology between the migration estimates based on the IPS and the more recent administrative data based series means that the two datasets are not comparable.

We are aware that the comparison of estimates to earlier time periods is of great importance to our users. This is why we have published a backseries of estimates based on our new methodology that covers the period of YE December 2018 to YE December 2022. While the previous (IPS-based) published estimates are not comparable with our new results, a backseries of data are available for our users to allow the analysis of long-term migration patterns.

We are working within the ONS to improve the coherence of our migration estimates in relation to other statistics, including population estimates and the Labour Force Survey. We are continuing to transform our population, migration and social statistics. This is to meet the need for more timely and frequent statistics that enable us to understand our population and how it changes on an ongoing basis. In July 2022, we announced our ambition for a new Dynamic Population Model (DPM), building on our earlier research to develop population and migration estimates based on administrative data; our LTIM estimates are an integral component of the DPM. This sets a clear path for the future of population and migration statistics to produce timely population estimates at a national and local authority level. More information can be found in our Dynamic population model for England and Wales: July 2022 article.

Accuracy and reliability

(The degree of closeness between an estimate and the true value. Reliability is the closeness of the initial estimated value to subsequent estimated values).

The LTIM estimates for non-British nationals are derived from administrative data collected and processed by the Department for Work and Pensions (DWP), HM Revenue and Customs (HMRC) and the Home Office (HO). These administrative data are sourced from operational systems and delivered to us for use in statistics production and statistical research. As such, these administrative data are reliant on the accuracy of information recorded and processed in these operational systems, which were not designed for statistical purposes. The LTIM estimates for British nationals continue to be based on data from the IPS.

The Office for Statistics Regulation (OSR) have recommended users need a clear understanding of uncertainty associated with international migration estimates and how the estimates can be used appropriately. We are committed to working on and providing uncertainty measures for our international migration estimates. We are progressing with research and will be publishing a working paper series soon that covers some of the main sources of uncertainty in international migration estimates. Specifically, the working paper series will focus on quantifying uncertainty associated with adjustments, modelling, and survey-based estimates.

Output quality

The estimates undergo quality assurance to ensure that they are plausible and meet the standards of experimental statistics set by the UK Statistics Authority. This includes comparing the trends with other comparable data and statistics, and consulting with the Government Statistical Service Migration Steering Group and the Migration Statistics Expert Group.

Accessibility and clarity

(Accessibility is the ease with which users are able to access the data, also reflecting the format in which the data are available and the availability of supporting information. Clarity refers to the quality and sufficiency of the release details, illustrations and accompanying advice).

LTIM estimates are available online and can be downloaded free of charge in CSV and Excel file types. Graphs, textual background information and supporting documents are provided as part of each statistical release.

Any additional enquires regarding international migration can be made via email to pop.info@ons.gov.uk or by telephone on +44 1329 444661. It may be possible to meet additional data requests, but these may be chargeable depending on the time required to produce the additional data requested. Metadata describing the limitations of additional data are provided with individual requests. User-requested data are also published on the ONS website.

For information regarding conditions of access to data, please refer to the following web pages:

• Terms and conditions (for data on the website)
• Freedom of information
• Accessibility statement for the Office for National Statistics

Concepts and definitions

(Concepts and definitions describe the legislation governing the output and a description of the classifications used in the output).

We use the United Nations (UN) recommended definition of a long-term international migrant: “A person who moves to a country other than that of his or her usual residence for a period of at least a year (12 months), so that the country of destination effectively becomes his or her new country of usual residence.” For more information, see Recommendations on Statistics of International Migration (PDF, 5.0MB).

We use the Government Statistical Service (GSS) harmonised principles. The main terms and definitions used in our outputs can be found in our International migration – terms, definitions and frequently asked questions report.

All sources of international migration statistics are valuable in their own right and provide us with an understanding of the flows of international migrants, as well as the numbers of international migrants in the resident population.

Geography

The LTIM estimates cover the whole of the UK and are currently only published at the UK level. The LTIM estimates are broken down to high-level nationalities of GB, EU and non-EU. The LTIM estimates are an important component in the production of mid-year population estimates and population projections and the methods used to break down our UK wide estimates to local authority level can be found in our Mid-year population estimates QMI.

Why you can trust our data

We are the UK’s largest independent producer of statistics. Our data policies detail how data are collected, secured and used in the publication of statistics. We treat the data that we hold with respect, keeping it secure and confidential, and we use statistical methods that are professional, ethical and transparent.

Here we summarise:

• how data are collected
• data sources
• current methodology
• assumptions

How we collect the data, main data sources and accuracy

The long-term international migration (LTIM) estimates are produced using a combination of administrative data and survey data. The main data sources consist of:

• Home Office Borders and Immigration data linking travel data, visa data and other sources in the Initial Status Analysis (ISA) system for non-EU nationals
• Home Office statistics on asylum and resettlement applications
• the Departments for Work and Pensions (DWP) and HM Revenue and Customs (HMRC) Registration and Population Interactions Database (RAPID) for EU nationals
• linked Higher Educational Statistics Authority (HESA) and Real Time Information (RTI) data to understand the number of students in employment to apply an adjustment to EU nationals
• International Passenger Survey (IPS) for GB nationals, the under 16 years of age coverage adjustment for EU nationals, and data for EU nationals forecasting

Home Office Borders and Immigration data

The Home Office Borders and Immigration data combines visa and travel information to link an individual's travel movements into and out of the country, with more information provided in the Home Office statistics on exit checks: user guide.

As with all large complex data collections based on administrative data, the data received may not always be complete and fully accurate. Records might not match in the ISA system because of administrative errors resulting in incomplete journeys. Incorrect status may be assigned to individuals where coverage for outbound journey is incomplete, particularly for train and maritime travel or travel through the Common Travel Area (CTA).

Registration and Population Interactions Database (RAPID)

RAPID is created by the DWP to provide a single coherent view of citizens’ interactions across the breadth of systems in the DWP, HMRC and local authorities (LAs) via housing benefit. RAPID covers everyone with a National Insurance number (NINo). For each person, the interactions within these systems are summarised in every tax year, from the year that ended in 2011 to the most recent available (year ending 5 April 2022).

In general, anyone arriving in the UK for the first time needs to apply for a NINo in order to work, claim benefits or apply for a student loan. The coverage in RAPID of the UK population is therefore extensive because of the wide range of data sources included. However, there are some populations where activities with the source datasets are less well covered or where a NINo would not have been issued, and we make adjustments to our EU estimates to account for this under-coverage.

International Passenger Survey (IPS)

The International Passenger Survey (IPS) is a sample survey of passengers arriving at, and departing from, UK air and sea ports and the Channel Tunnel. It is carried out by the Office for National Statistics (ONS) for a range of public and private sector organisations. In particular, it provides figures used for the travel account of the Balance of Payments, captures data on international tourism, and provides data on the numbers and characteristics of long-term international migrants.

It is important to note that these data are intentions-based. For example, the survey asks how long each migrant intends to remain in or out of the UK, as opposed to recording what they have done on their later journeys.

How we process the data

We have a published methodology piece Methods to produce provisional long-term international migration estimates which goes through in detail how we process our data sources to generate long-term international migration estimates.

How we analyse and interpret the data

The individual migration flows are combined to provide total immigration and emigration estimates for the UK, alongside the calculation of net migration (immigration minus emigration).

Alongside the nationality grouping breakdowns of the migration estimates, we also provide reasons for migration. The use of Home Office administrative data means we can analyse the types of visas that LTIM non-EU nationals are granted. This allows us to infer the reason for migration. The International Passenger Survey (IPS) asks people their reason for migration and we can apply the proportions obtained from respondents to our Registration and Population Interaction Database (RAPID)-based EU estimates and estimates of British nationals where visa records are not available. Research by the Swiss Federal Statistical Office shows that when administrative and survey data records are linked by individuals, the self-reported reason for migration does not always match the reason they have been granted permission to enter. This is discussed in their Exploring immigration reasons by linking administrative and survey data (PDF, 370KB). Users should therefore exercise caution when comparing breakdowns between non-EU, and EU and British nationals.

How we quality assure and validate the data

Rigorous quality assurance (QA) is carried out at all stages of production. Specific procedures include:

• communicating with data suppliers to improve and confirm our understanding of the data and their QA checks
• processing the input data to check that any values that seem abnormal are accurate
• building regular quality assurance checks into the analytical code and carrying out these checks to ensure the code is working as it should
• scrutinising trends in other data sources that can be considered signal data to international migration estimates
• comparing estimates from different sources to check for outliers
• checking output tables to ensure that there are no errors or inaccuracies during the creation of published tables
• consulting with the Government Statistical Service Migration Steering Group and the Migration Statistics Expert Group

How we disseminate the data

LTIM estimates are available online for the UK for the period between 2018 to 2022 using the new administrative system. Previous sets of LTIM estimates for the UK are also available on our website. The GSS dashboard released in August 2022 provides information on international migration releases and datasets.

How we review and maintain the data processes

We are continually improving the LTIM estimates throughout our feasibility research. As such, we regularly review and update our data processes.

Future revisions to the LTIM estimates may be required to reflect occasional or post-census revisions. This is in line with our Population and International Migration Statistics Revisions Policy.