This article summarises the methods used to produce experimental and provisional estimates of long-term international migration flows, published in our Long-term international migration, provisional: year ending June 2022 bulletin. It also summarises previous methods implemented in April 2021; further details are available in our Using statistical modelling to estimate UK international migration methodology, updated November 2021. We explain how our methods have changed and why future revisions of the estimates will be an important part of producing migration estimates going forward.
The research detailed in this paper is carried out as part of an ongoing programme of work to transform population and migration statistics. This work has been accelerated in response to the coronavirus (COVID-19) pandemic. For more details, please see our How we are improving population and migration statistics article, published 15 November 2021.Back to table of contents
We have long acknowledged that the International Passenger Survey (IPS), which underpins our previous estimates of migration, has been stretched beyond its original purpose. We need to consider all available sources and methods to estimate international migration flows (from here on referred to as migration).
In March 2020, the IPS was suspended because of the coronavirus (COVID-19) pandemic. In response, we accelerated our approach for transforming migration statistics using new methods and administrative data, supported by statistical modelling. The IPS resumed operation in January 2021 but, because of its known limitations in measuring migration, we continue to focus on using administrative data supported by statistical modelling.
Going forward, the revision of long-term international migration statistics will be an important part of the production of these estimates. Provisional estimates are released with the expectation they may be revised as more complete data become available. In addition, our methods are still experimental, and we will therefore revise the estimates as our methods mature. Further information on our approach to revisions can be found in our Population and international migration revisions policy (May 2022).Back to table of contents
A common misconception is that it is easy to measure international migration simply by counting people in and out as they cross the border. There are many reasons why it is difficult to count migrants by monitoring cross-border travel data using passport scans at airports. For example:
some people hold two passports and use different passports for incoming and outgoing journeys
the UK and Ireland belong to a free travel zone called the Common Travel Area; people can travel freely between the two countries and movements across the land border between Northern Ireland and the Republic of Ireland are not tracked
Countries across the world have encountered similar measurement challenges and we work closely with many of them to learn and share best practice.
The Office for National statistics (ONS) is committed to exploring all sources of information to produce migration estimates. This includes Advance Passenger Information, which provides passenger information for a large proportion of inbound and outbound air passengers.Back to table of contents
Our latest experimental estimates (for year ending (YE) June 2022, June 2021 and YE June 2020) are produced using a new method that relies less on International Passenger Survey (IPS) data and statistical modelling, and makes greater use of administrative data. This is in line with our ambitions published in April 2021, moving from intentions-based estimates from the IPS to estimates based on actual observed activity in administrative data.
Our latest estimates use different data sources and methods for each nationality grouping. We currently publish estimates on immigration, emigration and net migration for non-EU nationals, EU nationals and British nationals.
Non-EU migration refers to estimates of migration for people who do not hold British or EU nationality. We use Home Office Border Systems Data, which combines visa and travel information to link an individual's travel movements into and out of the country. This dataset is also known as the exit checks dataset, with more information provided in the Home Office statistics on exit checks: user guide.
We continue to use the United Nations' (UN) definition of a long-term migrant: a person who moves to a country other than that of their usual residence for at least a year. So, our first step is to identify which travellers meet the definition of a long-term migrant, filtering out those on long-term visit visas.
We use arrival and last departure dates within a visa period as an approximation for length of stay in the UK. Short trips abroad over the course of an extended period of residence are excluded. If either the first arrival or last departure information is missing, then visa start or end dates are used as a proxy.
Visa periods are constructed by linking together any consecutive or concurrent visas held. If there is a gap of more than seven days between visas, then a new visa period is started. We look at any previous visa period to determine if this is a new long-term immigrant or one who has previously been in the country. If no presence is identified in the country during the 12 months preceding first arrival on a given visa, or the previous visa period had a length of stay of less than 12 months, then this pattern of travel will be considered as identifying a new long-term immigrant.
To measure emigration, we identify previous long-term immigrants with a last departure from the UK during the reference period and record them as a long-term emigrant if they do not return to the UK within 12 months, or if they only return for a short-term stay.
An adjustment is also applied to account for potential over-estimation of both immigration and emigration during the most recent time periods. This relates to where we do not yet have enough data to see a long-term stay of 12 months or more, or a long-term return to the UK within 12 months of a previous departure. Further details are given in our Long-term international migration, provisional: year ending June 2022 bulletin.
In our previous reports (Exploring international migration concepts and definitions with Home Office administrative data from February 2020 and April 2021), we set out our journey to better understand these complex data, including the caveats. For example, we acknowledge that it excludes a minority of non-EU migrants that do not require a visa. Since 2016, the Home Office has published an annual report on exit checks data.
We cannot apply this same method to estimate EU nationals. During the reference period, there was free movement between the EU and UK, where EU nationals were not required to hold a visa to live in the UK.
The latest methodology to estimate the migration of EU nationals is based on previous research to measure international migration using the Registration and Population Interaction Database (RAPID). RAPID currently provides the best insight into the migration of EU nationals.
RAPID is created by the Department for Work and Pensions (DWP) to provide a single coherent view of citizens' interactions across the breadth of systems in the DWP, HM Revenue and Customs (HMRC) and local authorities via Housing Benefit. RAPID covers everyone with a National Insurance number (NINo) and for each person, the number of weeks of "activity" within these systems is summarised in each tax year. Records are then categorised as either long-term or short-term by looking for patterns of interactions with the tax and benefits system.
RAPID is a financial YE March dataset, whereas estimates of migration published in May 2022 are provided on a YE June basis. Therefore, we need to disaggregate the financial year estimates into quarters, and then predict the final quarter (April to June 2022) not covered in RAPID.
The disaggregation and prediction are based on an alternative measure of EU migration derived from IPS data. Long-standing issues with the IPS for measuring migration make it a lower-quality source for estimating levels, but its monthly frequency of reporting does enable estimates of within-year seasonal trends. We derive a monthly EU migration time series, taking IPS data collected before its suspension, and predicting a more current series up to June 2022.
Using the Denton-Cholette method, we applied the predicted IPS series to the financial year RAPID estimates to both disaggregate it to a monthly series and then predict this RAPID-based measure for April to June 2022.
These EU migration estimates are limited by the population coverage of RAPID. Anyone arriving in the UK needs to apply for a NINo to work, claim benefits or apply for a student loan. The coverage is extensive for most migrants because of the wide range of data sources included. However, there are some populations who have less or no interaction with the source datasets, for example, those aged under 16 years. Also, students are only included on RAPID if they are working or claiming benefits. Using findings from previous research linking HMRC's Real Time Information (RTI) data to Higher Education Statistics Agency (HESA) data, we have applied an adjustment that estimates the non-UK born students who would not normally appear on RAPID. In addition, we have included an adjustment to account for under 16s.
The latest release of RAPID uses a combination of migrant registration data from both the MWS and CIS. The CIS data captures certain registrations that historically didn't appear in the MWS, for example registrations for women aged 60+ and men aged 65+. RAPID now includes these registrations as part of the migrant registration measure. Going forward the automation of Apply for a NINo system and improvements to the MWS data extract will reduce disparity.
In addition, the NINo registration service was disrupted during the coronavirus (COVID-19) pandemic. Operational services were partially suspended in March 2020 for certain customers, with a phased return to normal operations completed by April 2021. Consequently, some of the trends seen in EU migration from RAPID may be affected, where some migrants may have arrived in the UK but were unable to register in the normal time frame for their NINo. Such records may not be included in these estimates from RAPID. More information on the suspension of the NINo application process is available in DWP's National Insurance numbers allocated to adult overseas nationals article. Further details of the noted coverage issues in RAPID can be found in our Methods for measuring international migration using RAPID administrative data article.
During the coronavirus pandemic, some EU workers may also have been furloughed under the Coronavirus Job Retention Scheme. It is likely that some of these workers will have left the country for more than 12 months (and therefore should be measured as a long-term emigrant) but will be missed in these estimates as they still appear active on RAPID.
Further research is required and planned for the methodology for determining migration estimates using the RAPID migration dataset.
Our research into British nationals is ongoing, but the complexity associated with identifying these migrants in administrative data means we cannot use such data at this time. For our latest estimates of migrants with British nationality, the IPS data are still our main source of information.
The IPS was reinstated in January 2021, and we use these data as our estimates for January 2021 to June 2022. To cover the period when the IPS was suspended (March to December 2020), we use the SSM time series analysis. This takes the available IPS and administrative data and uses the relationship between them to estimate the missing IPS data. We assume that the pattern of British nationals' immigration to the UK is equivalent to non-EU nationals' emigration from the UK (measured using visa data as described above) and the other way around.
The full technical details of the SSM can be found in our Using statistical modelling to estimate UK international migration methodology, updated November 2021.Back to table of contents
The estimates undergo quality assurance to ensure they are plausible and meet the standards of experimental statistics set by the UK Statistics Authority. This includes comparing the trends with other comparable data and statistics, and consulting with the Government Statistical Service Migration Steering Group and the Migration Statistics Expert Group.
For our most recent Long-term international migration, provisional year ending June 2022 bulletin, the experimental methodology used to produce our estimates means that we are currently not able to quantify this uncertainty. Our most recent estimates therefore have an unknown degree of uncertainty around them and are experimental and provisional.
Confidence intervals were calculated in our previous versions of the model to indicate the degree of uncertainty around each estimate. They assumed that the underlying assumptions in the model were true and made no account for the uncertainty in the assumptions themselves. Users reviewing previous outputs of the model should therefore exercise caution when interpreting estimates of uncertainty.Back to table of contents
The way we measure international migration is constantly developing, taking account of other new data sources, improved methods and the changing needs of our users. Detailed information about our previous methods is given in our Using statistical modelling to estimate UK international migration methodology.
We summarise four different methods we have used to calculate international migration, both developed since the suspension of the International Passenger Survey (IPS).
The first version of the model was published April 2021 and provided migration estimates for March 2020 to June 2020.
Since 2019, our methodology has been exploring time series modelling as an approach to estimate migration using administrative data. Time series modelling was selected because of the strong seasonal trends that are evident in international migration over time.
The state space model (SSM) was introduced in the first version. We projected the trends and seasonality of the previous IPS data forward and then adjusted it by the structural shift seen in the Home Office visa data for non-EU citizens. This version includes assumptions on EU nationals having different travel options during lockdowns. When airports were closed, EU nationals were still more able to travel by ferries and Euro Tunnel.
This version was published November 2021 and produced estimates of migration for Quarter 3 (July to Sept) and Quarter 4 (Oct to Dec) of 2020. Estimates produced from version one of the model (March to June 2020) were also revised because of data updates and improved methodology since April 2021.
The model in version two remained relatively similar to version one, with the main change being that we created an EU proxy series for estimating migration of EU nationals. This new EU trend used in the SSM was based on historical movements of EU nationals, rather than an assumption that EU nationals behave the same as non-EU nationals (as in version one).
Published May 2022. This version relied more on admin data and bench marked the EU estimate to the RAPID data for the first time. Also, for the first time we used the Denton Cholette method to disaggregate the annual data to a quarterly time series.
In Version three, record-level Home Office Border Systems Data were not available for the final eight months of the time series (November 2020 to June 2021). This missing time period was estimated using an aggregated version of the data. We applied the pattern of change observed in the aggregate dataset with the Denton-Cholette method to predict the record level data set for the missing period. This provided us with a rate of change for arrivals on a month-to-month basis. See the Temporal disaggregation of time series guidelines (PDF, 2.3MB) for more information on the Denton-Cholette method).
Also, at the time we did not have an equivalent method for measuring non-EU emigrants. Therefore, we calculated a ratio between emigration (numerator) and immigration (denominator) on a monthly basis from the aggregate dataset, which was applied to the calculated non-EU immigration estimates to estimate emigration. This assumed the trends in the aggregate dataset for the immigration series and emigration series are similar to the trends in the record-level data. This appeared a reasonable assumption at the time, as both are derived from the same source. However, subsequent estimates of emigration using record-level data (introduced in Version four) entail average revisions of around 200,000 emigrants compared with the Version two approach. These revisions are presented in Section 8, Revisions to migration statistics, in our Long-term international migration, provisional: year ending June 2022 bulletin.
This version was published in November 2022 to produce estimates for the year ending June 2020 to the year ending June 2022. The GB estimate for the period of the IPS suspension was updated in light of new data since the IPS reinstatement in January 2021.
Changes to EU estimation
To identify EU students immigrating into the UK long term, we continue to use Higher Education Statistics Authority (HESA) data as the best available data source. Our latest method links this to newly acquired HMRC Pay as You Earn Real Time Information (PAYE RTI) data to better understand how many international students are in employment alongside their studies.
For the outflow adjustment, we have continued to use the method first developed in April 2021, described in our Methods for measuring international migration using RAPID administrative data methodology.
In April 2021 we acknowledged the coverage gap in RAPID for those aged under 16 years. Since then, we have only published migration estimates for EU nationals for those aged 16 years and over. In November 2022, we have introduced a new adjustment to RAPID to help fill this coverage gap. The adjustment uses an adult-to-child ratio derived from the International Passenger Survey (IPS). Where IPS data are not available (namely in 2020), a five-year average ratio (2016 to 2019, 2021) is applied. This ratio is then applied to RAPID to estimate the number of under 16s to add to the RAPID estimate. This ratio is calculated separately for both inflow and outflow.
Changes to non-EU estimation
We will use record-level Home Office Border Systems Data for the whole time series.
We will use the new method for estimating long-term emigration (equivalent to that used for immigration).
We will make adjustments to account for potential over-estimation of both immigration and emigration for most recent time points.
Further detail on these changes can be found in International migration research progress update: November 2022.Back to table of contents
The current approach, as published in November 2022:
provides a calculated estimate of migration flows at a time when traditional data collection through the International Passenger Survey (IPS) was absent and migration patterns were changing because of the coronavirus (COVID-19) pandemic and Brexit
uses a wide range of existing and new data sources to observe the behaviour of migrants, greatly helped by the data-sharing powers of the Digital Economy Act 2017
provides timely, but provisional estimates that are subject to revision as data are finalised and methods refined
includes estimates increasingly derived by administrative data sources that capture actual rather than intended migration patterns (as was measured by the International Passenger Survey (IPS))
includes constantly evolving methods, taking account of other new data sources, improved processing methods and the changing needs of our users; it will join up with other Office for National Statistics (ONS) transformation work on population and migration statistics
involved developing our measures, which has supported and encouraged greater involvement and cooperation of experts and other government departments in the process of producing migration statistics; this leads to enhanced trust in, and an understanding of, these statistics
The current approach, as published in November 2022:
has discontinuity with migration flow estimates prior to 2020
can only produce headline figures for migration, by direction of flow and broad nationality groups (British, EU and non-EU); this does not meet all our users' needs
includes estimates that do not currently cover all types of migrants, as some migrants are harder to identify within administrative data sources
includes estimates that are currently experimental; there is a degree of uncertainty around them, which we are unable to quantify at this point in time
requires users to adapt to the new practice of regular revisions of migration estimates
The Office for Statistics Regulation's (OSR's) review on migration statistics encouraged us to improve and broaden our user engagement as well as ensuring that we have coherent plans across our transformation work. We recognise the need to continuously improve our methods with our users. If you would like to find out more or have any feedback, please email us at email@example.com.
We continue to explore other data sources as they become available and existing sources as they are updated. In the coming year, we plan to:
- compare our outputs to final Census 2021 data, which will provide the most robust and comprehensive picture of the England and Wales population
- explore insights available from Advance Passenger Information
- investigate new visa requirements for EU nationals, with the intention to incorporate associated data into our method
- investigate how other types of migrants, including refugees and asylum seekers, can be better incorporated into future versions of our method
- explore how these models can be improved to provide more timely and frequent measures of migration, and with breakdowns, including by reason of travel, geography, sex and age
- work with other government departments to ensure published migration statistics are coherent
Office for National Statistics (ONS), released 24 November 2022, ONS website, methodology, Methods to produce provisional long-term international migration estimates: November 2022
Contact details for this Methodology
Telephone: +44 1329 444661