2019 collection rehearsal evaluation report for Census 2021, England and Wales

This article summarises our evaluation of the 2019 Collection Rehearsal for Census 2021 in England and Wales. It details the main findings and lessons learned that we will be carrying forward to the main Census in 2021 from both the collection rehearsal (which covered households and communal establishments), and the Census Coverage Survey (CCS) rehearsal that immediately followed it. It does not cover the 2021 Census processing rehearsal, which was completed at the end of summer 2020.

It is part of a series of articles published on 01 October 2020 updating our design and plans for Census 2021. These plans take into account our past experience and development of our end-to-end statistical design, including feedback from local authorities on our quality assurance plans. We have adapted our plans and are planning contingencies in light of the coronavirus (COVID-19) pandemic. More information is available in our overview.

The census is the largest statistical exercise that the Office for National Statistics (ONS) undertakes, producing statistics that inform all areas of public life and underpin social and economic policy. It provides a wealth of information at small geographies to inform local planning and decision-making. It is, therefore, vital that everyone takes part and is counted and that the statistics produced are accurate and meet user needs.

Proposals for Census 2021 were published in a Government White Paper in 2018, Help Shape Our Future: The 2021 Census of Population and Housing in England and Wales. This set out quality objectives, the most important of which is achieving a high and consistent response rate (See Section 2: Census quality objectives and operational plans). The Census 2021 statistical design article sets out how the census has been designed to meet the quality objectives and deliver results that are fit for purpose and meet user needs.

The 2019 Rehearsal is an integral part of testing our systems and processes for Census 2021 in England and Wales and enabled us to test:

• the quality of our preparations (including the address frame underpinning the rehearsal)
• the main processes and systems for the operations for carrying out the census for households and communal establishments
• our community engagement and communications strategies
• the Census Coverage Survey (CCS)

In addition, it enabled us to work with our suppliers in a live-operation simulation.

Between September and December 2019, we undertook a rehearsal in four local authority areas - Carlisle, Ceredigion, Hackney and Tower Hamlets - as part of the preparations for our first "digital-first census" on 21 March 2021. These locations were selected so that we could rehearse in different types of areas, including urban and rural; densely populated and hard-to-count areas; areas with a number of hard-to-access properties (for example, managed apartment blocks); areas with poor internet connection; areas with a large population from a single minority ethnic group, along with areas with diverse ethnic groups; and an area with Welsh language speakers.

The rehearsal was not planned as a statistical test and therefore no statistical outputs will be published.

Main successes

The rehearsal successfully tested our preparations, our operational processes and systems, our digital platform, our engagement and communications strategies and the Census Coverage Survey with no significant problems. Main successes include:

• our publicity and communications campaign and our community engagement, which reached the public and specific population groups
• the performance of the electronic questionnaire (eQ), which enabled us to exceed our target of 75% online returns set for the census in March 2021
• the successful printing and delivery of questionnaires
• the receipting, tracking and automatic data capture of questionnaires

Main areas of improvement

While the high-level results and return rates are encouraging, it is the underlying analysis that has provided us with a detailed understanding of how our systems and processes have worked and where we need to make improvements to the design for Census 2021. The main areas where we plan to make improvements for Census 2021 include:

• earlier and more focused engagement and the further development of our engagement model
• the further development of our management information (MI) to identify issues earlier to ensure engagement has the most positive impact
• making better use of our field force - training them to engage and not just knock on doors, providing additional technology training for field officers, and using field officers' time more efficiently

Detailed tables that summarise the improvement lessons, and the resultant actions taken, can be found at the end of this article as an improvement lessons summary.

The rehearsal covered 331,359 households across the four rehearsal areas. In total, we received 101,774 household returns (return rate of 30.7%), of which 83,316 were submitted online (81.9% online share).

It took place between September and December 2019. We launched in September 2019 with our campaign website going live. 2019 Rehearsal day was on 13 October 2019 with field follow-up continuing until 14 November 2019.

This collection rehearsal is core to ensuring the success of Census 2021. It is an integral part of testing our processes and systems. Our four core objectives were to:

• enable us and our suppliers to undertake Census 2021 with the experience of having done something similar before
• prove that all the different parts of the operation worked and were integrated
• evaluate our ability to scale up to deliver a full census (operations and technology)
• test our designs including our ways of working, field staff, suppliers, return rates, logistics, management information, incident response, digital technology, and data security

The collection rehearsal, which covered households and communal establishments, was followed by a Census Coverage Survey (CCS) rehearsal. The CCS is a survey taking a 1% sample of the census population. Instead of being a self-completion survey, as in the census, the CCS is interview driven. Respondents are asked a range of similar questions to those in the collection rehearsal. This survey helps us find households we have missed, as well as individuals within known households that may have been missed. The data is used during the processing to assist with under-enumeration and statistical quality.

We have also been monitoring census experiences in other parts of the world, including those of our partners in Northern Ireland, and most notably those of New Zealand (PDF, 704KB)d. Where possible we have incorporated what we have learnt from our 2019 Rehearsal with lessons from the experience of others to build for 2021.

What the 2019 Rehearsal tested

Following the positive impact of the 2009 Rehearsal on the 2011 Census, we undertook a similar rehearsal for Census 2021. A significant difference since 2011 is that Census 2021 will be a digital-first census.

The 2019 Rehearsal enabled us to test:

• the quality of our preparations (including the address frame underpinning the rehearsal)
• the key processes and systems for the operations for carrying out the census for households and communal establishments
• our community engagement and communications strategies
• the Census Coverage Survey (CCS)

Similarly to 2009, we worked with a number of external suppliers, who also had the opportunity to rehearse their processes and systems as part of the 2019 Rehearsal.

While the 2019 Rehearsal replicates the actual census as closely as possible, there are a number of areas which we cannot rehearse fully. For example:

• the small scale and localised areas prevent us from rehearsing communications and engagement on a national scale
• the rehearsal is not compulsory and therefore we cannot rehearse non-compliance processes
• systems and processes which have been proved to work on a localised level will need to be scaled up
• the campaign for the rehearsal is local rather than national; raising awareness of the census and ensuring that people are aware of what they need to do, and why, is limited when rehearsing in just four local authorities compared with undertaking a national campaign where more communication channels will be open

While not part of this article, following lessons from the 2011 Census, we also rehearsed our processing - the process for creating outputs and dissemination of data. This was undertaken separately and ran between February and July 2020.

Geographic scope and size of the 2019 Rehearsal

The 2019 Rehearsal areas were selected to give a range of area types

Ceredigion is an area which has a high proportion of Welsh language speakers and includes a large rural population. The area helped us understand what mobile connectivity issues residents and our field officers may experience. Ceredigion also includes a student population, a range of second homes, and some areas with digitally excluded populations.

Carlisle is an area that represents a range of conditions for our field staff. It includes rural areas, urban areas, areas with low broadband take-up, a varying pattern of internet connectivity, and some student populations.

Tower Hamlets and Hackney are two neighbouring local authorities in London, which enabled us to rehearse working across a local authority border. Tower Hamlets and Hackney enabled us to rehearse the census operation in urban, densely populated and traditionally hard-to-count areas. Tower Hamlets also includes a student population, a number of hard-to-access properties (for example, managed apartment blocks) and a large population from a single minority ethnic group. Hackney is home to a number of smaller populations of diverse ethnic minority groups and enabled us to rehearse our community engagement and communications strategies.

Timing of the 2019 Rehearsal

The rehearsal was conducted over several weeks. Following a successful low-key launch of the census rehearsal campaign website in the second week of September, the 2019 Rehearsal started in the third week of September, with letters inviting households to take part posted in the final week of September. The rehearsal date was 13 October 2019 with the rehearsal officially closing on 14 November 2019.

As with the 2009 Rehearsal, which was also held in October, the time of year for the 2019 Rehearsal does not match that of the actual census (with Census 2021 to be held in March). The timing of the rehearsal can have impacts on return rates. For example, field operations on dark nights have different results to field work on Spring evenings. This is one reason why the return rates for rehearsals should not be directly compared with the actual census return rates.

The address frame

The address frame underpins our entire census design and operation. We are using the AddressBase Premium product (provided by Ordnance Survey and GeoPlace) as the core dataset for the address frame for Census 2021, as was the case for the 2019 Rehearsal.

The address frame was used to print addresses on letters of invitation to participate in the collection rehearsal. It was also used to populate the questionnaire tracking system. This system enables the tracking of every questionnaire from delivery to processing, through a unique identification code.

331,359 letters of invitation to participate in the collection rehearsal were addressed and posted successfully, with 90% (298,071) containing a Unique Access Code (UAC) and 10% (33,288) containing a paper questionnaire (which also includes a UAC). Of these, 3,473 (1% of the total) were "undelivered as addressed" and some addresses for blocks of flats had no flat numbers making them difficult to follow-up.

To ensure the address frame is of the highest quality possible, we undertook field address checks, clerical checks, and a check of addresses within the Census Coverage Survey (CCS). For the CCS, field staff compile their own address list for a small selection of postcodes as a way to analyse under-coverage. This list is matched against the list of addresses from the address frame and assists the location of any missing households.

For 2021, further work is needed to improve the address frame to ensure it is of the highest quality possible. We will be substantially increasing the number of clerical checks performed on the frame. This will use all available admin data and a number of other data sources. We will also be introducing new quality checks.

The systems

Our digital systems, which were built and integrated in-house, were thoroughly tested during the 2019 Rehearsal. These included the campaign website; the electronic questionnaire (eQ); and response management, which allows for electronic tracking of questionnaires.

While some systems were scaled down versions, the rehearsal was able to fully test them and demonstrated they worked and integrated with suppliers' systems well. There were no major problems or issues during rehearsal and minor incidents were resolved in live.

Enhancements to systems (including those recorded during the rehearsal) will now be incorporated where possible and systems will be scaled up to perform at full census capacities. We are also now developing some systems that were not available for the rehearsal, including non-compliance and telephone capture.

External suppliers

Relationships with our suppliers have been developed since the award of contracts. For the rehearsal, we worked collaboratively with them at both a strategic and operational level. There were some minor delays in supplier on-boarding and some decisions about system build, which meant that the full logistics supply chain could not be fully tested. The recruitment of field force was a challenge, both in terms of recruitment, job descriptions, the target numbers and the training of field officers once in post. The assisted digital service was also not exploited to its full potential.

For Census 2021, to reduce the recruitment challenge we have realigned our recruitment tranches to smooth out the numbers of personnel required, removing a late peak in requirements. We are planning to employ a greater proportion of field staff in the early stages of the census to help mitigate against recruitment shortages later. We also plan to improve recruitment processes with improved job descriptions and with mobile flexibility built into a proportion of the contracts and for much improved training. Changes are also being made to better exploit the assisted digital service, which include ensuring field staff are able to fully and appropriately signpost respondents who need additional help.

Security

Our approach to security was successfully tested in the 2019 Rehearsal. This was based on a staged set of assurance design and assurance activities that highlighted the relevant security requirements, identified security risks in the design and operations of 2019 Rehearsal services and oversaw the implementation of mitigations to reduce these risks. These assurance stages applied to all internal and supplier-led rehearsal activity. The reporting of this assurance gave us confidence in the design of security and its implementation in the eventual services operation.

To reinforce the security approach further, a range of reviews for elements of the rehearsal were undertaken from external security organisations including the National Cyber Security Centre (NCSC) and specialist technical testing at component and service levels. An independent security review was commissioned, performed by an NCSC-accredited organisation, to validate the security plans, design and assurance for the rehearsal. This review recognised the good security implemented and also provided areas for development that were addressed prior to the rehearsal.

Significant positive and developmental lessons were learned from the rehearsal and the opportunity it gave to test the security approach. For 2021, improvements to the end-to-end staged assurance have been implemented, including on supplier assurance activities, the overall risk reporting at both census and corporate levels and the closer integration of security activities within the census workstreams, particularly focused on any operational changes in programme development timescales.

Recruitment, pay and training of the field force

The recruitment, pay and training of the field force staff, including for the CCS, was managed by an external supplier. This is an area where there will be a substantial scale-up of operations for 2021.

Recruitment

The 2019 Rehearsal required the recruitment of just over 400 field staff compared with up to 40,000 required for Census 2021. These field officers were employed in three different tranches, with the peak coming in the third tranche. The target number of field staff to be deployed on "day one" was not met and this impacted on the later targets. This was coupled with the field force not being sufficiently skilled, partly due to job descriptions not being clear and lack of training in the appropriate skills required. 

To help with the recruitment challenge we have realigned the tranches to smooth out the numbers required, reducing the late peak in requirements. As well as planning to employ a greater proportion of field staff in the early stages of Census 2021 to help mitigate against recruitment shortages later, there are plans to improve recruitment processes with improved job descriptions and with mobile flexibility built into a proportion of the contracts. We are also exploring more generic job titles and roles to enable any gaps in the field structure to be covered. Together with our supplier, we are investigating an over-recruiting contingency option in order to account for under-recruitment experienced in the 2011 Census and 2019 Rehearsal. In addition, we are developing better management information to enable the identification and management of recruitment gaps. The job descriptions for 2021 will also be clearer on the level of technology use that is required for each job and will clearly state the precise work locations and the level of mobility required. 

Pay

The payroll worked well. Staff were paid on time although there were some issues with inconsistency between payroll week and working week. As a result, the census field force payroll week will move, from the Saturday to Friday used in the 2019 Rehearsal, to Monday to Sunday to align with the business reporting week for Census 2021.

Training

The training of field staff needs to be improved for 2021. The training failed to equip field staff with all the required role-specific knowledge and there was little ability to develop staff knowledge when frequent queries arose.

For 2021, the training will be split between pre- and post-employment and more robust training plans are being developed. Pre-training will focus on the use of the technology with time for the field staff to familiarise themselves with it, the importance of using the technology correctly; and behaviours on how to increase return rates.

Once the field staff are operational, further training will be developed, incorporating learning from the rehearsal. This new training will ensure staff are given enough notice to join training webinars and that they can regularly update their knowledge (for example, by video clips going out to all staff when frequent queries arise).

Households

Prior to the start of the collection rehearsal, various activities were undertaken to engage the public with the forthcoming rehearsal in the four local authority areas. These are discussed further in the Communications, campaign and community engagement section of this article.

The delivery of the questionnaire

All 331,359 households in the 2019 Rehearsal areas received an initial letter informing them that the collection rehearsal was taking place. The majority (90%) received a unique access code (UAC) to enable them to undertake the census online, with the remainder (10%) being sent a paper questionnaire (which also included a UAC). In Ceredigion, both English and Welsh language initial letters (and paper questionnaires in both languages where these were the initial contact) were sent to each household.

Electronic questionnaire (eQ)

Households successfully accessed the electronic questionnaires using their UACs. Positive feedback was received from respondents on the household electronic questionnaire. The average completion time for the household electronic questionnaire was 21 minutes 13 seconds, with minimal difference between devices:

• desktop time: 20 minutes 48 seconds
• mobile time: 21 minutes 54 seconds
• tablet time: 21 minutes 36 seconds

It is expected that changes due to be implemented for 2021, such as address look-ups and Search-As-You-Type, should reduce response average times.

Respondents within rehearsal areas showed a relatively high willingness to complete online with 83,316 (81.9% of the total 101,774) household returns.

Paper questionnaires

In total, 18,461 paper returns were submitted. For paper questionnaires, no specific issues were raised on usability, layout, or accessibility. The highest peaks of household paper questionnaire requests occurred following planned and unplanned interventions (particularly following the deployment of initial contact letters and reminder letters).

Among those whose initial contact included a paper questionnaire, 26.7% chose to complete by electronic questionnaire. To aid the switch from paper to the electronic questionnaire, a UAC is provided on the paper questionnaire that respondents can then use to complete the electronic questionnaire instead.

Individual questionnaires

In Census 2021, if any member of a household does not want to disclose their information to others in the household, they can request an individual questionnaire, a personal form that asks questions about one person only. Individual questionnaires are also used by people living in managed communal establishment accommodation.

For the 2019 Rehearsal, a small number of individual questionnaires were requested, with 88 online and 210 paper individual forms returned. Of the people who returned a paper individual form, very few had also submitted a household form, raising concerns that paper individual forms were being returned by people who instead should have returned a household form. A more substantive communications package is being developed along with new processes to monitor where we are on target for household returns but have fewer than expected individual forms. The option to request an individual form from within the eQ is also being developed.

Welsh language questionnaires

Welsh language questionnaires were well-received by the vast majority of Welsh speakers who gave us feedback. The eQ in Wales had a language toggle functionality, which allowed respondents to switch between Welsh and English, and was successfully used by respondents. Analytic data from the eQ revealed that respondents switched to Welsh 489 times with 252 instances of switching back to English. For returns in Welsh, the average completion time was 20 minutes and 15 seconds.

The questions

Respondents also fed back on the questions, resulting in some changes for Census 2021. For example, respondents' feedback that the "Type of accommodation" question had no clear response option for respondents living in a building that has been converted into housing - for example, schools, warehouses, textile mills. Following this feedback an additional response option: "Part of another converted building (for example, former school, church or warehouse)" will be included in Census 2021. As a result of feedback on the Welsh questionnaire, the wording on the ethnic group question has been made more inclusive for Welsh ethnic minority groups.

Processing questionnaires

Processing of the questionnaires returned was successfully carried out using the in-house Data Access Platform (DAP). The processing systems underwent development during the 2019 Rehearsal to add new functionality and to implement changes as a result of lessons being learned. By the end of the collection rehearsal, processing of eQ responses was carried out automatically overnight, with manual processing of paper questionnaires and resolution of multiple returns. The iterative nature of the development of the processing systems during the rehearsal meant that response data was not available until the end, making it difficult to target specific groups with low return rates. For Census 2021, we will ensure these vital data are available in the form of management information and business intelligence during the census operation. For example, this will allow field staff to be directed to areas where they can have most impact.

Return rates

An indicator of how well our processes and systems worked is achieved return rates. However, these cannot be taken as an indicator for the success of Census 2021 as the collection rehearsal was voluntary and took place without a national advertising campaign and at a different time of year. In a similar way to the 2009 Rehearsal, the rehearsal return rates are therefore significantly lower than we would expect for the actual census itself. For Census 2021, our target is 94% nationally, which was the target reached in 2011 Census.

The 2019 Rehearsal return rates do, however, enable us to assess:

• the public's willingness to take part in the census
• how easy it was for them to participate
• the effectiveness of our local communications and community engagement
• the effectiveness of the field staff to make contact with the public and how effective they were in increasing return rates

Overall household return rate

The overall return rate for the collection rehearsal was 30.7%, which compares with a return rate of 41% in the 2009 Rehearsal. Individual return rates for the four local authorities which took part in the rehearsal are shown below along with the rehearsal results for the local authorities who took part in the 2009 Rehearsal:

While there cannot be a direct comparison between the two rehearsals, it is clear that the London local authorities present more challenges than the other areas. The London local authorities made up around 74% of households in the rehearsal and all London authorities are expected to make up around 16% of Census 2021.

How different households responded

The hard-to-count (HTC) index is developed for the whole of England and Wales. All lower super output areas (LSOAs) are ranked according to their likely propensity to respond without intervention (determined using a model based on 2011 levels of self-response and various administrative data sources), and then classified. The 40% predicted best responding areas are classed as HTC1; the next 40% as HTC2; the next 10% as HTC3; the next 8% as HTC4; and the remaining 2% where we expect the lowest return rates as HTC5.

Online share

Census 2021 will be the first primarily online census, and this was an important factor in the collection rehearsal. 81.9% of the returns were online. Ceredigion had the lowest online share as expected from an area where mobile or internet connectivity is poor, and where the highest proportion of initial contacts included the paper questionnaire.

Interventions used to increase return rates

By rehearsal day, 14.7% of households had responded. Following the success of the "Wave of Contact" (a plan of interventions to increase return rates) in the 2011 Census, we used a similar plan for the 2019 Rehearsal. The "Wave of Contact" included set dates for different campaigns, set dates to invite households to participate in the census and reminder letters to be issued to non-responding households, and set dates when the field staff started operations. These influenced the pattern of response.

Reminder Letters

There were a number of follow-ups for households that had not returned a questionnaire, to encourage them to respond. All non-responding households received one "planned" reminder letter, which were sent in three batches over the three weeks following rehearsal day. All reminder letters included a new UAC with the final batch of letters being split into some receiving a letter with a new UAC and the remainder receiving a paper questionnaire (also with a UAC). Three additional batches of reminder letters (response driven reminders) were sent to non-responding households in the worst performing LSOAs over the three weeks following rehearsal day.

While in the census non-respondents are subject to non-compliance procedures, as the rehearsal is voluntary no further action was taken if a householder refused to participate.

The planned reminder letters had an impact on increasing the return rates in all areas. Each of the planned batches of reminder letters resulted in a spike of at least 5% and when paper questionnaires were sent out as a reminder (reminder group R3B), a spike in returns of 8.2% was observed.

Targeted reminder letters for households who had started but not completed the electronic questionnaire were also successful in doubling the likelihood of a questionnaire being submitted. Households that received the incomplete reminder letters had a 10% return, compared with just 5% from those who did not receive the incomplete reminder letter (but may still have received field visits).

The impact of the field force

During live operations, 267,491 field visits were recorded. We selected, in advance, a random sample of 91,899 addresses for field follow-up. Those addresses flagged for field follow-up received field visits if they did not respond, while addresses not flagged for field follow-up did not receive field visits even if they had not responded. Hard-to-count groups were represented equally in the field (follow-up) and non-field (non-follow-up) samples.

The field force in action

Rehearsing the field operation enabled testing of digital services for field staff. This included online training as well as training for the digital maps and listing tool with the field work management tool (FWMT). A response chasing algorithm (RCA) was used to determine the areas for additional field staff hours to reduce shortfalls in return rates.

While most households were expected to respond online, we nevertheless deployed the field force in a similar way to the 2011 Census, with the field force visiting non-responding households to provide assistance with completing the paper questionnaire or persuading them to do it themselves. However, given households had a unique access code to complete electronically on their personal digital devices for this rehearsal, it made it difficult for the field force to directly help households fill in their returns, and they could only persuade households to respond. The field force remained on the doorstep and did not cross the threshold of households.

The overall impact of the field force

The impact of the field force on return rates was not as high as anticipated. The 267,491 field visits improved return rates overall by 8.8 percentage points, with improved return rates across all local authorities.

While field officers are important for increasing responses in the hardest-to-count populations and to minimise variability in response, the rehearsal showed the biggest impact on return rates was made by posted material.

For 2021, we will be improving both the recruitment and training of the field force. In particular, we will improve their training in the use of technology and put more focus on how to encourage households to fill in the census questionnaire rather than on activity (knocking on doors). We will also be making improvements to the FWMT, in particular in the listing of addresses to visit, to increase return rates. In addition, we will be improving the use made of field staff's time, in particular working with community engagement staff, with hard-to-reach groups and with those who need assistance.

Assistance

The pattern of self-response return rates was similar to that in the 2009 Rehearsal, which gives us confidence that we will meet the target of 70% of households filling in the 2021 questionnaire without help. For those households who needed help, there were a number of ways they could request it:

Online help

The online help facility worked well and provided a range of information, including help to answer specific questions. Some further improvements are planned for Census 2021, including revising the guidance for some of the questions. 

Assisted Digital Service

The Assisted Digital Service, providing help to fill in questionnaires online at selected centres, was not well used. Some of this was due to poor signposting and the siting of centres. 117 returns were submitted through Assisted Digital. This equated to 0.12% of household returns.

We are reviewing our offer of support to those who need assistance, including working with the field force and working with the supplier to improve the assisted digital offer. Where appropriate, we will include more effective signposting of the assisted digital and other community services in the training of field officers and public contact centre staff. We are also planning to better integrate the telephone capture and assisted digital services into our offer of support.

Telephone helpline and social media channels

The public contact centre provided an omni-channel service. This included:

• phone

• SMS (text message)

• webchat via the census website or social media

• twitter

The telephone helpline was successful. A large number of respondents reported that the contact centre staff were very knowledgeable and that the information they received was easy to understand, and that their calls were answered more quickly than they expected.

The telephone capture service planned for Census 2021 was not available during the rehearsal. People calling the contact centre were given the option to complete a survey that asked if they would have completed a telephone capture were the option available. For Census 2021, contact centre staff will be encouraged to offer telephone capture and other channels first, prior to offering paper.

The take-up on webchat was lower than expected as was traffic to the other social media channels. We are reviewing the relevant feedback to address any issues for 2021.

Communal establishments

The communal establishments (CEs) tested during the rehearsal were halls of residence, hotels, care homes and hostels, and were all in London. A reduced scope was required due to some technological solutions not being ready for the CE operation. The rehearsal in communal establishments tested paper questionnaires only, with no electronic questionnaires access options available.

The communal establishment questionnaire (CE1), which asks questions about the accommodation characteristics (including the number of residents), was delivered by hand by a communal establishment officer (part of the census field force) and was then completed by communal establishment managers.

Paper individual questionnaires (IQs) for the residents were mainly given out by the managers of the establishment to residents to complete, having been hand-delivered to the establishment. While the communal establishment officers returned to collect completed questionnaires, some residents and some communal establishment managers chose to put the questionnaires in the post. 

In the rehearsal the communal establishment officers successfully recorded their calls and visits using the FWMT. However, a technical issue arose around the FWMT not synchronising accurately and this affected the management information used to assess performance.

The response management systems planned for CE operation in Census 2021 were not available for the rehearsal, so manual workarounds had to be put into place to determine the return rates. The return rate for communal establishment forms (CE1) completed by establishment managers was 58%. For paper individual questionnaires, the return rate was 19%, with 44% of care home residents and 14% of students living in halls of residence returning completed forms.

In the rehearsal, 30 UACs were issued despite there being no online access for communal establishments in the rehearsal. The training of field staff and census co-ordinators will be improved for Census 2021. For 2021, all staff involved in the census will be trained in the correct processes for the different types of CEs, including the field contact centre, public contact centre, Logistics and other departments. Field staff will also be trained to utilise field self-help to its full potential.

Operations

Management information

Daily management information (MI) dashboards were produced throughout the rehearsal (for the household operation but not the CE operation) using Tableau, Python and PowerPoint. These supported the daily operational governance meetings. However, the information they contained was not suitably specific enough to inform decisions in the precise way planned. For 2021, we are planning a single visualisation tool and datastore to ensure consistency in data and reporting. The response chasing algorithm (RCA) was automated for 2019, and it is planned that, for 2021, all MI reports will be automated where possible to better analyse activities and allow the direction of the field force more specifically to maximise return rates.

The MI dashboard for the Census Coverage Survey (CCS) was developed during the operation and the quality of information will need to be improved 2021.

Operational governance

Our operational governance meetings worked well, providing good forums for managing the operational services including managing the field resources to maximise response. While these meetings worked effectively to manage response during the collection rehearsal, they proved to be less effective for the CCS (in part, due to the quality of the MI).

Our decision making at times was not as quick as will be required for Census 2021. We are therefore increasing resources to ensure consistent governance support, that actions are managed in a timely way and the flow of information is improved. The terms of references for our governance meetings will also be reviewed.

As the Census 2021 operation will be delivered in partnership with external suppliers, for 2021, we plan to widen the scope of the operational governance to include the live commercial position, in order to aid decision making and to identify any supplier problems at an early stage.

Managing the variability of return rates across local authorities, and between and within hard-to-count groups

Reducing variability in return rates across local authorities is one of our key quality objectives for 2021. The field prioritisation algorithm (FPA) is an automated tool for optimising field visits routing within a specific area. The FPA has been specifically designed to help with the target of minimising variability across local authorities, and between and within hard-to-count groups. The FPA flags the worst performing output areas as the highest priority and the best performing ones as the lowest priority.

The FWMT should have ordered the address lists seen by the officers so that the highest priority visits were at the top of their list. However, the FWMT was incorrectly configured, resulting in addresses ordered by case reference, which effectively amounted to random sorting. Once the problem was identified, near the end of the rehearsal, manual instructions were given to the field staff to re-order by priority code. However, the codes were set as 0 (low priority), 1 (default priority) or 2 (high priority) and re-sorting by priority code put priority code 1 at the top of their list, so the priority order was still not correct. As a result, variability was not reduced. This is being addressed for Census 2021, alongside improved training for the field staff.

Communications and campaign

The rehearsal communications campaign comprised advertising, public relations, digital and social media (including a campaign website) and the primary school education programme "Let's Count" in the four rehearsal areas. It was also an opportunity to test the effectiveness and accessibility of the new Census 2021 visual identity across a full suite of communications products.

The communications in the run up to rehearsal day were very successful, helping to lead to the initial spike in questionnaires returned. Independent pre- and post-campaign tracking research showed that census awareness increased from 91% pre-rehearsal to 97% afterwards in the rehearsal areas, with 75% of people confirming that they were very likely to participate in Census 2021 after the rehearsal.

While communications ran across a full range of channels, rehearsal communications were focused at the regional and local levels, purposely avoiding national coverage which may have resulted in public confusion about who should respond. As a result, it was not possible to test the impact of a fully integrated campaign operating nationally, regionally and locally, which would be expected to have a significantly greater impact in combination. In addition, the number one reason people cited for not participating in the collection rehearsal was the fact that it was not a legal requirement. The absence of national media and the voluntary nature of the rehearsal are two critical differences to Census 2021.

Public engagement was particularly strong in both Ceredigion and Carlisle, with positive local media coverage and community events. Response rates in in Hackney and Tower Hamlets were predictably lower, reinforced by the pre- and post-campaign tracking research, which highlighted that urban areas with a higher proportion of young respondents are harder to engage. For Census 2021 we plan to build upon the segmentation and wider research we have completed, identifying harder-to-count groups and building tailored campaigns addressing motivations and barriers to completion. There will also need to be heavier weight media campaigns in urban areas to cut through the congestion of competing communications activity.

The post-campaign tracking research showed that census information by post was by far the biggest driver of awareness, followed by out of home and TV advertising. Building on the success of the rehearsal campaign, specifically the initial spike in returns, we are introducing postcard to all households prior to the initial letter as a clear postal call-to-action for respondents.

We will also be building upon the public relations activity run during rehearsal, which saw 92 pieces of coverage, including the Deputy National Statistician's cycle ride to each of the rehearsal areas, underlining the value of putting a human face to the census. 

The Census 2021 website had more than 120,000 users by the end of November 2019. Activity across digital and social media saw over 1500 queries dealt with through the census Facebook and Twitter accounts. Both channels grew more quickly than forecast, providing a solid foundation on which to build for Census 2021.

58% of primary schools across the rehearsal areas took part in the "Let's Count" primary schools programme in support of the census, equating to 148 schools. The programme was enthusiastically received with over two-thirds of pupils in each school participating and over 90% stating that it made a positive educational impact. The effects of "Let's Count" also extended to the home, with 58% of parents directly informed. As a result, we will build on the early success of "Let's Count" and develop a companion secondary school education programme for Census 2021.

Community engagement

The aim of local community engagement in the rehearsal was to raise awareness, encourage participation and support communities complete their census rehearsal questionnaire.

We employed community engagement staff in each of the four rehearsal areas. Community engagement across all population groups was undertaken by census engagement managers, who worked in partnership with local authorities to identify and prioritise population groups. These staff worked well in their communities and were effective in raising awareness and getting responses. Identifying and working with intermediaries, such as community, faith and housing leads, that are trusted by communities, were vital in gaining wider community engagement and participation. Particular success was seen in Carlisle and Ceredigion through staff building relationships with those supporting economically and educationally disadvantaged population groups.

In addition, two community advisers worked with specific communities - the Bangladeshi and Somali communities in Tower Hamlets and, to a lesser extent, Hackney.

Feedback from communities was that they appreciated the face to face engagement with our engagement staff and that they valued the time taken to explain the relevance of the census to them and their community.

The engagement staff were successful in organising community completion activity where people were given support to complete their questionnaires. Events which had already been organised by communities, in places people normally attend and that engagement staff were invited to attend, were particularly successful. For example, with the kind support of the East London Mosque, we were able to help people complete their returns before and after Friday prayers.

In light of the success of local engagement in the 2011 Census and the rehearsal, for 2021 we are increasing the number of community engagement staff. Other field staff will be better integrated with our community engagement and we are planning to establish local engagement teams who will help co-ordinate community activities and help with completion locally.

We thank the local authorities (both officers and elected members) and the organisations, community and faith leaders that gave their support and helped us run a successful rehearsal.

The Census Coverage Survey (CCS) is a voluntary, interviewer-administered, doorstep survey, delivered immediately after census. Its purpose is to improve the accuracy of census results by identifying households that were missed in census, or individuals within households that were missed. It enables us to make accurate estimates of the number and characteristics of people not captured in the census.

The CCS is a face-to-face survey administered by field officers, who ask households some, but not all, of the census questions. The CCS rehearsal ran from 20 November to 20 December 2019 (which covered the 2019 General Election period).

CCS design, scope, and main improvements

The CCS operation was designed to run from 20 November to 20 December. After an initial property listing phase to accurately assess the addresses in the selected sample area, the selected households were asked, face-to-face, some of the same questions that were asked during the rehearsal. A telephone capture service was available in the final phases of the CCS.

The predominantly digital operation successfully rehearsed smart phones enabling online workload allocation, property listing, maps, interviewing, and support services, including Human Resources (HR). Household enumeration using the electronic questionnaire (eQ) was successfully tested and problems were identified and recorded. For Census 2021, we need to improve business process (and training) including ensuring that we use paper questionnaires as a contingency where connectivity is poor and there is no offline technical solution. In addition, improvements in management information quality to assist with the field operation are required.

Supporting services ranging from health and safety, recruitment, training, pay, HR, tech and non-tech kit deliveries and collections, field staff support, and incident management were also successfully rehearsed. For 2021, we need to improve the training, including providing additional technology training for field officers. The on-boarding programme also needs to be improved so that field staff are effective from day one.

CCS overall return rates

All operational phases were successfully rehearsed despite an unpredicted General Election taking place during the interview phase. The overall return rate (calculated as the number of successful interviews as a percentage of valid recorded addresses) was 40.6%. The return rates for each area are shown in Table 7.

Addresses

Main lessons: A large number of "undelivered as addressed" invitations indicated issues with the address frame and a significant number of blocks of flats had no numbers on them, making it difficult for field staff to follow up.

Resultant actions:

• the use of Census Coverage Survey (CCS) data to improve address frame quality

• an increase in clerical checks to improve quality

• the use of admin data and other data sources to improve address frame quality

• improved and increased data quality checks using new technology including google maps and street view

Systems

Main lesson: Systems enhancement requirements were recorded during the rehearsal, as planned.

Resultant action:

• to enhance systems, incorporating specific feedback

Suppliers and commercial

Main lesson: Minor delays in supplier onboarding resulted in us being unable to test the full logistics supply chain.

Resultant action:

• to plan to ensure sufficient time is programmed-in to allow for further testing

Main lesson: Recruiting sufficient field staff was challenging.

Resultant action:

• the re-profiling of tranches of recruitment to remove a late peak in requirements

Main lesson: Processes for recruitment need improving to ensure we employ the right people with the right skills.

Resultant actions:

• to work closely with supplier to ensure processes are more effective and efficient

• to improve job descriptions

• to introduce mobile flexibility into a portion of contracts

Main lesson: Onboarding of field staff needs to be improved.

Resultant action:

• to update the field officer training package, focussing on important messages

Security

Main lesson: Although the rehearsal did not highlight any significant issues, ways to improve security were identified.

Resultant action:

• to revise the end-to-end security management process to rationalise security risk management with improved management information

Households

Main lesson: Number of individual response returns was lower than expected.

Resultant action:

• to develop clearer messaging and route for those wanting to use an individual response

Main lesson: Processing enhancements are needed to improve speed of information to field operations to allow field staff to target areas of low response.

Resultant action:

• to further develop, as planned, the processing capability to ensure real-time data can feed management information and business intelligence

Main lesson: The pattern of response we saw was that easy-to-count (that is, those willing to complete) households were easier to count than expected, while the harder-to-count (that is, those less willing to complete) households, particularly in London, proved more difficult to count than expected.

Resultant actions:

• to update the Wave of Contact model to encourage response

• to improve the use of the field force, including their participation at community engagement events to help people fill in questionnaires

• to invest in additional community engagement staff to encourage those who are less willing to complete the questionnaires

• the further development of a full under-enumeration strategy to ensure that maximising response rates is part of the design in every stage and that that the use of admin data is maximised

Field force

Main lesson: Technical provision to field staff for their training was delayed.

Resultant action:

• to ensure online training is enhanced to complement the improved onboarding

Main lesson: Field force should focus on outcomes rather than activity.

Resultant actions:

• to update the model of field force engagement to ensure maximum coordination

• where direct contact is required, to ensure this is targeted and offset against the likelihood of a response through other means such as through the contact centre or posted material

Main lesson: The default sorting order within field work management tool (FWMT) was set to case reference requiring manual resorting.

Resultant action:

• to reset the default ordering to preferred address sorting as opposed to case reference

Assistance for households

Main lesson: The contact centre provided support through phone, interactive voice response, text message, social media and webchat. The uptake on social media and webchat was lower than expected.

Resultant action:

• to review uptake and test with users to determine the decision points for each channel and how to encourage usage

Main lesson: Assisted Digital Service was not utilised as expected.

Resultant actions:

• to review the offering and test with users to determine whether the service is accessible and fit for purpose

• to integrate with field force and contact centre, and signpost the availability for users including supporting groups and associations

Communal establishments (CEs)

Main lesson: Response rate for communal establishments (CEs) was 58% for CE1 manager-filled forms. For paper individual questionnaires it was 19%. The rate was low in halls of residence, at 14%.

Resultant actions:

• to utilise enhanced induction and training to ensure field staff are aware of the process for different types of communal establishment

• contact centre staff to be included to ensure an integrated approach

Operations

Main lesson: Management dashboards were provided both in Tableau, Python and PowerPoint which were not integrated and at times provided different vintages of numbers.

Resultant action:

• to use unified reporting to ensure "one version of the truth" in a consistent format

Main lesson: Operations reporting was manual, placing a burden on resources and importing risk of data error.

Resultant action:

• to automate reporting for 2021 to better enable teams to analyse activities to maximise return rates

Main lesson: Operational governance worked effectively though was constrained by the lack of management information (MI) especially during CCS.

Resultant action:

• to review the provision of MI to ensure the information is readily available

Communications and campaign

Main lesson: Lower level of engagement within Hackney and Tower Hamlets (which have highly diverse population groups and communities) than in Ceredigion and Carlisle.

Resultant actions:

• to improve the range of translated information (as planned) to support multiple languages

• to introduce a greater degree of communications targeting, by message and channel to reach and engage with harder-to-reach populations

• to update campaign messaging which needs to be clearer around the purpose and importance of the census and the call-to-action for respondents

• to implement a specific ethnic minority group advertising campaign

Main lesson: Website traffic data was not available.

Resultant action:

• to make para-data available to assist with marketing and engagement, as well as direction of field force

Community engagement

Main lesson: Community engagement events worked well at a local level but were in isolation of the other activities in the area.

Resultant actions:

• to develop a more strategic and integrated approach, including engaging at national levels to support the local-level engagement both with local authorities and community groups, in particular around hard-to-reach groups

• to establish local engagement teams who will integrate and co-ordinate community sector services and help with completion locally

• to overlay maps showing return rates with maps that show the geographic distribution of the key population groups and by commonly used geographic terms

Census Coverage Survey (CSS)

Main lesson: Lack of effective training for CCS field staff.

Resultant actions:

• updated training and on boarding programme (in line with main field operation) to improve effectiveness from day one

• to provide additional technology training for field officers (as for Census 2021)

Main lesson: Use of electronic questionnaire was hampered by some poor internet connectivity.

Resultant action:

• to improve business process (and training) including ensuring that we continue to use paper questionnaires as a contingency where connectivity is poor and there is no offline technical solution

Main lesson: Management information (MI) was not to the standard required to assist with directing the operation.

Resultant action:

• to improve management information quality to assist with the field operation