1. Methodology background


 National Statistic   
 Survey name   Opinions and Lifestyle Survey (OPN)
 Frequency   Annual
 How compiled   Sample based survey
 Geographic coverage   UK
 Last revised   1 May 2012

Back to table of contents

2. Executive summary

This report is part of a programme of quality reports from the Office for National Statistics (ONS). Quality and Methodology Information reports are overview notes that pull together important qualitative information on the various dimensions of quality as well as providing a summary of methods used to compile the output.

This report relates to the internet access estimates, which are calculated from data collected in modules of the ONS Opinions and Lifestyle Survey in Great Britain. The survey aims to provide users with information on households’ and individuals’ use of the internet and other information and communication technologies (ICTs). The estimates have been published annually since 2006.

The Opinions and Lifestyle Survey (previously the Opinions Survey) is a multi-purpose household survey designed to be a fast, cost-effective and reliable way of obtaining information on a variety of topics too brief to warrant surveys of their own.

The internet access module is collected in the January, February and March Opinions and Lifestyle Surveys.

Until 2010, the survey was designed to cover the UK, but from 2011 the coverage was reduced to Great Britain. This had minimal impact on the published results because most of the published estimates relate to the current period. Where estimates for previous years are published they have been adjusted to be on a Great Britain basis.

This report contains the following sections:

  • Output quality

  • About the output

  • How the output is created

  • Validation and quality assurance

  • Concepts and definitions

  • Other information relating to quality trade-offs and user needs

  • Sources for further information and advice

Back to table of contents

3. Output quality

This report provides a range of information that describes the quality of the output and details any points that should be noted when using the output.

We have developed Guidelines for Measuring Statistical Quality; these are based upon the five European Statistical System (ESS)

Quality Dimensions. This report addresses these quality dimensions and other important quality characteristics, which are:

  • relevance

  • timeliness and punctuality

  • coherence and comparability

  • accuracy

  • output quality trade-offs

  • assessment of user needs and perceptions

  • accessibility and clarity

More information is provided about these quality dimensions in the following sections.

Back to table of contents

4. About the output

Relevance

(The degree to which the statistical outputs meet users’ needs.)

The Internet Access Survey collects data on households’ access to, and use of, the internet. Different questions are asked each year in order to obtain new, relevant information, for example, on e-skills or e-commerce.

The internet access estimates are collected to meet the requirements of the Regulation (EC) Number 808/2004 of the European Parliament and of the Council.

Eurostat is the principal user of the statistics. The UK provides statistics to Eurostat measuring households’ and individuals’ use of the internet and other forms of information and communication technologies (ICTs), in accordance with the ICT Regulation.

The internet access estimates are used to provide information that is consistent with other European Union (EU) member states, as part of progress towards measuring EU benchmarking indicators. These EU indicators compare the development and use of ICTs in the EU member states and help to provide a better understanding of the adoption of ICTs and the internet by households and individuals at an EU level.

ICTs are considered as critical for improving the competitiveness of European industry and, more generally, to meet the demands of society and the economy. Broadband is considered to be important when measuring access to, and use of, the internet, as it offers users the possibility to rapidly transfer large volumes of data and keep access lines open. The take-up of broadband is an important ICT policy-making indicator. Widespread access to the internet via broadband is regarded as essential for the development of advanced services on the internet, such as e-business, e- government or e-learning.

Since 2005, the EU policy framework for ICT has been the i2010 initiative called A European information society for growth and employment, which sought to boost efficiency throughout the EU economy by means of the wider use of ICT. Having undergone a mid-term review, an updated i2010 strategy was presented in April 2008, addressing the main challenges for the period 2008 to 2010.

Within the UK there is wide interest in the statistics from researchers, public bodies, charities and academics. In recent years there has been particular interest in statistics about adults who don’t use the internet, as part of the debate about social and digital exclusion. Given the need for more frequent estimates of internet users and non-users than could be provided by the annual Internet Survey, a new publication, the Internet access quarterly update, was developed. This was published between Quarter 1 (January to March) 2011 and Quarter 1 2014 and then replaced by an annual Internet access – households and individuals publication.

One of the strengths in the collection process of the annual internet access data, is that the vast majority of interviews are carried out face-to-face. The interviewer will attempt to call at an address three times over a 4-day period to obtain an interview. A proportion of those where no contact is made are referred to the ONS Telephone Unit. This results in approximately 2% of the interviews being conducted over the telephone.

The fundamental limitation of the Opinions and Lifestyle Survey is its sample size, which is relatively small compared with other ONS social surveys, although it is designed to be a sample survey of the Great Britain population, rather than at regional level. This only allows limited geographical analysis at regional level. The Opinions and Lifestyle Survey has a maximum interview time limit of 60 minutes each month and of that, the internet access module runs for between 15 and 20 minutes. Therefore, there is limited time for in-depth probing on specific elements of the module subject.

Timeliness and punctuality

(Timeliness refers to the lapse of time between publication and the period to which the data refer. Punctuality refers to the gap between planned and actual publication dates.)

The internet access data are collected by the Opinions and Lifestyle Survey in January, February and March. This is over a 19-week period, including 4 weeks for developing and testing the questionnaire in BLAISE (a computer-assisted survey design and processing system), producing showcards when required and preparing derived variables. During the final 5 weeks of the survey period the data are cleaned, weighted, analysed and tabulated by the research team.

The estimates are published in the Internet access – households and individuals statistical bulletin on the ONS website in August of the year that the data are collected. EU aggregates are published around the end of the year on the Eurostat website.

For more details on related releases, the GOV.UK release calendar provides 12 months’ advance notice of release dates. If there are any changes to the pre-announced release schedule, public attention will be drawn to the change and the reasons for the change will be explained fully at the same time, as set out in the Code of Practice for Official Statistics.

Back to table of contents

5. How the output is created

The Opinions and Lifestyle Survey is used for the collection of the internet access data. This is a multipurpose survey developed by ONS to be a fast, cost-effective and reliable way of obtaining information on a variety of topics too brief to warrant surveys of their own.

Sample design

A new sample of 67 postal sectors is selected for each month of the Opinions and Lifestyle Survey. They are stratified by region; the proportion of people who are aged 65 and over and by National Statistics Socio-economic Classification (NS-SEC). The postal sectors are selected with probability of selection proportional to size. Within each sector, 30 addresses (delivery points) are selected randomly.

If an address contains more than one household, the interviewer uses a standard ONS procedure to randomly select where to interview – this may be at one, two or three households depending on the exact circumstances. Within households with more than one adult member, just one person aged 16 or over is selected with the use of a Kish Grid. The Kish Grid is a tool used to randomly select individuals in the household that uses the address number and number of people aged 16 and over in the household to randomly select the individual for interview.

Until 2008, Northern Ireland was covered by the survey and data were collected separately through the Northern Ireland Omnibus Survey. The Great Britain and Northern Ireland results were then combined by ONS to produce UK results. When the Northern Ireland collection ceased, the 2009 and 2010 internet access results continued to be published on a UK basis by estimating the results for Northern Ireland. In 2011, when there appeared no prospect of the resumption of the collection of Northern Ireland data, primarily for financial reasons, the process of estimating for Northern Ireland was discontinued and the results were published for the first time on a Great Britain basis.

While Internet Access Survey data are not collected in the Northern Ireland Omnibus Survey, results for Northern Ireland will still be estimated so that ONS can provide estimates to Eurostat on a UK basis, as required by the ICT regulation.

Weighting

Weights are applied to Opinions data to correct for unequal probability of selection caused by interviewing only one adult per household. The weighting system also adjusts for non-response bias by calibrating the Opinions sample to ONS population totals.

Despite the considerable efforts made by interviewers to maximise response rates, approximately 40% of selected individuals decline to take part or cannot be contacted. Differential non- response among main sub-groups in the population is especially problematic because it can result in biased estimates. Consequently, respondents belonging to sub-groups that are prone to high levels of non-response are assigned higher weights.

For example, young males living in London have a lower response rate and are therefore assigned higher weights than males living in other regions. Weighting the data by age, sex and by region to estimate ONS population totals reduces the standard errors of survey estimates if the survey variable is correlated with age, sex and region.

For the household analysis, the weights are influenced by the characteristics of all the members of the household. As the data are weighted to population estimates, weighted totals of individuals by age group, sex and region from the survey are guaranteed to match the fixed population estimates. However, the weighting does not include control totals for the total number of households.

Therefore, the total number of households calculated by summing across the categories shown in the tables is a survey estimate and subject to random variation. Over time, this estimate will fluctuate around the true number of households in the population, but is not guaranteed to match the true value exactly (or estimates from other sources), nor show the same year-on-year change.

Further information about the Opinions and Lifestyle Survey is available.

Response rates

The response rates for the internet access module are published as part of the quality and methodology information in the Internet access statistical bulletin. The response rate is approximately 54% for the Internet Access Survey.

As the survey is collected as part of the Opinions and Lifestyle Survey, the methodology for calculating the response rate is the same as used for the Opinions and Lifestyle Survey. The full details of the methodology are available in the Opinions and Lifestyle Survey Quality and Methodology Information report.

Definitions of response outcome categories

The Opinions and Lifestyle Survey provides details for all non-responders. The categories are split by:

  • full response – an individual who has answered all the relevant questions on behalf of the household and all applicable individual questions

  • outright refusal – an individual who refuses to respond to the survey and the interviewer feels that there is no chance of an interview

  • circumstantial refusal – an individual refuses to respond because of a temporary circumstance (for example, going on holiday, too busy during the field period)

  • refusal to HQ – an individual who contacts headquarters to refuse to participate in the survey in response to the advance letter

  • non-contact – an address is occupied but it has not been possible to contact the selected individual in the field period

National Statistics Socio-economic Classification

The National Statistics Socio-economic Classification (NS-SEC) replaces previous classifications that were based on social class and social and economic group.

Proxy response

The Opinions and Lifestyle Survey does not collect proxy responses.

Back to table of contents

6. Validation and quality assurance

Accuracy

(The degree of closeness between an estimate and the true value.)

The internet access results, as with other sample surveys, are subject to error consisting of two elements: the sampling error and the non-sampling error.

Sampling error

The Opinions and Lifestyle Survey is a sample survey and therefore estimates are subject to sampling variability. Sampling variability is dependent on several factors, including the size of the sample, clustering and the effect of weighting on the variable of interest. Standard errors, which give an indication of the amount that a given estimate deviates from a true population value, are supplied for some main variables.

Non-sampling error

The Opinions and Lifestyle Survey is also subject to non-sampling errors such as non-response bias. The survey population is all Great Britain adults with a postal address. The risk of the survey not being representative is likely to increase with every refusal or non-contact with a sampled household (survey non-response). The voluntary nature of the survey means that people who do not wish to take part in the survey can refuse to do so. Non-sampling error may also be affected by whether having internet access affects the decision to participate. These types of errors cannot be measured but can be minimised through good questionnaire design, data validation and well-trained interviewers.

Comparability and coherence

(Comparability is the degree to which data can be compared over time and domain, for example, geographic level. Coherence is the degree to which data are derived from different sources or methods, but refer to the same topic.)

Before 2006, the Internet Access Survey did not cover Northern Ireland. Results were published four times a year for Great Britain, on the (then) National Statistics website, as Topic Based Summaries. These were each based on one module of the Opinions Survey, from a sample of approximately 3,000 households.

In 2006, the Opinions Survey sample was reduced to approximately 2,000 households each month, resulting in around 1,200 interviews. From 2006 onwards, the data from three separate monthly collections each year were combined to produce the dataset for the annual Internet access statistical bulletin. Coverage was extended to include Northern Ireland in 2006, to enable UK estimates to be produced, so that the annual bulletins for the years 2006 to 2010 were published for the UK.

In 2009, the Northern Ireland Omnibus Survey did not include the internet access module because of funding constraints. Therefore, in order to produce results for the UK, data for Northern Ireland were estimated for 2009 and 2010. In 2011, ONS decided to stop estimating for Northern Ireland and the coverage of the publication reverted back to being on a Great Britain basis.

The majority of the internet access estimates take the form of proportions of households or adults. As Northern Ireland only comprises approximately 3% of the UK adult population, the proportion estimates are usually the same, whether the coverage is Great Britain or UK. When there are differences, these have been in the order of only one percentage point.

Back to table of contents

7. Concepts and definitions

(Concepts and definitions describe the legislation governing the output and a description of the classifications used in the output.)

Eurostat is the principal user of Internet Access Survey results. The supply of results to Eurostat is a legal requirement under European Regulation 808/2004 and it is unlikely that the survey would operate without the legal basis set out by this Regulation. This Regulation is a framework regulation that sets out a process whereby the specific data requirements of the survey are defined annually; an Implementing Regulation is passed each year to set out these annual data requirements.

An annual consultation process takes place each year, in which users and policy makers within Eurostat discuss data requirements and proposed questionnaire changes in Task Force meetings, with delegates from a selection of EU countries. Involvement in these meetings is from those countries particularly interested in influencing the survey. Once the details of the questionnaire have been discussed in the Task Force meetings, the final data requirements are agreed each year by a Working Group of all EU member states before the annual Implementing Regulation is passed.

ONS takes part in this annual process and we are fully engaged in the development of the questionnaire for each year. This is to ensure, as far as possible, that those selected for the survey will be able to understand and respond to the survey questions.

Back to table of contents

8. Other information

Output quality trade-offs

(Trade-offs are the extent to which different dimensions of quality are balanced against each other.)

The data collected in the three Opinions and Lifestyle Survey modules (January, February and March) give a sample size large enough to provide good quality results at Great Britain level, but not large enough for detailed regional estimates. Most of the published results therefore relate to Great Britain. A sample size large enough for detailed regional estimates would require the internet access module to be run in additional months of the Opinions and Lifestyle Survey. However, the timetable for publishing the results on our website in August and preparing the results for submission to Eurostat by the October deadline, does not allow this. Funding limitations would also make it difficult to carry out more monthly collections.

Assessment of user needs and perceptions

(The processes for finding out about users and uses, and their views on the statistical products.)

As explained in “About the output”, Eurostat is the principal user of the statistics. Eurostat’s data requirements are determined by policy needs driven by various Directorates within the EU Commission.

We receive ad hoc data queries and requests for additional data from a wide range of users, such as researchers, students, academics and public bodies. The organisation RaceOnline2012 was established to help more people get online for the first time by 2012 and their need for more frequent statistics on people who had never used the internet, could not be met by the annual Internet Access Survey. Therefore, we have started publishing a series of updates on internet users and non-users. These estimates are compiled from the Labour Force Survey.

Back to table of contents

9. Sources for further information or advice

Accessibility and clarity

(Accessibility is the ease with which users are able to access the data, also reflecting the format in which the data are available and the availability of supporting information. Clarity refers to the quality and sufficiency of the release details, illustrations and accompanying advice.)

Our recommended format for accessible content is a combination of HTML webpages for narrative, charts and graphs, with data being provided in usable formats such as CSV and Excel. Our website also offers users the option to download the narrative in PDF format. In some instances other software may be used, or may be available on request. Available formats for content published on our website but not produced by us, or referenced on our website but stored elsewhere, may vary. For further information please refer to the contact details at the beginning of this report.

For information regarding conditions of access to data, please refer to the following links:

In addition to this Quality and Methodology Information, basic quality information relevant to each release is available in the quality and methodology section of the relevant annual Internet access statistical bulletin.

Back to table of contents

Contact details for this Methodology