FAQ on Quality Measurement and Quality Assurance
Quality measurement is not Quality Assurance. Quality measurement is about providing supplementary information about data enabling the user to ascertain the strengths and limitations of the data.
- What is Quality Measurement?
- What sort of information can you use to measure data quality?
- Why standardise quality measurement?
- Is it the same as Quality Assurance?
What is Quality Measurement?
Quality measurement is about providing information on each of the quality dimensions of outputs. The purpose for providing quality measures is to equip the user with sufficient information about the Office for National Statistics (ONS) outputs so they can assess the strengths and limitations of the data and determine for themselves their appropriate use. Quality measures can be qualitative or textual information, or quantitative information.
What sort of information can you use to measure data quality?
Examples of quantitative quality measures are: standard error, imputation rates, non-response rates, editing rate, proxy response rate, time lag between data collection and data release. Qualitative measures consist of results from user satisfaction surveys, descriptions of methods used for calculating standard errors and sources of error, for example measurement error. This list is not exhaustive.
Why standardise quality measurement?
A standard approach to quality measurement and reporting is critical to the successful delivery of the ONS Quality Strategy and plans are set out in the ONS paper ‘Quality: directions, strategy and progress.’
Comprehensiveness quality measurement and reporting is vital within ONS to monitor improvement and to provide support to implementing changes to facilitate continuous improvement.
Is it the same as Quality Assurance?
Quality Assurance is concerned with ensuring procedures associated with the data area are auditable, for example, making sure all documentation is up-to-date or ensuring that no data are lost when transferring from one data system to another. Quality Assurance is also concerned with ensuring products and outputs are of acceptable quality.
The definition of Key Quality Measures (KQMs) is ‘the smallest set of important, informative, achievable quality measures and indicators which provide a summary of the overall quality output'.
As well as providing a snapshot of output quality, the KQMs are used for monitoring the health of the office. In this use, they are referred to as ‘Statistical Key Performance Indicators’, meaning that they are key indicators of the statistical performance of ONS. 11 KQMs have been identified:
-
standard error for key estimates
-
overall unit response rate
-
total contribution to key estimates from imputed values
-
key item response rates
-
editing rate (for key items)
-
for outputs revised on a regular basis: estimate the likely revision between provisional and final estimates and include known reasons for differences
-
compare estimates from other sources, and include any known reasons for the differences
-
describe uses supported and where possible how the data relate to needs of users
-
identify known gaps between key user needs, in terms of coverage and details, and current data
-
provide a statement of the national or international agreed definitions and standards used
-
time lag: reference period to release of provisional or final output
It is expected that the list will expand to take account of the needs of non-survey outputs, for example administrative data and National Accounts.
