Data sources can be evaluated using the following criteria, based on New Zealand’s Official Statistical System definitions of data quality (SNZ 2007) and ONS (2008).
Relevance
- What is the concept being measured?
- Is the concept clearly defined, and by whom?
- Why do we want to measure it?
- Is the concept likely to change?
- Who are the key users, and how does this concept relate to their needs?
- How well do the available data fit the concept?
Accuracy
- What is the proportional coverage of the available data?
- How are missing data dealt with what is the impact on the estimates?
If sample survey
- What is the sample frame and how does it fit the population of interest?
- What is the sample size?
- What is the response rate?
- How is the population parameter established?
- What is the estimated sampling error?
If administrative data
- Are there concerns about completeness?
- How well does the original collection purpose fit the current concept?
- How likely are distortions from changing definitions over time or incentives on improving completeness of data recording?
Timeliness
- What is the time lag from reference date to supply?
- What time periods are the data available for?
- Are the data punctual?
- What additional data would be required to forecast as required to match other sources?
Accessibility
- Are data available for the whole country? Are the definitions consistent?
- Are data available for a back series?
- Is the source expected to be available in the future?
- Are the data readily available?
Coherence/Consistency
- Are other data available for the same concept? Do they have the same story?
Interpretability
- What metadata are available to help interpret the data?
- Are there current or expected changes in definitions or methodology within the time series?