Respondent Load Strategy

Action	Implementation date
Develop parameters for the identification of load ‘hotspots’	December 2007
Set time limits for surveys	December 2007
Audit existing surveys	June 2008
Develop other metrics	June 2008
Measure impact of Statistics NZ surveying on subpopulations	June 2008
Adopt consistent actual time taken measures	September 2008
Set load limits for individuals	September 2008
Measure how often respondents are selected for surveys	March 2009

Strategy

Effective management of respondent load is only possible through consistent measuring and reporting. The measurement of respondent load is fundamental to ensuring we develop policies that balance the need for official statistics against the load on respondents. It enables us to understand and report on existing levels of load, as well as providing a benchmark against which to measure changes in trends and/or the effect of new policies or initiatives over time. Consistent measurement of respondent load will enable us to compare respondent load across the Official Statistics System.

Currently, the measurement of respondent load is patchy and ad hoc both here at Statistics NZ and across the Official Statistics System. In particular:

• there are no standards for measures of load
• the measures that Statistics NZ and other agencies adopt for measuring load vary between surveys; they are not consistent
• we only measure the time taken to complete a survey in some surveys
• the measures of load that are currently used are relatively simple and do not measure all the dimensions of load.

Our long-term strategy will be to develop and implement statistical standards for the measurement of load and advocate their adoption across the Official Statistics System. This will ensure that there are consistent measures against which to benchmark trends and assess the load of individual surveys.

The statistical standards will provide comprehensive measures of load. We will develop standards for:

• traditional measures of surveys, such as the time taken by respondents to complete a questionnaire and response rates
  other measures of load associated with different modes of collection including telephone/face-to-face interviews, and electronic questionnaires.
• We will undertake an audit of surveys in line with the Official Statistics System protocols, the purpose of which is to re-evaluate existing collections and to establish an ongoing review programme.

We will develop and publish commitments to business and individual respondents. This will include:

• setting parameters for the maximum amount of load that can be imposed on different sizes and types of business, and the maximum amount of time that can be taken to complete a survey
• investigating the possibility of including time taken estimates on our surveys
• reporting the results of performance against these targets through the annual report to parliament.

By being open about the load that direct surveying activity imposes and what constitutes a reasonable level of respondent load, we will be providing guidelines to help respondents put their survey obligations in context.

Currently, we have a range of internal practices for minimising respondent load and reports on load in our annual report. However, explicit targets for load and time taken are not publicly available or reported on, therefore, our expectations of respondents are not as clear as they could be.

Back to top

Implementing actions

Develop parameters for the identification of load ‘hotspots’

Many businesses, particularly small businesses, are not surveyed in any given year. However, for the business respondents that we survey there should be clear and reasonable expectations about the commitment that we expect them to make to complete our surveys.

Relatively few small- and medium-sized businesses are currently receiving more surveys and taking more time completing surveys than is reasonable. Table 1 sets out the load parameters that we will apply to identify ‘hotspots’ where we will consider remedial action.

The targets discussed in this section should not be considered as absolute. They represent a level of load that we will try not to exceed – but always with the understanding that because of the nature of the New Zealand economy and the small size of some important sectors, there may be times when we exceed load targets.

Initially, we will apply these parameters to small- and medium-sized business only and we will view them as targets that we will try to meet. Because of the small size of New Zealand’s economy it is not practical to set meaningful load targets for very large businesses without impacting on the quality of outputs.

To ensure transparency, we will report publicly on progress in relation to these targets and on the impact new surveys have on our ability to meet the load targets.

Table 1

^{(1) Monthly and quarterly surveys are counted as one survey per year rather than 12 or four. The time requirement is based on the total for all instances of that survey in a year (for example, five minutes per form for a monthly survey would equal one hour).}

Back to top

Set time limits for surveys

Every Statistics NZ questionnaire will include an estimate of how much time would normally be required to complete it. This estimate will be developed according to a standardised method.

We also need to consider the impact of surveying multiple respondents in the same household.

The Government Statistician can approve an exemption to the prescribed limit in exceptional circumstances. If this occurs, it is noted in the annual report.

Audit existing surveys

We will conduct an audit of existing surveys to determine if we have reviewed them recently as per the Official Statistics System protocol.

Regular collections are reviewed periodically to establish:

• whether the survey is still necessary
• whether appropriate data sources have become available
• that the views of respondents and users of data are being taken into account
• whether variables are effectively utilised and only necessary information is collected.

The audit will identify any surveys that need reviewing immediately, and it will ensure that there is an ongoing review programme for all surveys.

Develop other metrics

To facilitate setting targets and assessing agency performance against those targets, it is helpful to provide a single indicator to assess load for an individual respondent. To achieve this, we will work towards developing a respondent load index. This index could combine time taken, how often respondents are selected for surveys and other variables such as business size, survey complexity and sensitivity, but initially we will develop each metric separately.

Measure impact of Statistics NZ surveying on subpopulations

We will begin to develop measures of the impact of our social surveying on subgroups of the population, particularly on more heavily sampled groups (for example, elderly Mäori). Attention will be paid to the extent that the sampling differentially impacts on population subgroups, or on particular regions of the country.

These measures will be in the annual report and they can be extended, by use of the Survey Notification System, to include social surveys run by other agencies.

Back to top

Adopt consistent actual time taken measures

We will implement a standard time-taken measurement methodology internally. This means that every Statistics NZ survey will:

• forecast the time taken for a survey to ensure the expected time it will take to complete a survey is transparent
• ask respondents to record the time taken to complete the questionnaire, including the time taken to compile the required information.

The time-taken measures will provide an overall annual measure across all of our surveys. They will enable us to improve surveys by providing information on whether initial estimates of time taken are realistic. They will also provide a tool for assessing the impact of each additional question in a survey.

Set load limits for individuals

As a general principle, individuals who are already participating in one social survey are exempt for selection into a second. However, it is possible that an individual may be selected for more than one survey based on two completely different selection criteria. If this happens then the individual may apply for relief, although we will choose which survey they will still have to complete.

The same principle applies to sole proprietors or owner-operators who have been selected for a business survey and a social survey. Again, we will choose which survey relief can be granted for.

It is important to note that because of privacy concerns, we do not attempt to match respondents between social and business surveys, so it will be important for any individual concerned to bring this to our attention.

Measure how often respondents are selected for surveys

We will develop ways of measuring how often businesses or individuals are selected for surveys. This will enable load to be easily assessed and analysed.

In the future, we would be able to identify businesses or individuals in other Tier 1 surveys administered by other agencies. For businesses, this could be done with reference to the Statistics NZ Business Frame. However, for social surveys the information should be stored against a central contact register of names and addresses, which would be more complex.

There may also be implications for the content and update processes associated with the Respondent Management System and the Survey Notification System as the existing repository for respondent load information associated with Statistics NZ and the Official Statistics System, respectively.