Top Down Review of Official Statistics System

Phase 2

Recommended Option for the future role of Statistics New Zealand and the Official Statistics System

Purpose
Recommended Model
Official Statistics
Tier 1 statistics
Application of standards, protocols, and policies
Independent advice and review - ACOS
Accessing and release of statistics
A single point to access unit record data - OSRDAC
Clear terms for contract survey work undertaken by Statistics New Zealand for departments
Managing respondent burden
Printable Version

Purpose

This report presents the Review Team's preferred model to make improvements to the official statistics system, and seeks Ministers’ approval for the model.

Under the Statistics Act 1975 the term "official statistics" is defined as any statistics that can be produced by a government department. The Act also enables any other "agency of the Executive Government of New Zealand" to de defined as a Government agency for the purposes of this Act.

Problems with the Current System

The Statistics Act 1975 had three major objectives (see section 3 of the Statistics Act 1975):

• To provide information for Executive Government, government departments, local authorities and businesses for making policy decisions and to enable these groups and also the general public to understand economic, social, demographic and other matters of interest
• To co-ordinate as effectively as possible and to render most useful the official statistics produced by government, and
• To avoid unnecessary duplication of requests for information by government.

These objectives remain as relevant today as they were back in 1975, and are consistent with the benefits of working across government agencies, using statistical evidence to support government decision-making and providing best value for money.

There have been three substantial changes that have limited the ability of the Statistics Act 1975 to work as intended:

• Substantial growth in data demands to inform public policy and describe economic, social, demographic and environmental change
• Major changes in information technologies during the 1980 and 1990s enabling the creation and storage of significantly more administrative data, and the ability to more readily process and disseminate data to produce statistics, and
• The significant changes in the public management system during the 1980 and 1990s made more explicit the role and responsibility of Chief Executives in managing their departments but the consequence, until recently, was a low priority given to inter-agency collaboration and working together across government agencies.

The Review Team found wide acceptance of the need for significant improvements in the overall co-ordination and management of the official statistics system:

• Management of the official statistics system. The current system is characterised by a weak and fragmented co-ordination and management model and is not consistently focussed on the important statistics. In line with Government’s direction outlined in the Review of the Centre, there needs to be stronger management and co-ordination of statistics to get the most value from the Government’s investment in statistical activity.
• Better management of access to statistical data. The current weak co-ordination of the official statistics system has resulted in inconsistent access to statistical data. This has resulted in significant under utilisation of existing data and statistics to inform policy decisions and evaluative activities.
• Differing standards and policies currently being applied by departments. These reduce the ability of researchers to utilise data from across the state sector. Greater consistency in the adoption of standards and in the documentation of existing data sources would facilitate access and location of data, and allow greater integration of data from different sources.
• Better use of existing government statistics. There is significant potential to better utilise the existing stock of information within government without undertaking new surveys.
• Contract surveys under the Statistics Act. Conflict has arisen in the past between departments and Statistics New Zealand over contract surveys. Departments have expectations as purchasers of Statistics New Zealand surveys to ‘own’ data provided by respondents and to have full access to the data and results. On the other hand, the Government Statistician has an obligation to maintain trust in Statistics New Zealand produced statistics and give effect to the Statistics Act 1975. This requires Statistics New Zealand to retain control of the full statistics production process including access and release protocols. These two conflicting interests have resulted in some heat over access to data between departments and Statistics New Zealand.

Recommended model

To address the problems outlined above, the model proposes defining a portfolio of statistics across the state sector (referred to as Tier 1 from this point on) that are important in their own right and, consequently, need to be produced, analysed and released to high statistical standards.

Other statistics still play a role in the official statistics system and the use of standards in their production will be strongly encouraged by the Government Statistician. However, there will not be the same obligations to meet standards for these statistics, as there can be legitimate reasons for not applying standards that need to be taken into account by the producer.

The model has the following 11 elements that together will form an official statistics system across the state sector that is capable of overcoming the problems of the current system:

• Broadening the coverage of official statistics to include a class of entities to be known as Crown Agents (Public Finance Bill (State Sector Management) 2003). These entities fulfil an important role of government that fit naturally into official statistics. Producers of Tier 1 statistics outside the proposed scope of departments and crown agents, need to be invited to come within the scope.
  
  
• Identification of a set of important statistics (Tier 1). Tier 1 statistics are a defined set of key official statisitcs that are performance measures of New Zealand. The identification of Tier 1 statistics is primarily based on the purpose of the statistic and not 'who' produces the statistics. The list of Tier 1 statistics will be developed by a committee of Chief Executives, chaired by the Government Statistician, who will make a recommendation to ministers that the list be published as a regulation.
  
  
• Application of a consistent set of standards, protocols, and policies for Tier 1 statistics. The Government Statistician will set standards, protocols, and policies to be applied to Tier 1 statistics and therefore maximise the ability to integrate, compare and use such statistics across the state sector.
  
  
• The Minister of Statistics will approve all new or substantially revised Tier 1 surveys to ensure standards as defined above are being met.
  
  
• Leadership by the Government Statistician of the development of the official statistics system through the development of standards, protocols and policies, the use of Statistics New Zealand's expertise and capability and a strong programme of statistical research.
  
  
• Independent Advice and Review. The Advisory Committee on Official Statistics (ACOS) will represent the interests of the wider statistical community, including users, producers, and suppliers of statistical data to ensure that official statistics, particularly Tier 1 are developed to meet the needs of users and producers.
  
  
• Accessing and releasing statistics
  1. Users will be able to access a full list of available statistics produced by government
  2. Users will be able to use a variety of means of methods to access statistics, and
  3. Release of statistics will be by Chief Executive of the producing agency according to a calendar of release dates published at least six months in advance.
     
     
• A single point of access to Tier 1 unit record data. A new business unit will be set up in Statistics New Zealand called the Official Statistics Research and Data Archive Centre (OSRDAC). OSRDAC will provide a single reference point for unit record data for Tier 1 statistics for government, university and other researchers. Statistics New Zealand will promote, manage transparently, and reduce current cost of access for OSRDAC.
  
  
• Clear terms for contract survey work undertaken by Statistics New Zealand for government agencies. These terms and conditions, including earlier cabinet decisions, must be widely available and understood by potential contracting parties, as knowing the ground rules up front should help remove some tension in this area.
  
  
• Notification process for all surveys, in order to produce a comprehensive register of all official surveys.
  
  
• A burden management process will be put in place for all surveys with a sample size over 2,500. Agencies will be encouraged to collaborate. The final decision will be made by the Chief Executive of the producing agency.
  
  

Once Ministers have approved this model, a detailed implementation plan will be developed. The Review Team envisages the implementation of the model will build on existing processes (e.g. e-GIF) and work (e.g. Social Statistics Programme). In addition, there will be a need for some:

• Cabinet directives to set in place new or modified parts of the system
• additions to the Statistics Act 1975 (e.g. to form ACOS), and
• additional funding for some of the government agencies to meet the requirements of the proposed model.

Official statistics

The current definition of official statistics in section 2 of the Statistics Act 1975 has two elements:

• The source of statistics (surveys and administrative records) which does not need to be changed; and
• Who produces them (government departments).
  This definition is limited in scope and yet too broad to manage effectively. It:
  1. excludes potentially valuable statistics produced by Crown entities because it is restricted to "departments", and
  2. encompasses a wide range of departmental data that might generally be expected to be useful only for internal management purposes or statistical purposes for a narrow range of users. (See next paragraph)

When the Statistics Act was introduced in 1975 the structure of government was very different to the modern arrangement. Significant activities have been moved out of the ambit of departments and therefore outside the scope of the Act. Some important statistics are now being produced by government agencies outside the scope of the Statistics Act (e.g. tertiary education performance statistics produced by TEC or road safety statistics produced by LTSA).

The coverage of official statistics needs to be broadened to include a class of entities to be known in the Public Finance Bill (State Sector Management) 2003 as Crown Agents, defined as "non-company Crown entities that have a close working relationship with the government of the day"¹. These entities fulfil an important role of government that fit naturally into official statistics.

¹ SSC paper Review of Centre - Crown entity governance

Other types of entities (ACEs, ICEs, CROCs, SOEs) have not been included as they tend to have regulatory, commercial or judicial functions and are unlikely to be producers of significant statistics.

The process to produce a list of Tier 1 statistics could include those produced by agencies outside the proposed scope of official statistics (departments and crown agents). These producers need to be invited to come within the scope of official statistics to ensure that there is confidence in the statistics which are important enough to be classified as Tier 1. Likely examples of these agencies are the Reserve Bank and QVNZ.

Tier 1 statistics

Tier 1 statistics are a defined set of key official statistics that are performance measures of New Zealand. The identification of Tier 1 statistics is primarily based on the purpose of the statistic and not 'who' produces the statistics. The Government Statistician will encourage the development of the set of Tier 1 statistics to achieve a consistent level of quality across all subject areas.

Tier 1 statistics will have most of the following attributes:

• Are essential to central government decision making
• Are of high public interest
• Need to meet public expectations of impartiality and statistical quality
• Require long term continuity of the data
• Provide international comparability or meet international statistical obligations.

The identification of Tier 1 statistics is central to the proposed model. In the consultation phase for this report, departments consistently asked which statistics would be Tier 1 so that they could determine the likely impact on their departments. The Review Team's view is that the major components of Tier 1 statistics will be:

• those currently produced by Statistics New Zealand - primarily economic and demographic statistics, but also includes some social statistics (e.g. HSS, SOFIE, Labour Market), and including statistics produced by data integration from other departments' databases
• a few economic statistics produced by other organisations outside the definition of producers of official statistics (e.g. Reserve Bank, Quotable Value New Zealand)
• some key social statistics: particularly health, education, justice sector and social development statistics produced from surveys and administrative databases, and
• a future set of environmental statistics now being developed.

The Review Team has produced a list of Tier 1 statistics as an indicative set to inform this stage of the Review. This list is not definitive. The Government Statistician will convene a group of chief executives, (or their delegates), who are likely to be producers of Tier 1 statistics, to advise Ministers, through the Minister of Statistics, on what should be included in the initial list of Tier 1 statistics. Subsequent changes to the list will be proposed to ministers, by a similar group convened for that purpose. The Government will consult users as part of the process. The indicative list is in Appendix 1.

Application of standards, protocols, and policies

Producers of Tier 1 statistics will apply the statistical standards, protocols, and policies set by the Government Statistician in order for the system to work effectively across the state sector and allow a whole of government statistical picture to be built up. Although some obligations exist on departments under the current system, the identification of Tier 1 statistics may require additional obligations to be placed on departments in order for their statistics to meet the quality requirements, including the potential for cross-agency integration and collaboration, for Tier 1 statistics.

In order to ensure standards policies and protocols (see paragraphs below) are applied consistently for Tier 1 statistics, and that Government maximises the value of its investment in Tier 1 statistics, the Minister of Statistics will approve all new or substantially revised Tier 1 surveys. This survey approval process will be a transparent, simple and timely process based upon a self-assessment completed by departments. Self-assessments that meet the Tier 1 standards, policies and protocols will be referred automatically by the Government Statistician for approval to the Minister of Statistics. A sample checklist for department's self-assessment is in Appendix 5.

Standards for surveys and administrative databases

The adoption of common standards for Tier 1 statistics across administrative databases and surveys will allow a greater utilisation of a vast pool of data and enable more comprehensive statistics to be produced. Increased utilisation of existing data can reduce the need to survey the public directly with a corresponding reduction in respondent burden. The Review Team found strong cross agency support for the adoption of a common set of internationally compatible standards and basic data definitions for administrative databases and surveys. This is also in line with the Government’s E-Government strategy of common metadata definitions.

Components of these standards include:

• Statistical standards
• definitions of statistical units, and
• classifications (e.g. ANZSIC) and coding processes.

The technical nature of standards and the need for one common set for Tier 1 statistics makes the setting of standards an appropriate role for an expert, centralised, independent decision-maker. At an international level many statistical standards are ratified by the United Nations Statistical Commission with Statistics New Zealand participating actively in this work. The Government Statistician should undertake this role in consultation with departments and key users.

The adoption of common standards in existing administrative databases and surveys could involve significant effort, time, and cost to government agencies. The process of migrating to common standards needs to be carefully managed and focused on leveraging off existing work in this area. Standards should be implemented in the following way:

• all new surveys and administrative databases should incorporate the relevant standards into the planning and initial implementation phase; and
• existing surveys and administrative databases should be updated to the standards at the time of their next major revision or upgrade, which in some cases may not be completed for some considerable time.

The E-Government Unit’s “New Zealand E-government Interoperability Framework (NZ e-GIF)” is a good starting point for making basic data definitions for administrative databases more widely known.

Protocols and Policies for Tier 1 statistics

Protocols and policies cover practices such as the handling of data, release of statistics, and access to data. Their purpose is to ensure that there is integrity in the handling of data, that statistical data is only used for its intended purpose, and, ultimately, that the confidence of users and suppliers of data is retained by the agency collecting and producing statistics.

There is a need for a common set of protocols and policies for Tier 1 statistics to maintain public confidence in these statistics. As with standards, the Government Statistician will set policies and protocols for Tier 1 statistics. However, protocols and policies differ from standards in that they are less technical in nature and, arguably, more subjective. The Government Statistician will be expected to go through a consultation process before setting protocols and policies. ACOS will also have a role in reviewing and reporting on the application of protocols and policies set by the Government Statistician (see Independent Advice and Review below).

Leadership of Official Statistics

The current statistical leadership role of the Government Statistician is set out in Section 14 of the Statistics Act 1975. For continuing development of the official statistics system it is important that emphasis is placed upon:

• The development of standards, protocols and policies described above
• the use of Statistics New Zealand's expertise and capability to support departments to improve the quality and reduce the cost of Tier 1 statistics, and
• a strong programme of research, to inform users and producers about the quality and effectiveness of Tier 1 statistics, and develop new statistical series from existing data sources. This research will be facilitated by the establishment of the Official Statistics Research and Data Archive Centre (OSRDAC).

Independent advice and review - ACOS

The Advisory Committee on Official Statistics (ACOS) will represent the interests of the wider statistical community, including users, producers, and suppliers of statistical data to ensure that Tier 1 official statistics are developed to meet the needs of users and producers.

Role of ACOS

ACOS is focussed upon improving the quality and relevance of official statistics by:

• producing an annual report on:
  1. the “health” of Tier 1 statistics
  2. the appropriateness of standards, policies and protocols, and
  3. recommends changes necessary to improvements to Tier 1 statistics.
• advising the Government Statistician on the management of the Official Statistics and Research Data Archive Centre.

In any report to the Minister of Statistics and Cabinet, ACOS must give an opportunity to the affected agencies to comment on ACOS's recommendations and opinions. Any dissenting views should be included in the report.

Accountability for the production and quality of Tier 1 statistics according to the appropriate statistical system standards, protocols and policies will be through normal departmental accountability processes and informed by ACOS’s “Health of Statistics” report.

Members of ACOS

The Minister of Statistics will appoint ACOS members, including the Chair, based on the advice of major users, suppliers, and producers of statistical data. Members of ACOS are appointed as individuals, and not representatives of any particular organisation or interest group. The Government Statistician will be an ex-officio member of ACOS.

ACOS should have 8 - 12 members. This has implications for the structure and operation of ACOS (discussed below).

The Chair of ACOS has an important role and should possess the following attributes:

• is respected by the statistics community; and
• is viewed as impartial and unaligned with any particular organisation or interest group.

Structure of ACOS

The structure of ACOS is defined through:

• the role that it will undertake, and
• the need for its advice to be authoritative and independent of the Government Statistician.

To undertake functions such as reporting on the health of Tier 1, ACOS requires secretariat support. ACOS will need to be briefed and gather information to make its decisions. While Statistics New Zealand will provide this secretariat support, it is likely that ACOS would look to contract specialist resources to look at particular areas and address specific issues.

Accessing and release of statistics

The construction of a single state sector statistics portal managed by Statistics New Zealand will enable users, producers and researchers to quickly find all available official statistics. This will be kept up-to-date through the notification system. (see "Managing respondent burden" section below)

Producers of Tier 1 statistics will need to have in place a range of ways of facilitating access to their statistics to help them to be better utilised. The Review Team identified six different methods that users should expect to be able to use with any government agency producing Tier 1 statistics and two additional ways which are currently being developed (Confidentialised Unit record Files (CURFs), and Remote Microdata Access Service (RMAS)( Refer to Appendix 4). Both these methods were recommended by the McLennan reviews in 2002 of the barriers to access unit record and small area data.
Statistics New Zealand already provides the six different methods. Other producers of Tier 1 statistics should look to work with Statistics New Zealand rather than provide their own solutions.

To maintain trust in official statistics Tier 1 statistics will be released under the principles that:

• There is regular publication of on-going Tier 1 statistics, with a calendar of release dates to be published at least six months in advance, and
• The Chief Executive of the producing agency is responsible for the publication of Tier 1 statistics according to this timetable. The Government Statistician will release surveys contracted by government agencies with Statistics New Zealand as the producer.

A single point to access unit record data - OSRDAC

A new business unit will be set up in Statistics New Zealand called the Official Statistics Research and Data Archive Centre (OSRDAC). OSRDAC will provide a single reference point for unit record data for Tier 1 statistics for government, university and other researchers. SNZ will promote, and manage OSRDAC transparently, and reduce the current cost of access of using unit record data. This will encourage researchers to make full use of its capabilities. Subject to normal budget processes, OSRDAC will provide:

• A metadata archive for Tier 1 surveys, administrative data and integrated statistical datasets.
• A unit record data archive for all Tier 1 surveys, and Statistics New Zealand's holdings of Tier 1 administrative datasets and integrated statistical datasets, including historical data (e.g censuses of particular interest to researchers).
• The infrastructure to promote data access and facilitate research.(e.g. Offsite access facilities for those unable to work in the research centre such as the provision of confidentialised unit record files (CURF's), and Remote access data laboratory (RADL)). This will include implementation of the McLennan Data Access review recommendations.
• Core CURF's from existing datasets such as HLFS, HES, longitudinal surveys health surveys and samples of a cross section from adminstrative data files.
• Metadata and unit record data from other statistics may also be held in OSRDAC at the discretion of the producer and the Government Statistician.

The following requirements to supply data will be established:

• Tier 1 Surveys. A copy of the unit record file and metadata will be deposited with OSRDAC.
• Tier 1 Administrative Data. Government agencies responsible for the administrative database will supply the metadata of the database (effectively a description of what data is contained in the database).

OSRDAC will be able to provide access to the data for users (under terms and conditions for unit record data) for all Tier 1 surveys (Statistics New Zealand and other agencies) and Tier 1 administrative data held by Statistics New Zealand. Other government agencies responsible for Tier 1 administrative databases will have two options for providing access to their data. The agency can provide access itself through its own facilities or allow OSRDAC to provide access. The Review Team envisages that, if OSRDAC provides access, the administrative data will remain with the agency and an arrangement will be entered into to allow OSRDAC access to necessary data, as requests arise. Access to data from non Tier 1 surveys will be the responsibility of the agency that ran the survey.

The formation of OSRDAC does not interfere with departments’ access and ownership of their own data. Administrative data can be subject to legislative or contractual obligations to protect privacy (e.g. IRD or Justice sector legislation). Consequently, unit record access may not be possible for all administrative databases producing Tier 1 statistics.

The level and nature of data stored in OSRDAC must be flexible enough to meet this duty whilst still offering a useful service to users of statistics.

Cabinet decision [CAB (97) M31/14] gave Statistics New Zealand the lead role in cross agency integration work to address privacy concerns, which is consistent with this proposal as OSRDAC will be a unit within Statistics New Zealand. OSRDAC would also address Cabinet concerns about the paucity of researchers’ access to unit record and small area data. These issues were identified as part of deliberations on the Information Knowledge Base for social policy.

Clear terms for contract survey work undertaken by Statistics New Zealand for departments

Contract Survey Terms of Reference

The Government Statistician will have a set of clear terms and conditions for undertaking contract surveys. These terms and conditions must be widely available and understood by potential contracting parties. Clarity of the 'ground rules' should help remove some tension in this area.

Role clarification:

The model also clarifies Statistics New Zealand’s role in the following ways. Statistics New Zealand will:

• restrict its contract survey work to the production of statistics to Tier 1 standards, policies and protocols
• provide advice, on request, on standards, policies and protocols for all official statistics
• provide specialised statistical services for Tier 1 statistics to other government agencies on a cost recovery basis and similar services may be available for other statistics if the capacity is available
• reaffirm that all Statistics New Zealand (including contract) surveys are carried out under the Statistics Act and subject to its provisions. In particular, this means that confidentiality be maintained and Statistics New Zealand surveys remain compulsory, and
• reaffirm that the principles for the release of surveys produced under contract by Statistics New Zealand, and noted by Cabinet (POL Min (02) 14/7), will continue to apply.

Access to statistics prior to first release –

Departments desire access to statistics from contract surveys prior to release, to enable officials and Ministers to respond promptly to public queries following the release of the data. The present arrangements are:

• For any new area of statistics involving a sponsored survey, the Government Statistician will grant the sponsoring agency(s) appropriate access to the work in progress and final survey results for a reasonable period, specifically to assist in the preparation of the first release and as a consequence to enable the agency(s) to make effective and immediate use of the final data following the first release. Such access will be in accordance with the provisions of section 21 of the Statistics Act, and
• Any other release of the data to other departmental employees or their Ministers will follow the normal Statistics New Zealand procedures for releasing embargoed data (i.e. a 'lock-up' prior to release at 10:45am).

Access to unit record data following release

Departmental employees will have access through normal Datalab access rules. Most contract surveys include some access to the Datalab at no additional cost.

Joint Collections

All information from a joint collection (section 9 of the Statistics Act) can be shared with the contracting government agency unless the respondent objects in writing.

Managing respondent burden

Respondent burden is becoming an increasing problem with the growth of statistical and other surveys, and creates compliance costs for both businesses and communities. Government needs to actively manage respondent burden by:

• Better utilisation of existing data (discussed above).
  1. The application of a common set of standards across administrative databases should indirectly reduce respondent burden by allowing greater utilisation of existing data
  2. The use of OSRDAC as a central repository of unit record data, and
  3. The survey notification system (see below).

• Burden Management Process - For all new or substantially revised surveys, which have a sample size of more than 2,500, departments will consult with the Government Statistician early in the planning cycle to manage respondent burden. The consultation will be on:
  1. Ways to minimise overlap of the people being surveyed
  2. The elimination of any duplication with existing data sources, and
  3. The collection of data through the survey for other departments (and vice versa).
  Departments make the final decision after consulting with the Government Statistician.
  
• Survey Notification Process - All agencies carrying out surveys of persons, or organisations will complete a simple notification process for any survey directly or indirectly funded by the agency (e.g. what information will be collected, who it is being collected from, and how results will be made available). For this purpose surveys are not restricted to those for statistical purposes, and include research and other descriptive surveys. The information will be published by Statistics New Zealand. This information will be used:
  1. To provide a single point of reference for users and producers of official statistics. This will increase usage of existing statistics and minimise duplication.
  2. To manage respondent burden. Statistics New Zealand will use it to help reduce instances of over-sampling, which could result in excessive respondent burden.
  3. To prepare summary respondent burden information to be included in Statistics New Zealand’s Annual Report to Parliament.

Printable version

The downloadable file is in Adobe Acrobat format. If you do not have the Adobe Acrobat Reader you may download the reader to view or print the contents of this file.

TDRPhase2Report2.pdf (262 KB)