Measuring government sector productivity in New Zealand: a feasibility study

Number

Recommendation

Page

G1

Any implementation of this study should be clear what the question(s) associated with any requested productivity measure is (are), with particular emphasis on the perspective of the measure.

12

G2

A first step in implementing this study should be to address the industry perspective, to provide estimates of government productivity that are consistent with Statistics NZ’s existing market sector productivity estimates.

12

G3

Consistent terminology should be adopted and consistently used to avoid ambiguity and confusion. In this feasibility study, the term ‘quality’ relates to change in the set of characteristics of the units being measured.

13

G4

A cautious approach should be taken in combining measures of quantity and quality change in health care and education output, with wide and transparent discussion of options and careful building of a consensus before decisions on methods are adopted. Until then, quality change should not be incorporated into measures of quantity change in output.

13

G5

Statistics NZ and the Ministries of Health and Education should explore further what level of disaggregation is most suitable in the New Zealand context, to understand the impact on estimates of output and productivity and to inform the choice of this level. The choice of which to adopt should be reached after wide discussion and consensus building.

15

G6

In order to weight together the growth rates of different types of health care and education in a composite measure of total output, the relative weights should be total cost weights. Examining the impact of other types of weight may be useful in understanding different perspectives, for example in cost / benefit analyses.

17

G7

Any measure of output should be as comprehensive as possible in terms of the coverage of the types of health care provided to patients or education provided to students.

18

G8

Where quantitative information on change over time is not available for some types of services, there may be qualitative information about change which can be used to make informed decisions about the use of proxy measures (for example, growth in some types of activity for which figures are available may be considered to be representative of the growth rates for other types of activity for which figures are not available). For those types of services for which neither quantitative nor qualitative information on change over time is available, growth should be assumed to be the same as growth in measured activity, or labelling would need to be clear about how partial the measure is.

18

G9

A staged approach to implementation is recommended, giving higher priority to those areas of measurement that take little resource and have large impact.

18

G10

Statistics NZ should garner user views on the relative priorities of the productivity-specific questions and decide which one(s) should to be answered.

19

G11

Statistics NZ should review the desirability of using different index number methodologies for the numerator and denominator of the productivity equation.

20

G12

Statistics NZ should consider how best to inform users about the statistical quality of any government productivity measures it publishes, bearing in mind both quantitative and qualitative means.

23

G13

Statistics NZ should consider what are the appropriate ways for ensuring on ongoing dialogue with users, to ensure that the statistics provide (at least part of) an answer to specific user questions, and that any external expertise and experience can be drawn on to improve the development work.

23

G14

In order to deal with complications associated with separating between government output and private sector output if the scope of the productivity measure is defined according to who is paying, then the distribution by source of financing should be used to calculate how much of the inputs and output are government and how much are private.

24

G15

The scope of inputs must match that of output in the productivity equation. Where apportionment is not feasible, inputs should be spread across the industry on a pro-rata basis.

25

G16

Measurement of productivity for the government sector should follow as closely as possible that of the market sector where data sources and user needs allow.

25

G17

To help improve statistical quality, where information exists to compile output estimates using both deflated expenditure and a direct volume approaches, the sources, methods, and results should be compared and contrasted with the better quality aspects of both approaches being drawn on, to form a single best method

26

G18

Statistics NZ should consider what the appropriate rate of return should be for calculating the user cost of capital used in the government sector.

26

Number

Recommendation

Page

H1

The available information on the number of day patients should be incorporated into the existing method of calculation of Statistics NZ’s health care output.

36

H2

Revisions to estimates of casemix-adjusted throughput should be incorporated into the existing method of calculation of Statistics NZ’s health care output.

36

H3

Changes in the number of day patient discharges should be broken down by type of service. Along with information on average costs of these different types of service, this information will help to introduce an element of quality change into Statistics NZ’s measure of day patient output.

36

H4

Consideration should be given to combining the number of inpatient and day patient activities, where these are substitutes, in order to improve the price / volume breakdown.

36

H5

Revisions to estimates of the number of day patients treated should be incorporated into the existing method of calculation of Statistics NZ’s health care output.

37

H6

Consistent with recommendation 5.3.5 on comprehensiveness and representativeness, consideration ought to be given to incorporating all of the available information on activities in hospitals and other settings in order to maximise the comprehensiveness of Statistics NZ’s measure of health care output.

37

H7

The number of bed-nights should not be used as part of a measure of health care output for all types of hospital patient. It might be appropriate to consider using number of bed-nights as an appropriate indicator of the volume of health care output associated with ‘boarders’.

37

H8

The weighting scheme should be updated, possibly as frequently as annually, to reflect the changing relative costs of providing the different services.

38

H9

The method for aggregating the different sub-components of the health care output index should conform to the standard method involving weighting together changes in the volume of different activities using relative weights (rather than weighting together different index series).

38

H10

Given the development infancy of system-level measures of change in the quality of health care provided in New Zealand, and until there is broad discussion and agreement on how to construct such measures and combine these with the existing quantity measures, care should be taken in presenting such information.

115

H11

New Zealand should draw on the guidance already available globally on how to construct system-wide measures of change in the quality of health care provided in New Zealand, in deciding exactly what specification is appropriate for New Zealand.

116

H12

Statistics NZ should formally register its interest in information on the effectiveness of hospital treatment as part of an information suite that could be used in measuring health care output at the national level.

117

H13

Statistics NZ and the Ministry of Health should study the relationship between the WIES weights and total hospital costs, including all sources of funding, to confirm whether use of WIES weights as the measure of relative importance of different types of hospital inpatient and day carte activities introduces any bias.

121

E1

Consideration must be given to consistently applying definitions of government and private education across all levels. The definition selected should fit the question government productivity measures are intended to answer.

50

E2

Most education involves a certain degree of co-financing through fees and donations, and integrated schools through privately owned capital. Care is required to treat this consistently in accordance with the principles laid out in this report.

52

E3

Statistics NZ’s volume measures for education should be aligned as closely as is practicable with recommendations representing international best practice.

61

E4

The appropriate output measure of ECE education should be full-time student equivalents, disaggregated by service type. The definition of full-time at the ECE level should be consistent over time and across service types.

63

E5

If quality-inclusive output measure is desired, data are available to compare ECE hours enrolled with ECE hours delivered in census weeks.

63

E6

Stakeholders should be engaged in defining the boundary between education and care in a manner consistent with the question these measures are intended to answer.

68

E7

At a minimum, full-time student equivalents by level should be used to estimate school output quantity.

68

E8

A decision is required to include or exclude international students in accordance with the question these measures are intended to answer. International students must be treated consistently on both the inputs and output side, and should be treated consistently at the school and tertiary level.

70

E9

Alternative education programmes and teen parent units represent a sufficiently different service from mainstream secondary education that they merit separate treatment. This requires identifying them in the data on the inputs and output side so that they can be included or excluded as required by scope.

71

E10

A decision is required on how to treat the Correspondence School. It should be applied consistently on both the inputs and output side so that it can be included or excluded as required by scope.

72

E11

The most desirable output measure available for New Zealand’s tertiary education is credits completed, broken down by: subsector (university, polytechnic, etc), qualification level, domestic/international, broad field of study, and public/private.

74

E12

Universities, polytechnics and wananga provide distinct and separable educational services, and should be treated as such. Care should be taken with the treatment of Auckland University of Technology, which moved from the polytechnic category to the university category.

75

E13

The funding of tertiary education is complex and involves a large amount of co-financing across government and across the public/private split. Care should be taken to define the scope in a manner consistent with the question these measures are intended to answer, and to treat it consistently in both inputs and output.

75

E14

Research is recognised as an important output of universities, with an income stream that is increasingly separate and identifiable. However, identifying research funding in a longer time series may be impossible at this time. Stakeholders should be engaged in discussion about whether to explicitly include or exclude research within the productivity estimates.

82

E15

Research is acknowledged as an important output of universities that involves extensive co-funding and co-production. Given the lack consistent data and the uncertainty of research’s treatment in the National Accounts, it would be difficult to create a robust measure for it at this time. A decision will be required to either include or exclude identifiable research on both the inputs and output side.

85

E16

Consideration must be given to consistently applying definitions of government and private education across all levels. The definition selected should fit the question government productivity measures are intended to answer. There is the strong possibility that no ‘other education’ providers should be legitimately included in the government sector.

87

E17

The most desirable output measure for industry and targeted training is credits completed by level.

88

E18

On the basis of its small size and poor data availability, it is recommended that adult and community education be excluded from productivity estimates.

88

E19

Labour devoted to tertiary research should be estimated and treated in a manner consistent with the treatment of research output.

94

E20

Student attainment data are available for estimating output quality, but pose challenges in continuity (qualifications) and periodicity (achievement tests). Stakeholder engagement is recommended around any decisions on the suitability of adjusting for attainment, and the correct distribution of point-in-time achievement over a pupil’s schooling career.

146

E21

If used, qualification achievements must not be treated as a continuous measure in which each level of attainment has equal value that can be summed together. Care must also be taken around the discontinuity in the qualifications series marked by the introduction of NCEA in 2002.

146

E22

The available quality assessments by the National Education Monitoring Project and the Educational Review Office are not sufficiently quantitative or longitudinally consistent to be used in indexes of quality change for adjustment of educational output quantity.

148