Fantastical words: Outcomes and Outputs

“When I use a word,” Humpty Dumpty said in rather a scornful tone, “it means just what I choose it to mean—neither more nor less.”

“The question is,” said Alice, “whether you can make words mean so many different things.”

“The question is,” said Humpty Dumpty, “which is to be master — that’s all.”

(Lewis Carroll, Through the Looking-Glass, 1865)

“Outcomes” is a crucial word in the Coalition Government’s lexicon if its first few months in office are anything to go by.

The emphasis on government by targets is a key plank of prime minister Christopher Luxon’s commitment to deliver better outcomes.

Outcomes is a very grabby word. Who doesn’t want better outcomes? Isn’t that why people become clinicians, organisational managers, and even politicians – because they want to improve something?

But the word has multiple meanings that don’t always overlap. In health policy, it has multifarious origins, and that gets us into Humpty Dumpty territory.

The language of clinical outcomes, such as five-year cancer survival rates, has long had a role in health policy. Similarly, population health outcomes such as life expectancy or infant mortality are well-established as part of the language of health outcomes.

In the broad field of policy evaluation, outcomes primarily refer to the intended effects of a defined policy programme. These can be assessed through measurable indicators or more qualitative judgement. This may include health outcome indicators but covers a much broader range.

Evaluators typically distinguish between long-term and intermediate outcomes. It’s usually impossible to distinguish the impact of a specific programme on a long-term outcome, such as the reduction in hospitalisation due to preventable falls.

Policy evaluators typically track a programme’s impact on an intermediate outcome, such as increasing enrolment in community-based strength and balance classes, that could plausibly (but might not) contribute to the long-term outcome.

The paradox is that the more valuable the outcome, the less amenable to measurement it is, and the things that are more easily measurable tend to be less meaningful or more ambiguous.

In the broad field of public management, outcomes had a much more specific meaning – to be understood in relation to other terms such as inputs, outputs and processes.

Most important here, is the difference between outputs and outcomes – where outputs refer to the things that are produced, and outcomes denote the effects or impacts that those things contribute to.

The Government points to the reintroduction of health targets as evidence of its commitment to outcomes. They certainly are outcomes if we consider them intended effects. But in the stricter public management sense, most targets are not outcomes at all – they are indicators of outputs or processes that might lead to valued outcomes.

But the whole point of this factory production language, when introduced into the New Zealand public sector in the 1990s, was that government agencies should be held accountable for outputs rather than outcomes, because outcomes were influenced by many factors beyond the control of individual organisations. Back then, the theory was that ministers should select which outputs could be “purchased” to achieve the desired outcomes.

This was fantasy on a grand scale, and governments quickly realised how untenable this idea was. But that didn’t stop it from being baked into the core of how the public service and much of the health sector operated.

All the contracts developed in the 1990s and 2000s between government funders and community-based providers reflected this doctrine. Yet, there was usually no clear theory about which “bundle of outputs” contributed to better outcomes beyond hope or wishful thinking.

For the Government, this linguistic universe of outputs, processes and outcomes has been abandoned. Outcomes now appear to mean simply intended effects. But in the recent announcement of revamped government targets (see panel) , the responsibility for the alchemical transformation of what the health sector does into valued outcomes lies with the public sector.

According to the prime minister’s press release announcing the nine public service targets on 8 April: “The targets are deliberately ambitious – they will be challenging and require the public sector to think differently, dig deeply into root causes, learn from other places, and be innovative and disciplined in directing resources to where they will have the greatest impact on outcomes.”

In other words – public sector, you work it out. And by the way, please do this with significantly fewer resources.

One key message of decades of public management and policy research into using targets to improve outcomes is that a target approach is only as good as the indicators chosen. If outcome targets are meant to be navigational aids, they are analogous to stars used by seafarers – and only a few stars are fit for that purpose.

Most of the Government’s announced targets across the public sector will either be prone to significant methodological and measurement problems and/or vulnerable to gaming. But their use reflects a long-standing approach that starts with saying, “This outcome is important, we need indicators – find something – anything!”

So, over the next six years, we can expect to see the many dynamics of outcome indicators used as stretch targets play out. These include the trumpeting of rubbery figures that invite scepticism, the puzzling and exasperation about lack of progress, or the appearance of unintended (yet sometimes predictable) consequences. Within all that, there may be some islands of real progress.

Of course, there is one more way that the word outcomes is used – the way that public policy practitioners and researchers have used it for decades. Here, the word outcomes means no more than “whatever happened”. This could be expressed in measures and indicators but does not need to be. An outcome is simply an arbitrarily defined endpoint of a policy process. In this sense, outcomes will always be achieved.

To return to Humpty Dumpty, “The question is – which is to be master—that’s all.” So, in which sense will Christopher Luxon’s Government use the word outcomes by the 2026 election? Watch this space.

Tim Tenbensel is professor, health systems, in the Faculty of Medical and Health Sciences at the University of Auckland

1. Shorter stays in emergency departments: 95 per cent of patients to be admitted, discharged, or transferred from an emergency department within six hours.

2. Shorter wait times for (elective) treatment: 95 per cent of people wait less than four months for elective treatment.

3. Reduced child and youth offending: 15 per cent reduction in the total number of children and young people with serious and persistent offending behaviour.

4. Reduced violent crime: 20,000 fewer people who are victims of an assault, robbery or sexual assault.

5. Fewer people on the Jobseeker Support Benefit: 50,000 fewer people on Jobseeker Support Benefit.

6. Increased student attendance: 80 per cent of students are present for more than 90 per cent of the term.

7. More students at expected curriculum levels: 80 per cent of Year 8 students at or above the expected curriculum level for their age in reading, writing and maths by December 2030.

8. Fewer people in emergency housing: 75 per cent reduction of households in emergency housing.

9. Reduced net greenhouse gas emissions: On track to meet New Zealand’s 2050 net zero climate change targets, with total net emissions of no more than 290 megatonnes from 2022 to 2025 and 305 megatonnes from 2026 to 2030.

Note: According to the media release announcing the Government Targets, delivery of the targets is the responsibility of a lead minister and lead public service agency chief executive, working in partnership with other ministers and their agencies as appropriate. Progress reports will be released publicly every quarter, starting from mid-2024.

Source: Media release from prime minister Chrisopher Luxon, 8 April 2024