Increasingly, in health and elsewhere, the principles of economic evaluation are being drawn on for help. Based on the premise that any expenditure involves a trade-off, economic evaluation provides a lens through which to ask whether money spent on one programme could be better spent on another. It commonly does this by comparing the costs and benefits of that programme with those of alternative options.

Using old tools in new places…

The various methods available have grown into a sub-discipline of their own over the last 20-30 years. However, this growth has been driven largely by the needs of stakeholders in higher-income countries to guide domestic public policy (often in health). International donors work in a different arena: they often have limited direct accountability to the beneficiaries; and health systems in lower-income countries operate under more severe resource constraints. This means fewer health workers, equipped facilities, drugs and medical consumables per person. Or, in the language of economic evaluation, the opportunity cost of expenditure becomes much higher.

Furthermore, some systemic differences are also apparent. Health systems in lower-income countries often rely more on out-of-pocket expenditure, which increases the risk of pushing households into poverty. There is often a wider variety of provider types (such as public, private, faith-based, unregulated and traditional providers). The global market for health workers makes it hard to retain expensively trained staff. And, of course, the prevalent diseases differ (communicable diseases remain major issues in lower-income countries, but the burden of non-communicable diseases is growing).

What does this all mean for the use of economic evaluation methods to guide health-sector decisions in such contexts? Here are some recommendations.

1.       Keep it local

There is no "one-size-fits-all" model, estimate or guideline. As the methods for health economic evaluation have gone global, there has been a corresponding shift towards top-down analysis using increasingly universal measures of value—applied to everything from discount rates and cost-effectiveness thresholds right through to fundamental ideas about what "good" health means. Two high-profile examples of this are the WHO-CHOICE tools (including Generalised Cost-Effectiveness Analysis), and the World Bank’s Disease Control Priorities publication of up-to-date information on intervention efficacy and programme effectiveness (now in its third edition).

There is a danger, however, that, without sufficient tailoring, evaluations based on generalised metrics lose their meaning at the local level. A regional estimate plugged into a generic model may produce results that do not reflect local realities. An overreliance on impressive-looking models risks undermining the need to understand local epidemics and to generate reliable, context-sensitive evidence on relative effectiveness and costs, ultimately affecting the credibility of the analysis.

 2.       Decide in advance what good value looks like (know your thresholds)

 Insufficient clarity on what good value is decreases the transparency and accountability of the process. For example, it is becoming common for donors to conduct routine retrospective "Value for Money" (VfM) analyses on programme expenditure, constructing a measure of cost-effectiveness out of economy, effectiveness and efficiency (and sometimes equity). However, much of the current guidance does not comment on the point at which something should be considered cost-effective or good VfM.

 As such, analysts are asked to make a "reasoned judgement" about the VfM with the information they have. They are told to think about economy, efficiency, effectiveness and equity, but not what levels are actually considered acceptable. This creates space for inconsistency across programmes, where projects with similar costs and effects are treated differently owing to the personal perspective of the analyst.

 3.       Be realistic

 Analysts often find themselves asked to comment on broad, system-wide interventions (such as the blanket removal of user fees or a new national information system). Furthermore, the data available in lower-income settings are often scarce and of insufficient quality. Put these two things together and what do you get? The principles of economic evaluation are applied, with sub-optimal data, to options so broad that they touch on the nature of the health system itself. How much trust would you put in such answers?

 Of course all questions should be considered, but when limited resources are available to spend on analysis, the methodological approach to answering the question should be both valid and feasible. There are plenty of important questions that can be answered with the limited data that are available.

Go forth and multiply

I absolutely encourage international donors to make use of economic evaluation. But consider some simple steps to make sure you get what you want. Inaccurate analyses, without a clear conception of good value, based on inadequate data, are of no use and can actually do more harm than good. Instead try to use the local data that are available, decide in advance what good value looks like, and be realistic about what you can answer.

The issues addressed in this blog post are currently being discussed at the iHEA Boston Congress.

The views and opinions expressed in this article are those of the authors and do not necessarily reflect the views of The Economist Intelligence Unit Limited (EIU) or any other member of The Economist Group. The Economist Group (including the EIU) cannot accept any responsibility or liability for reliance by any person on this article or any of the information, opinions or conclusions set out in the article.