development includes evaluation as a follow-up and review principle for the agenda
(UN 2015, p. 37).
Evaluations at higher scales and at the global level are not often done and are
difficult to design, implement and report on. Many problems could be mentioned,
such as reliability and comparability of data, external validity of evidence of
causality, but a particular problem that raises its head in relation to impact evidence,
is the problem whether evidence at local levels and lower scales translates into
evidence at global levels, at higher scales and over longer time periods. The first
chapter of this book has dealt with this issue in detail. In 2013 I argued that a
“micro-macro paradox”, which points to successes at the micro level that seem not
to be reflected in trends at the macro level, is particularly relevant to the linkage
between environment and development and thus to sustainable development which
aims to achieve a balance between society, the economy and the environment (van
den Berg 2013, pp 41–43). Climate Change provides good examples for this. Many
climate change related interventions are successful and achieve what they set out to
do. However, the success of individual activities has not affected global climate
change substantially. As the Intergovernmental Panel on Climate Change concluded in its 2014 report: “without additional mitigation efforts beyond those in
place today, and even with adaptation, warming by the end of the twenty-first
century will lead to high to very high risk of severe, widespread and irreversible
impacts globally (high confidence)” (IPCC 2014, p. 77). Notice the use of the term
impact for global phenomena.
3.2 Demand for Impact Evidence
Although the evidence movement has aimed to narrow down and reduce the
meaning of the term “impact” as referring to what can be found through counterfactual testing, the term impact is an ordinary word in the English language, the
meaning of which varies according to context. While science and in this case
evaluation may prefer a precise definition and a narrow meaning of terminology,
in general this will not change how terminology is used in conversation and debates.
When the public demands to see proof of impact, they will use the term impact in an
undefined way. To correct the public tends to be rather difficult if not impossible.
The question thus emerges whether narrowing the definition of impact is helpful
and whether another approach would not be more appropriate, which is to identify
how the term is used, what kind of evidence would be required to meet the demand
and to identify clearly what the advantages and disadvantages are of the tools and
thus of the reliability, validity and credibility of the evidence.
A good example of the discrepancy between what works and does not work at
the local level and whether “impact” is achieved according to the way the public
thinks about it, is climate change. At the level of individual activities good, solid
evidence is found on what works, especially on mitigation of climate change.
Mitigation activities aim to reduce the level of greenhouse gas emissions and thus
3 Mainstreaming Impact Evidence in Climate Change and Sustainable Development
39
(UN 2015, p. 37).
Evaluations at higher scales and at the global level are not often done and are
difficult to design, implement and report on. Many problems could be mentioned,
such as reliability and comparability of data, external validity of evidence of
causality, but a particular problem that raises its head in relation to impact evidence,
is the problem whether evidence at local levels and lower scales translates into
evidence at global levels, at higher scales and over longer time periods. The first
chapter of this book has dealt with this issue in detail. In 2013 I argued that a
“micro-macro paradox”, which points to successes at the micro level that seem not
to be reflected in trends at the macro level, is particularly relevant to the linkage
between environment and development and thus to sustainable development which
aims to achieve a balance between society, the economy and the environment (van
den Berg 2013, pp 41–43). Climate Change provides good examples for this. Many
climate change related interventions are successful and achieve what they set out to
do. However, the success of individual activities has not affected global climate
change substantially. As the Intergovernmental Panel on Climate Change concluded in its 2014 report: “without additional mitigation efforts beyond those in
place today, and even with adaptation, warming by the end of the twenty-first
century will lead to high to very high risk of severe, widespread and irreversible
impacts globally (high confidence)” (IPCC 2014, p. 77). Notice the use of the term
impact for global phenomena.
3.2 Demand for Impact Evidence
Although the evidence movement has aimed to narrow down and reduce the
meaning of the term “impact” as referring to what can be found through counterfactual testing, the term impact is an ordinary word in the English language, the
meaning of which varies according to context. While science and in this case
evaluation may prefer a precise definition and a narrow meaning of terminology,
in general this will not change how terminology is used in conversation and debates.
When the public demands to see proof of impact, they will use the term impact in an
undefined way. To correct the public tends to be rather difficult if not impossible.
The question thus emerges whether narrowing the definition of impact is helpful
and whether another approach would not be more appropriate, which is to identify
how the term is used, what kind of evidence would be required to meet the demand
and to identify clearly what the advantages and disadvantages are of the tools and
thus of the reliability, validity and credibility of the evidence.
A good example of the discrepancy between what works and does not work at
the local level and whether “impact” is achieved according to the way the public
thinks about it, is climate change. At the level of individual activities good, solid
evidence is found on what works, especially on mitigation of climate change.
Mitigation activities aim to reduce the level of greenhouse gas emissions and thus
3 Mainstreaming Impact Evidence in Climate Change and Sustainable Development
39