impact evaluations should be “based on a thorough analysis of an intervention’s
theory of change”
2 as there may be other links in the causal chains that should be
tested or taken into account. Adopting the new technology is a change of behaviour,
but it could potentially lead to unintended consequences which may lead to an
overall increase of greenhouse gas emissions, if energy use increases overall. Other
changes in the context may make a specific behaviour change redundant, as for
example where new markets emerge and take over functions that are done more
efficiently through new technology. However, the focus remains on checking for
evidence of the behaviour change, as this is the causal mechanism that can be
checked in a traditional impact evaluation. Let us explore whether a deeper
understanding of the theory of change would lead to different and new questions.
Let us take a typical mitigation intervention as an example: the introduction of a
new technology that would lower greenhouse gas emissions. The Hilly Hydel
project in India was a typical project funded by the Global Environment Facility
and the Government of India, supported through UNDP, which took place from
1995 to 2003. This has been a particularly well evaluated project (see Ratna Reddy
et al. 2006). It was the object of a case study for a major GEF study on local benefits
generated through support for global benefits (GEFEO 2006), has an end-of-project
evaluation including a counterfactual impact assessment (Ittyerah et al. 2005) and
was further studied for the GEF impact evaluation of mitigation projects in emerging economies (GEFIEO 2013). For a total amount of $ 14.6 million this project led
to the introduction of small hydroelectrical power plants in hilly regions in India,
mostly in remote villages without access to the main grid. The reduction of
greenhouse gas emissions was supposed to be achieved through using a renewable
source of energy (hydro power) and reducing the need for wood as a source for fuel,
thus leading to a secondary but important benefit: reduced deforestation. The outputs of the project were a national strategy and master plan for hydro electrical
power generation, 20 stand-alone small hydel power generating water mills,
upgrading of 100 existing water mills to incorporate power generation and institutional and human capacities to ensure sustainability. In general these outputs were
achieved or surpassed – upgrading of no less than 143 water mills took place. All in
all this led to direct greenhouse gas emission reductions of 1900 tons CO 2 equivalent per year. If the potential for installation of these small-scale hydroelectric
water mills would be fulfilled throughout India, the total amount of reductions per
year would calculate as 4 million tons CO 2 per year (GEFIEO 2013, table 24 p. 70).
The theory of change of the project focused on introducing a technology that was
new for the villages in the hilly areas, that would lead to a source of energy that
would be more reliable and would lead to a halt to deforestation because of energy
needs, reduced greenhouse gas emissions as a result and given its benefits, would
convince villages to invest in this kind of technology. This would lead to a change
in the market for rural energy in hilly areas, where hydroelectricity would take the
2 See From Influence to Impact. 3ie strategy 2014–2016, p. 2, found at http://www.3ieimpact.org/
media/filer_public/2014/09/07/3ie_strategy_summary_final_rgb.pdf, on September 4, 2015.
3 Mainstreaming Impact Evidence in Climate Change and Sustainable Development
41
theory of change”
2 as there may be other links in the causal chains that should be
tested or taken into account. Adopting the new technology is a change of behaviour,
but it could potentially lead to unintended consequences which may lead to an
overall increase of greenhouse gas emissions, if energy use increases overall. Other
changes in the context may make a specific behaviour change redundant, as for
example where new markets emerge and take over functions that are done more
efficiently through new technology. However, the focus remains on checking for
evidence of the behaviour change, as this is the causal mechanism that can be
checked in a traditional impact evaluation. Let us explore whether a deeper
understanding of the theory of change would lead to different and new questions.
Let us take a typical mitigation intervention as an example: the introduction of a
new technology that would lower greenhouse gas emissions. The Hilly Hydel
project in India was a typical project funded by the Global Environment Facility
and the Government of India, supported through UNDP, which took place from
1995 to 2003. This has been a particularly well evaluated project (see Ratna Reddy
et al. 2006). It was the object of a case study for a major GEF study on local benefits
generated through support for global benefits (GEFEO 2006), has an end-of-project
evaluation including a counterfactual impact assessment (Ittyerah et al. 2005) and
was further studied for the GEF impact evaluation of mitigation projects in emerging economies (GEFIEO 2013). For a total amount of $ 14.6 million this project led
to the introduction of small hydroelectrical power plants in hilly regions in India,
mostly in remote villages without access to the main grid. The reduction of
greenhouse gas emissions was supposed to be achieved through using a renewable
source of energy (hydro power) and reducing the need for wood as a source for fuel,
thus leading to a secondary but important benefit: reduced deforestation. The outputs of the project were a national strategy and master plan for hydro electrical
power generation, 20 stand-alone small hydel power generating water mills,
upgrading of 100 existing water mills to incorporate power generation and institutional and human capacities to ensure sustainability. In general these outputs were
achieved or surpassed – upgrading of no less than 143 water mills took place. All in
all this led to direct greenhouse gas emission reductions of 1900 tons CO 2 equivalent per year. If the potential for installation of these small-scale hydroelectric
water mills would be fulfilled throughout India, the total amount of reductions per
year would calculate as 4 million tons CO 2 per year (GEFIEO 2013, table 24 p. 70).
The theory of change of the project focused on introducing a technology that was
new for the villages in the hilly areas, that would lead to a source of energy that
would be more reliable and would lead to a halt to deforestation because of energy
needs, reduced greenhouse gas emissions as a result and given its benefits, would
convince villages to invest in this kind of technology. This would lead to a change
in the market for rural energy in hilly areas, where hydroelectricity would take the
2 See From Influence to Impact. 3ie strategy 2014–2016, p. 2, found at http://www.3ieimpact.org/
media/filer_public/2014/09/07/3ie_strategy_summary_final_rgb.pdf, on September 4, 2015.
3 Mainstreaming Impact Evidence in Climate Change and Sustainable Development
41