13.7 Methodological Implications
The purpose of this meta-analysis was to apply a critical realism philosophical lens
and realist approach proposed by Pawson and Tilley (1997, 2004). Concretely, the
purpose thus was to introduce and apply the method to extracting and hypothesising
theories of change and contextual conditions under which programmes are expected
to generate results through an underlying mechanism. In addition to a focus on
programme theories, the realist approach pays close attention to the kinds of
contextual conditions which enable (but not necessarily determine) a programme’s
IOs to be realised. Therefore, the first implication of adopting a realist approach in a
meta-analysis of CCA programmes is its focus on enabling contextual conditions. It
can be a significant element since non-realistic evaluations often focus on the
aspects that are only related to programme interventions and their programme
theories and not such contexts.
Second, the contextual conditions that are identified and hypothesized in this
meta-analysis can be useful for future CCA programming, particularly since similar
types of interventions are often designed without necessarily thinking of the
contexts. A realistic approach can provide explanations (rather than deterministic
“answers”) as to what type of programme interventions may work under what type
of conditions, and for whom. CCA programmes are embedded in quite a complex
environment, e.g. involving a number of stakeholders and beneficiaries,
implementing partners, funding sources and their requirements, and differing
programme goals and local priorities, on top of the five types of challenges
identified by Valencia (2009). All of these aspects can further be influenced by
the country’s culture, history and socio-economic conditions. These are also important context aspects to explore in further deepening the CMO configurations for
CCA programming. By paying close attention to such contextual conditions, the
realist approach can thus be considered useful for knowing how, when and where to
place the relevant interventions in a relevant context.
Third, this type of meta-analysis based on a realist approach may be able to shed
new light onto a number of ex-post evaluations that have been already prepared.
Though it will be difficult to prove quantitatively, there seems to be a tendency in
the development practitioner’s community to pay inadequate attention to such
ex-post evaluations, since they may be simply perceived as a mere requirement
routinely asked by sponsoring agencies and donors. Since only in recent years have
we started to complete ex-post evaluations of multilateral CCA programmes, a
realist approach can provide a good analytical lens in fully utilizing those evaluations to better inform future CCA programming.
13.8 Conclusion
This paper presents a case of meta-analysis using a realist approach, the evaluation
approach based on a philosophy of science called critical realism. The authors have
adopted this approach in the meta-analysis of the nine CCA programme terminal
13 What Do Evaluations Tell Us About Climate Change Adaptation? Meta-analysis. . .
251
The purpose of this meta-analysis was to apply a critical realism philosophical lens
and realist approach proposed by Pawson and Tilley (1997, 2004). Concretely, the
purpose thus was to introduce and apply the method to extracting and hypothesising
theories of change and contextual conditions under which programmes are expected
to generate results through an underlying mechanism. In addition to a focus on
programme theories, the realist approach pays close attention to the kinds of
contextual conditions which enable (but not necessarily determine) a programme’s
IOs to be realised. Therefore, the first implication of adopting a realist approach in a
meta-analysis of CCA programmes is its focus on enabling contextual conditions. It
can be a significant element since non-realistic evaluations often focus on the
aspects that are only related to programme interventions and their programme
theories and not such contexts.
Second, the contextual conditions that are identified and hypothesized in this
meta-analysis can be useful for future CCA programming, particularly since similar
types of interventions are often designed without necessarily thinking of the
contexts. A realistic approach can provide explanations (rather than deterministic
“answers”) as to what type of programme interventions may work under what type
of conditions, and for whom. CCA programmes are embedded in quite a complex
environment, e.g. involving a number of stakeholders and beneficiaries,
implementing partners, funding sources and their requirements, and differing
programme goals and local priorities, on top of the five types of challenges
identified by Valencia (2009). All of these aspects can further be influenced by
the country’s culture, history and socio-economic conditions. These are also important context aspects to explore in further deepening the CMO configurations for
CCA programming. By paying close attention to such contextual conditions, the
realist approach can thus be considered useful for knowing how, when and where to
place the relevant interventions in a relevant context.
Third, this type of meta-analysis based on a realist approach may be able to shed
new light onto a number of ex-post evaluations that have been already prepared.
Though it will be difficult to prove quantitatively, there seems to be a tendency in
the development practitioner’s community to pay inadequate attention to such
ex-post evaluations, since they may be simply perceived as a mere requirement
routinely asked by sponsoring agencies and donors. Since only in recent years have
we started to complete ex-post evaluations of multilateral CCA programmes, a
realist approach can provide a good analytical lens in fully utilizing those evaluations to better inform future CCA programming.
13.8 Conclusion
This paper presents a case of meta-analysis using a realist approach, the evaluation
approach based on a philosophy of science called critical realism. The authors have
adopted this approach in the meta-analysis of the nine CCA programme terminal
13 What Do Evaluations Tell Us About Climate Change Adaptation? Meta-analysis. . .
251