13.3 Realist Approach
This meta-analysis was conducted using a philosophical lens called critical realism.
In evaluation, the realist approach emphasizes underlying assumptions about the
way certain interventions are expected to yield certain outcomes in a certain context
(Pawson and Tilley 2004). It thus defies the deterministic worldview which is
symbolized as “if X happens, it automatically produces outcome Y.” Such a linear,
sequential worldview is considered deterministic or positivistic, in that hypothesized theories of change are thought to work regardless of the context within which
theories of change are situated. In other words, deterministic theory of change does
not give us the explanations as to “for whom such interventions may work, in what
circumstances, and how” (Pawson and Tilley 1997). Moreover, although the deterministic findings can tell us what interventions may have worked in certain countries under certain conditions (“there”), they may not tell us for whom these
successful interventions are expected to work, under what circumstances, and
how (“here”). The realist approach thus resonates with evidence-based policy
making in that it is thought to be useful in answering the important evaluation
question, i.e. “it worked there, but will it work here?” (Cartwright and Hardie
2012).
The following sections, however, first present the results of the meta-analysis
that are considered deterministic in nature, immediately followed by
non-deterministic ones and how the realist approach is applied. The intention
behind this structure is to emphasize the characteristics of critical realism philosophy. Deterministic findings appear to help evaluators to know whether certain
interventions work or not for achieving key outcomes, but such a deterministic
approach is what a realist approach attempts to defy.
The realist approach belongs to the school of theory-based evaluation (Stern
et al. 2012). The realist approach is based on a school of thought in a philosophy of
science, called critical realism. The concept of critical realism has been most
significantly developed by Roy Bhaskar.
4 Critical realism can provide a useful
lens especially in social sciences for the world that is “structured, differentiated,
stratified and changing,” and recognizes the shift of emphasis “to what produces the
events – not just to the events themselves.” (Danermark et al. 2002). An evaluation
approach based on critical realism is thus an “intuitively appealing approach to
those trying to expose and unpack the complexities of contexts and interrelated
mechanisms underlying implementation activity” (Rycroft-Malone et al. 2012).
The use of this evaluation approach is thus considered appropriate in the complex
experience of CCA projects. Adoption of critical realism in evaluation field (principally in public health and criminology) has significantly progressed thanks to the
work of Pawson (2013), Pawson et al. (2004), Pawson and Tilley (1997, 2004), and
Wong et al. (2013) and other scholars.
4 His most notable works include The Possibility of Naturalism (1979) and A Realist Theory of
Science (2008).
13 What Do Evaluations Tell Us About Climate Change Adaptation? Meta-analysis. . .
239
This meta-analysis was conducted using a philosophical lens called critical realism.
In evaluation, the realist approach emphasizes underlying assumptions about the
way certain interventions are expected to yield certain outcomes in a certain context
(Pawson and Tilley 2004). It thus defies the deterministic worldview which is
symbolized as “if X happens, it automatically produces outcome Y.” Such a linear,
sequential worldview is considered deterministic or positivistic, in that hypothesized theories of change are thought to work regardless of the context within which
theories of change are situated. In other words, deterministic theory of change does
not give us the explanations as to “for whom such interventions may work, in what
circumstances, and how” (Pawson and Tilley 1997). Moreover, although the deterministic findings can tell us what interventions may have worked in certain countries under certain conditions (“there”), they may not tell us for whom these
successful interventions are expected to work, under what circumstances, and
how (“here”). The realist approach thus resonates with evidence-based policy
making in that it is thought to be useful in answering the important evaluation
question, i.e. “it worked there, but will it work here?” (Cartwright and Hardie
2012).
The following sections, however, first present the results of the meta-analysis
that are considered deterministic in nature, immediately followed by
non-deterministic ones and how the realist approach is applied. The intention
behind this structure is to emphasize the characteristics of critical realism philosophy. Deterministic findings appear to help evaluators to know whether certain
interventions work or not for achieving key outcomes, but such a deterministic
approach is what a realist approach attempts to defy.
The realist approach belongs to the school of theory-based evaluation (Stern
et al. 2012). The realist approach is based on a school of thought in a philosophy of
science, called critical realism. The concept of critical realism has been most
significantly developed by Roy Bhaskar.
4 Critical realism can provide a useful
lens especially in social sciences for the world that is “structured, differentiated,
stratified and changing,” and recognizes the shift of emphasis “to what produces the
events – not just to the events themselves.” (Danermark et al. 2002). An evaluation
approach based on critical realism is thus an “intuitively appealing approach to
those trying to expose and unpack the complexities of contexts and interrelated
mechanisms underlying implementation activity” (Rycroft-Malone et al. 2012).
The use of this evaluation approach is thus considered appropriate in the complex
experience of CCA projects. Adoption of critical realism in evaluation field (principally in public health and criminology) has significantly progressed thanks to the
work of Pawson (2013), Pawson et al. (2004), Pawson and Tilley (1997, 2004), and
Wong et al. (2013) and other scholars.
4 His most notable works include The Possibility of Naturalism (1979) and A Realist Theory of
Science (2008).
13 What Do Evaluations Tell Us About Climate Change Adaptation? Meta-analysis. . .
239