evaluation community tends to keep its distance from the ex ante evaluators, as
there is widespread concern that any involvement of ex post evaluators in ex ante
evaluation will lead to a conflict of interest when the activity needs to be evaluated
later on. If design and implementation characteristics were decided upon because of
an ex ante evaluation’s outcomes, an ex post evaluator would in fact be required to
evaluate his or her own judgments in the ex ante evaluation. In actual practice the
two communities of practice hardly mingle. Ex ante evaluators have their own
conferences and their own literature and good practice standards. What Fig. 3.1
shows is that they are the first to delve into the question of impact and aim to
provide evidence, even if hypothetical at that stage, for what an intervention would
set out to do.
During implementation monitoring and evaluation often become management
tools. If the project needs to be steered through difficult circumstances and react
adequately to changes, it needs to set up an adequate monitoring system, either
collecting its own data or using data from available statistical services. Relatively
new is the inclusion of real time evaluation, which on impact tends to take the form
of randomized controlled trials that need to be included in the design of the project
and need to be adhered to during implementation, in order to come to valid
conclusions about the causal mechanism tested out. Other evaluations during
implementation (such as mid-term evaluations) tend to look at processes and
efficiency and are not represented in this matrix. Randomized controlled trials
tend to be “local” in nature; rarely will we see RCTs at the national level and
even more rarely at the regional level, as they would become very costly to reach a
sufficient level of data (large “n”) to allow for conclusions at that level.
In the ex post columns we tend to see two varieties of evaluation that provide
impact evidence. First of all, end-of-project evaluations may present results of
experiments or provide data on impact; usually these evaluations also contain
Fig. 3.1 The time and space dimensions of demand for impact evidence (Source: Author)
3 Mainstreaming Impact Evidence in Climate Change and Sustainable Development
45
there is widespread concern that any involvement of ex post evaluators in ex ante
evaluation will lead to a conflict of interest when the activity needs to be evaluated
later on. If design and implementation characteristics were decided upon because of
an ex ante evaluation’s outcomes, an ex post evaluator would in fact be required to
evaluate his or her own judgments in the ex ante evaluation. In actual practice the
two communities of practice hardly mingle. Ex ante evaluators have their own
conferences and their own literature and good practice standards. What Fig. 3.1
shows is that they are the first to delve into the question of impact and aim to
provide evidence, even if hypothetical at that stage, for what an intervention would
set out to do.
During implementation monitoring and evaluation often become management
tools. If the project needs to be steered through difficult circumstances and react
adequately to changes, it needs to set up an adequate monitoring system, either
collecting its own data or using data from available statistical services. Relatively
new is the inclusion of real time evaluation, which on impact tends to take the form
of randomized controlled trials that need to be included in the design of the project
and need to be adhered to during implementation, in order to come to valid
conclusions about the causal mechanism tested out. Other evaluations during
implementation (such as mid-term evaluations) tend to look at processes and
efficiency and are not represented in this matrix. Randomized controlled trials
tend to be “local” in nature; rarely will we see RCTs at the national level and
even more rarely at the regional level, as they would become very costly to reach a
sufficient level of data (large “n”) to allow for conclusions at that level.
In the ex post columns we tend to see two varieties of evaluation that provide
impact evidence. First of all, end-of-project evaluations may present results of
experiments or provide data on impact; usually these evaluations also contain
Fig. 3.1 The time and space dimensions of demand for impact evidence (Source: Author)
3 Mainstreaming Impact Evidence in Climate Change and Sustainable Development
45