components, some cause-to-effect relationships were either non-existent or had
been overlooked, and several key drivers and assumptions had been neglected.
As explained above, attribution of large-scale, global changes to UNEP’s work
was difficult due to the largely normative nature of UNEP’s work. Casual pathways
from UNEP outputs to impact on the environment and human living conditions
tended to be very long, with many external factors coming into play all along the
causal pathways. The reconstructed ToC was used to assess the likelihood of
impact by considering four distinct elements:
• UNEP’s effectiveness in achieving its expected direct outcomes. This included
verification of progress on output delivery and, most importantly, of the extent to
which UNEP outputs led to increased stakeholder capacity, for instance:
enhanced access to information and technological know-how, enabling policies
and regulatory frameworks, or increased access to climate change finance.
• The validity of the ToC. The purpose was to prove the causal connection
between UNEP direct outcomes and results higher-up the causal pathways.
This was done by applying logic, through interviews with key stakeholders,
and through analysis of evaluative evidence of progress towards impact at the
country or lower geographical levels.
• The presence of drivers and validity of assumptions. The evaluation had to
collect adequate evidence, mostly through desk review and key informant
interviews, to verify the presence of an adequate enabling environment in
supported countries.
• Early signs of large-scale progress on medium-term outcomes, intermediate
states and impact. In itself this was not evidence of UNEP’s contribution to
higher-level changes, but was still necessary to inform stakeholders about global
trends. Also, if UNEP’s contribution to direct outcomes had been established,
the ToC was very likely to be valid, and the required drivers were present and
assumptions were valid, then the likelihood of UNEP’s contribution to impact
was very high even though it remained unquantifiable.
6.5 Data Sources
The evaluation team conducted a comprehensive desk review spread over the
inception and main evaluation phase. During the inception phase, it helped to
reconstruct the ToC of the components and the sub-programme as a whole, and
to refine key areas of analysis and the evaluation approach highlighting evaluation
challenges and information gaps. During the main evaluation phase, it was essential
to collect information on achievements, impact, sustainability and upscaling, and
the main factors affecting performance, while also leaving room for unanticipated
results. The evaluation team conducted an in-depth analysis of CCSP key documents: background documents on climate change science and technology, the
UNFCCC process and Climate Change finance, UNEP strategy and planning
106
M. Carbon
been overlooked, and several key drivers and assumptions had been neglected.
As explained above, attribution of large-scale, global changes to UNEP’s work
was difficult due to the largely normative nature of UNEP’s work. Casual pathways
from UNEP outputs to impact on the environment and human living conditions
tended to be very long, with many external factors coming into play all along the
causal pathways. The reconstructed ToC was used to assess the likelihood of
impact by considering four distinct elements:
• UNEP’s effectiveness in achieving its expected direct outcomes. This included
verification of progress on output delivery and, most importantly, of the extent to
which UNEP outputs led to increased stakeholder capacity, for instance:
enhanced access to information and technological know-how, enabling policies
and regulatory frameworks, or increased access to climate change finance.
• The validity of the ToC. The purpose was to prove the causal connection
between UNEP direct outcomes and results higher-up the causal pathways.
This was done by applying logic, through interviews with key stakeholders,
and through analysis of evaluative evidence of progress towards impact at the
country or lower geographical levels.
• The presence of drivers and validity of assumptions. The evaluation had to
collect adequate evidence, mostly through desk review and key informant
interviews, to verify the presence of an adequate enabling environment in
supported countries.
• Early signs of large-scale progress on medium-term outcomes, intermediate
states and impact. In itself this was not evidence of UNEP’s contribution to
higher-level changes, but was still necessary to inform stakeholders about global
trends. Also, if UNEP’s contribution to direct outcomes had been established,
the ToC was very likely to be valid, and the required drivers were present and
assumptions were valid, then the likelihood of UNEP’s contribution to impact
was very high even though it remained unquantifiable.
6.5 Data Sources
The evaluation team conducted a comprehensive desk review spread over the
inception and main evaluation phase. During the inception phase, it helped to
reconstruct the ToC of the components and the sub-programme as a whole, and
to refine key areas of analysis and the evaluation approach highlighting evaluation
challenges and information gaps. During the main evaluation phase, it was essential
to collect information on achievements, impact, sustainability and upscaling, and
the main factors affecting performance, while also leaving room for unanticipated
results. The evaluation team conducted an in-depth analysis of CCSP key documents: background documents on climate change science and technology, the
UNFCCC process and Climate Change finance, UNEP strategy and planning
106
M. Carbon