5.5 Challenges and Lessons Learnt
5.5.1 In General
Preparing such a report on the effectiveness of the international cooperation in
climate change is indeed a challenging exercise: one has to deal with a mass of
information, with a wide and highly diverse portfolio and with a variety of actors;
moreover developing a method for assessing adaption is a crucial challenge.
Another demanding task is to inform a larger public on the results presented in a
comprehensive and highly technical report in a synthesised but still relevant
manner. The elaboration of the report has shown clear limits that must be balanced
with too high expectations. Given the lack of comprehensive and reliable data as
well as efficient and agreed methodologies to collect quantitative data, mainly in
the field of adaption, there is a risk that the results are either too generic at a
portfolio level or that “show cases” are reduced to a few examples.
5.5.2 For Evaluators
• Resources: The numerous challenges for evaluators in a complex exercise start
with the allocation of sufficient resources for such a pioneering assessment. The
expectation to conduct a pure accountability exercise in a most efficient way
often leads to the allocation of insufficient resources. The absence of welldeveloped methodologies, the huge size of data and information to be assessed
in a large portfolio, combined with the expected lack of direct evidence of
climate effectiveness has to be taken into account.
• Expectations: The expectations have to be aligned with the size of the investment. The ex-post reconstruction of baselines and the assessment of quantitative
results is an intensive and time consuming process. If rigorous climate change
related quantitative and qualitative data are not available in final reports or
evaluations of the assessed projects, it is illusive to think that an assessment
covering a portfolio of several hundred projects is able to fill that gap and to
produce aggregated quantitative data, for example on mitigated GHG emissions.
When producing data on proxies or qualitative assessments, the expectations
must be realistic, not to say modest.
• Independence is one of the most important principles in evaluations focusing on
accountability. Ensuring this independence of the consultants in such an innovative approach is however challenging. It could either undermine a constructive
exchange between the consultant and the commissioner if implemented too
strictly, leaving the consultants too isolated. Or it could lead to in-depth involvement and micro-management by the commissioner especially if there are disparate perceptions on how to approach and address the upcoming challenges in
developing the methodology from the very beginning.
5 Lessons from Taking Stock of 12 Years of Swiss International Cooperation. . .
89
5.5.1 In General
Preparing such a report on the effectiveness of the international cooperation in
climate change is indeed a challenging exercise: one has to deal with a mass of
information, with a wide and highly diverse portfolio and with a variety of actors;
moreover developing a method for assessing adaption is a crucial challenge.
Another demanding task is to inform a larger public on the results presented in a
comprehensive and highly technical report in a synthesised but still relevant
manner. The elaboration of the report has shown clear limits that must be balanced
with too high expectations. Given the lack of comprehensive and reliable data as
well as efficient and agreed methodologies to collect quantitative data, mainly in
the field of adaption, there is a risk that the results are either too generic at a
portfolio level or that “show cases” are reduced to a few examples.
5.5.2 For Evaluators
• Resources: The numerous challenges for evaluators in a complex exercise start
with the allocation of sufficient resources for such a pioneering assessment. The
expectation to conduct a pure accountability exercise in a most efficient way
often leads to the allocation of insufficient resources. The absence of welldeveloped methodologies, the huge size of data and information to be assessed
in a large portfolio, combined with the expected lack of direct evidence of
climate effectiveness has to be taken into account.
• Expectations: The expectations have to be aligned with the size of the investment. The ex-post reconstruction of baselines and the assessment of quantitative
results is an intensive and time consuming process. If rigorous climate change
related quantitative and qualitative data are not available in final reports or
evaluations of the assessed projects, it is illusive to think that an assessment
covering a portfolio of several hundred projects is able to fill that gap and to
produce aggregated quantitative data, for example on mitigated GHG emissions.
When producing data on proxies or qualitative assessments, the expectations
must be realistic, not to say modest.
• Independence is one of the most important principles in evaluations focusing on
accountability. Ensuring this independence of the consultants in such an innovative approach is however challenging. It could either undermine a constructive
exchange between the consultant and the commissioner if implemented too
strictly, leaving the consultants too isolated. Or it could lead to in-depth involvement and micro-management by the commissioner especially if there are disparate perceptions on how to approach and address the upcoming challenges in
developing the methodology from the very beginning.
5 Lessons from Taking Stock of 12 Years of Swiss International Cooperation. . .
89