– A further critical requirement was the availability of an adequate documentation of the selected projects.
– Finally, some projects should have produced visible effects that would allow
an attractive visualization of achievements.
• Key questions: The Climate Change Report on Effectiveness investigates the
achieved results of the selected 423 climate-relevant projects carried out by the
SDC and the SECO in the areas of climate change adaptation and climate change
mitigation. The key questions for the analysis were: What contribution did the
projects make towards improving people’s ability to cope with the negative
effects of climate change (adaptation)? What contribution did the projects make
towards reducing greenhouse gas emissions (mitigation)? To what extent did
Switzerland’s engagement for enabling frameworks contribute to the development of fair and binding climate-sensitive political frameworks at the international level and in partner countries?
• Detailed investigations: The second step comprised more detailed investigations
of these 30 projects during field visits to the selected six countries (Nepal,
South Africa, Peru, Mongolia, Serbia and Albania). The desk study of 31 additional projects (including an in-depth study of projects in Vietnam) ensured the
balanced coverage across the various themes and modalities within the portfolio.
The project documentation included planning and reporting documents such as
Credit Proposals, Annual Reports, Progress Reports as well as Evaluation
Reports. The detailed investigations involved direct interviews with knowledge
holders at project level.
• Portfolio analysis: The third step was to analyse the complete portfolio of
423 projects, and to determine adaptation and/or mitigation effectiveness scores,
with the aim of estimating the overall effectiveness of each thematic approach
and of the whole portfolio. This assessment drew on the portfolio appraisal,
detailed project reviews, questionnaires, interviews, and focus group discussions. Overall effectiveness scores for the 423 projects for which sufficient
information was available were distributed across all themes. These scores
were either ‘tentative’ or ‘confirmed’ and both represented the reviewer’s judgement on the project’s effectiveness, from ‘extremely strong’ (score 7) to ‘none’.
Tentative scores were based on the information presented in key documents,
informed by similar projects that have been reviewed through in-depth assessments, as well as sectoral specific reputation of the implementing partner.
Confirmed scores were based on the findings of the 61 detailed desk and field
studies, and replaced the tentative scores in each of these cases. The distribution
of effectiveness scores in the sample of confirmed scores (n ¼ 61) was compared
with that in the larger sample of tentative scores (n ¼ 362), and the distributions
were found to be significantly correlated. Though not as perfect as an in-depth
study of all 508 projects would have been, the use of tentative scores in the
overall assessment was necessary. The portfolio was far too diverse to yield
meaningfully representative results or aggregate results statements for the whole
portfolio.
5 Lessons from Taking Stock of 12 Years of Swiss International Cooperation. . .
87
– Finally, some projects should have produced visible effects that would allow
an attractive visualization of achievements.
• Key questions: The Climate Change Report on Effectiveness investigates the
achieved results of the selected 423 climate-relevant projects carried out by the
SDC and the SECO in the areas of climate change adaptation and climate change
mitigation. The key questions for the analysis were: What contribution did the
projects make towards improving people’s ability to cope with the negative
effects of climate change (adaptation)? What contribution did the projects make
towards reducing greenhouse gas emissions (mitigation)? To what extent did
Switzerland’s engagement for enabling frameworks contribute to the development of fair and binding climate-sensitive political frameworks at the international level and in partner countries?
• Detailed investigations: The second step comprised more detailed investigations
of these 30 projects during field visits to the selected six countries (Nepal,
South Africa, Peru, Mongolia, Serbia and Albania). The desk study of 31 additional projects (including an in-depth study of projects in Vietnam) ensured the
balanced coverage across the various themes and modalities within the portfolio.
The project documentation included planning and reporting documents such as
Credit Proposals, Annual Reports, Progress Reports as well as Evaluation
Reports. The detailed investigations involved direct interviews with knowledge
holders at project level.
• Portfolio analysis: The third step was to analyse the complete portfolio of
423 projects, and to determine adaptation and/or mitigation effectiveness scores,
with the aim of estimating the overall effectiveness of each thematic approach
and of the whole portfolio. This assessment drew on the portfolio appraisal,
detailed project reviews, questionnaires, interviews, and focus group discussions. Overall effectiveness scores for the 423 projects for which sufficient
information was available were distributed across all themes. These scores
were either ‘tentative’ or ‘confirmed’ and both represented the reviewer’s judgement on the project’s effectiveness, from ‘extremely strong’ (score 7) to ‘none’.
Tentative scores were based on the information presented in key documents,
informed by similar projects that have been reviewed through in-depth assessments, as well as sectoral specific reputation of the implementing partner.
Confirmed scores were based on the findings of the 61 detailed desk and field
studies, and replaced the tentative scores in each of these cases. The distribution
of effectiveness scores in the sample of confirmed scores (n ¼ 61) was compared
with that in the larger sample of tentative scores (n ¼ 362), and the distributions
were found to be significantly correlated. Though not as perfect as an in-depth
study of all 508 projects would have been, the use of tentative scores in the
overall assessment was necessary. The portfolio was far too diverse to yield
meaningfully representative results or aggregate results statements for the whole
portfolio.
5 Lessons from Taking Stock of 12 Years of Swiss International Cooperation. . .
87