4.4 Getting to the Right Mix
CCAFS was clearly committed to an outcome-focused R4D program from its
inception. It became increasingly clear that a logframe approach (LFA) was not
the most ideal way of doing R4D differently. In particular, when moving from a
Challenge Program to a CGIAR Research Program with increasing complexities of
partnerships, engagement and CGIAR integration, the limits of a logframe became
apparent (as described in section 4.5.1). The program’s vision of contributing
towards development outcomes increasingly required a different approach: one
that acknowledged the importance of stakeholder engagement and capacity development. As a result, monitoring the annual contribution of CCAFS and its partners
towards development outcomes becomes increasingly complex.
While a wide range of MEL approaches and methodologies with an outcome
focus exist (e.g. PIPA, Outcome Mapping, Outcome Harvesting), none provides a
blue-print solution that can just be rolled out. The approaches were designed to
address the particular needs of a specific program or organization. Thus, to adapt
these approaches to a new program, it is key to select the right mix of elements
creating a conceptual framework in support of the program’s specific TOC and
MEL requirements. Springer-Heinze et al. (2003) advocate a holistic approach to
impact evaluation and program monitoring with quantitative and qualitative elements, based on an impact pathway that can accommodate different stakeholder
views, allows for reflection, and emphasizes capacity of research organizations.
Cummings (1997) compares RBM, LFA and Project Lifecycle Management and
would welcome more discussions and learning among the different approaches.
According to Bazeley (2004), ‘The “mixing” may be nothing more than a side-byside or sequential use of different methods, or it may be that different methods are
being fully integrated in a single analysis’. Applying a mix of methodologies in a
programmatic MEL framework raises certain terminological, definitional, paradigmatic and methodological issues, including over-interpretation of numbers, single
dimensionality, and disregarding ‘outliers’ from the analysis (Bazeley 2004). However, mixed methods also provide opportunities to address the respective shortcomings of any single method as applied in practice.
CCAFS in its early years worked with various logframe elements in planning
and reporting. Limitations of the more traditional LFA resulted in experimentation
with elements of TOC that were integrated within the logframe, in order to more
effectively capture the complexity of activities, partners and anticipated outcomes
of the program. The limitations of this single method approach resulted in CCAFS
deciding to operationalize a modular MEL approach, described in the next section.
The findings and analysis section explains CCAFS’s approach over time. With the
limitations in mind CCAFS is aiming for a more holistic approach in line with
Springer-Heinze et al. (2003).
4 Pathway to Impact: Supporting and Evaluating Enabling Environments for. . .
61
CCAFS was clearly committed to an outcome-focused R4D program from its
inception. It became increasingly clear that a logframe approach (LFA) was not
the most ideal way of doing R4D differently. In particular, when moving from a
Challenge Program to a CGIAR Research Program with increasing complexities of
partnerships, engagement and CGIAR integration, the limits of a logframe became
apparent (as described in section 4.5.1). The program’s vision of contributing
towards development outcomes increasingly required a different approach: one
that acknowledged the importance of stakeholder engagement and capacity development. As a result, monitoring the annual contribution of CCAFS and its partners
towards development outcomes becomes increasingly complex.
While a wide range of MEL approaches and methodologies with an outcome
focus exist (e.g. PIPA, Outcome Mapping, Outcome Harvesting), none provides a
blue-print solution that can just be rolled out. The approaches were designed to
address the particular needs of a specific program or organization. Thus, to adapt
these approaches to a new program, it is key to select the right mix of elements
creating a conceptual framework in support of the program’s specific TOC and
MEL requirements. Springer-Heinze et al. (2003) advocate a holistic approach to
impact evaluation and program monitoring with quantitative and qualitative elements, based on an impact pathway that can accommodate different stakeholder
views, allows for reflection, and emphasizes capacity of research organizations.
Cummings (1997) compares RBM, LFA and Project Lifecycle Management and
would welcome more discussions and learning among the different approaches.
According to Bazeley (2004), ‘The “mixing” may be nothing more than a side-byside or sequential use of different methods, or it may be that different methods are
being fully integrated in a single analysis’. Applying a mix of methodologies in a
programmatic MEL framework raises certain terminological, definitional, paradigmatic and methodological issues, including over-interpretation of numbers, single
dimensionality, and disregarding ‘outliers’ from the analysis (Bazeley 2004). However, mixed methods also provide opportunities to address the respective shortcomings of any single method as applied in practice.
CCAFS in its early years worked with various logframe elements in planning
and reporting. Limitations of the more traditional LFA resulted in experimentation
with elements of TOC that were integrated within the logframe, in order to more
effectively capture the complexity of activities, partners and anticipated outcomes
of the program. The limitations of this single method approach resulted in CCAFS
deciding to operationalize a modular MEL approach, described in the next section.
The findings and analysis section explains CCAFS’s approach over time. With the
limitations in mind CCAFS is aiming for a more holistic approach in line with
Springer-Heinze et al. (2003).
4 Pathway to Impact: Supporting and Evaluating Enabling Environments for. . .
61