pathways that cannot be exactly predicted at the outset of a project.
18 One of the
potential applications of theory-based evaluation is to identify measures that can be
used for monitoring over time.
19 But in complex systems relevant to CCA interventions, “SMART” measures may not be able to be developed in advance, making
pre-and post-comparisons difficult.
Whilst some versions of program theory evaluation rely on close adherence at a
detailed level to the initial theory to guide the evaluation, we do not take this strict
definition. The complex character of CCA interventions far from makes theorybased evaluation redundant. Rather, flexible application is needed, and a balance
struck between evaluation questions that are closely guided by the (initial) theory of
change, and an approach that is open to outcomes, and the means to achieving them,
emerging during implementation itself.
20
16.3.2 Developmental Evaluation, or, Learning in Complex
Systems
‘Developmental evaluation’ was coined by Paton to describe the types of evaluations applicable in complex situations where outcomes are emergent, where activities are not set in stone, and where it is not exactly known how, why or where
activities will lead.
21 Developmental evaluation aims to “support real-time learning
in complex and emergent situations” where the focus is on “adaptive learning rather
than accountability.”
22
Development practitioners and researchers have widely recognised that for projects in complex environments to be successful, self-evaluation and ongoing learning is key. Developmental evaluation can be understood by considering what its
purpose is not – it is not summative, in that it doesn’t aim to evaluate at the end of a
program and make a judgement about whether and how the program will continue;
18 Rogers, Patricia J. 2011. “Implications of Complicated and Complex Characteristics for Key
Tasks in Evaluation.” in Evaluating the Complex: Attribution, Contribution, and Beyond, edited by
K. Forss, M. Marra and R. Schwartz. New Brunswick, New Jersey: Transaction.
19 Funnell, SC and PJ Rogers. 2011. Purposeful Program Theory: Effective Use of Theories of
Change and Logic Models. San Francisco: John Wiley and Sons.
20 Rogers, P. J. 2008. “Using Programme Theory to Evaluate Complicated and Complex Aspects of
Interventions.” Evaluation 14(1):29–48. doi: 10.1177/1356389007084674
21 Gamble, Jamie A.A. 2008. A Developmental Evaluation Primer: The J.W. McConnell Family
Foundation.
22 Dozois, Elizabeth, Marc Langlois and Blacnhet-Cohen. 2010. A Practitioner’s Guide to Developmental Evaluation: The J.W. McConnell Famliy Foundation and the International Institute for
Child Rights and Development.
296
J. Chong et al.
18 One of the
potential applications of theory-based evaluation is to identify measures that can be
used for monitoring over time.
19 But in complex systems relevant to CCA interventions, “SMART” measures may not be able to be developed in advance, making
pre-and post-comparisons difficult.
Whilst some versions of program theory evaluation rely on close adherence at a
detailed level to the initial theory to guide the evaluation, we do not take this strict
definition. The complex character of CCA interventions far from makes theorybased evaluation redundant. Rather, flexible application is needed, and a balance
struck between evaluation questions that are closely guided by the (initial) theory of
change, and an approach that is open to outcomes, and the means to achieving them,
emerging during implementation itself.
20
16.3.2 Developmental Evaluation, or, Learning in Complex
Systems
‘Developmental evaluation’ was coined by Paton to describe the types of evaluations applicable in complex situations where outcomes are emergent, where activities are not set in stone, and where it is not exactly known how, why or where
activities will lead.
21 Developmental evaluation aims to “support real-time learning
in complex and emergent situations” where the focus is on “adaptive learning rather
than accountability.”
22
Development practitioners and researchers have widely recognised that for projects in complex environments to be successful, self-evaluation and ongoing learning is key. Developmental evaluation can be understood by considering what its
purpose is not – it is not summative, in that it doesn’t aim to evaluate at the end of a
program and make a judgement about whether and how the program will continue;
18 Rogers, Patricia J. 2011. “Implications of Complicated and Complex Characteristics for Key
Tasks in Evaluation.” in Evaluating the Complex: Attribution, Contribution, and Beyond, edited by
K. Forss, M. Marra and R. Schwartz. New Brunswick, New Jersey: Transaction.
19 Funnell, SC and PJ Rogers. 2011. Purposeful Program Theory: Effective Use of Theories of
Change and Logic Models. San Francisco: John Wiley and Sons.
20 Rogers, P. J. 2008. “Using Programme Theory to Evaluate Complicated and Complex Aspects of
Interventions.” Evaluation 14(1):29–48. doi: 10.1177/1356389007084674
21 Gamble, Jamie A.A. 2008. A Developmental Evaluation Primer: The J.W. McConnell Family
Foundation.
22 Dozois, Elizabeth, Marc Langlois and Blacnhet-Cohen. 2010. A Practitioner’s Guide to Developmental Evaluation: The J.W. McConnell Famliy Foundation and the International Institute for
Child Rights and Development.
296
J. Chong et al.