New evaluation approaches emerging from Traditional evaluation… (focused on accountability) to Complexity-based evaluation… (in support of innovation and adaptation)
Traditional Evaluation… Render definitive judgments of success or failure ‘Developmental’ Evaluation… Provide feedback, generate learnings, support direction or affirm changes in direction in real time
Traditional Evaluation… Measure success against predetermined goals Linear cause/effect models ‘Developmental Evaluation’… Develop new measures and monitoring mechanisms as goals emerge & evolve Seek to capture system dynamics
Traditional ‘Developmental Evaluation… Evaluation’… Evaluator external, independent, objective oEngender fear of failure Evaluator part of a team, a facilitator and learning coach bringing evaluative thinking to support organization’s goals Feed hunger for learning Adapted from: Patton, Michael Q., 2006, “Evaluation for the Way We Work”, The Nonprofit Quarterly, Spring.
Relying on Indicators can be dangerous: 1.lulling: create a false sense of security 2.corrupting: they can become our objectives 3.biased: are not use, user, context or value neutral; subject to interpretation 4.static: do not show complex, incremental change 5.misleading: may be chosen for ease of access or need to aggregate rather than relevance
why behaviour changes? • development is done by and for people • a program can influence the achievement of outcomes - it cannot control them • ultimately responsibility rests with the people affected
The International Model Forest Network (Mexico, Russia, Chile)
Progress Markers for local communities 1. Participating in regular model forest meetings 2. Establishing a structure for cooperation 3. Acquiring new skills for managing model forests 4. Contributing resources to get the MF operational 5. Articulating a locally relevant vision for the MF 6. Promoting their MF nationally 7. Expanding the partnership 8. Calling upon external experts for advice 9. Requesting new opportunities for training 10. Publishing examples of benefits achieved through MF 11. Seeking out new partners for the MF 12. Obtaining funding from different national sources 13. Helping other communities establish MFs 14. Sharing lessons learned internationally 15. Influencing national policy debates on resource use
Another example… http://www.idrc.ca/en/ev-81099-201-1-DO_TOPIC.html
principles of use Flexible: modular to be adapted to use & context Complementary: use with other methodologies Participatory: seeks dialogue and collaboration with partners Evaluative thinking: culture of reflection, results oriented thinking, and promotes social & organizational learning
OM Strengths and Weaknesses (according to workshop participants...) OM is useful when: •the human component is very important •there is high likelihood of unexpected results •community development / action projects •project involves learning in the public sector •program involves capacity building •flexibility is needed •participatory processes are desired
OM Strengths and Weaknesses (according to workshop participants...) OM is not useful when: •looking at quantitative “hard“ science projects •conducting feasibility studies •primary goal is emergency relief •infrastructure projects with purely physical outputs •research with political agenda/constraints •large project with rigid M&E system in place •contract compliance is the focus
http://web.idrc.ca/en/ev-27705-201-1-DO_TOPIC.html http://www.outcomemapping.ca many more examples of OM use: “Not everything that matters can be counted, and not everything that can be counted matters” Albert Einstein
OMLFA Used as appropriate Required Includes tools & methods Defines cells Indicators are directional Unitary result indicators Focus on partners & partners’ partners Intervention focused OM แตกต่างจาก LFA อย่างไร
OMLFA Claims limited results Claims all results Rejects ‘impact” Includes “impact” Open to, negotiates issues Sets standard issues PM&E from bottom up Top-down oriented Learning orientedAccountability oriented OM แตกต่างจาก LFA อย่างไร
Your consent to our cookies if you continue to use this website.