Empowerment and Accountability in difficult settings: What are we learning? - Gaventa, John; Oswald, Katy (Key messages emerging from theAction for Empowerment and Accountability Programme, 2019)
Empowerment and Accountability in Difficult Settings: What Are We Learning? Key Messages Emerging from the Action for Empowerment and Accountability Programme Empowerment and accountability have long been part of the international development vocabulary and a core part of governance, social development and civil society programmes. Yet, much of what has been learnt about these approaches has been drawn from studies in somewhat stable, open and middle-income places around the world. Less is known about how empowerment and accountability are achieved through social and political action in more difficult settings – those faced by institutional fragility, conflict, violence, and closing civic space. This document highlights key messages emerging from the work of the Action for Empowerment and Accountability Research Programme (A4EA), and the implications for how donors, policy makers and practitioners support strategies for empowerment and accountability in fragile, conflict and violence affected settings (FCVAS). Our eight key messages have strong implications for the theories of change used for effective programming in the field.

Empowerment Evaluation - Better Evaluation (Better Evaluation)
Empowerment Evaluation is an approach which provides communities with the tools and knowledge that allows them to monitor and evaluate their own performance. First developed by David Fetterman who describe it as “the use of evaluation concepts, techniques, and findings to foster improvement and self-determination.”

Empowerment evaluation - Fettermen, David (2014)

Empowerment Evaluation Blog - Fetterman, David (2006)
This blog, by Dr David Fetterman, provides a range of resources on empowerment evaluation theory and practice. The blog includes links to videos, guides and relevant academic literature that provide a detailed analysis and discussion of using empowerment evaluation.

Enabling citizen-driven improvement of public services. Leveraging Technology to Strengthen Accountability in Nigerian Healthcare - ICT4SA (World Bank and Reboot, 2015)
Effective public service delivery begins with knowing whether the services offered are working as intended. We launched a system to enable citizen input on the delivery of public services.

Enabling collaboration and synthesis: in brief - Cochrane, Logan; Cundill Kemp, Georgina; Landry, Marie-Eve; Lee, Rebecca (INBRIEF, 2017)
Collaborative Adaptation Research Initiative in Africa and Asia (CARIAA) aims to build the resilience of vulnerable populations in climate change hotspots by building new knowledge and capacities to support better informed policy and practice. The program connects more than 450 researchers and practitioners from over 40 organizations. Initially, CARIAA was not directive in terms of the specific topics or forms of synthesis. This approach allowed new ideas to emerge from interactions between members of the network. However, as the brief outlines, lessons learned at mid-term regarding cross- consortia collaboration and synthesis indicate that greater programmatic leadership is required.

Enabling collaborative synthesis in multi-partner programmes - Cochrane, Logan; Cundill, Georgina (Development in Practice, 2018)
Multi-partner consortia have emerged as an important modality for knowledge generation to address complex sustainability challenges. Establishing eﬀective multi-partner consortia involves signiﬁcant investment. This article shares lessons from the Collaborative Adaptation Research Initiative in Africa and Asia (CARIAA), which aims to support policy and practice for climate change adaptation through a consortium model. Key lessons include the need to facilitate collaborative spaces to build trust and identify common interests, while accepting that this is not a guarantee of success; the importance of programmatic leadership to achieve synthesis; and the value of strategic planning in supporting motivation and alignment between partners.

Enabling Innovation: A Practical Guide to Understanding and Fostering Technological Change - Douthwaite, M. B. (Zed Books, 2002)
Enabling Innovation is an engrossing look at some of the disaster—and success—stories surrounding technological development and diffusion in industrialized and developing countries. The book tells the story of widely divergent technologies—agricultural appliances, wind turbines, Green Revolution high yielding seeds, the Linux computer operating system, and Local Economic Trading Systems. Boru Douthwaite has constructed a "how to do it" guide to innovation management that runs counter to so many current "top-down", "big is good", and "private sector is best" assumptions.
Keywords: Political science

Energy Governance in Developing Countries — A New Approach - Mcculloch, Neil (The Policy Practice & Chemonics, 2021)
In 2015, leaders from around the world agreed to 17 Sustainable Development Goals (SDGs) to be achieved by 2030. The seventh goal (SDG7) is: “Ensure access to affordable, reliable, sustainable and modern energy for all.” In the same year, the world’s leaders concluded the Paris Agreement to tackle climate change, which will require a global transition in the energy sector away from the use of fossil fuels. Yet, despite growing investments in clean energy in many developing countries, the transition is happening much more slowly than needed. The central reason for this is poor energy governance. This technical brief shows how poor energy governance damages energy access and efforts to improve the quality and reliability of power. It explains the political reasons why energy governance is so bad in many countries and contrasts this with the current system of procuring technical assistance, which largely ignores the energy governance challenge. It shows that a new approach to tackling energy governance is emerging that is better matched to the nature of the problems faced and provides recommendations on how to implement it.

Engage. Empower. Enact. - Citizen Engagement & Democratic Innovation Programme White Paper - Doyle, Linda; Smith, Bethan (Cognitive Edge, 2021)
The Cynefin Centre’s Citizen Engagement & Democratic Innovation programme provides tools for collective sense-making in the areas of community development and youth work; civic engagement and democratic innovation; collaborative service/policy design and evaluation; housing/tenant engagement; futures and planning; shared learning and peer to peer knowledge exchange.

Engaging Farmers in Climate Change Adaptation Planning: Assessing Intercropping as a Means to Support Farm Adaptive Capacity - Himanen, Sari J.; Mäkinen, Hanna; Rimhanen, Karoliina; Savikko, Riitta (Agriculture, 2016)
Agriculture is one of the most vulnerable and adaptation-prone sources of livelihood facing climate change. Joint adaptation planning by farmers and researchers can help develop practically feasible and environmentally and economically sound adaptation actions as well as encourage the proactive building of farm adaptive capacity. Here, the perceptions of Finnish farmers and rural stakeholders regarding intercropping, the cultivation of two or more crop genotypes together in time and space, as a means to prepare for climate change, were collected in an open workshop. Our aim was to identify the potentials and challenges associated with intercropping, its role as an adaptation strategy, and in farm adaptive capacity. Qualitative analysis revealed better yield security, increased nutrient and protein self-sufficiency, soil conservation and maintenance, reduced pathogen pressure and regulation of water dynamics as the main perceived potentials of intercropping. Potentials relating to the farm economy and environment were also recognized. The main challenges associated with intercropping were related to the lack of information on crop variety performance and optimal yielding in mixtures, industry and policy requirements for seed purity, more complicated crop management and harvesting, and the economic risks associated with experimenting with novel mixtures. Nitrogen-fixing legumes; deep-rooted species, such as lucerne (Medicago sativa L.); special crops, such as herbs in forage mixtures; and autumn-sown winter oilseeds and cereals were highlighted as the most promising intercrops. Because the recognized potentials relate to the safeguarding of field cropping from anticipated climate change and the associated weather variability, we conclude that intercropping can serve as one adaptation strategy to strengthen the adaptive capacity of Finnish farms. However, assuring markets and policies that allow the development of intercropping, performing experiments to assess the benefits and implement options in practice, and providing farmers and farm advisors with more knowledge on the method represent the critical prerequisites for the broader adoption of intercropping.
Keywords: Adaptation planning; Adaptive capacity; Climate change; ecological intensification; intercropping; yield security

Engaging scientists through institutional histories - Prasad, Shambu; Hall, Andrew; Thummuru, Laxmi (ILAC Brief, 2006)
An institutional history is a narrative that records key points about how institutional arrangements – new ways of working – evolve over time creating more effective ways to achieve goals. It can be used to document institutional innovations in projects and to highlight barriers to change. An institutional history draws out and synthesizes lessons for research organizations and partners as well as for others in similar circumstances.

Engaging Stakeholders in Planning for Sea Level Rise and Resilience - Yusuf, Juita-Elena (Wie); John, Burton St; Covi, Michelle; Nicula, J. Gail (Journal of Contemporary Water Research & Education, 2018)
This case study describes a region-wide, multi-sectoral, and whole-of-community stakeholder engagement approach for addressing sea level rise (SLR) and flooding. This approach was implemented through a university-led community engagement event, the Hampton Roads Resilient Region Reality Check (H4RC), which allowed an examination of its effectiveness as a mechanism for capturing community-wide perceptions regarding SLR, flooding, and associated risks; engaging stakeholders in discussion within and across different groups; and assessing community willingness to address flooding and SLR. The results show that the event helped participants broaden their perspectives and understanding of flooding and SLR. In an approach that called for participants to engage in social learning across social networks, the event had some effect on individual efficacy. However, there was little impact on participants' already-established perception that the region does not possess significant willingness to take action.
Keywords: Social learning; flooding; resilience; social capital; stakeholder engagement

Engineering Complex Scaled Up Innovations in the Humanitarian and Development Sector - McClure, Dan; Gray, Ian (ThoughtWorks, 2015)

Engines of Change: What Civic Tech can learn from Social Movements - Purpose; Omidyar Network (Omidyar Network and Purpose, 2016)

Enhancing partner and system-level learning: 8 Tips from MEL Managers - Gover, Dun; Nasution, Zulka; Okutu, David; Bolder, Meghan; Henao, Lina (MSD in MEL Brief, 2023)
Effective learning is a key driver of market systems change, with the potential to enhance system competitiveness, resilience, and inclusiveness. Shifting the Locus of Learning: Catalyzing Private Sector Learning to Drive Systemic Change recently outlined a rationale for enhancing the scale and quality of learning in a system and identifying 10 strategies programs can contextualize to catalyze learning. These strategies are also backed with robust examples from 13 programs doing this work across 11 countries. To deepen insights on what MSD Monitoring, Evaluation and Learning (MEL) Managers have experienced in putting several of those strategies into practice, the Feed the Future Market Systems and Partnerships (MSP) Activity convened a series of peer discussions as part of a larger initiative (see Figure 1). This brief shares the collective learning and experience on this topic of three senior MEL Managers who were interested in and had experience with this topic. The group represented full-time, program-based MEL Leads working on MSD programs funded by USAID and DFAT, based in Fiji, Albania, and Kosovo working for Adam Smith International, SwissContact, and DT Global, respectively. From those discussions, this paper synthesizes eight tips from MEL Managers for practically enhancing partner and system-level learning: 1. Identify the right decision-maker(s) at potential partners. 2. Use a co-creation process to identify learning opportunities. 3. Use diagnostics and assessments to strengthen partner and system capacity for actionable learning. 4. Use a phased capacity strengthening process tied to behavior change. 5. Measure partners’ continued investment in and use of learning—not the continuation of specific learning activities. 6. Work with sector-level institutions for scale but be aware of risks. 7. Leverage informal communities of practice to share learning. 8. Use the right terminology to talk about partner and system-focused learning.

EPE Week: Lee-Anne Molony on Collaborative Outcomes Reporting - Molony, L A (AEA365 a Tip-a-day by and for Evaluators, 2013)
This AEA365 blog, written by Lee-Anne Molony a Principal Consultant at Clear Horizon, provides a brief overview on Collaborative Outcomes Reporting (COR).

Escaping Capability Traps through Problem-Driven Iterative Adaptation (PDIA) - Andrews, Matt; Pritchett, Lant; Woolcock, Michael (HKS Working Paper, 2012)
Keywords: IMPORTANT

Essentials of Utilization-Focused Evaluation - Patton, Michael Quinn (SAGE Publications, 2011)
Based on Michael Quinn Patton's best-selling Utilization-Focused Evaluation, this briefer book provides an overall framework and essential checklist steps for designing and conducting evaluations that actually get used. The new material and innovative graphics present the utilization-focused evaluation process as a complex adaptive system, incorporating current understandings about systems thinking and complexity concepts. The book integrates theory and practice, is based on both research and professional experience, and offers new case examples and cartoons with Patton's signature humor.
Keywords: Reference / Research; Research

EvalC3 - Davies, Rick (2019)
Tools for developing, exploring and evaluating predictive models of expected outcomes

Evaluability assessment for Impact Evaluation: Guidance, checklists and decision support - Peersman, Greet; Guijt, Irene; Pasanen, Tiina (ODI, 2015)
This guidance note focuses on the utility of, and guidance for, evaluability assessment before undertaking an impact evaluation. The primary audience for this guidance note is evaluators conducting an evaluability assessment for impact evaluation. The secondary audience is people commissioning or managing an evaluability assessment for impact evaluation, as well as funders of an impact evaluation. Sections one and two provide an overview of evaluability assessment and how it can be used for impact evaluation. Section three provides guidance for planning to undertake an evaluability assessment for impact evaluation. This is informative for all intended users of the guidance note. Section four includes checklists and decision support for evaluability assessments. The checklist is geared to those conducting the evaluability assessment and can be adapted to suit a particular context or purpose. The decision support provides those conducting an evaluability assessment with evidence-based recommendations for impact evaluation funders and commissioners, about whether, when and how to proceed with the evaluation. Sections five and six provide guidance on what to do after the assessment is concluded, and offer lessons learned from evaluability assessments in practice.

Evaluability Checklist for Post Project Evaluation - Zivetz, Laurie; Cekan, Jindra (Valuing Voices, 2020)
Considerations for planning a post project evaluation during the project, at the end, or after it has closed.

Evaluating CLARISSA: Innovation Driven by a Participatory Learning Agenda - Apgar, Marina; Snijder, Mieke; Kakri, Shanta; Macleod, Shona; Paul, Sukanta; Sambo, Anna; Ton, Giel (CLARISSA working paper, 2020)
Children end up in child labour as a result of many, often unknown or hidden, interactions between multiple actors and multiple factors within households, communities, and labour systems, leading to unpredictable outcomes for children and other sector stakeholders and sometimes resulting in the worst forms of child labour (WFCL). It is a complex problem, and interventions aimed at tackling it are also, inevitably, complex and challenging. The way they influence change is non-linear, causality is uncertain, and unintended consequences may result. Programmes such as the Child Labour: Action-Research-Innovation in South and South-Eastern Asia (CLARISSA) that are engaging with such intractable challenges and aim to reach the most left behind (children in WFCL) are operating in conditions of complexity. This complexity poses significant challenges to the way programmes are designed, planned, implemented, and evaluated, and requires a move away from linear and predetermined models. In this Working Paper, we share our experience and early learning about how to design and implement monitoring, evaluation and learning that intentionally embraces the challenge of complexity.

Evaluating Community Impact: Capturing and Making Sense of Community Outcomes - Tamarack Institute (2017)

Evaluating complex interventions: A theory-driven realist-informed approach - Douthwaite, Boru; Mayne, John; McDougall, Cynthia; Paz-Ybarnegaray, Rodrigo (Evaluation, 2017)
There is a growing recognition that programs that seek to change people’s lives are intervening in complex systems, which puts a particular set of requirements on program monitoring and evaluation. Developing complexity-aware program monitoring and evaluation systems within existing organizations is difficult because they challenge traditional orthodoxy. Little has been written about the practical experience of doing so. This article describes the development of a complexity-aware evaluation approach in the CGIAR Research Program on Aquatic Agricultural Systems. We outline the design and methods used including trend lines, panel data, after action reviews, building and testing theories of change, outcome evidencing and realist synthesis. We identify and describe a set of design principles for developing complexity-aware program monitoring and evaluation. Finally, we discuss important lessons and recommendations for other programs facing similar challenges. These include developing evaluation designs that meet both learning and accountability requirements; making evaluation a part of a program’s overall approach to achieving impact; and, ensuring evaluation cumulatively builds useful theory as to how different types of program trigger change in different contexts.

Evaluating complex interventions in international development - Masset, Edoardo; Shrestha, S.; Juden, M. (CEDIL Methods Working Paper 6, 2021)
This paper reviews promising methods for the evaluation of complex interventions that are new or have been used in a limited way. It offers a taxonomy of complex interventions in international development and draws on literature to discuss several methods that can be used to evaluate these interventions. Complex interventions are those that are characterised by multiple components, multiple stakeholders, or multiple target populations. They may also be interventions that incorporate multiple processes of behavioural change. While such interventions are very common and receive a large proportion of development aid budgets, they are rarely subject to rigorous evaluations. The CEDIL Methods Working Paper, ‘Evaluating Complex Interventions in International Development’, reviews promising methods for the evaluation of complex interventions that are new or have been used in a limited way. It offers a taxonomy of complex interventions in international development and draws on literature to discuss several methods that can be used to evaluate these interventions. The paper focuses its attention on methods that address causality and allow us to state conclusively whether an intervention works or not. It shows that several rigorous methods developed in different disciplines can be adapted and used to evaluate complex interventions in international development.

Evaluating complex interventions: What are appropriate methods? - Masset, Eduardo (CEDIL, 2022)
In the CEDIL Methods brief, ‘Evaluating complex interventions: What are appropriate methods?’ we identify four types of complex development interventions: long causal chain interventions, multicomponent interventions, portfolio interventions, and system-level interventions. These interventions are characterised by multiple activities, multiple outcomes, multiple components, a high level of interconnectedness, and non-linear outcomes.

Evaluating Complexity. Propositions for improving practice - Preskill, Hallie; Gopal, Srik (FSG, 2014)
9 propositions can help evaluators measure progress on complex social problems.
Keywords: IMPORTANT

Evaluating complexity, simplistically - Aston, Thomas (Medium, 2022)
A critical appraisal's of CEDIL papers on Evaluating Complex Interventions... A study was recently published by the Centre of Excellence for Development Impact and Learning (CEDIL) entitled Evaluating complex interventions in international development. This is the sort of title that raises great expectations. Complexity is a hugely popular theme and many of us are keen to know more about how to evaluate efforts that seek to achieve results amid complexity. In April 2021, CEDIL conducted a webinar on the paper, and in July 2021 CEDIL published a blog. In September 2021, I wrote a blog expressing some reservations regarding the focus of the study; its apparent over-emphasis on interventions, under-emphasis on context, as well as its choice of some supposedly under-used methods. These methods were: (1) factorial designs; (2) adaptive trials; (3) Qualitative Comparative Analysis (QCA); (4) synthetic control; (5) agent-based modelling, and system dynamics.

Evaluating Countering Violent Extremism: Practice and Progress - Fink, Naureen Chowdhury; Romaniuk, Peter; Barakat, Rafia (Center on Global Counterterrorism Cooperation, 2013)

Evaluating development projects: exploring a synthesis model of the logical framework approach and outcome mapping - Yang, Ting (2018)
Under the current results-driven development agenda, sound evaluation, and a corresponding evaluation toolkit, need to be in place to examine whether and to what extent development interventions have achieved their targeted objectives and results, and to generate lessons for further development learning and improvement. My review of the literature shows that innovative and appropriate evaluation approaches are needed to address key challenges in evaluation such as the tension between learning and accountability objectives, the need to unpack the mechanisms linking outputs and outcomes or goal, and to add an actor perspective. Irrespective of project type, the Logical Framework Approach (LFA) is often a standard requirement of major official donor agencies on projects they fund, so as to fulfil bureaucratic imperatives. However, it is often considered inadequate in addressing key challenges in development evaluation. Given the dominant status of the LFA with such strong support from donors, it is helpful to seek a ‘middle way’: a combination of the LFA with other approaches in order to address some of its inadequacies, while satisfying donor agencies’ requirements. A synthesis of the LFA and Outcome Mapping (OM) is one such option. This thesis explores the practical value and usefulness of a synthesis model empirically. Applying the model in two case study aid projects, I found that it serves well as a theory-based evaluation tool with a double-stranded (actor strand and results chain) theory of change. The model helps reconcile learning and accountability and add explanatory power and an explicit actor perspective. It also helps establish causation and enable attribution claims at various results levels with its different elements. The model has some limitations but my results suggest it can be usefully adopted. The choice of its application depends on project evaluation context and purpose in specific cases.

Evaluating Humanitarian Innovation - Obrecht, Alice; Warner, Alexandra; Dillon, Neil (HIF-ALNAP working paper, 2017)

Evaluating interventions that prevent or counter violent extremism: A practical guide - Hofman, Joanna; Sutherland, Alex (RAND Corporation, 2018)

Evaluating megaprojects: From the ‘iron triangle’ to network mapping - Lehtonen, Markku (Evaluation, 2014)
Evaluation literature has paid relatively little attention to the specific needs of evaluating large, complex industrial and infrastructure projects, often called ‘megaprojects’. The abundant megaproject governance literature, in turn, has largely focused on the so-called ‘megaproject pathologies’, i.e. the chronic budget overruns, and failure of such projects to keep to timetables and deliver the expected social and economic benefits. This article draws on these two strands of literature, identifies shortcomings, and suggests potential pathways towards an improved evaluation of megaprojects. To counterbalance the current overemphasis on relatively narrowly defined accountability as the main function of megaproject evaluation, and the narrow definition of project success in megaproject evaluation, the article argues that conceptualizing megaprojects as dynamic and evolving networks would provide a useful basis for the design of an evaluation approach better able to promote learning and to address the socio-economic aspects of megaprojects. A modified version of ‘network mapping’ is suggested as a possible framework for megaproject evaluation, with the exploration of the multiple accountability relationships as a central evaluation task, designed to reconcile learning and accountability as the central evaluation functions. The article highlights the role of evaluation as an ‘emergent’ property of spontaneous megaproject ‘governing’, and explores the challenges that this poses to the role of the evaluator.

Evaluating programming that thinks and works politically: Challenges and emerging practice - Jacobstein, David; Swift, Sarah (New Directions for Evaluation, 2022)
Issues of power are not new to program evaluation. What is new is a consideration of how programming uses insights into incentives that shape and adapt implementation. How should one evaluate in a way that explicitly assesses the ways in which a program considers power? One of the innovative topics deriving from the democracy and governance space is the approach of thinking and working politically (TWP) which is seeing increased use in development programming. TWP suggests different mental models and practical approaches to achieving development objectives in ways that are more contextually grounded and informed by power dynamics. This article describes several of the core challenges to evaluation of TWP and also a rubric of considerations for more effective evaluation practices in this emerging field.

Evaluating Research for Development: Innovation to Navigate Complexity - Apgar, Marina; Snijder, Mieke; Higdon, Grace Lyn; Szabo, Sylvia (The European Journal of Development Research, 2023)
Large publicly funded programmes of research continue to receive increased investment as interventions aiming to produce impact for the world’s poorest and most marginalized populations. At this intersection of research and development, research is expected to contribute to complex processes of societal change. Embracing a co-produced view of impact as emerging along uncertain causal pathways often without predefined outcomes calls for innovation in the use of complexity-aware approaches to evaluation. The papers in this special issue present rich experiences of authors working across sectors and geographies, employing methodological innovation and navigating power as they reconcile tensions. They illustrate the challenges with (i) evaluating performance to meet accountability demands while fostering learning for adaptation; (ii) evaluating prospective theories of change while capturing emergent change; (iii) evaluating internal relational dimensions while measuring external development outcomes; (iv) evaluating across scales: from measuring local level end impact to understanding contributions to systems level change. Taken as a whole, the issue illustrates how the research for development evaluation field is maturing through the experiences of a growing and diverse group of researchers and evaluators as they shift from using narrow accountability instruments to appreciating emergent causal pathways within research for development.

Evaluating Systems Change Results: an inquiry framework - Cabaj, Mark (Tamarack Institute, 2018)

Evaluating the Design of Behavior Change Interventions: A Case Study of Rhino Horn in Vietnam - Olmedo, Alegría; Sharif, Vian; Milner‐Gulland, E. J. (Conservation Letters, 2018)
Behavioral change interventions are increasingly widely used in conservation. Several projects addressing rhino horn consumption were recently launched in Vietnam. We used key informant interviews, document analysis, and marketing theory to explore their strategies for intervention design, implementation, monitoring, and evaluation. We developed a framework to evaluate whether they followed best practice and identify implementation challenges. Interventions could make greater use of key project design steps, including basing interventions on robust research to understand the behavior in question, identifying the target audience whose behavior interventions aim to change, and developing measures that can provide reliable evidence of success or not. Challenges include the need for law enforcement to complement campaigns; improving cooperation between NGOs; and clearly defining aims of demand-reduction initiatives. Using best practice from other fields and considering demand reduction within the wider context of wildlife, trade policy will help address these challenges.
Keywords: Awareness; Ilegal wildlife trade; Project design; demand reduction; social marketing

Evaluating the impact of flexible development interventions using a ‘loose’ theory of change: Reflections on the Australia-Mekong NGO Engagement Platform - Davies, Rick (Methods Lab, 2016)
• For some interventions, tight and testable theories of change are not appropriate – for example, in fast moving humanitarian emergencies or participatory development programmes, a more flexible approach is needed. • However, it is still possible to have a flexible project design and to draw conclusions about causal attribution. This middle path involves ‘loose’ theories of change, where activities and outcomes may be known, but the likely causal links between them are not yet clear. • In this approach, data is collected ‘after the event’ and analysed across and within cases, developing testable models for ‘what works’. More data will likely be needed than for projects with a ‘tight’ theory of change, as there is a wider range of relationships between interventions and outcomes to analyse. The theory of change plays an important role in guiding the selection of data types. • While loose theories of change are useful to identify long term impacts, this approach can also support short cycle learning about the effectiveness of specific activities being implemented within a project’s lifespan.

Evaluating the ‘logical framework approach’ towards learning-oriented development evaluation - Gasper, Des (Public Administration and Development, 2000)
The logical framework approach has spread enormously, including increasingly to stages of review and evaluation. Yet it has had little systematic evaluation itself. Survey of available materials indicates several recurrent failings, some less easily countered than others. In particular: focus on achievement of intended effects by intended routes makes logframes a very limiting tool in evaluation; an assumption of consensual project objectives often becomes problematic in public and inter-organizational projects; and automatic choice of an audit form of accountability as the priority in evaluations can be at the expense of evaluation as learning.

Evaluating with the Success Case Method - Gram, Tom (Performance X Design, 2011)
The method was developed by Robert Brinkerhoff as an alternative (or supplement) to the Kirkpatrick approach and its derivatives. It is very simple and fast (which is part of it’s appeal) and goes something like this: Step 1. Identify targeted business goals and impact expectations Step 2. Survey a large representative sample of all participants in a program to identify high impact and low impact cases Step 3. Analyze the survey data to identify: a small group of successful participants a small group unsuccessful participants Step 4. Conduct in-depth interviews with the two selected groups to: document the nature and business value of their application of learning identify the performance factors that supported learning application and obstacles that prevented it. Step 5. Document and disseminate the story report impact applaud successes use data to educate managers and organization The process produces two key outputs In-depth stories of documented business effect that can be disseminated to a variety of audiences Knowledge of factors that enhance or impede the effect of training on business results. Factors that are associated with successful application of new skills are compared and contrasted with those that impede training. It answers practical and common questions we have about training and other initiatives: What is really happening? Who’s using what, and how well? Who’s not using things as planned? What’s getting used, and what isn’t? Which people and how many are having success? Which people and how many are not? What results are being achieved? What value, if any, is being realized? What goals are being met? What goals are not? Is the intervention delivering the promised and hoped for results? What unintended results are happening? What is the value of the results? What sort of dollar or other value can be placed on the results? Does the program appear to be worthwhile? Is it producing results worth more than its costs? What is its return on investment? How much more value could it produce if it were working better? How can it be improved? What’s helping? What’s getting in the way? What could be done to get more people to use it? How can everyone be more like those few who are most successful?

Evaluation and learning in complex, rapidly changing health systems: China’s management of health sector reform - Xiao, Yue; Husain, Lewis; Bloom, Gerald (Globalization and Health, 2018)
Healthcare systems are increasingly recognised as complex, in which a range of non-linear and emergent behaviours occur. China’s healthcare system is no exception. The hugeness of China, and the variation in conditions in different jurisdictions present very substantial challenges to reformers, and militate against adopting one-size-fits-all policy solutions. As a consequence, approaches to change management in China have frequently emphasised the importance of sub-national experimentation, innovation, and learning. Multiple mechanisms exist within the government structure to allow and encourage flexible implementation of policies, and tailoring of reforms to context. These limit the risk of large-scale policy failures and play a role in exploring new reform directions and potentially systemically-useful practices. They have helped in managing the huge transition that China has undergone from the 1970s onwards. China has historically made use of a number of mechanisms to encourage learning from innovative and emergent policy practices. Policy evaluation is increasingly becoming a tool used to probe emergent practices and inform iterative policy making/refining. This paper examines the case of a central policy research institute whose mandate includes evaluating reforms and providing feedback to the health ministry. Evaluation approaches being used are evolving as Chinese research agencies become increasingly professionalised, and in response to the increasing complexity of reforms. The paper argues that learning from widespread innovation and experimentation is challenging, but necessary for stewardship of large, and rapidly-changing systems.

Evaluation at IDRC - IDRC (IDRC, 2017)

Evaluation Building Blocks: A Guide - McKegg, K.; Oakden, J.; Wehipeihana, N.; King, Julian (Kinnect Group, 2018)
This guide presents the Kinnect Group’s approach to evaluation. Developed through ten years of collaborative work, it has been well tested. While there are many other guides to evaluation and different ways to do it, our approach emphasises the place of evaluative reasoning in evaluation. The guide draws on the work of many evaluation theorists, as well as our practice-based body of knowledge. We hope that using the building blocks as explained in the guide will help you in your endeavour to do credible and useful evaluation. We find this approach works for us. Our clients tell us it gives them meaningful and insightful findings that they can use to take action. Our view to intellectual property is simple. What you read is yours to use. We just ask that you cite the guide when you draw from it. For those of you who engage with this guide and use it in your practice, we look forward to hearing from you. We are particularly interested in feedback where you find aspects of the guide work well for you. In that way we can all continue to build and evolve our practice.

Evaluation Feedback for Effective Learning and Accountability - OECD (Evaluation and Aid Effectiveness, 2001)
Evaluation is a key tool in improving the quality and effectiveness of development co-operation. The Development Assistance Committee (DAC) Working Party in Aid Evaluation is the only international forum where bilateral and multilateral evaluation experts meet regularly. Its Members meet to improve evaluation practice and to learn lessons from development co-operation activities and experiences. A series "Evaluation and Aid Effectiveness" was launched to share selected pieces of work with a broader public. This publication is composed of two parts: The Workshop Report, based on the fore-mentioned meeting, highlights the various issues raised, topics of discussion and different feedback systems, and outlines the areas identified by participants as most relevant for improving evaluation feedback. The Background Synthesis Report, intended as a starting point for discussion at the workshop, outlines the main concerns and challenges facing evaluation feedback and the means to address these. The report is based on an analysis of questionnaire results, and a review of previous initiatives in this area. The print version of the above report is available free of charge from the Development Co-operation Directorate; please send an email to dac.contact@oecd.org to get a copy.

Evaluation for equitable development results - Segone, M (UNICEF, 2011)
This document is made up of a range of Evaluation Working Papers (EWP) focused on evaluation for equitable development. Put together by evaluation specialists they present strategic evaluation findings, lessons learned and innovative approaches and methodologies. Part 1:Evaluation and equity Evaluation to accelerate progress towards equity, social justice and human rights 2 Human rights and gender equality in evaluation 13 When human rights is the starting point for evaluation 25 Strengthening Equity- focused evaluations through insights from feminist theory and approaches 39 Decolonizing evaluation in a developing world. Implications and cautions for Equity-focused evaluations 59 Part 2: Methodological implications for Equity-focused evaluations Methodological issues to design and implement equity-focused evaluations 86 Developmental evaluation for Equity-focused evaluations 102 Systems thinking and Equity-focused evaluations 115 Methodological challenges in using programme theory to evaluate pro-poor and equity-focused programmes by Patricia Rogers, Royal Melbourne Institute of Technology University and Richard Hummelbrunner, Independent consultant 142 Case Study and equity in Evaluationby Saville Kushner, University of the West of Englan 172 Values-Engaged Evaluationsby Jennifer Greene, University of Illinois at Urbana-Champaign 192 Part 3: Examples of Equity-focused evaluations Evaluating the contribution of UNDP to equity-focused public policies in Brazil and China 210 Using a human rights approach to evaluate ILO’s discrimination strategy 222 CONEVAL experience in evaluating interventions for Indigenous populations in Mexico 244 UNICEF supported evaluations with elements of equity-focused evaluations 258

Evaluation for Improvement: A Seven Step Empowerment Evaluation Approach for Violence Prevention Organizations - Cox, PJ; Keener, D; Woodward, T; Wandersman, A (2009)
This guide, written by Pamela J. Cox, Dana Keener, Tifanee L. Woodard, & Abraham H. Wandersman for the Centers for Disease Control and Prevention (CDC) outlines a seven step process for hiring an evaluator to implement an empowerment evaluation. The process begins with preparing for the hiring and concludes with an assessment of an evaluation to ensure its sustainability. Excerpt "...concerns and experiences with independent evaluation led to the development of participatory evaluation approaches as a way to promote an organization’s use of evaluation for the improvement of its strategies. Although there are many participatory evaluation approaches, empowerment evaluation places an explicit emphasis on building the evaluation capacity of individuals and organizations so that evaluation is integrated into the organization’s day-to-day management processes. Through empowerment evaluation, both individual and organizational evaluation capacity are increased through a “learn-by-doing” process, whereby organizations and their staff evaluate their own strategies. Specifically, organizations hire an evaluator to work with them in conducting an evaluation of their strategies. Rather than evaluating an organization’s strategies and presenting an evaluation “report card,” empowerment evaluators coach individuals and organizations through an evaluation of their own strategy(ies) by providing them with the knowledge, skills, and resources they need to conduct just such an evaluation." Contents Empowerment Evaluation: An Overview 11 Principles of Empowerment Evaluation 11 Step 1: Preparing for the Hiring Process 23 Step 2: Writing a Job Announcement 31 Step 3: Finding Potential Empowerment Evaluators 39 Step 4: Assessing the Candidates 45 Step 5: Writing An Evaluation Contract 55 Step 6: Building an Effective Relationship With Your Evaluator 61 Step 7: Assessing and Sustaining the Evaluation 65 Appendix A: Resources for General Evaluation and Empowerment Evaluation 73 Appendix B: Worksheets for Hiring an Empowerment Evaluator 75 Appendix C: Sample Hiring Committee Confidentiality Statement 89 Appendix D: Sample Job 90 Appendix E: Sample Request For Proposals 91 Appendix F: Sample Interview Questions 93 Appendix G: Sample Budget and Narrative for an Evaluation Team 95

Evaluation of development programmes - OECD - OECD (2019)
Data & research on evaluation of development programmes inc. paris declaration, budget support, multilateral effectiveness, impact evaluation, joint evaluations, governance, aid for trade

Evaluation of International Development Interventions: An Overview of Approaches and Methods - Vaessen, Jos; Lemire, Sebastian; Befani, Barbara (World Bank, 2020)
This guide provides an introductory overview of a range of methods that have been selected for their actual and potential use in the field of international development evaluation. For each method, a detailed guidance note presents the method’s main features and procedural steps, key advantages and disadvantages, as well as its applicability. Each guidance note includes references for relevant background readings (basic and advanced) as well as references to other additional resources of interest. Both the choice of approaches and methods and the associated guidance are by no means definitive. IEG plans to periodically update the guide as evaluation practices evolve.

Evaluation of the market systems development approach: Lessons for expanded use and adaptive management at Sida Volume I: Evaluation Report - Ruffer, Tim; Bailey, Helen; Dahlgren, Stefan; Spaven, Patrick; Winters, Mark (Evaluation of the market systems development approach, 2018)
This report presents the findings of an evaluation of Sida’s management of the market systems development (MSD) approach. It aims to inform thinking on how Sida can best manage its growing portfolio of MSD programs. Beyond this, it provides insights relevant to Sida’s wider support to complex and adaptive programs. The evaluation identified several factors that affect Sida’s ability to ensure that conducive conditions are in place for effective MSD programs and good development programming more generally. Sida’s relatively flexible framework of rules, guidelines and systems for project management provide the space needed for staff to innovate and manage adaptively. But for this to happen consistently and effectively, Sida needs to invest more deliberately in building the capacity of its staff in relevant areas. In addition, leadership and incentives are key to shaping a culture of active experimentation and learning. This needs to be supported with clearer guidance for those involved in the design and appraisal of MSD projects; and strengthened oversight of project performance, including through adjustments to Sida’s contracts and funding agreements.

Evaluation of the market systems development approach: Lessons for expanded use and adaptive management at Sida Volume II: Case studies - Ruffer, Tim; Bailey, Helen; Dahlgren, Stefan; Spaven, Patrick; Winters, Mark (Evaluation of the market systems development approach, 2018)
This report presents the findings of an evaluation of Sida’s management of the market systems development (MSD) approach. It aims to inform thinking on how Sida can best manage its growing portfolio of MSD programs. Beyond this, it provides insights relevant to Sida’s wider support to complex and adaptive programs. The evaluation identified several factors that affect Sida’s ability to ensure that conducive conditions are in place for effective MSD programs and good development programming more generally. Sida’s relatively flexible framework of rules, guidelines and systems for project management provide the space needed for staff to innovate and manage adaptively. But for this to happen consistently and effectively, Sida needs to invest more deliberately in building the capacity of its staff in relevant areas. In addition, leadership and incentives are key to shaping a culture of active experimentation and learning. This needs to be supported with clearer guidance for those involved in the design and appraisal of MSD projects; and strengthened oversight of project performance, including through adjustments to Sida’s contracts and funding agreements.

Evaluative reasoning in complexity - King, Julian (Evaluation and Value for Investment, 2023)

Everyday patterns for shifting systems - GCSI (GCSI, 2022)

Everyday patterns for shifting systems - Right scaling - GCSI (GCSI, 2022)

Everyday Political Analysis - Hudson, David; Marquette, Heather; Waldock, Sam (Developmental Leadership Program, 2016)
This short note introduces a stripped-back political analysis framework designed to help frontline development practitioners make quick but politically-informed decisions. It aims to complement more in-depth political analysis by helping programming staff to develop the 'craft' of political thinking in a way that fits their everyday working practices. Everyday Political Analysis involves two steps: Understanding interests: What makes people tick? Understanding change: What space and capacity do people have to effect change? For each step five questions, accompanied by prompts, aim to help staff to conduct quick political analysis. The EPA framework can be used at any stage of the aid management cycle, and can help users to respond rapidly to unexpected change. We are keen to hear back from people on their experience of using EPA to help us adapt the framework. Was it useful (or not)? Do people tend to use just one or both steps? Are there missing statements or prompts that would improve the analysis? Please email us at info [at] dlprog.org.

Evidence Base for Collaborating, Learning, and Adapting: summary of the literature review (2016.08) - Learning Lab (USAID, 2016)
LEARN and USAID/PPL are managing an area of work known as Evidence Base for Collaborating, Learning, and Adapting (EB4CLA). The purpose of this work is to answer key learning questions:

Evidence Base for Collaborating, Learning, and Adapting: summary of the literature review (2017 update) - Learning Lab (USAID, 2017)
LEARN and USAID/PPL are managing an area of work known as Evidence Base for Collaborating, Learning, and Adapting (EB4CLA). The purpose of this work is to answer key learning questions:

Evidence base for Collaborating, Learning, and Adapting: summary of the literature review (2020 update) - Learning Lab (USAID, 2020)
The LEARN contract and the United States Agency for International Development/Bureau of Policy, Planning, and Learning (USAID/PPL) are managing an area of work known as the Evidence Base for Collaborating, Learning, and Adapting (EB4CLA). The purpose of this work is to answer the following key learning questions: ● Does an intentional, systematic, and resourced approach to collaborating, learning, and adapting (CLA) contribute to organizational effectiveness and/or development outcomes? ● If so, how? And under what conditions? ● How do we know? How do we measure any contribution that CLA makes to development results? As we began this work, we identified the need to conduct a literature review looking at these questions to understand what is known, what remains unknown, and how others have tried to answer these questions to date. We were primarily interested in answering these questions: ● What evidence is there, if any, that collaborating, learning, and/or adapting contributes to organizational effectiveness, development outcomes, or both? What are the strongest pieces of evidence? ● Does the literature identify any factors critical to CLA that are not currently included in the CLA framework? ● Who else is working on measuring the impact of collaborating, learning and adapting? ● What methods and measures did researchers use to study the effects of collaborating, learning, and adapting? ● Where are there gaps in the research relevant to collaborating, learning, and adapting? ● When taken together, what practical guidance does the evidence for collaborating, learning, and adapting offer to practitioners and policy makers to improve organizational effectiveness and development outcomes?

Evidence, hierarchies, and typologies: horses for courses - Petticrew, M; Roberts, H (Journal of epidemiology and community health, 2003)
Debate is ongoing about the nature and use of evidence in public health decision making, and there seems to be an emerging consensus that the "hierarchy of evidence" may be difficult to apply in other settings. It may be unhelpful however to simply abandon the hierarchy without having a framework or guide to replace it. One such framework is discussed. This is based around a matrix, and emphasises the need to match research questions to specific types of research. This emphasis on methodological appropriateness, and on typologies rather than hierarchies of evidence may be helpful in organising and appraising public health evidence.
Keywords: Decision Making; Policy making; Public Health; Public policy

Evidence-led adaptive programming: Lessons from MUVA - Sharp, Samuel; Riemenschneider, Nils; Selvester, Kerry (ODI, 2022)
Calls for more ‘adaptive programming’ have been prominent in international development practice for over a decade. Learning-by-doing is a crucial element of this, but programmes have often found it challenging to become more learning oriented. Establishing some form of reflective practice, against countervailing incentives, is difficult. Incorporating data collection processes that generate useful, timely and practical information to inform these reflections is even more so.This paper explores MUVA - an adaptive female economic empowerment programme in Mozambique. MUVA, we suggest, is atypically evidence-led. It combines systematic, inclusive reflective practice with extensive real-time data collection. We describe the fundamental features of MUVA’s monitoring, evaluation and learning (MEL) approach that supported this. One, how data collection and analysis are synchronised with set cycles for learning and adapting projects. Two, how MEL systems are designed to prioritise actionable learning, with data collection oriented more to the needs of implementing staff than to the reporting requirements of funders.This approach was enabled by building collective ownership over the programme’s objectives and the purpose of MEL from the outset. Implementers are asked about their motivations, and these are related to the programme’s Theory of Change. The evidence culture is supported by the proximity of MEL staff to implementing staff; and through structuring upwards accountability to funders around justifying evidence-based adaptations instead of reporting on more narrow indicators. We conclude by considering the relevance, or not, of MUVA’s approach to programmes in other contexts or issue areas trying to replicate a similarly evidence-informed approach to adaptive management. Key messages Learning-by-doing is essential to adaptive programming, but it can be challenging to establish data collection processes that generate useful, timely and practical information. MUVA – a female economic empowerment programme in Mozambique – has an atypically evidence-led adaptive management approach. This has two fundamental features. One, data collection and analysis are synchronised with set cycles for learning and adapting projects. Two, data collection is oriented more to the needs of implementing staff than to the reporting requirements of funders. Monitoring, evaluation and learning (MEL) systems are designed to prioritise actionable learning. This approach was enabled by building collective ownership over the programme’s objectives and the purpose of MEL from the outset. Implementers are asked about their motivations, and these are related to the programme’s Theory of Change. The evidence culture is supported by the proximity of MEL staff to implementing staff; and through structuring upwards accountability to funders around justifying evidence-based adaptations instead of reporting on more narrow indicators.

Evidence to Action: Research to Address Illegal Wildlife Trade - Milner-Gulland, E. J.; Cugniere, Laure; Hinsley, Amy; Phelps, Jacob; Rolfe, Michael 't Sas; Veríssimo, Diogo (2018)
Tools and expertise to improve the evidence base for national and international Illegal Wildlife Trade policy already exist but are underutilised. Tapping into these resources would produce substantive benefits for wildlife conservation and associated sectors, enabling governments to better meet their obligations under the Sustainable Development Goals and international biodiversity conventions. This can be achieved through enhanced funding support for inter-sectoral research collaborations, engaging researchers in priority setting and programme design, increasing developing country research capacity and engaging researchers and community voices in policy processes. This briefing, addressed to policy makers and practitioners, is part of the 2018 Evidence to Action: Research to Address Illegal Wildlife Trade event programme, organised by five of the UK’s most active IWT research institutions, to support the London 2018 IWT Conference.

Evolutionary Delivery Versus the "Waterfall Model" - Gilb, Tom (SIGSOFT Softw. Eng. Notes, 1985)
The conventional wisdom of planning software engineering projects, using the widely cited "waterfall model" is not the only useful software development process model. In fact, the "waterfall model" may be unrealistic, and dangerous to the primary objectives of any software project.The alternative model, which I choose to call "evolutionary delivery" is not widely taught or practiced yet. But there is already more than a decade of practical experience in using it. In various forms. It is quite clear from these experiences that evolutionary delivery is a powerful general tool for both software development and associated systems development.Almost all experienced software developers do make use of some of the ideas in evolutionary development at one time or another. But, this is often unplanned, informal and it is an incomplete exploitation of this powerful method. This paper will try to expose the theoretical and practical aspects of the method in a fuller perspective. We need to learn the theory fully, so that we can apply and learn it completely.

Examining the results and adaptation ideas in foreign aid - Janus, Heiner (University of Manchester, 2020)
This thesis applies ideational and institutional theories to analyse how two specific ideas, results and adaptation, have changed the theory and practice of development cooperation. The thesis addresses the question of why the results and adaptation ideas are often treated as binaries and how this debate has evolved historically. In a first theoretical paper, the evolution of results and adaptation is conceptualised as a combination of institutional layering and diffusion within development organisations. The second theoretical paper applies ideational theory, in particular, the coalition magnet framework, to China as a donor country. The empirical papers apply ideational and institutional theories to study aid projects funded by the World Bank and China in the Rwandan agriculture sector. The third paper analyses through which mechanism, results-based principal-agent relationships or problem-driven iterative adaptation, the World Bankâs Program for Results in the agriculture sector in Rwanda has led to increased agricultural productivity. The paper combines causal process tracing and contribution analysis to investigate two underlying theories of change of the Program for Results. The fourth paper applies the same framework and methodology to the Chinese Agricultural Technology Demonstration Center in Rwanda. The fifth paper compares both projects, the World Bank project and the Chinese project. The thesis finds that the ideas of results and adaptation are often presented as mutually exclusive mainly at the general level of public philosophies or paradigms, but show overlap and potential for integration on the level of framing policy problems and policy solutions. The thesis also demonstrates that there is unexplored potential for convergence between China and Development Assistance Committee donors around âcoalition magnetâ ideas. The empirical part of the thesis reveals how results-based and adaptive causal mechanisms co-exist within given aid interventions by the World Bank and China, how these interact and how they ultimately contribute to achieving development outcomes. The key finding is that the broader political context of the Rwandan agricultural sector is the main factor for determining development outcomes, which neither the World Bank project nor the Chinese projects take into account. The comparison of the World Bankâs and Chinaâs interventions finds that donor organisations need to address how results-based ideas in combination adaptive development ideas can be better tailored to fit into the specific context of the Rwandan agriculture sector.

Exit for Sustainability Checklists - Cekan, Jindra (Valuing Voices, 2020)
These Checklists for Exit along the Sustainability Project Cycle are based on lessons from ex-post evaluations that Valuing Voices has done or researched. Ex-post project completion evaluations are rare and even more so those that consulted partners and participants in the field about sustained impacts. Over $3.5 trillion has been spent on public foreign aid projects in the past 70 years (OECD 2019) yet we have evaluated less than 1% of these projects for sustainability. Our Valuing Voices ex-post research of 39 organizations’ evaluations of sustainability shows that most project results decrease (20-90%) as early as two years ex-post in addition to An Asian Development Bank study of post-completion sustainability found that “some early evidence suggests that as many as 40% of all new activities are not sustained beyond the first few years after disbursement of external funding,” Most project exits are in the last quarter and sustainability handover assumptions are not validated expost. Learning from what was sustained helps us know how to exit for sustainability from the very onset of the project (green slices) as compared to the typical project cycle (orange), above. We encourage those tasked with funding, designing, implementing, monitoring & evaluating projects to use these longer checklists and view the full recording shared with participants. A partial PowerPoint can also be found on „Sustaining All of our Hard Work“ presentation for the Vienna Evaluation Network (10/20). These checklists are aimed at donors/designers and implementers of foreign aid projects outcomes and impacts and can be adapted by local NGOs, national governments, private sector, academics, to create exit plans. Local participation in creating these and feedback on how well exit is going will help them sustain results.

Experiential Learning: Experience as the Source of Learning and Development - Kolb, David (Pearson FT Press, 2015)

Experiential Learning: Experience as the Source of Learning and Development - Kolb, David A. (Prentice Hall, 1983)
Drawing from the intellectual origins of experiential learning in the works of John Dewey, Kurt Lewin, and Jean Piaget, this comprehensive and systematic book describes the process of experiential learning. The author proposes a model of the underlying structures of the learning process based on research in psychology, philosophy, and physiology, and bases its typology of individual learning styles and corresponding structures of knowledge in different academic disciplines and careers on this structural model. He also applies experiential learning to higher education and lifelong learning, particularly with regard to adult education.

Experiential Learning: Fostering lasting behavioural change - Norton, Brittney M. (Prospects practice paper, 4, 2016)
The term ‘experiential learning’ is often mistakenly used to describe any learning experience that involves participatory activities. This paper presents an overview of what true experiential learning is and how this form of learning allows for real-world application - thus promoting lasting behavioural change. Furthermore, it explains how experiential learning was adopted as the methodological core of the Prospects psychosocial programme in Liberia, which builds ‘pre-employment skills’ in 3,500 vulnerable youth aged 15-24 years. The goal of the paper is to provide other youth and education development practitioners with a resource that can be used to strengthen youth and behaviour change programmes through the incorporation of the experiential learning theory. In addition, this paper explains its application in the Prospects programme, before concluding with key considerations of such a design. 1

Explaining positive deviance in public sector reforms in development - Andrews, Matt (World Institute for Development Economic Research (UNU-WIDER), 2013)
Public sector reforms are commonplace in developing countries. Much of the literature about these reforms reflects on their failures. This paper asks about the successes and investigates which of two competing theories best explain why some reforms exhibi
Keywords: Development; Innovation; Reform; case survey; governance; growth; leadership

Exploring a new governance agenda: What are the questions that matter? - Nixon, Nicola; Verhulst, Stefaan; Matin, Imran; Vermonte, Philips J. (From Poverty to Power, 2021)
The 100 Questions Initiative, pioneered by the GovLab, seeks to overcome the chasm between supply and demand. It begins not by searching for what data is available, but by asking important questions about the biggest challenges societies and countries face, and then seeking more targeted and relevant data solutions. In doing this, it narrows the gap between policy makers and constituents, providing opportunities for improved evidence-based policy and community engagement in developing countries. As part of this initiative, we seek to define the ten most important questions across several domains, including Migration, Gender, Employment, the Future of Work, and—now–Governance. On this occasion, we invited over 100 experts and practitioners in governance and data science –whom we call “bilinguals”– from various organizations, companies, and government agencies to identify what they see as the most pressing governance questions in their respective domains. Over 100 bilinguals were encouraged to prioritize potential impact, novelty, and feasibility in their questioning — moving toward a roadmap for data-driven action and collaboration that is both actionable and ambitious.

Exploring new territories for evaluation - Sharp, Cathy (Collective Leadership for Scotland, 2018)
Drawing on action research, this paper recasts evaluation as ‘action inquiry’, an embedded evaluative learning practice that can help navigate complexity when enacting collective leadership. It is offered as an invitation to inquiry amongst a reasonably well-informed audience of policy makers and practitioners who work in and for public services. It will particularly interest those who provide research, evaluation and facilitation support, and those seeking to develop a more relational approach to research and evaluation. Action inquiry is a model of practising change together in environments where ‘nothing is clear, and everything keeps changing’ that significantly challenges the prevailing discourse on evaluation. Action inquiry can be wrapped around and enmeshed within initiatives and programmes that work with complexity - anywhere where success will depend on the quality of relationships that can be developed. The paper reviews some important interrelated concepts that underpin the ideals of collective leadership and public service reform and which confront deeply embedded traditional notions of leadership, expertise and participation. These offer important challenges to ideas about how change happens and recognise that relationships are at the heart of practising change. This warrants a re-examination of the high expectations of evidence-based or informed practice. Collective leadership makes new demands of evidence as it rests on help to determine ‘wise actions’ in real-life situations. This confronts the practical reality of how to work together in conditions often expressed as ‘dynamic’ or ‘turbulent’ and the added human complexities of power, emotions and relationships; too often these elements are denied or avoided aspects of a change process. Facilitated action inquiry makes these elements part of the conversations, in the midst of ‘work-as-we-are-doing-it’, to increase areas of choice for individuals and a group as a whole. Within public policy, there is an overwhelming amount of evidence about ‘what works’ yet change seems to be stubborn and slow. The paper explores some of the deep-rooted vestiges of a ‘hierarchy of evidence’ and assumptions about standardisation and generalisability that act as a ‘barrier to transformation’. These include the narrow framing of what counts as evidence and consequent relegation of community perspectives, lived experience and practice-based evidence. The paper acknowledges the clear appetite for different approaches to evaluation, especially those that better reflect deeply held values and avoid creating a culture of ‘gaming’, rooted in fear of failure and loss of funding, at the expense of learning. The need for new forms of developmental evaluative thinking, collaborative inquiry and action research to create embedded learning is well overdue. Action inquiry is a desirable and necessary response to the complex situations and challenges of human services and recognises the essentialness of knowledge co-production. It is a model of co-creation at every stage and endorses the idea that people learn from participation in evaluation and by testing theories of change through action. Action inquiry builds on the idea of inquiry, or a moment-to-moment awareness and quality of attention and draws from several elements of action research practice. It sees inquiry as an intervention in itself, one that furthermore, explicitly seeks to enhance the probability of the success of a programme, focus on learning, the collaborative development of practice-based knowledge and positive relationships. The paper highlights the importance of building inquiry into living systems, the role of facilitation, systemic inquiry, and evaluative thinking. It proposes an expansion of ideas of appreciation as a relational and collaborative practice that is a driver of emergence. Social recognition that acknowledges someone’s social value to the community and implies mutual moral obligations to cooperation and participation is particularly crucial in a work context that requires successful coordination and multiple contributions to achieve results across hierarchies of position, professional rank and sectors. Hence, appreciation goes beyond the idea of positivity to include social recognition, valuing more explicit forms of inquiry, building participants’ aspirations to design new social systems and acting in new ways to embed change. In developing this discussion, the paper contributes to emerging dialogues about the need for a model of ‘5th generation evaluation’. Such a model would be based on the idea that appreciative and challenging inquiry that is contextual, relational and open-minded will create better opportunities for change and development. The paper sets out some ‘provocative propositions’ that can help us to navigate this terrain, perhaps of a fledgling ‘5th generation approach’ to inquiry. Facilitated action inquiry can hold the key to developing both new knowledge and an adaptive, collaborative and improvisational skill-set, able to respond in new ways to systemic and complex issues on the ground. It’s common to hear the expression ‘it’s all about relationships’ and it is clearly time to shift our focus to relationships; not relationships as ‘things’, but as co-created and dynamic relational processes in which we are embedded. In this way we can bring new qualities to our talking to each other about our various and shared visions of a better future.

Exploring new territories for evaluation - Provocative propositions - Sharp, Cathy (2018)

Exploring the new world: practical insights for funding, commissioning and managing in complexity - Lowe, Toby; Plimmer, Dawn (Collaborate & Newcastle University, 2019)
The world is complex. If we want to contribute to creating positive social outcomes, we must learn to embrace this complexity. This is the New World that funders and commissioners are discovering: • People are complex: everyone’s life is different, everyone’s strengths and needs are different. • The issues we care about are complex: issues – like homelessness – are tangled and interdependent. • The systems that respond to these issues are complex: the range of people and organisations involved in creating ‘outcomes’ in the world are beyond the management control of any person or organisation. Building on the findings from our previous report, A Whole New World, we have spent the last 12 months working with a growing movement of funders, public sector commissioners, and organisations working on the ground to begin to explore this New World: to find examples of practice that will help people to navigate it effectively. This report explores the key features of their response: working in a way that is human, prioritises learning and takes a systems approach.

Exploring the potential for interactive radio to improve accountability and responsiveness to small-scale farmers in Tanzania - Gilberds, Heather; Handforth, Calum; Leclair, Mark (IDS, 2016)
What are the possibilities of using new digital technologies alongside radio to help ensure that agricultural development projects are farmer-centred, and meet the needs of the rural citizens they intend to serve? This research assesses Farm Radio International’s Listening Post – a model that combines radio and digital technologies with the aim of collecting and aggregating farmer feedback to aid decision-making and adaptive project implementation. The research shows that linking a mobile-based crowd-sourcing tool with radio is effective at ensuring engagement from a large number of farmers, who felt it was a useful way of raising their questions and concerns to NGOs, policy-makers and experts. The model has clear potential to strengthen the chain of relationships between citizens, extension services suppliers, projects and policymakers. It has also demonstrated its potential to collect real-time feedback from farmers that could be used to aid decision-making and improve accountability in agricultural development initiatives, helping to ensure they are more responsive to farmers. However, it also concludes that ‘closing the feedback loop’ – ensuring that farmer’s comments, questions and concerns are responded to – is a challenge for the Listening Post. Sometimes, the interactive radio programmes close the loop by disseminating answers to questions or concerns raised by farmers, or by connecting farmers to extension services – but only when a solid and systematic process had been developed for this to happen. Therefore is critical to identify and incentivise actors who are capable of responding during the design phase of a Listening Post, and to ensure that they are involved in every stage of the process. It also important that project partners who are interested in beneficiary feedback identify the flexible or actionable points in their project frameworks from the outset, rather than collecting data that they are not able to use to adapt their programmes

Exploring the prospects for adaptive governance in marine transboundary conservation in East Africa - Tuda, Arthur Omondi; Kark, Salit; Newton, Alice (Marine Policy, 2019)
This article explores the prospects for adaptive governance in a proposed marine transboundary conservation initiative in East Africa. Adaptive governance that involves interdependent state and non-state actors learning and taking action on joint environmental problems is suggested for effective transboundary resource governance. Using the concept of adaptive co-management, the current multi-stakeholder marine governance systems in southern Kenya and northern Tanzania are compared to illuminate opportunities and constraints for adaptive marine transboundary conservation governance between Kenya and Tanzania. The concept of networks and the formal method of social network analysis (SNA) are applied as the main methodological device. Using questionnaire and semi-structured interviews, social network data of 70 organizations (local resources users, government agencies and NGOs) was generated from Kenya (n = 33) and Tanzania (n = 37). Results show the existence of strong collaboration networks for marine resource governance in both Kenya and Tanzania. Social proximity is the common driver of network formation. Collaboration networks in Kenya and Tanzania have contributed to enhanced learning among marine resource managers. Conclusions point to the need to focus on common challenges relating to low levels of rule-compliance, limited access to information on the state of resources and poor integration of science into marine management decisions. Finally, differences in views regarding the state of marine ecosystems need to be addressed to improve prospects for joint problem-solving in marine transboundary conservation.
Keywords: Adaptive co-management; Adaptive governance; Collaboration; Marine transboundary conservation; Social network analysis

Exploring the science of complexity: Ideas and implications for development and humanitarian efforts - Ramalingam, Ben; Jones, Harry; Reba, Toussaint; Young, John (Working Paper 285, 2008)
This paper draws on the science of complexity to outline alternative approaches to analysing and understanding problems faced in humanitarian and development work.

Exploring ways to reconcile accountability and learning in the evaluation of niche experiments - Regeer, Barbara J.; de Wildt-Liesveld, Renée; van Mierlo, Barbara; Bunders, Joske F. G. (Evaluation, 2016)
While evaluation is seen as a mechanism for both accountability and learning, it is not self-evident that the evaluation of niche experiments focuses on both accountability and learning at the same time. Tensions exist between the accountability-oriented needs of funders and the learning needs of managers of niche experiments. This article explores the differences in needs and expectations of funders and managers in terms of upwards, downwards and internal accountability. The article shows that as the multi-stakeholder contexts in which niche experiments take place give rise to various requirements, tensions in evaluation are essentially a specific manifestation of tensions between niche experiments and their multiple contexts. Based on our findings, an adjusted accountability framework is proposed, including several strategies that can reconcile a learning approach with accountability needs in niche experiments aiming to change current practices in a more sustainable direction.

Extending epistemology for programme evaluation – can After Action Reviews become spaces for critical reflection? - Apgar, Marina; Snijder, Mieke (Methodspace - Sage, 2020)
The term After Action Review (AAR) is becoming more common in the world of development evaluation, particularly in programmes that focus on evaluation as a formative learning process. As structured and facilitated learning moments, AARs take many shapes always built to support a specific team in a specific moment on their journey. Across such a diversity of practice how do we understand the evolution of the AAR as a method with an action research orientation? Being mindful always of the risk of instrumentalising and co-opting participatory methods, there is danger that AARs become an empty programme ritual, remaining at the surface and failing to achieve critical reflection and so falling short of their intention of pushing for deeper change in our practice. As the evaluation and learning team of two large complex, multi-partner projects using research to address development challenges, we have been applying the AAR method while adapting to virtual COVID working with partners across the world. As action researchers we have been thinking about what critical reflection means within a programme AAR process and whether and how they can open up a second person inquiry space.

External Review of the Building Effective and Accessible Markets (BEAM) programme - Snedden, Ewan; Freer, Gordon; Moffatt, Michelle (WYG International, 2016)

External validity and policy adaptation. From impact evaluation to policy design - Williams, Martin J. (BSG Working Paper Series, 2017)
With the growing number of rigorous impact evaluations worldwide, the question of how best to apply this evidence to policymaking processes has arguably become the main challenge for evidence-based policymaking. How can policymakers predict whether a policy will have the same impact in their context as it did elsewhere, and how should this influence the design and implementation of policy? This paper introduces a simple and flexible framework to address these questions of external validity and policy adaptation. I show that failures of external validity arise from an interaction between a policy’s theory of change and a dimension of the context in which it is being implemented, and develop a method of “mechanism mapping” that maps a policy’s theory of change against salient contextual assumptions to identify external validity problems and suggest appropriate policy adaptations. In deciding whether and how to adapt a policy in a new context, I show there is a fundamental informational trade-o↵ between the strength and relevance of evidence on the policy from other contexts and the policymaker’s knowledge of the local context. This trade-o↵ can guide policymakers’ judgments about whether policies should be copied exactly from elsewhere, adapted, or invented anew.

Extreme Teaming: Lessons in Complex, Cross-Sector Leadership - Edmondson, Amy C.; Harvey, Jean-Francois (Emerald Group Publishing Limited, 2017)
Extreme Teaming Lessons in Complex, Cross-Sector Leadership Today’s global enterprises increasingly involve collaborative work by teams of experts operating across different professions, organizations, and industries. Extreme Teaming provides new insights into the world of complex, cross industry projects and the ways they must be managed. Leading experts Amy Edmondson and Jean-François Harvey analyze contemporary cases that expose the complex demands of cross-boundary collaboration on management, and inform our understanding of teams. Containing powerful insights and practical guidelines that allow managers to bridge professional divides and organizational boundaries in order to work together effectively, this is a new exploration of the challenges involved in today’s global enterprises. The authors demonstrate that the work done in the modern organization is less and less about looking inward and creating strong teams inside the company, and more about teaming across boundaries – that often are in flux. Extreme Teaming is a must-read book for all courses related to leading open innovation; teamwork and collaboration; project management; and cross-boundary work.

Facilitating collaborative problem solving with human-centred design: the Making All Voices Count governance programme in 12 countries of Africa and Asia - Jackson, Carl (Knowledge Management for Development Journal, 2015)
Keywords: IMPORTANT

Facilitator's Guide to Participatory Decision-Making - Kaner, Sam (John Wiley & Sons, 2014)
Unleash the transformative power of face to face groups The third edition of this ground–breaking book continues to advance its mission to support groups to do their best thinking. It demonstrates that meetings can be much more than merely an occasion for solving a problem or creating a plan. Every well–facilitated meeting is also an opportunity to stretch and develop the perspectives of the individual members, thereby building the strength and capacity of the group as a whole. This fully updated edition of The Facilitator′s Guide to Participatory Decision–Making guides readers through the struggle and the satisfaction of putting participatory values into practice, helping them to fulfill the promise of effective group decision–making. With previous editions already embraced by business and community leaders and consulting professionals around the world, this new book is even more insightful and easy to use. New for this edition: 60 pages of brand new skills and tools Many new case examples Major expansion and reorganization of the advanced sections of the book. New chapter: Teaching A Group About Group Dynamics Doubled in size: Classic Facilitator Challenges. Substantially improved: Designing Realistic Agendas now three chapters, with wise, insightful answers to the most vexing questions about meeting design.

Facilities deserve a place in development - Lacy, Jacqui de (Devpolicy Blog from the Development Policy Centre, 2017)
There are different ways to deliver aid programs, DFAT often uses facilities in its larger relationships, what are the rationales and risks behind this?

Failing Forward: How CARE is focusing on what goes wrong to improve impact - Janoch, Emily (2019 CLA Case Competition, 2019)
CARE's Failing Forward initiative is sparking opportunities to showcase the ideas that don't work so we can spend more time implementing the ones that do. It's changing the conversation inside the organization, and leading to changes in the way we design and implement programs. It's also allowing us to make connections across a global portfolio of more than 900 projects, and with new and different partners to learn from their experience.

Failing Forward: How CARE is Focusing on What Goes Wrong to Improve Impact - Janoch, Emily (CLA Case Competition, 2019)
CARE's Failing Forward initiative is sparking opportunities to showcase the ideas that don't work so we can spend more time implementing the ones that do. It's changing the conversation inside the organization, and leading to changes in the way we design and implement programs.

Failure, Success and Improvisation of Information Systems Projects in Developing Countries - Heeks, Richard (Development Informatics Working Paper Series, No.11/2002, 2002)
Research on the Information Society, the Digital Divide and Information and Communication Technologies for development

Fall in love with the solution, not the problem - Quaggiotto, Giulio; Leurs, Bas; Hazeldine, Shaun (Nesta, 2016)
Three strategies for international development organisations to solve problems without defining them.

Farewell DFID … a personal obituary - Mason, Phil (LinkedIn - Phil Mason, 2020)
Reflection on how DFID was created.

Fast Forwarding to Systems Maps of Corruption: Getting to Usable Analysis More Quickly - Woodrow, Peter (CJL, 2023)
For AC practitioners, systems mapping is essential but also difficult. To lessen obstacles, Peter Woodrow proposes a scaffolded approach. An Experiment in “Fast Forwarding” Drawing upon our near-decade work on corruption, we recently decided to try an experiment: we would present “common patterns” of corruption as tentative models to adapt and add to—rather than try to teach people to do systems mapping from scratch. In this teaching experiment, each common pattern would function as a kind of “scaffolding” or framework that participants can build on to generate a cogent systems map as the basis for identifying possible points of intervention and subsequent program planning.

FCDO Beneficiary Engagement - FCDO (FCDO, 2021)
This guide has been developed to help build confidence and capability, distilling useful tips and considerations that may help teams think through programme delivery issues and interpret elements of the PrOF Rules. This PrOF Guide lays out: - The definition of beneficiary engagement. - The case for beneficiary engagement. - FCDO’s approach to beneficiary engagement. - Practical tips for how to integrate beneficiary engagement throughout the programme cycle, including guiding questions to ask, rules of thumb to apply, tools to use and challenges and special topics to consider during Design, Mobilisation, Delivery and Closure phases. At its core, beneficiary engagement is about processes that recognise the dignity and support the agency of the people whose lives we are trying to improve. It is about beneficiaries and programme constituents having a say over what assistance they receive and how they receive it. It is about engaging beneficiaries and programme constituents as people with valuable insights and capabilities, rather than a compliance exercise. It is about empowering all beneficiaries and programme constituents to improve their lives by engaging them in helping us make better design and delivery decisions for the programmes that affect their lives. It’s about ensuring that a diverse set of voices are heard. Harnessing the power of beneficiary engagement can also improve outcomes and help programmes reach them more efficiently. It helps define and promote Value for Money, improve transparency and ensure that beneficiaries are safe from harm and empowered to speak out wherever harm does occur. Beneficiary engagement is supported by FCDO policy commitments, PrOF Rules, internal guidance and key international commitments. Beneficiary engagement is applicable to, and valuable in, a range of contexts, including humanitarian contexts. Beneficiary engagement requires time and resources, but programmes can help ensure the benefits of engagement outweigh the costs by Doing No Harm, Engaging Early and Closing the Loop. It is ultimately the Programme Responsible Owner’s responsibility to determine what beneficiary engagement is suitable and feasible for a programme. Quality beneficiary engagement is not about applying the one “right” approach but rather thoughtfully considering key questions and applying key principles and proven tools to the programme’s context to achieve a programme that supports the dignity and agency of all beneficiaries as much as possible.

FCDO Programme Operating Framework - FCDO (FCDO, 2021)
The Programme Operating Framework (PrOF) sets the standard for how the Foreign, Commonwealth & Development Office (FCDO) delivers its programmes and projects.

Feasibility Study: Crowdsourcing High-frequency food price data in rural Indonesia - UN Global Pulse (2015)

Feedback Commons (Keystone Accountability, 2018)

FeverBee’s Online Community Strategy Template - FeverBee (FeverBee, 2018)

FeverBee's Strategic Project Plan Template - FeverBee (FeverBee)
Gantt Chart [ client] COMMUNITY STRATEGY- PROJECT PLAN [ client] Community Strategy, COMPANY NAME,[ client] BY RICHARD MILLINGTON, DATE, 25/ 7/ 2018 No, TASK TITLE, TASK OWNER, START DATE, DUE DATE, DURATION, PCT OF TASK COMPLETE, RESEARCH AND DEVELOPMENT, DEVELOPING THE STRATEGIC PLAN, BUILD...

Field Theory in Social Science: Selected Theoretical Papers - Lewin, Kurt (Harper & Brothers, 1951)

Finding and using causal hotspots: a practice in the making - Apgar, Marina; Snijder, Mieke (Institute of Development Studies, 2021)
This is the second blog in our series ‘Lessons on using Contribution Analysis for impact evaluation’. In our first blog, we introduced Contribution Analysis (CA) as an overarching approach to theory-based evaluation and the idea of causal hotpots as a way to zoom in, unpack and make the hard choices of where to focus evaluation research. Identifying specific links in the theory of change (ToC) with specific evaluation questions enables you to then choose appropriate methods. We have applied the approach in diverse settings, testing how robust Contribution Analysis can really be. Here we walk you through an example.

Finding out fast: investigative skills for policy and development - Thomas, Alan; Chataway, Joanna; Wuyts, Marc (Sage, 1998)

Finding the Spaces for Change: A Power Analysis - Gaventa, John (IDS Bulletin, 2006)

Five Misunderstandings About Case-Study Research - Flyvbjerg, Bent (Qualitative Inquiry, 2006)
This article examines five common misunderstandings about case-study research: (a) theoretical knowledge is more valuable than practical knowledge; (b) one cannot generalize from a single case, therefore, the single-case study cannot contribute to scientific development; (c) the case study is most useful for generating hypotheses, whereas other methods are more suitable for hypotheses testing and theory building; (d) the case study contains a bias toward verification; and (e) it is often difficult to summarize specific case studies. This article explains and corrects these misunderstandings one by one and concludes with the Kuhnian insight that a scientific discipline without a large number of thoroughly executed case studies is a discipline without systematic production of exemplars, and a discipline without exemplars is an ineffective one. Social science may be strengthened by the execution of a greater number of good case studies.

Five tips for evaluating your impact in international development - Pasanen, Tiina (The Guardian, 2015)
How can impact evaluations actually be helpful? These top tips will save development professionals time, energy and money. International development can be messy – with uncertain, complex settings, and multiple partners with different interests, goals and capacities. At the same time, we are under increasing pressure to demonstrate impact. We have to show that our projects have made a real change in people’s lives and that donor or taxpayer money hasn’t been wasted. But impact evaluations, which are often seen as a solution to this, aren’t always used. And in some cases they are unhelpful. So how can we improve the quality of impact evaluations, so that they produce results that are useful? This is where evaluability assessments come in. By asking whether we should evaluate a project – and if so, when and how – these assessments can improve the quality of impact evaluations. Here are five other reasons to assess evaluability before starting an impact evaluation.

Five Ways to Make a Difference: Perceptions of Practitioners Working in Urban Neighborhoods - Durose, Catherine; van Hulst, Merlijn; Jeffares, Stephen; Escobar, Oliver; Agger, Annika; de Graaf, Laurens (Public Administration Review, 2015)

FOCEVAL – Promoting Evaluation Capacities in Costa Rica:: Smart(er) Implementation with Capacity WORKS? - Storm, Sabrina (Transformation, Politics and Implementation, 2017)
The National Monitoring and Evaluation System of Costa Rica and its corresponding laws were established during the 1990s. Since then, the country has endeavored to implement monitoring and evaluation (M&E) activities as part of its public policy framework. Nevertheless, hardly any systematic evaluations had been conducted, and monitoring activities had been reduced mainly to the institutional self-reporting of implementation compliance. Persisting regional disparities and growing levels of inequality among the population raised the level of pressure on the government to present reliable information on the effectiveness of public interventions. Hence, results-oriented evaluations were promoted by some Costa Rican departments as

Follow the rabbit: Field Guide to Systemic Design - CoLab (Government of Alberta, 2018)
This Field Guide is designed to support budding systemic designers to facilitate and lead systemic design projects. It’s designed for systemic designers who have basic familiarity with SD concepts and are looking for some practical tips and tools to put theory into action. While this Guide was designed with Government of Alberta staff in mind, the content applies across a range of issue areas, sectors, and intersections. The Guide goes through a systemic design project from concept to implementation. It takes you through the workshop planning process, and discusses workshop roles and client relations. In the FAQs, you’ll find explanations to some commonly asked questions about systemic design concepts to help you introduce others to SD and bring them along with you. You’ll get the most out of this field guide if you approach it with a systemic design mindset – open, willing to try things, a desire to learn, and a ‘yes, and’ orientation. Think of this field guide as an early stop in your systemic design journey: it’s not the end of your learning – it’s a jumping off point. In the Additional Resources section, you can connect to more in-depth explorations of concepts and practical applications.

Food security indicators - FAO (ESS Website, 2018)

Foreign Aid and Its Unintended Consequences - Koch, Dirk-Jan (Routledge, 2023)
Foreign aid and international development frequently bring with it a range of unintended consequences, both negative and positive. This book delves into these consequences, providing a fresh and comprehensive guide to understanding and addressing them. The book starts by laying out a theoretical framework based on complexity thinking, before going on to explore the ten most prevalent kinds of unintended effects of foreign aid: backlash effects, conflict effects, migration and resettlement effects, price effects, marginalization effects, behavioural effects, negative spillover effects, governance effects, environmental effects, and ripple effects. Each chapter revolves around a set of concrete case studies, analysing the mechanisms underpinning the unintended effects and proposing ways in which policymakers, practitioners, and evaluators can tackle negative side effects and maximize positive side effects. The book also includes personal testimonies, a succinct overview of unintended effects, and suggestions for further reading. Providing a clear overview of what side effects to anticipate when planning, executing, and evaluating aid, this book will be an important resource for students, development practitioners, and policymakers alike.

Foreign policy and complex adaptive systems: Exploring new paradigms for analysis and action - Frej, William; Ramalingam, Ben (Santa Fe Institute, 2011)

Foresight and International Development* - Bingley, Kate (IDS Bulletin, 2016)
This article provides an overview of the use of foresight-type approaches and techniques in policy-related work in international development. It draws primarily on published and grey literatures, as well as select interviews with foresight practitioners. It begins with a brief introduction to the approaches and tools used in the field of strategic foresight, and then a broad mapping of the foresight landscape as relevant to international development. It provides reflections on the evidence of use and impact of foresight initiatives, and makes suggestions around future directions for foresight in international development.

Forget the money: De-linking technical assistance - Manuel, Clare (Synthesis Paper, 4, 2016)

Forward Look - A vision for the World Bank Group in 2030 - World Bank (World Bank, 2016)

Fostering Innovation and Entrepreneurialism in Public Sector Reform - Cummings, Clare (Public Administration and Development, 2015)
Summary There is growing recognition within the international development sector that there is a need for a new, more effective approach to engaging in public sector reform. This article builds on an emerging body of work that advocates more entrepreneurial and adaptive public sector reform programming. Drawing on knowledge and theory from public sector management, psychology and entrepreneurialism, this article aims to understand what motivates public sector workers to work entrepreneurially and suggest how these ideas can be applied to the way in which international development agencies engage in public sector reform work. This requires a shift in thinking from predesigned, large, externally led programmes promoting international best practice to interventions in which agencies adopt the role of a facilitator, helping to establish the enabling conditions for local partners to work entrepreneurially, developing their own solutions to the problems that they identify in their work. Copyright ? 2015 John Wiley & Sons, Ltd.
Keywords: Development; Entrepreneurship; Innovation; Public sector reform; motivation

Fostering Learning and Effectiveness through Collaboration: A New Collaborative on Transparency, Accountability, and Participation (TAI, 2018)
What are the features, values, and practices of effective learning organizations? How do learning practices contribute to more effective programming? And, how can collaborations between academics, researchers and practitioners better support learning organizations in the global South?  These are just a few of the questions that a new global learning collaborative seeks to explore. In …

Fostering Learning in Large Programmes and Portfolios: Emerging Lessons from Climate Change and Sustainable Development - Harvey, Blane; Pasanen, Tiina; Pollard, Alison; Raybould, Julia (Sustainability, 2017)
In ﬁelds like climate and development, where the challenges being addressed can be described as “wicked”, learning is key to successful programming. Useful practical and theoretical work is being undertaken to better understand the role of reﬂexive learning in bringing together different knowledge to address complex problems like climate change. Through a review of practical cases and learning theories commonly used in the areas of resilience, climate change adaptation and environmental management, this article: (i) reviews the theories that have shaped approaches to reﬂexive learning in large, highly-distributed climate change and resilience-building programmes for development; and (ii) conducts a comparative learning review of key challenges and lessons emerging from early efforts to promote and integrate reﬂexive learning processes in programmes of this nature. Using a case study approach, the authors focus on early efforts made in four large, inter-related (or nested) programmes to establish, integrate and sustain learning processes and systems. Eight themes emerged from the review and are considered from the perspective of learning programmes as emergent communities of practice. By investigating how these themes play out in the nested programming, the paper contributes to the limited existing body of evidence on learning in large climate change programmes and identiﬁes areas where future efforts might focus.

Fostering Systems Change - Gopal, Srik; Kania, John (2015)
Five simple rules for foundations seeking to create lasting social change.

Foundations of the After Action Review Process - Morrison, John E; Meliza, Larry L (Special Report, 1999)
The U.S. Army has adopted the After Action Review AAR as its primary method of providing feedback after unit collective training exercises. The AAR is an interactive discussion in which unit members decide what happened, why it happened, and how to improve or sustain collective performance in future exercises. other services and organizations outside the military are also beginning to employ the AAR as a feedback tool. This report describes the twenty-five year history of AAR research and development and the major behavioral research areas contributing to AAR development and refinement. In addition, this report defines goals for future AAR research.

Four ways development practitioners can borrow from private sector adaptive approaches - Pett, Jamie (LearnAdapt, 2020)
Agile, lean startup and human-centred design can be an answer — if you’re asking the right questions

Fractal approaches to scaling transformations to sustainability - O’Brien, Karen; Carmona, Rosario; Gram-Hanssen, Irmelin; Hochachka, Gail; Sygna, Linda; Rosenberg, Milda (Ambio, 2023)
Responses to sustainability challenges are not delivering results at the scale and speed called for by science, international agreements, and concerned citizens. Yet there is a tendency to underestimate the large-scale impacts of small-scale, local, and contextualized actions, and particularly the role of individuals in scaling transformations. Here, we explore a fractal approach to scaling sustainability transformations based on “universal values.” Universal values are proposed as intrinsic characteristics that connect humans and nature in a coherent, acausal way. Drawing on the Three Spheres of Transformation framework, we consider how enacting universal values can generate fractal-like patterns of sustainability that repeat recursively across scales. Fractal approaches shift the focus from scaling through “things” (e.g., technologies, behaviors, projects) to scaling through a quality of agency based on values that apply to all. We discuss practical steps involved in fractal approaches to scaling transformations to sustainability, provide examples, and conclude with questions for future research.
Keywords: Agency; Fractals; Relational paradigms; Scaling; Three Spheres of Transformation; Universal values

Framework for the Biodiversity Cross-Mission Learning Program - Environmental Incentives; Foundations of Success; ICF International (Measuring Impact, 2015)
USAID’s Bureau for Economic Growth, Education, and the Environment/Office of Forestry and Biodiversity (E3/FAB) launched the five-year Measuring Impact program (MI) to help strengthen USAID’s biodiversity programs by building the capacity of the Agency to design, monitor, and evaluate effective programs and by enhancing the evidence base that informs programming decisions. A core programmatic strategy of MI is to “Improve Biodiversity Conservation Approaches” with the life-of-project (LOP) objective that E3/FAB and the Missions engaged with MI have a greater understanding of conditions under which commonly deployed conservation strategic approaches are effective. To accomplish this, MI is working with E3/FAB to design and implement a Biodiversity Cross-Mission Learning Program (Learning Program) to systematically capture and share lessons on theories of change1 (TOCs) for common conservation strategic approaches in the USAID biodiversity portfolio.

From a last mile century to a first mile revolution: the future is OSLO - Indy Johar (Medium, 2013)
The Future is OSLO

From analysis to action: operationalising learning and adaptation in Savings at the Frontier - Salehi, Yusef (2016)
Keywords: IMPORTANT

From best practice to best fit: understanding and navigating wicked problems in international development - Ramalingam, Ben; Laric, Miguel; Primrose, John (Working Paper, 2014)

From Data to Wisdom - Ackoff, Russell (Journal of Applied Systems Analysis, 1989)

From Design Thinking to Systems Change - Conway, Rowan; Masters, Jeff; Thorold, Jake (RSA, 2017)
This report takes the Small Business Research Initiative (SBRI), a twophase pre-procurement innovation programme that aims to match social challenges with new ideas, as its primary case study. It suggests augmenting the excellent design thinking deployed through SBRI with a think like a system, act like an entrepreneur lens in order to drive better social outcomes from SBRI-originating innovations. Programmes like SBRI have great potential to drive change and address pressing challenges, but must be guided by a more developed understanding of how change happens. The stakes are too high to not raise our game when it comes to social innovation. Wicked problems can be overcome but will require sophisticated theories of change able to account for the complexity and unpredictability of modern life. We offer think like a system, act like an entrepreneur as a contribution to this effort. This report is based on the following research: •• Desk based review of relevant literature; •• A series of structured interviews with participants in SBRI competitions; and •• An expert round table that brought together policymakers, commissioners, procurement experts and entrepreneurs to consider and develop initial findings.
Keywords: ⛔ No DOI found

From ‘Feedback Loops’ to ‘Responsive Governance’ - McGee, Rosie (Making All Voices Count, 2014)
Part of a sequenced learning process for MAVC's Research and Evidence Component, the summary of a thematic discussion led by researcher Rosemary McGee.

From mouthset to mindset shifts in co-creating systems change - Blomkamp, Emma; Snow, Thea; Burkett, Ingrid (Griffith Centre for Systems Innovation, 2023)

From political economy analysis to doing development differently: a learning experience - Booth, David; Harris, Daniel; Wild, Leni (ODI, 2016)
A study of how development efforts can be strengthened by an awareness of political economy, reflecting on the experiences of PoGo policy researchers.
Keywords: IMPORTANT

From Political Won't to Political Will: Building Support for Participatory Governance - Malena, Carmen (Kumarian Press, 2009)
* Geographically diverse examples of participatory governance in action* Practical case studies show how citizens can participate more fully in the political process Despite a recent wave of democratization around the world, traditional systems of representative democracy seem to be in crisis. Citizens in both the North and the South lack opportunities, rights and access to information and have expressed growing disillusionment with their governments. Ordinary citizens (especially women, poor people and other marginalized groups) are largely excluded from the political processes that directly affect their lives. There is now growing consensus that good governance is participatory governance (PG), but public officials are often reluctant to adopt such an approach. From Political Won’t to Political Will addresses the particular challenge of encouraging these officials to involve citizens in the political process. The book presents contributions by participants from CIVICUS’ 2008 conference on building political will for PG. Representing the perspectives of both civil society and government actors, they propose a number of strategies and lessons such as demonstrating the benefits of PG to governments, complementing formal democratic institutions, building trust, supporting PG champions and using strategic political timing. Their practical and highly original findings will interest anyone eager to see the empowerment of people around the world.

From poverty to power: how active citizens and effective states can change the world - Green, Duncan (2012)
From Poverty to Power From Poverty to Power argues that a radical redistribution of power, opportunities, and assets rather than traditional models of charitable or government aid is required to break the cycle of poverty and inequality. The forces driving this transformation are active citizens and effective states. Published in association with Oxfam GB. Full description

From PRA to PLA and pluralism: practice and theory - Chambers, Robert (2007)
PRA (participatory rural appraisal) and the more inclusive PLA (participatory learning and action) are families of participatory methodologies which have evolved as behaviours and attitudes, methods, and practices of sharing. During the 1990s and 2000s PRA/PLA has spread and been applied in most countries in the world. Among the multifarious domains of application, some of the more common have been natural resource management and agriculture, programmes for equity, empowerment, rights and security, and community-level planning and action. Related participatory methodologies which have co-evolved and spread widely as movements include farmer participatory research, Integrated Pest Management, Reflect, Stepping Stones and Participatory Geographic Information Systems. Ideologically and epistemologically PRA/PLA seeks and embodies participatory ways to empower local and subordinate people, enabling them to express and enhance their knowledge and take action. It can be understood as having three main components: facilitators’ behaviours, attitudes and mindsets linked with precepts for action; methods which combine visuals, tangibles and groups; and sharing without boundaries. The interplay of these resonates with theories of chaos, complexity, emergence and deep simplicity, especially self-organising systems on the edge of chaos. Good practice has moved towards an eclectic pluralism in which branding, labels, ownership and ego give way to sharing, borrowing, improvisation, creativity and diversity, all these complemented by mutual and critical reflective learning and personal responsibility. Keywords: participatory methodologies; networks; pluralism; practice; theory.

From principle to practice: Implementing the Principles for Digital Development - Waugaman, Adele (2016)
The Principles for Digital Development (download PDF here) find their roots in the efforts of individuals, development organizations, and donors alike who have called for a more concerted effort by donors and implementing partners to institutionalize lessons learned in the use of information and communication technologies (ICTs) in development projects.
Keywords: IMPORTANT

From purposeful to meaningful adaptive programming: how about adaptive operations? - Nixon, Nicola (Devpolicy Blog from the Development Policy Centre, 2019)
Embracing adaptive programming approaches but sticking with rigid, traditional systems-based operations? It's time to take a long hard look at all the boring stuff, says Nicola Nixon.

From Silos to Synergy: Learning from politically informed, gender aware programs - Derbyshire, Helen; Siow, Orlanda; Gibson, Sam; Hudson, David; Roche, Chris (DLP Program, 2018)

From Thinking Politically To Working Politically: Are We Really Doing Development Any Differently? (2019)
In June it will be seven years since the Center for Global Development published the Problem Driven Iterative Adaptation (PDIA) paper. Few academic papers have had such an impact on development thinking. The question is whether the paper – and subsequent debate and experimentation - have had a demonstrable and beneficial impact on development practice. Experience to date suggests thinking politically is easier than working politically. Practitioners in development agencies, governments and contractors find that convincing theoretical literature can be tricky to operationalize in a fractious environment. Still, a growing number of programs have set out to be politically informed in design and politically savvy in implementation. Engaging the reality of politics creates the potential to promote change. To consider these issues, Abt Associates is hosting a workshop for international development practitioners in Washington, DC, on June 18th, 2019, to discuss the successes and failures in operationalizing this agenda. The session will allow practitioners to discuss experiences in TWP and propose solutions or approaches. Please join us in this Innovations in Governance series finale where four internationally renowned thinkers and doers will join us to offer their perspectives.

From Users and Choosers to Makers and Shapers. Repositioning Participation in Social Policy - Cornwall, Andrea; Gaventa, John (IDS Bulletin, 2000)

Future Democracies - Laboratory of Collective Intelligence for Participatory Democracy - Yago Bermejo (MediaLab Prado, 2019)
The Laboratory of Collective Intelligence for Participatory Democracy (2016-2019) is a project that arose out of Medialab Prado in coordination with the Government Area of Citizen Participation, Transparency and Open Government of the city of Madrid. Its work has been very connected with the analysis, reflection and innovation found on the digital participation platform Decide Madrid. The project has also organised many workshops and conferences that have brought together hundreds of people from the world of participatory democracy. This publication summarises the project’s core working principles as well as the open activities it has hosted over the course of these three years. These pilot experiences offer a possible vision of the future of democratic governance.
Keywords: Democracy

Future oriented conservation: knowledge governance, uncertainty and learning - Wyborn, Carina; van Kerkhoff, Lorrae; Dunlop, Michael; Dudley, Nigel; Guevara, Oscar (Biodiversity and Conservation, 2016)
Despite signiﬁcant progress in understanding climate risks, adaptation efforts in biodiversity conservation remain limited. Adaptation requires addressing immediate conservation threats while also attending to long term, highly uncertain and potentially transformative future changes. To date, conservation research has focused more on projecting climate impacts and identifying possible strategies, rather than understanding how governance enables or constrains adaptation actions. We outline an approach to futureoriented conservation that combines the capacities to anticipate future ecological change; to understand the implications of that change for social, political and ecological values; and the ability to engage with the governance (and politics) of adaptation. Our approach builds on the adaptive management and governance literature, however we explicitly address the (often contested) rules, knowledge and values that enable or constrain adaptation. We call for a broader focus that extends beyond technical approaches to acknowledge the sociopolitical challenges inherent to adaptation. More importantly, we suggest that conservation policy makers and practitioners can use this approach to facilitate learning and adaptation in the context of complexity, transformational change and uncertainty.

Future Positive: International Co-operation in the 21st Century - Edwards, Michael (Routledge, 1999)
An optimistic assessment of the prospects for a new international order - acting as a counter-blast to global pessimism. The text explains how the international system operates, the pressures it faces and the changes it must undergo, and offers concrete ideas to re-frame international relations, foreign aid and humanitarian intervention, without using jargon or simplistic judgements.

Gambling on Development: Why Some Countries Win and Others Lose - Dercon, Stefan (C Hurst & Co Publishers Ltd, 2022)

Gendered voices for climate action - Fisher, Susannah; Shakya, Clare (iied Working Paper, 2018)
Including local women and men who have experienced the impacts of climate change in decision-making forums is key to developing a just process hearing local voices and experiences. The Paris Agreement signs up the international community to delivering countrydriven, gender-responsive climate action. Including local women and men will be critical to this transition and to achieving the scale and pace of change needed through both procedural and distributional justice. However, there is scarce evidence on how these voices and experiences can impact on and strengthen climate responses, or how best to enable and facilitate this participation.

Gestión de la complejidad para el impacto sistémico: respuestas a entornos VUCA y BANI - Peña-López, Ismael (SociedadRed, 2023)
Research on the Information Society, the Digital Divide and Information and Communication Technologies for development

Getting beneath the surface in program planning, monitoring and evaluation: Learning from use of participatory action research and theory of change in the CGIAR Research Program on Aquatic Agricultural Systems - Apgar, J Marina; Allen, Will; Albert, Joelle; Douthwaite, Boru; Paz Ybarnegaray, Rodrigo; Lunda, Jeston (Action Research, 2017)
Many rural poor and marginalized people strive to make a living in social-ecological systems that are characterized by multiple and often inequitable interactions across agents, scale and space. Uncertainty and inequality in such systems require research and development interventions to be adaptive, support learning and to engage with underlying drivers of poverty. Such complexity-aware approaches to planning, monitoring and evaluating development interventions are gaining strength, yet, there is still little empirical evidence of what it takes to implement them in practice. In this paper, we share learning from an agricultural research program that used participatory action research and theory of change to foster learning and support transformative change in aquatic agricultural systems. We reflect on our use of critical reflection within participatory agricultural research interventions, and our use of theory of change to collectively surface and revisit assumptions about how change happens. We share learning on the importance of being strengths-based in engaging stakeholders across scales and building a common goal as a starting point, and then staging a more critical practice as capacity is built and opportunities for digging deeper emerge.

Getting good at disruption in an uncertain world: Insights from Southern NGO leaders - Buckley, Lila; Ward, Halina (International Institute for Environment and Development, 2016)
Keywords: IMPORTANT

Getting intentional about M&E: choosing suitable approaches for adaptive programmes - Management, Global Learning for Adaptive (GLAM Blog, 2020)
Does the choice of monitoring and evaluation (M&E) approaches and tools matter for adaptive programmes? In short, yes: monitoring, evaluation and learning (MEL) and adaptive management (AM) are intertwined. While programme monitoring data and evaluation results are not the only sources of evidence that programmes use for learning and iteration, they often are amongst most important ones — or at least they should be. Selecting what type of information to collect and analyse — and how — is critical for any type of programme. However, what AM especially focuses on is intentionally building in opportunities for structured and collective reflection, ongoing and real-time learning, course correction and decision-making in order to improve effectiveness.

Getting Past the Rhetoric #1: Tendering for ‘Thinking and Working Politically’ in Large Facilities - Tyrrel, Lavinia (Abt Associates - Governance Soapbox, 2018)

Getting Past the Rhetoric #2: Managing for ‘Thinking and Working Politically’ in Large Facilities - Tyrrel, Lavinia (Abt Associates - Governance Soapbox, 2018)

Getting Practical With Causal Mechanisms: The application of Process-Tracing Under Real-World Evaluation Constraints - Raimondo, Estelle (New Directions for Evaluation, 2020)
Over the past decade, the field of development evaluation has seen a renewed interest in methodological approaches that can answer compelling causal questions about what works, for whom, and why. Development evaluators have notably started to experiment with Bayesian Process Tracing to unpack, test, and enhance their comprehension of causal mechanisms triggered by development interventions. This chapter conveys one such experience of applying Bayesian Process Tracing to the study of citizen engagement interventions within a conditional cash transfer program under real-world evaluation conditions. The chapter builds on this experience to discuss the benefits, challenges, and potential for the applicability of this approach under real-world evaluation conditions of time, money, and political constraints.

Getting Real about Unknowns in Complex Policy Work - Andrews, Matt (Research on Improving Systems of Education (RISE), 2021)
As with all public policy work, education policies are demanding. Policy workers need to ‘know’ a lot—about the problems they are addressing, the people who need to be engaged, the promises they can make in response, the context they are working in, and the processes they will follow to implement. Most policy workers answer questions about such issues within the structures of plan and control processes used to devise budgets and projects. These structures limit their knowledge gathering, organization and sense-making activities to up-front planning activities, and even though sophisticated tools like Theories of Change suggest planners ‘know’ all that is needed for policy success, they often do not. Policies are often fraught with ‘unknowns’ that cannot be captured in passive planning processes and thus repeatedly undermine even the best laid plans. Through a novel strategy that asks how much one knows about the answers to 25 essential policy questions, and an application to recent education policy interventions in Mozambique, this paper shows that it is possible to get real about unknowns in policy work. Just recognizing these unknowns exist—and understanding why they do and what kind of challenge they pose to policy workers—can help promote a more modest and realistic approach to doing complex policy work.

Getting Strategic about Technology and Accountability: MAVC Learning and Inspiration Event in Manila - Halloran, Brendan (Politics, Governance and Development, 2016)
Two years ago, Jonathan Fox released a widely read and well-received paper looking at the evidence for citizen-led accountability.  In it, he noted the proliferation of short-term, isolated interve…

Getting strategic: vertically integrated approaches - Aceron, Joy; Isaac, Francis (Institute of Development Studies, 2016)
Holding power to account requires understanding where power lies and how it is exercised. It entails understanding how decisions are made, who makes them and what decision criteria are used to make them. Vertically integrated civil society action takes into account how power is exercised and how decisions are made in a given policy, programme or process.

GIZ's Evaluation Policy: Principles, guidelines and requirements of our evaluation practice - GIZ (GIZ, 2018)

GIZ’s evaluation system - GIZ Evaluation Unit (GIZ GmbH, 2018)
This provides a general description of GIZ’s new evaluation system, which is designed to implement our new policy and to achieve the objectives of evaluation reform. This general description is accompanied by two other documents that provide supplementary details of GIZ’s key evaluation instruments: project evaluations for BMZ business and corporate strategic evaluations. They are geared in particular to GIZ staff members who commission, implement or support evaluations and to evaluators who conduct evaluations on behalf of GIZ as well as repre-sentatives of commissioning parties, clients and cooperation partners. The current versions are very much a ‘work in progress’ in many regards. We are well aware that we will only be able to meet our own high expectations of our evaluations if we continue to develop their quality in dialogue with project staff, our commissioning parties, clients and partners and with evaluation practitioners and experts from the academic, scientific and research community. We look forward to taking on this task and to sharing our experience with all interested parties.

GLAM Monitoring and Learning Strategy (internal document) - GLAM (ODI, 2018)

Glass Half Full: Civic Space and Contestation in Bangladesh, Sri Lanka and Nepal - Surie, Mandakini D.; Saluja, Sumaya; Nixon, Nicola (The Asia Foundation, 2023)
The past decade has witnessed a surge of interest in and concern over the global trend toward democratic regression. In South Asia, regulatory and institutional frameworks have become increasingly restrictive, curbing the ability of citizens and civil society organizations to occupy and use civic spaces to organize, express themselves, and participate in decisions that affect the lives of people whose interests they serve. Of course, this is only one half of the story. The other half is how citizens, collectives, and organizations adapt by carving out spaces where they can maintain—or even expand—the boundaries of their engagement in local and national civic spaces. In this paper, we examine how civic spaces are evolving in three South Asian countries—Bangladesh, Sri Lanka, and Nepal—drawing on the experiences and perspectives of civil society representatives. The paper concludes with recommendations for donors and development practitioners on how they can make relevant investments that will strengthen civic spaces and support democratic resilience in the region.

Global Affairs Canada: an approach to development innovation - GAC (Global Affairs Canada, 2018)
For Global Affairs Canada, development innovation includes new or improved business models, policy practices, approaches, technologies, behavioral insights or ways of delivering products and services that benefit and empower the poorest and most vulnerable people in developing countries.

Global Delivery at the World Bank Group - GDI (Global Delivery Initiative, 2014)

Global governance and local peace: accountability and performance in international peacebuilding - Campbell, Susanna (Cambridge University Press, 2018)
Local peacebuilding and global accountability -- The country context--Burundi from 1999 to 2014 -- Ingos in peacebuilding--globally unaccountable, locally adaptive -- International organizations in peacebuilding--globally accountable, locally constrained -- Bilateral development donors--accountable for global targets, not local change

Global Learning for Adaptive Management (GLAM) - DFID (DFID Development Tracker, 2016)
To improve the value for money and impact of development interventions by promoting the use of better monitoring, evaluation and learning strategies that support development programmes to gather and respond to emergent evidence on the effects of interventions, as well as changes in the development context. The Global Learning for Adpative Management programme will make available technical assistance to DFID, USAID and partners to develop this ‘adaptive management’ approach, as well as build an evidence base to understand links between the use of adaptive management and more effective, efficient, relevant and sustainable development interventions.

Global mapping of technology for transparency and accountability - Avila, Renata; Feigenblatt, Hazel; Heacock, Rebekah; Heller, Nathaniel (Transparency & Accountability Initiative, 2010)

Glossary of Key Terms in Evaluation and Results Based Management - OECD (OECD, 2009)

Glossary of Knowledge Management - Serrat, Olivier (Knowledge Solutions: Tools, Methods, and Approaches to Drive Organizational Performance, 2017)
The knowledge management discipline can be cryptic. These Knowledge Solutions define its most common concepts in simple terms.
Keywords: Definitions; Knowledge management; Taxonomy; Terminology

Going Local 2.0: How to Reform Development Agencies to Make Localized Aid More Than Talk - Ang, Yuen Yuen (Stanford Social Innovation Review, 2018)
More and more global aid agencies believe they should replace one-size-fits-all best practices with locally tailored solutions, but they must shift from just agreeing to “go local” to preparing development experts for the task by taking on three major problems with their internal practices.

Going vertical: citizen-led reform campaigns in the Philippines - Aceron, Joy; Isaac, Francis (Making All Voices Count Research Report, 2016)
The Philippines has a long history of state–society engagement to introduce reforms in government and politics. Forces from civil society and social movements have interfaced with reform-oriented leaders in government on a range of social accountability initiatives – to make governance more responsive, to introduce policy reforms, and to make government more accountable. Several theoretical propositions on which strategic approaches work best for social accountability initiatives have been put forward – including the idea of vertically integrated civil society monitoring and advocacy. This multi-authored research report uses vertical integration as a framework for examining seven successful civil society social accountability initiatives in the Philippines, looking at what made them successful, and how the gains they realised can be deepened and sustained.

Good Enough Governance Revisited - Grindle, Merilee S. (Development Policy Review, 2007)
The concept of good enough governance provides a platform for questioning the long menu of institutional changes and capacity-building initiatives currently deemed important (or essential) for development. Nevertheless, it falls short of being a tool to explore what, specifically, needs to be done in any real world context. Thus, as argued by the author in 2004, given the limited resources of money, time, knowledge, and human and organisational capacities, practitioners are correct in searching for the best ways to move towards better governance in a particular country context. This article suggests that the feasibility of particular interventions can be assessed by analysing the context for change and the implications of the content of the intervention being considered.

Governance and Service Delivery: Practical Applications of Social Accountability Across Sectors - Wetterberg, Anna; Brinkerhoff, Derick W.; Hertz, Jana C. (RTI Press, 2016)
The six case studies in Governance and Service Delivery: Practical Applications of Social Accountability Across Sectors illustrate the multiple ways citizen participation in accountability – called social accountability – can lead to positive effects on governance, citizen empowerment, and service delivery. Drawing on their extensive experience implementing international donor-funded programs and projects, the authors examine six recent RTI International projects in Africa and Asia. The analysis focuses on both intended and actual effects, and it unpacks the influence of context on implementation and outcomes achieved, as well as on prospects for sustainability. Donors, academics, policy makers, practitioners and others interested in sustainable development and governance will find this systematic review invaluable.

Governance Diaries: An Approach to Governance Research from the Ground Up - Loureiro, Miguel; Joshi, Anuradha; Barnes, Katrina; Chaimite, Egídio (IDS, 2020)
Research on empowerment and accountability tends to focus on collective action and its potential for empowering citizens undertaking the action and on achieving state accountability. In fragile, conflict and violence-affected settings (FCVAS) collective action is rare and risky. So how do citizens, particularly the chronically poor and most marginalised, interact and make claims on the different public authorities that exist in these settings, and how do these interactions contribute to citizens’ sense of empowerment and accountability? Given the current agenda of ‘leave no one behind’, an understanding of how such populations interact with public authorities to meet their governance needs can help identify the constraints to achieving development for all in these challenging settings. We developed ‘governance diaries,’ a cross between a panel survey and multi-sited ethnographies, as an iterative approach to capture their experiences around governance issues over time. We explain here how this approach works, and the challenges and opportunities it offers for research.

Governance for Growth in Vanuatu: Review of a decade of thinking and working politically - Hadley, Sierd; Tilley, Helen (ODI, 2017)

Grounded Theory Method in HCI and CSCW - Muller, Michael J; Kogan, Sandra (IBM Center for Social Software, 2010)

Grow - Scale - Impact: How to help inclusive business achieve scale - Pérez Castro, Andrea A.; Tora, Krisztina (GIZ, 2016)

Growth of human factors in application development - Cockburn, Alistair (Humans and Technology Technical Report, 1995)

Guidance Note for Adaptive Management - Ministry of Foreign Affairs of Denmark (Government of Denmark, 2022)
This note explains what adaptive aid management is; why and when it should be considered; and how it should be applied. It covers all Danish development support channels and modalities, including bilateral country assistance, assistance to and through civil society, the private sector and to and through multilateral organisations. This guide has three chapters. Chapter 1 provides an executive overview of what adaptive management is. Chapter 2 goes deeper into five key operational principles of adaptive management. Chapter 3 details the main tenets of adaptive management processes during the programming cycle.

Guidance Note: Practical introduction to adaptive management - DT Global (DT Global, 2022)
DT Global is proud to introduce our new Guidance Note: Practical Introduction to Adaptive Management There is a growing consensus around adaptive management as an effective (even necessary) approach when programs are tackling complex development problems. While there is no standard definition of adaptive management, there is general agreement that such programs need to routinely engage with and respond to program context; constantly test what works in that context; and adjust approaches, plans, and activities based on continuous learning. However, there remains a more limited body of evidence about what this looks like in practice—the enabling conditions, systems, resourcing, skills, and attitudes to effectively operationalise adaptive management. There is also limited guidance around when adaptive management is required, and to what extent—both critical and often overlooked considerations when planning for successful adaptive management. This Guidance Note draws together lessons and good practice in adaptive management from across DT Global’s diverse portfolio of donor-funded programs. It outlines our conceptual framework for adaptive management, with practical guidance on how it can be applied by our program teams. It is also designed to help our teams distinguish adaptive management from good (non adaptive) project management, consider when adaptive management is most useful on a program, and how adaptive a program (or part of a program) should be.

Guidance on tasks and deliverables for different evaluation phases - Methods Lab (2015)
This tool describes the five key phases of evaluation, from planning and design, to implementation and communication of results. It provides a list of the main tasks and deliverables for each phase, intended for use by anyone managing an impact evaluation. This tool was developed by Irene Guijt, Simon Hearn, Tiina Pasanen and Patricia Rogers for use in Methods Lab projects. It follows to some extent the BetterEvaluation Rainbow Framework.

Guidance on using the revised Logical Framework - How to note - DFID (DFID, 2011)
The principal changes to the logframe from the earlier (2008) 4 x 4 matrix are: - Objectively Verifiable Indicators (OVIs) have been separated into their component elements (Indicator, Baseline and Target), and Milestones added. - Means of Verification has been renamed ‘Source’. - Inputs are now quantified in terms of funds (expressed in Sterling for DFID and all partners) and use of DFID staff time (expressed as annual Full-Time Equivalents (FTEs). - A DFID Share box now indicates the financial value of DFID’s Inputs as a percentage of the whole. - Assumptions are shown at Purpose and Output level only. - Risks are shown at Activities level only, but also rated at Output level; - At the Output level, the Impact Weighting is now shown in the logframe together with a Risk Rating for individual Outputs. - Activities are now shown separately (so do not normally appear in the logframe sent for approval), although they can be added to the logframe if this is more suitable for your purposes. - A renewed emphasis on the use of disaggregated beneficiary data within indicators, baselines and targets. The Logical Framework (logframe) was significantly re-designed in February 2009. In January 2011 a slightly amended logframe template was introduced at the same time as the launch of the new DFID Business Case. Given the extent of changes that took place in February 2009 it was only necessary to make minor amendments in January 2011. These amendments are as follows: - Results Chain terminology aligned across DFID (the terms Goal and Purpose in the old logframe template have been replaced by Impact and Outcome) - Rows added to allow achieved results to be captured alongside the planned results determined at project design stage - Word version of the logframe template removed – excel is the preferred format for logical frameworks from January 2011 - Indicator numbering introduced within logframe template

Guidance: Rigor and Reproducibility in Grant Applications | grants.nih.gov - NIH (National Institutes of Health, 2018)

Guide for Adopting Remote Monitoring Approaches During COVID-19 - USAID (USAID, 2020)
USAID maintains staff and operations in more than 80 countries around the world, all of which the COVID-19 pandemic will disrupt. USAID remains committed to protecting the health and safety of our staff, while continuing appropriate oversight of our programs and ensuring the accountable and effective use of U.S. taxpayer funds. In the current operating environment, USAID and implementing partners face new challenges in implementing activities, monitoring progress, collecting data, and tracking indicators. As we adapt our approaches, we will work with implementing partners to find innovative, responsible, and safe ways to monitor and evaluate programming. Digital tools can support novel approaches to remote monitoring. Responsible use of digital tools also supports operating unit alignment with the Digital Data Collection mandate in the Agency’s new Digital Strategy. This guide provides information for Agency staff and implementing partners on remote monitoring techniques and when they can be employed. We encourage use of this guide to identify and pursue appropriate remote-monitoring approaches for your needs. CORs/AORs and Activity Managers should work with implementing partners to document updated approaches in each agreement’s plans for monitoring, evaluation, and learning (MEL), and should upload these amended Activity MEL Plans into the Agency Secure Image and Storage Tracking System (ASIST) as soon as possible. This can be done in collaboration with Mission and Washington Bureau M&E Specialists, as appropriate

Guide to Evaluating Participatory Processes - Practical guide - Parés, Marc (2013)
Guide to Evaluating Participatory Processes - Practical guide
Keywords: Academia; Biology; Computer Science; Earth Sciences; Geography; History; Law; Math; Medicine; Philosophy; Physics; Political science; Psychology; Religion; Research; economics; universities

Guide to Innovation Partnerships – A United Nations Ghide - Halse, Michelle; Ginsberg-Fletcher, Gabriella; Balbi, Luisa (UN Global Pulse, 2023)

Guidelines and good practices for evidence-informed policy-making in a government department - Wills, Alf; Tshangela, Mapula; Shaxson, Louise; Datta, Ajoy; Matomela, Bongani (2016)

Guidelines for Country Strategic Frameworks Programmes and Projects - Ministry of Foreign Affairs of Denmark (Government of Denmark, 2020)

Guidelines for designing and monitoring social accountability interventions - Franco, Erika Lopez; Shankland, Alex (IDS, 2018)

Guidelines for Drawing Causal Loop Diagrams - Daniel H., Kim (The Systems Thinker, 1992)
The old adage “if the only tool you have is a hammer, everything begins to look like a nail” can also apply to language. If our language is linear and static, we will tend to view and interact with our world as if it were linear and static. Taking a complex, dynamic, and circular world and linearizing it into a set of snapshots may make things seem simpler, but we may totally misread the very reality we were seeking to understand. Making such inappropriate simplifications “is like putting on your brakes and then looking at your speedometer to see how fast you were going” says Bill Isaacs of the MIT Center for Organizational Learning. Articulating Reality Causal loop diagrams provide a language for articulating our understanding of the dynamic, interconnected nature of our world. We can think of them as sentences which are constructed by linking together key variables and indicating the causal relationships between them. By stringing together several loops, we can create a coherent story about a particular problem or issue.

Guidelines for good market development program design. A managers’ perspective - Bekkers, Harald; Roggekamp, Peter (BEAM Exchange, 2014)
Good programme design is a key factor contributing to the success of market development programmes. Unfortunately, too many current designs have major flaws which prevent programmes becoming successful even before they get started. This document considers what makes market development programmes consistently successful and how to prevent programme design from being a hurdle to sustainable and efficient impact as scale.

Guidelines for the Monitoring System for Development-Related Changes (MERV) - SDC (SDC, 2013)

Guidelines for writing a case study on implementation (Book Project: Smart Implementation in Governance) - Kirsch, Renate (GIZ GmbH, 2015)

Guiding questions to help narrow the scope of an evaluation - Methods Lab (2015)
Time and budget constraints can mean that programmes are not able to assess all possible evaluation questions; this is especially true for multi-component or multi-site programmes operating in challenging environments. This tool identifies areas of enquiry to help programmes prioritise the number of questions and measurement indicators used. This tool was developed by Anne Buffardi for use in in Methods Lab projects.

Habits of a system thinker - WCST (Waters Center For Systems Thinking, 2020)
From the Waters Center For Systems Thinking. See: https://thinkingtoolsstudio.waterscenterst.org/courses/habits https://thinkingtoolsstudio.waterscenterst.org/cards

Hacking by the prompt: Innovative ways to utilize ChatGPT for evaluators - Ferretti, Silva (New Directions for Evaluation, 2023)
“Hacking by the prompt”—writing simple yet creative conversational instructions in ChatGPT's message window—revealed many valuable additions to the evaluator's toolbox for all stages of the evaluation process. This includes the production of terms of reference and proposals for the dissemination of final reports. ChatGPT does not come with an instruction book, so evaluators must experiment creatively to understand its potential. The surprising performance of ChatGPT leads to the question: will it eventually substitute for evaluators? By describing ChatGPT through four personality characteristics (pedantic, “I know it all,” meek, and “speech virtuoso”), this article provides case examples of the potential and pitfall of ChatGPT in transforming evaluation practice. Anthropomorphizing ChatGPT is debatable, but the result is clear: tongue-in-cheek personality characteristics helped hack ChatGPT more creatively while remaining aware of its challenges. This article combines practical ideas with deeper reflection on evaluation. It concludes that ChatGPT can substitute for evaluators when evaluations mostly focus on paperwork and conventional approaches “by the book” (an unfortunate trend in the sector). ChatGPT cannot substitute engagement with reality and critical thinking. Will ChatGPT then be a stimulus to rediscover the humanity and the reality we lost in processes?

Handbook of Democratic Innovation and Governance - Elstub, Stephen; Escobar, Oliver (Edward Elgar Publishing, 2019)
Democratic innovations are proliferating in politics, governance, policy, and public administration. These new processes of public participation are reimagining the relationship between citizens and institutions. This Handbook advances understanding of democratic innovations, in theory and practice, by critically reviewing their importance throughout the world. The overarching themes are a focus on citizens and their relationship to these innovations, and the resulting effects on political equality. The Handbook therefore offers a definitive overview of existing research on democratic innovations, while also setting the agenda for future research and practice.
Keywords: Democracy; Public Affairs & Administration

Harnessing Complexity: Organizational Implications of a Scientific Frontier - Axelrod, Robert M.; Cohen, Michael D. (Free Press, 1999)
Recent advances in the study of complexity have given scientists profound new insights into how natural innovation occurs and how its power can be exploited. Now two pioneers in the field, Robert Axelrod and Michael D. Cohen, provide leaders in business and government with a guide to complexity that will help them make effective decisions in a world of rapid change. Building on evolutionary biology, computer science, and social design, Axelrod and Cohen have constructed a unique framework for improving the way people work together. Their approach to management is based on the concept of the Complex Adaptive System, which can describe everything from rain forests to the human gene pool, and from automated software agents to multinational companies. The authors' framework reveals three qualities that all kinds of managers must cultivate in their organization: "Variation" What is the best way to manage the development of software? Should the problem be broken up into small pieces for programmers working independently, thus enhancing variation, or should there be a centralized hierarchy of programmers ruled by a chain of command? The authors show how the decentralized creation of variation combined with the centralized maintenance of standards was the key to the success of the Linux "open source software" project, which brought together thousands of volunteers in cyberspace to produce an operating system that can outperform Microsoft's. "Interaction" Why did northern Italy prosper while southern Italy remained poor? Recognizing the internal interactions of a Complex Adaptive System -- be it a national region, a company, or a nonprofit group -- reveals vitalnetworks of trust. Axelrod and Cohen explain that in successful adaptive systems, rich networks of horizontal linkages foster cooperation and provide an advantage over other less cooperatively networked groups. In the case of Italy, voluntary associations created networks of trust in the Middle Ages that became northern Italy's critical advantage over the south. "Selection" Is a Pulitzer Prize better than a National Book Award? How can foundations and corporations design competitions that have a positive effect on the evolution of excellence? The authors' framework makes clear that the worst selection processes are mired in orthodox standards that have not adapted to a new environment. The best selection processes, on the other hand, are created and run by leaders who understand how the standards they use can transform their organization and its environment. This simple, paradigm-shifting analysis of how people work together will transform the way we think about getting things done in a group. "Harnessing Complexity" is the essential guide to creating wealth, power, and knowledge in the 21st century.
Keywords: Organizational Behavior

Has Kenya’s ICT revolution triggered more citizen participation? - Salome, Nyambura (MAVC Research Briefing, 2016)
Lessons from policy in Ghana, Kenya, South Africa and Tanzania
Keywords: IMPORTANT

Help! I’m hiring new staff and I want them to work adaptively. A guide to hiring adaptive employees. - Salib, Monalisa (USAID LEARN, 2019)
If you are involved in hiring, this tool can help you increase your chances of selecting staff members skilled in adaptive management. It will help answer the questions: Which competencies should I recruit for in order to hire more adaptive employees? Which desired qualifications should I incorporate into position descriptions to attract adaptive employees? Which interview questions should I ask to screen for adaptive employee competencies? You should use this tool when you decide to hire a new individual - whether that be a direct hire, contractor, or consultant - or when you are changing an individual’s current position description or scope of work. Specifically, this tool will help you determine which qualifications are most important for the position and offers interview questions you can use with candidates. --- You’re in luck! We just released a new guide for hiring adaptive employees. This visual and user-friendly tool will help increase your chances of selecting staff skilled in adaptive management. Here are the details: What do you mean by “work adaptively” or an “adaptive employee”? An adaptive employee is an individual who, in collaboration with relevant stakeholders, systematically acquires and uses knowledge to make decisions and adjustments in their work in order to achieve greater impact. Why focus on adaptive employees? Why does it matter? With industries, technologies, and organizations in a near constant state of flux, leaders are recognizing the importance of adaptability as a critical capacity. Because of this, it’s not surprising that a recent LinkedIn report found that adaptability was the most important soft skill hiring managers were screening for. In addition, evidence shows that teams that apply more data-driven and adaptive leadership practices perform better than those that focus less on these practices. It makes sense, then, that managers want to recruit more team members who are skilled in adaptive management - it helps achieve results. Who should use this tool? Anyone involved in hiring, responsible for developing or approving position descriptions, participating in interviews, and/or approving new hires. When should I use this tool? Use it as soon as you decide to hire a new staff person or when you’re adjusting existing position descriptions. What does the tool help me do? It will help you think through: Which competencies should I recruit for in order to hire more adaptive employees? Which desired qualifications should I incorporate into position descriptions to attract adaptive employees? (You can even copy and paste qualification language from the tool into scopes of work!) Which interview questions should I ask to screen for adaptive employee competencies? (You can copy and paste interview questions into your screening, interview, and reference check protocols!) Has the tool been tested? Yes, staff within USAID’s Global Development Lab and on the USAID LEARN contract tested the tool, providing feedback on the minimum viable product and subsequent versions. Testers confirmed finding the tool useful, and gave it a 9/10 score when asked if they would recommend it to colleagues. Some specific feedback from testers included: "The tool prompts deeper thought on what you might need and want in a job candidate" and "All [of the interview questions I used from the tool] worked well." What about the existing staff? How can they gain greater skills in adaptive management? Existing USAID staff and implementing partners can access online training in collaborating, learning, and adapting (CLA) and find resources in the CLA Toolkit. USAID staff also have access to an internal CLA community of practice and five-day, in-person CLA training.

HERMES (Highly Extensible Resource for Modeling Event-Driven Supply Chains) - HERMES (hermes.psc.edu, 2015)
Systems modeling is a tool for policymakers and program managers to capture all the direct and indirect effects of changes to a supply chain, identify sustainable solutions to the root causes of issues and save time, effort and resources in costly trial and error. Without systems modeling, evaluating the supply chain, identifying gaps, and implementing solutions can be insufficient, unsustainable and costly. Stock2 Vaccine supply chains are complex systems, comprising all the equipment, personnel, policies and processes needed to deliver a vaccine from its point of origin to the population. Understanding how the various components of a vaccine supply chain interact with each other is critical to evaluating supply chain function, identifying the root causes of issues and formulating sustainable solutions. HERMES (Highly Extensible Resource for Modeling Event-Driven Supply Chains) is a software program that allows users to generate a detailed computer simulation model of a supply chain

Herramientas de Conocimiento y Aprendizaje: Una Guía para Organizaciones Humanitarias y de Desarrollo - Ramalingam, Ben (ODI, 2006)
This toolkit presents entry points and references to the wide range of tools and methods that have been used to facilitate improved knowledge and learning in the development and humanitarian sectors.
Keywords: spanish

Hidden Potential: The Science of Achieving Greater Things - Grant, Adam (WH Allen, 2023)
#1 New York Times bestseller "This brilliant book will shatter your assumptions about what it takes to improve and succeed. I wish I could go back in time and gift it to my younger self. It would've helped me find a more joyful path to progress."-Serena Williams, 23-time Grand Slam singles tennis championThe #1 New York Times bestselling author of Think Again illuminates how we can elevate ourselves and others to unexpected heights.We live in a world that’s obsessed with talent. We celebrate gifted students in school, natural athletes in sports, and child prodigies in music. But admiring people who start out with innate advantages leads us to overlook the distances we ourselves can travel. We can all improve at improving. And when opportunity doesn’t knock, there are ways to build a door.Hidden Potential offers a new framework for raising aspirations and exceeding expectations. Adam Grant weaves together groundbreaking evidence, surprising insights, and vivid storytelling that takes us from the classroom to the boardroom, the playground to the Olympics, and underground to outer space. He shows that progress depends less on how hard you work than how well you learn. Growth is not about the genius you possess – it’s about the character you develop. Grant explores how to build the character skills and motivational structures to realize our own potential, and how to design systems that create opportunities for those who have been underrated and overlooked.This book reveals how anyone can rise to achieve greater things. The true measure of your potential is not the height of the peak you’ve reached, but how far you’ve climbed to get there.

Historical roots of Agile Methods: Where did “Agile Thinking” come from? - Abbas, Noura; Gravell, Andrew M.; Wills, Gary B. (Springer Berlin Heidelberg, 2008)
The appearance of Agile methods has been the most noticeable change to software process thinking in the last fifteen years [16], but in fact many of the “Agile ideas” have been around since 70’s or even before. Many studies and reviews have been conducted about Agile methods which ascribe their emergence as a reaction against traditional methods. In this paper, we argue that although Agile methods are new as a whole, they have strong roots in the history of software engineering. In addition to the iterative and incremental approaches that have been in use since 1957 [21], people who criticised the traditional methods suggested alternative approaches which were actually Agile ideas such as the response to change, customer involvement, and working software over documentation. The authors of this paper believe that education about the history of Agile thinking will help to develop better understanding as well as promoting the use of Agile methods. We therefore present and discuss the reasons behind the development and introduction of Agile methods, as a reaction to traditional methods, as a result of people’s experience, and in particular focusing on reusing ideas from history.
Keywords: Agile methods; Software Development

HIV, food and drugs: Livelihoods, nutrition and Anti-retroviral Therapy (ART) in Kenya and Zambia - Samuels, Fiona; Rutenberg, Naomi (Briefing Paper, 2008)
Key points • ART has restored the health of many people living with HIV, but their livelihoods lag behind • Good nutrition is important for people on ART. Food supplementation can help, but is no substitute for sustainable livelihoods • The livelihoods of people on ART can be bolstered through skills, livelihood networks, assets and cash or food transfers

Holistic Flexibility for Deploying Systems Thinking as a Cognitive Skill - Chowdhury, Rajneesh (Systemic Practice and Action Research, 2023)
Considering Systems Thinking (ST) as a cognitive skill can create greater acceptability of and openness to the discipline from practitioners and researchers outside operations research and management science. Rather than associating ST with frameworks and methodologies, ST as a cognitive skill can help popularize and democratize the discipline. This paper highlights how the conceptual lens of Holistic Flexibility can help practitioners deploy ST as a cognitive skill without the application of any traditional systems methodology. Holistic Flexibility is defined as the dynamic interplay between a state of mind that has the ability to absorb systemic complexity and a state of practice that has the ability to embrace flexibility, both in intent and in form. Through two case-studies, discussions in this paper highlight how Holistic Flexibility can serve as a conceptual lens for systems practitioners. The case-studies demonstrate the importance of a practitioner’s ability to seamlessly manage and work with multiple variables, stakeholders, and factors to deliver responsible outcomes with the aid of learning loops. The main contribution of this paper lies in the case-studies and analyses presented that provide use cases for Holistic Flexibility in ST, which will help address recent calls in the discipline for ST to be considered as a cognitive skill.

Horizontal Evaluation - Better Evaluation (Better Evaluation)
Horizontal evaluation is an approach that combines self-assessment by local participants and external review by peers. Originally developed to evaluate new methodologies for agricultural research and development, horizontal evaluation has wider potential for application. In its original setting, the focus of horizontal evaluation is the actual R&D methodology itself rather than the project per se or the team or organisation that developed it

Horizontal Evaluation: Stimulating social learning among peers - Thiele, G; velasco, C; Manrique, K (ILAC Brief, 2006)
Horizontal evaluation is a flexible evaluation method that combines self-assessment and external review by peers. We have developed and applied this method for use within an Andean regional network that develops new methodologies for research and development (R&D). The involvement of peers neutralizes the lopsided power relations that prevail in traditional external evaluations, creating a more favourable atmosphere for learning and improvement. The central element of a horizontal evaluation is a workshop that brings together a group of ‘local participants’ who are developing a new R&D methodology and a group of ‘visitors’ or ‘peers’ who are also interested in the methodology. The workshop combines presentations about the methodology with field visits, small group work and plenary discussions. It elicits and compares the perceptions of the two groups concerning the strengths and weaknesses of the methodology; it provides practical suggestions for improvement, which may often be put to use immediately; it promotes social learning among the different groups involved; and it stimulates further experimentation with and development of the methodology in other settings.

Hospicing The Old - Robinson, Cassie (Stewarding Loss, 2019)
In 2010 I was introduced to the Berkana Institutes’s Two Loop model, and I come back to it again and again. As I’ve moved across different…