What is social impact? How do you measure it? Last week I attended “Metrics from the Ground Up” – a two-day event hosted by the Aspen Network of Development Entrepreneurs (ANDE) in Washington DC. Participants included entrepreneurs, investors, foundations, accelerators, and academics, all with an interest in examining the concept of impact and how to measure it.
This year’s ANDE event focused on how to make sense of data from the perspective of the entrepreneur (and their investors) who are interested in refining their business models to achieve greater impact, as well as from the perspective of the sector as a whole with an interest in aligning priorities and perspectives from multiple stakeholders to facilitate greater good.
The randomized control trial (RCT) is the traditional gold standard for evaluating the impact of an intervention. Recently, Esther Duflo (and others) has touted the RCT as an effective approach for assessing the economic impact of development-oriented interventions. Conducting an RCT however, is resource intensive and often not feasible, particularly in resource-constrained contexts.
The dialogue of this year’s conference was predicated on the understanding that alternative approaches are required where RCTs are not possible or appropriate. Two frameworks were presented at this year’s conference that provide alternative approaches to conducting evaluations with consideration for the constraints typically experienced by organizations undertaking research in this field:
1) The Goldilocks Principles provide a set of guidelines for organizations to use to build robust systems of data collection regardless of an organization’s capacity to assess impact. Developed by Innovations for Poverty Action, the principles are:
- Credible – Only collect data that reflect what you intend to measure.
- Actionable – Only collect data that you will use. For each metric, identify how the information could be used to change course of action.
- Responsible – Data collection should match the systems and resources available.
- Transportable – Apply what you learn to other programs and contexts.
As Mary Kay Gugerty and Dean Karlan argue in their SSIR article, measuring impact isn’t for everyone and “sometimes impact is simply not measurable in a credible way.” In some cases it may be best for organizations to focus on appropriate, decision-driven measurement rather than striving to account for impact.
2) The Lean Research Framework, developed at the Massachusetts Institute of Technology’s Development Lab (MIT D-Lab) outlines principles for human-centered research and evaluation. This approach focuses on increasing the benefits of evaluation for all involved. Lean Research is described as:
- Rigorous, regardless of methodology.
- Respectful and Relevant to all engaged in the process.
- Right-sized with regard to scope and methods given the research objectives and usefulness of findings.
Lean Research principles focus the research process on creating value for all engaged. For example, Root Capital, a social investment fund with a focus on agriculture, has applied Lean Research methods as part of their program assessments to generate data that is of use to the farmers they partner with. This approach enables data-driven decisions among the enterprises they support, potentially increasing the productivity of the farmers and serving their own mission of strengthening small-scale agricultural businesses.
Though I left the conference with more uncertainty about how best to measure impact than when I arrived, I left feeling assured that we can address evaluation questions in a way that is sensitive to everyone engaged in the process, and helps rather than hinders those we seek to support. By working together to ensure that each relevant stakeholder has a part in the process, we increase the potential to create value for all involved and better position ourselves to achieve our shared mission of generating social good.
To learn more about Innovations in Healthcare and how we’re thinking about impact metrics, please contact any member of the Research and Knowledge team: Lauren Westervelt (lauren.westervelt@duke.edu), Erin Escobar (erin.escobar@duke.edu) or Andrea Taylor (andrea.d.taylor@duke.edu).