Contents
Introduction
How do you measure the impact of activism and advocacy work? It can be tricky. Here are some ideas, resources and organisational examples to get you started.
Impact is nonlinear and combinational, reflecting the efforts of many actors working together to incite change….Impact is transformational and not much transformation happens in front of you. – Zehra Mizra
What sort of impacts are there?
Many activists, organisations and academics have developed processes for measuring different aspects of ‘impact’. The most important first step in the process of measuring your organisation’s impact is to establish what type of impact you are trying to achieve.
There are a range of potential impacts an organisation can seek to achieve. These can be roughly split into three categories: external, organisational and individual impact.
‘External’ examples of impact
This category refers to groups or entities external to the organisation – such as the media, policy makers or a corporation – that are affected by your activities.These could include achieving outcomes such as:
- Gaining more media coverage/attention
- Influencing policy makers and/or politicians
- Influencing companies or challenging corporate decisions
‘Organisational’ examples of impact
This category refers to some aspect of your organisation that is affected by your activities. These could include achieving outcomes such as:
- Increasing the number of volunteers in your organisation
- Organising more activities or growing local groups/branches
- Increasing your organisation’s financial resources
‘Individual’ examples of impact
This category refers to how your activities might affect individuals; either people within your organisation or movement, or people in the wider community. These could include achieving outcomes such as:
- Improving the enjoyment and satisfaction of employees/volunteers
- Stepping individuals up from one-off participation through to active participation and leadership roles
- Changing values, beliefs and behaviours of friends, family and/or individuals in the wider community
Once your organisation has established what impact it seeks to achieve, you can turn to the question of how to measure these impacts.
How can these impacts be measured?
Once you have identified what sort of impact you are seeking to achieve, you can then consider how this could be measured. To measure impact you need to collect data. Data can include numbers, reports, notes from post program evaluation meetings, reflections, interviews, and feedback.
Some impacts are easier to measure than others, mostly because some data is easier to obtain than other types of data. For example, some organisations already collect data on their supporters, volunteers and donors. This allows them to measure whether their organisation is attracting new supporters, retaining activists over time and growing local groups.
It will likely be more difficult to measure the impact of your activities on external groups or individuals.This is because the data may be more difficult to collect, and it may take skills and experience to then analyse it.
The following images give examples of each of these impact categories, and the data that could be collected to measure them.
External impact examples and measures
External impact is often what activists seek to achieve the most. However, it can be difficult to measure. This is because it may be difficult to obtain data; particularly data around the influence of activism on policy and corporate behaviours. Despite this difficulty, identifying external impact can be very helpful for communicating successes to build support amongst activists, other organisations and potential funders.
Examples of measuring external impact
Many organisations track their external impact by tracking social media metrics. These metrics include ‘reach’ (the number of people who see your content), ‘impressions’ (the number of times people saw your content), and audience ‘growth rate’ (how many new followers you get). You can read more about social media metrics here. You can also read a detailed report on measuring communication impact by the Bill and Melinda Gates Foundation by clicking here.
Other organisations track external impact on mainstream media. An example of this is the analysis of media coverage of Extinction Rebellion, School Strike for Climate and Stop Adani coverage in 2019. This analysis demonstrated that Extinction Rebellion protests received the most coverage, however School Strike for Climate received far more positive coverage. You can read about this and download the report here.
Other examples of external impact have been compiled in an academic study of the outcomes of environmental collective action published in 2022. This study found 47 different types of external impact that had been achieved by environmental campaigns. Most of these external impacts were related to corporate behaviour, such as changing a corporation’s policy or developing a new corporate standard. Political impacts included creating new laws and policies, or increasing the number of pro-environmental candidates. Other external impacts included stopping or delaying development projects like dams and coal mines, improving the environment by reforesting areas or cleaning up pollution, and achieving legal successes such as winning cases. You can read the full paper by clicking here.
Organisational impact examples and measures
Of the three impact categories, organisational impact is often the easiest to measure and track. This is because many organisations already have data on what their organisation does and how this might change over time.
Example of measuring organisational impact
Sunrise movement, in the United States, has a database of supporters and actions. This data infrastructure enabled Sunrise to train over 12,000 people during the pandemic, reach 6.5 million voters during the 2020 election, and mobilise tens of thousands of members in actions. The data they collect on supporters and actions allows them to track:
- What events are more popular and attract more attention
- What sort of events attract more first time participants
- How people move through the participation ladder, from crowd – member – leader – top leader
- What training programs attract more participants and are then associated with ongoing participation
You can find more information about how Sunrise Movement measures their impact by reading ‘A Practitioner’s Guide for Measuring Movements’ or watching their webinar.
Individual impact examples and measures
Like external impact, individual impact can be challenging to measure. Gathering data will likely require organising dedicated surveys or interviews, or otherwise searching for existing data which is connected to the particular impact you are seeking to achieve.
Example of measuring individual impact
The ‘See Me’ campaign run by the Mental health Foundation wanted to change people’s values, beliefs and behaviours around mental health. The campaign ran for three years and trained volunteers as ‘Community Champions’, media spokespeople and community speakers. The campaign wanted to increase the number of people who challenge self-stigma, stigma and discrimination, feel confident about speaking up and increase the broader community’s understanding about stigma and discrimination.
To measure their impact they organised a staged plan of evaluations. These included:
- Training evaluation forms (pre and post training)
- Community surveys (through Survey Monkey)
- Regular volunteer monitoring surveys
- Interviews and two focus groups
This data enable them to measure their impact. They found that the campaign increased volunteers skills and knowledge and successfully built networks and social contacts. Their data also demonstrated that sharing lived experiences of mental health is a powerful way of normalising mental health concerns. Through their analysis they were also able to find ways to improve campaigns in the future, such as by building leadership amongst volunteers and sharing successes, learnings and challenges across teams.
You can find more information about the campaign and the way they measured their impact by reading about the program here or downloading their evaluation report.
Tips for Measuring the Impact of Your Influencing Work
These tips come from Measuring your Campaigning Impact: An Introduction, Katie Boswell and Anne Kazimirski, New Philanthropy Capital, 2016 pgs 5 – 6
- Spend time and effort on planning
This could involve properly developing your theory of change. Ask yourself: Does your plan make sense? Is it realistic? Do you have the necessary resources to carry out (and evaluate) the campaign? Or does your focus need to shift in order to get most out of your resources? - Use what you already have
Think about the information your organisation already collects, and how that might help inform the evaluation of your campaign. This could include media hits, meetings held with policymakers, twitter engagement levels and website visits. - Be proportionate
Ensure you are prioritising measurement for the most important outcomes. Your campaign may result in a number of outcomes—intended or otherwise—but impact measurement must be proportionate, so focus on what is instrumental to your work. Similarly, do not feel you have to collect all of the required measurement data yourself. You may be able to obtain the information you need elsewhere eg, from government data or academic research. - Focus on learning
Learning and improving is a key reason that charities should be seeking to measure the impact of their work, and campaigns are no exception. A good way to do this is to have a thorough debrief with your team in which you can thoroughly scrutinise the results. Think about what your results show, what went well and what didn’t, and what you will change next time. - Differentiate between contribution and attribution
Most campaign successes happen because all the pieces in a large puzzle come together. For example, legislative change may occur as the result of your campaigning, but it is most likely that it was also brought about by a combination of shifting political factors, changing public opinion, and external events. You may not be able to attribute this change to your campaign, but you need to at least be clear on what your contribution was, whether that was influencing the media, mobilising specific groups, or building a wider coalition.
Resources and Tools
Metrics can help organizations articulate where they are going, what road they are taking, and what they expect to find along the way. They can help groups strike the right balance in the trade-offs above, allocating time and energy to be maximally effective. They can serve as a guide and tool for lifting up lessons for the field and for funders. p.8
See pg 24 – 25 for Sample metrics
“In this paper, the authors outline the argument that advocacy Monitoring, Evaluation, and Learning (MEL) needs to accommodate the uncertainty, the unpredictability, and the complexity around how social and political change actually happens. While there is no “royal road,” the paper aims to raise practical solutions for advocates, evaluators, and funders to approach MEL in a way that makes it credible, reliable, and instructive.” – p. 2
- See pg. 36 for Elements in a Campaign Victory Framework
- See pg 51 for Table 4: Advocacy “fitness for purpose” metrics
This guide is for charities who seek to evaluate their work, or measure their impact. Measuring means using data and evidence, both quantitative and qualitative, to gain useful insights about your charity’s work. It’s about understanding whether your project or service made a difference in people’s lives. If yes, how so? If not, why not?
Here are other great resources from New Philanthropy Capital
- Result! What good impact reporting looks like
- Stories and numbers: Collecting the right impact data
- Keeping it in proportion: Impact measurement for small charities
Impact Assessment: Understanding and assessing our contributions to change, INTRAC, 2010
In spite of a large increase in the production of tools and frameworks to assess impact, there remains a lack of clarity about what ’impact assessment’ actually means, and how it differs from ‘evaluation’. There also appears to be a lack of confidence in organisational ability to carry out effective impact assessments. This paper addresses these issues in a practicable way, aiming to provide clarity on what impact assessments are, and how they differ from and complement processes of monitoring and evaluation (M&E). It is written for staff and development practitioners working with or for international NGOs or other complex organisations. It considers some of the key challenges and practical difficulties that these practitioners face in carrying out impact assessments; and offers some good practice guidelines. – INTRAC for Civil Society
“For impact assessment to be useful for learning and accountability, we need to look beyond project logic and focus on changes in relation to our target groups. The question to ask is not ‘What did we achieve’? but rather ‘What has changed in relation to our efforts’?
Types of enquiry which will support our understanding include:
- Who or what was involved in the change? (e.g. individual actors or state institutions)
- What strategies were used to bring about the change? (e.g. reform, mass mobilisation)
- What were the contexts that affected how the change happened? (e.g. urbanisation, power relationships)
- What was the process or pathway of change? (e.g. demonstration effects, cumulative progress)
- How were our efforts connected to this?” – pg. 5
Organisational and Campaign Examples
This post highlights some of the challenges experienced during an in-depth project seeking to measure the impact of climate change activism in Australia. The authors measured the number of groups, their events, campaigns, and campaign outcomes across a 10 year period to detect which campaigns were the most successful and how the environmental movement changed over time. They used a variety of metrics which were then mapped onto three social movement frameworks: the political process model, the great turning and the insurgency model. You can download the free monograph by clicking here.
Their analysis demonstrated that most climate change campaigns do not achieve their stated goal. However, as they write in the blog post, this does not mean they are unsuccessful. Instead, multiple sub-wins can occur over the short, medium and long term, before the overarching goal may be achieved. Organisations need to ensure that they know what kinds of impacts they want to achieve and how these may link together. This will help them track their achievements over time.
- Amnesty International – Impact and Learning System, 2017
- Save the Children – Measuring our Impact – Guidances and Templates
- Unicef – Impact Evaluation
Explore Further
- Measuring Transformational Impact in Human Rights Advocacy, Zehra Mizra, Stanford Social Innovation Review, 2021
- A Guide to Measuring Advocacy and Policy, Organizational Research Services, 2007
- The Art of Assessing the Impact of Advocacy Work
- How are we Measuring People Power in 2020+, and where do we go from here?
- Campaign Monitoring and Evaluation: Measuring What Matters
- Toolkit for Demonstrating the Impact of Independent Advocacy, Scottish Independent Advocacy Alliance
