Testing our narrative ideas
Following the last chapter, you’ve (hopefully) now got a set of narrative ideas to take to the next stage in the process: testing!
When you communicate, we usually have a good idea of what we want to say (chapter #6) and the change we want to make (chapter #5). But when it comes to crafting a message, how do you know whether it will work? Do you just close your eyes and hope for the best? That’s where testing comes in.
We believe that every campaigner can improve their communications with testing, and we know that it’s possible to do this even on a low budget.
“The old model of lengthy testing before messaging and strategy are implemented has not served the field and certainly does not serve the political moment. Now is a time to experiment, to hold developing narratives lightly, test in real time, and be okay with failing and trying again.”—Narrative Initiative
Any testing is better than no testing
Campaign messages are often developed under huge pressure, in a small team, and sent straight out into the world. These messages are based on assumptions of how an audience will react rather than evidence and they are therefore likely to be hit-or-miss.
When messages miss, they can leave a lasting negative impression on how people think about your issue. This can set you back in time and resources, and make it harder to realise your vision.
Practicing our values whilst testing
Testing messages should not be a substitute for spending time engaging directly with people. We can learn a lot through research, but it’s in community with people that we can understand others better and therefore listen and communicate more effectively.
All too often, folks carrying out this kind of testing work tend to follow what we understand to be an ‘extractive’ research process, informed by western academic standards. In this type of research you simply arrive, extract the relevant information from participants, leave without any further contact, and draw your own independent conclusions for your work.
There’s a strong power imbalance in each stage of this process in favour of the researcher, and most especially the ‘conclusion-drawing’ stage: who gets to make sense of the data is a deeply political question.
What might a testing process look like that honours the people who we would like to learn from, and contributes to collective power-building across our communities?
Some ideas from us*, as a starting point:
- Collect informed consent: Let participants know the general purpose of the research, inform them of their right to withdraw at any time, and provide a point of contact.
- Do more participatory knowledge-creation over traditional research methodologies: Involve the people you’re wanting to hear from in your testing/research in the design of the process. For this collaboration to work, you’ll need to prioritise relationships, care and trust building.
- Debrief with participants: sense-check how participants are doing after the research activities, provide opportunities for people to learn more out about the research.
- Analyse the findings together: Invite participants to collectively reflect with you on the findings of the research, and explore possible conclusions together.
*Building on this beautiful guest blog from our friend Elena Blackmore on how we can decolonise the work of narrative change—follow the link for more.
“The alternative to extractivism is deep reciprocity. It’s respect, it’s relationship, it’s responsibility, and it’s local.”—Leanne Simpson, Indigenous Idle No More activist
How to test
There are lots of ways to test narrative ideas: from focus groups and street interviews to surveys and informal workshops. We are also big believers in testing on the go: trying your ideas out in your day-to-day communications work, learning what works and doesn’t work, then trying again. You can do all of these on big or small budgets.
Let’s walk through our handy 5-step process:
- Know what you are looking for: form the right research questions before you begin, drawing on your narrative goal(s) from chapter #5.
- Choose your methodology: decide whether focus groups, interviews, surveys or another method might be the best match for your research question.
- Prepare your messages to test: follow some basic principles to get your messages ready to test and compare.
- Find the right people: find out about different types of ‘samples’ and how you can recruit for them.
- Understand your results: know how to make sense of your results.
Check out these slides for some top tips for how to take each of these steps, one at a time:
And if you’re hungry for more, download a copy of our Testing Guide, where you’ll find detailed advice, practical examples and signposts to further tools and resources—whether you’re on a shoestring budget or have some cash to splash: publicinterest.org.uk/TestingGuide.pdf
CASE STUDY
Following this 5-step process, we worked closely with an LGBTI organisation in Slovenia called Legebitra. Together we organised some low cost focus groups to test their messages about LGBTI discrimination, and we used the results to develop the final campaign. Testing helped Legebitra to find effective and humorous ways of appealing to shared identities and emphasising the common ground between LGBTI people and non-LGBTI people. The campaign proved successful in Slovenia, with the film getting 50,000 views in just the first few days of its release.
Iterating your narrative ideas
It’s really common on the back of any testing process to learn some pretty fundamental lessons about what works and doesn’t work when trying to frame the specific issue we’re working on. With that new understanding, we go back to our original narrative ideas and iterate them, redesigning and honing where we need to to help us meet our narrative goals.
CASE STUDY
During our Framing Climate Justice project, we found very little existing research and guidance about how to frame specific elements of climate justice, like how to talk about historic responsibility and unequal impacts. So we had a go at creating narrative ideas that we thought might help with these elements, but we weren’t sure if they’d work. So we took them out to test using street interviews and a big survey experiment.
Some of the things we learnt during testing:
- Solidarity framing generated hope, and agreement that those least responsible are the most affected.
- Including economic design in the framing was a helpful way in to talking about historic responsibility, and the role of colonialism and capitalism.
- Emergency framing, in contrast, made people more likely to say that climate change has no impact on existing injustices, and also led people to call for more top-down solutions.
With that stronger understanding of what did/didn’t work, project participants went back to the drawing board to strengthen their campaign messaging.
For more of the story, check out this slidedeck summarising the project findings.
Is there some iteration that you can already have a go at before you get stuck into testing?
Give one or two of these resources a scan to see if there’s any top framing tips for your issue that will save you having to test specific things that someone else has already experimented with:
Race
Contains Strong Language – Reframing Race
Race-Class Narrative UK – CLASS & ASO
Avoiding Antisemitic and Islamophobic Hate Speech – Solutions Not Sides
The Economy
Framing the Economy – PIRC, NEON, NEF, FrameWorks, +
Wellbeing Economy – Positive Money & NEON
Tax, Public Services and Cost-of-Living – NEON
Talking About Poverty – FrameWorks UK
Conflict & Peace
Building the Bridge to Peace – FrameWorks
Palestine and Israel – ASO
Policing & Safer Communities
Holding Our Own – Liberty & NEON +
The Silencing Bill – NEON
Violence Against Women
Media Guidelines – Level Up
Health
NHS privatisation – NEON
Nurses pay – NEON
Pandemic Response – PIRC
Housing
Rent Caps – NEON
Climate
Climate Justice – PIRC, 350.org & NEON +
Climate visuals – Climate Outreach
Air Pollution – Inquest & NEON
Reframe Aviation – Stay Grounded
Nature & Conservation
Framing Nature – PIRC
Animal Freedom – Animal Think Tank
LGBTQ+ Equality
Framing Equality – PIRC, ILGA-Europe, +
LGBTQ+ Asylum – PIRC+
Transgender Freedom – ASO Communications
Anti-Trans violence – Transgender Law Centre
Gypsy & Traveller lives
Media That Moves – PIRC +
Migration & Asylum
Channel Crossing – NEON
Asylum Accommodation – Minnie Rahman & NEON
Connecting Climate & Migrant Justice – many!
Resources – iMix
Protest
How to talk about strikes – NEON
How to talk about demos – NEON
For more issue guides and general good-practice recommendations, checkout these libraries from NEON, FrameWorks UK and ASO Communications (including Messaging This Moment).
Let’s give it a go…
To help you work out how to test your narrative ideas, we recommend sitting down with your group to work through steps 1-5 above, giving yourself some space and time to work it out together.
For now though, why not have a go at a simple testing method to get a feel for this work in action:
Quick-and-easy interviews
- Find 1-3 people to talk about your narrative ideas. If you can, find people outside of your close circle—for example, someone’s parent, random people in a cafe, or your local librarian—so much the better. If you can get others to have a conversation about it in front of you, even better!
- Show them the material you want to test. This might be options for a poster, a number of images, or even a speech.
- Ask them some questions to see whether what you’ve produced meets your narrative goal (from chapter #5). For example: How do they feel when they see or hear this? What do they understand the material to be about? What does it make them think of? Can they see a solution to a problem here? Would they act on this, and what would they do?
Let’s get reflective…
Reflective question
Does anything you’ve learnt in this chapter (or any testing activities you’ve tried out already) suggest you should pick one narrative idea over another, or change the material you developed last week in some way?
Learning links
If you’ve got 15 minutes:
- Read (5-60+ mins): How to test your comms—from PIRC: This is the most in-depth resource we know of on how to carry out your own communications testing, and we’ve packed it full of practical examples. Have a quick scan, or dive in one chapter at a time for the full low-down.
If you’re keen for more:
- Read (2 mins): Organising with Elicitive Questions—from 350.org: A helpful resource for designing research questions that will help you to test, whilst avoiding ‘leading’ questions.
- Listen (16+ mins): How the left can suck less at messaging—from Volts and ASO: The first 16 minutes especially focusses on the challenges and pitfalls of certain testing methodologies. Keep listening if you’re into it and you’ve got the time!
Got loads of time? Great:
- Read (5-30ish mins): Survation: Our Services—from Survation: Get a sense of what a research agency might be able to support you with, or give them a call to chat through your ideas. (Although, a cautionary note about extractivism (see above) and segmentation!)
- Long-read: What they see matters—from Resource Media: A practical guide for working out how to test visuals and imagery.
In the next chapter, we’ll explore different platforms and formats for spreading your narrative ideas—from video, graphics and email content to interview techniques, door-knocking scripts and press stunts. Meanwhile, if you’ve got any questions or feedback you’d like to share, feel free to drop us an email on courses@publicinterest.org.uk.
