Episode 37: Getting creative with impact evaluation

This week, Mark talks through a series of good practice principles for engagement and impact, published in his recent blog post- Add your own feedback and ideas hereThe blog builds on a number of sources, including:Mark’s Impact Culture book (see the bottom of this page for his impact culture toolkit)NCCPE engagement guidanceMark’s “Re-thinking impact…” paperYou can download a written transcript of this episode here

This week, Mark explores what the recent REF2029 guidance means for impact, including a discussion of what we might be expected to write about in the new engagement and impact narrative, and an evidence-based approach to writing a 4* impact case study.NCCPE engagement guidanceMy “Re-thinking impact…” paperWhat made a 4* impact case study in REF214 paperMy 4* REF2021 impact case studyImpact templates (in the Google Sheets menu, go to File > Make a copy to save an editable version)3i analysis templateFast Track Impact Planning TemplateYou can download a written transcript of this episode here

This week, Mark talks to Michael Parker, Director of Operations at The Conversation, a news outlet that specialises in working with researchers, giving them editorial control on a global platform. Drawing from their work together creating the Media Impact Guide and Toolkit, Mark and Michael discuss how researchers can harness the media to get more impact from their research and provide evidence of both the reach and significance of the impacts that arise.Read the Media Impact Guide and ToolkitFind out more about The Conversation Sign up for next year’s Exchange conferenceYou can download a written transcript of this episode here

Eric Jensen is an expert in impact evaluation who has written a textbook on many of the most useful methods you'll need and leading two companies that specialise in impact evaluation consultancy and training alongside his academic research on environmental issues. In this interview, he provides tips to help you design a more effective survey to evaluate your impact, and explains how you can repurpose widely used impact planning tools as evaluation tools, that will help you improve your practice as much as they will give you evidence of impact.Find out more about the Institute for Methods Innovation via his Methods for Change website Learn about Qualia Analytics software to help you both collect and analyse impact evaluation dataWatch Eric training with Mark and others on the use of Theory of Change to evaluate impactRead Eric's book, Doing Real Research: A Practical Guide to Social Research You can download a written transcript of this episode here

This week, Mark interviews Laura Tucker (Vertigo Ventures), Tobias Schoep (GrowImpact), and Sarah Morton (Matter of Focus) about the platforms they have developed to help researchers keep track of their impacts. Find out more about each platform:GrowImpactOutNav from Matter of FocusImpact Tracker, part of the impact ecosystem from Vertigo VenturesYou can download a written transcript of this episode here

This week, Mark interviews Rachel Blanche from Queen Margaret University Edinburgh to find out how arts-based methods can provide depth and rigour to an impact evaluation. They discuss a range of approaches including visual, performative and narrative methods, the types of evidence these methods can generate, how these approaches can empower participants in determining what’s meaningful, and how evaluating in this way can itself generate further impacts.Rachel shares two examples of arts-based methods used to evaluate impact in healthcare research – a theatre project capturing data on dementia care (citing this paper) and (a participative creative inquiry on osteoporosisThe paper we mention writing together about evaluating impact with arts-based (among other) methods can be accessed hereWatch the training Rachel and I ran on evaluating impactAnd you can find out more about Rachel's work hereYou can download a written transcript of this episode here

Mark discusses a range mixed methods evaluation designs that can help you collect data to evidence impacts arising from industry, policy, media, and public engagement. The methods are easy to use without any specialist training or experience, and can generate useful data in some of the trickiest areas of impact evaluation.Find out more about the "postcard to your future self" methodFind out more about the Media Impact Guide and ToolkitRead Mark's REF2021 case study in which he evidenced a range of policy impactsYou can download a written transcript of this episode here

How do you design an impact evaluation? There is no blueprint, but in this episode, Mark gives you the tools you will need to choose an evaluation design with methods that can deliver convincing evidence while giving you win-wins for your research.Evidencing impact paper | Decision treeYou can download a written transcript of this episode here

In this first of a series of episodes on monitoring, evaluating and evidencing impact, Mark discusses one of the biggest mistakes that people make when the evaluate their engagement instead of their impact, and provides simple tools anyone can use to keep track of their impact with minimal time and effort. He concludes by explaining the difference between monitoring and evaluation and introducing impact evaluation concepts and approaches. The episode draws on this paper about evaluating impact: https://www.sciencedirect.com/science/article/pii/S0048733320302225?via%3DihubYou can download a written transcript of this episode here

This week, Mark interviews George Hope, Communications Manager at Oxford Net Zero and Greenhouse Gas Removal Hub, based at University of Oxford, about how you can reduce the risks of engaging with social media and achieve more impact online. They discuss challenges of engaging with skeptics and conspiracy theorists, and strategic tools for achieve more impact with less risks to your time, reputation and mental health.Connect with George on LinkedIn or follow his work on Twitter via his personal account, Oxford Net Zero and the CO2RE Hub.You can download a written transcript of this episode here