Introduction to the toolkit

Introduction

Until now there has been no guidance for those seeking to design, develop and evaluate interventions that aim to educate children and young people about animals and their welfare, prevent harm, and promote positive relationships through their interactions. This document provides a framework for understanding the purpose of evaluation and how it might be approached - from the intervention planning stage right through to reporting and sharing the findings. It will be most useful for those who are new to evaluation or wish to gather more comprehensive data to provide stronger evidence of impact. It is also highly relevant to those working within humane education.

The toolkit builds on the findings from a Delphi study of 31 animal welfare education professionals, 27 of whom were UK-based (Muldoon & Williams, 2021a; 2021b). This study highlighted areas of practitioner consensus with respect to the goals and important components of successful interventions. However, there were differing views with respect to structuring an intervention so that it can be evaluated effectively, and also recognition that measuring impact is a significant challenge for most animal welfare organisations. Illustrative quotations from our participants are used throughout the toolkit to highlight their perspectives. All names are pseudonyms. We would like to thank everyone who participated in our study and those who provided feedback on a final draft of the toolkit.

 The core aim of this toolkit is to support practitioners to develop strong evidence-based practice. There is a pdf to save or print out that reproduces the guidance here, but does not include the worked through example and details of the measures we recommend (these are currently being developed). The toolkit provides some simple guidance on the steps organisations can take to plan, evaluate, adapt, and report on, their intervention work. It also provides links to a range of resources that have been developed to support practitioners with their evaluation activities. Importantly, it highlights the processes involved, providing a step-by-step guide to ensuring high quality evaluation and, consequently, a comprehensive understanding of an intervention’s impact. The online toolkit is designed to be a responsive resource that we update periodically. This work has been funded by the Economic and Social Research Council and has been undertaken in partnership with the Scottish Society for the Prevention of Cruelty to Animals (Scottish SPCA).

What is an ‘animal welfare education (AWE) intervention’?

Animal welfare education/cruelty prevention teams carry out a range of activities with the ultimate aim of enhancing human treatment of animals, as the quotation below illustrates. They may choose to focus on helping others develop their knowledge, understanding, skills, attitudes, empathy and perspective-taking, or reflect more fully on their own behaviours and/or moral and ethical values. From our perspective, an ‘intervention’ is a structured, planned and integrated set of activities designed to have specific types of impact on recipients. Ideally, those developing an intervention will describe it in detail, identify the goals and intended impact of each activity, and illustrate how their input will effect change.

Why is it important to evaluate interventions?

If interventions are carried out without due attention to what is working well and what is not, it is impossible to establish with any certainty if the work being undertaken is successful and, if not, what needs to be done to ensure success in the future. The practitioners in our study described the enormous potential of animal welfare education, moving beyond just improving life for animals. To ensure this potential is achieved and to persuade others of its significance, it is essential that organisations clearly demonstrate the value and effectiveness of their work. This is particularly critical as we enter an era where priorities are changing, competition for funding is likely to be stronger than ever and sources of financial support more limited. There are a range of benefits associated with systematically evaluating an intervention. These include:

1. providing evidence of impact, demonstrating if, how, and when (under what circumstances) an intervention has been successful; highlighting the specific changes that children/young people experience as a result of participating;
2. pinpointing where improvements need to be made, ensuring an intervention is the best it can possibly be;
3. helping to show others what has been achieved, stimulating support for the cause, and
4. allowing others to replicate successful intervention and evaluation work, leading to wider reach and impact

Evaluation is most effective when it is a continuous reflective process, not a one-off assessment. There should be opportunities built in for all those involved to self-evaluate; systematically documenting and reflecting on what they are doing, what is working well and why, and what problems are being encountered along the way. It also involves refining or making changes to elements that are not working smoothly or achieving intended outcomes. Piloting an intervention and evaluation on a small scale is therefore extremely useful prior to rolling out the full programme. The reflective process can be informal, focusing on everyday experiences of working with children and young people, or more formal and comprehensive.

However, it is important to produce robust evidence of effectiveness. Children/young people may engage and enjoy a programme, but this does not necessarily lead to improvements (e.g., in perceptions of animals, knowledge, understanding, skills, attitudes, empathy, or intent to behave differently).

Image