Capturing experiences

Finding out how participants feel the intervention has helped

This burden of uniform evaluation can be one imposed by funders too though thankfully some are now coming to value 'stories' over statistics.

Louise

Finding out how people feel about an intervention, and the impact they feel it has had on them is extremely useful. This ‘qualitative’ evidence helps to bring the intervention to life when describing it to those unfamiliar with the programme. Depending on how comprehensive this process is, it can also shed light on change mechanisms, thus allowing re-evaluation of initial logic models and assessment as to whether the intervention has worked in the ways anticipated or in a different manner.

Participants can be asked open-ended or close-ended questions relating to how they think the intervention has changed their thoughts, feelings or intended behaviours. However, questions should not lead participants into responding in particular ways. Instead, they should allow those responding to express a view that the intervention did not result in change for them personally. This is also useful information that needs to be incorporated in any evaluation – when things do not work, as well as when they do.

Most organisations collect data like this already, but if not, it is relatively easy to implement procedures to do so. The following represent different ways in which participants’ perceptions and experiences can be accessed:

  • Feedback forms – given to participants at the end of the intervention, capturing how much they enjoyed taking part, if and how they feel the intervention has had an impact on them, and any recommendations they have for changes. This can include rating scales as well as open-ended questions for participants to provide written comments.
  • Case studies – personal stories about ways in which the intervention has impacted a child’s or young person’s thinking, attitudes or behaviour can be captured through interviews, participants’ writing, drawing or photographs, or through videos. Case studies are stronger if they include a range of perspectives, i.e., the child, their teacher, a parent/caregiver, etc.
  • Testimonials – accounts from other stakeholders (such as parents, carers, peers, teachers, youth leaders or other organisations) about the impact of the intervention; how it has made a difference and to whom. These could be gathered in person or drawn from social media, such as tweets about the success of a project in different settings.
  • Images/objects – drawings or photographs of young people taking part in your intervention, objects, posters or other outputs that have been created throughout the intervention process.

It is vitally important that organisations provide detailed information for participants and stakeholders about the type of data they will be collecting and what they will ask those involved to do; obtaining consent where necessary (especially if images of participants are to be used). This type of evidence provides compelling cases and clear examples of children’s and young people’s experiences. However, it is not possible to generalise the findings to the population as a whole.

Other types of outcome/impact evaluation