Outlines two broad types of evaluation that should be undertaken as part of a comprehensive assessment |Outcome or impact evaluation This type of evaluation assesses the results of the intervention; the changes brought about by the programme, activity or series of sessions. It entails collection and analysis of data relating to the outcomes that are expected; the areas the intervention is targeting and trying to improve. The findings will demonstrate if the intervention is successful in achieving those outcomes - changes in children and young people’s thinking, attitudes, behaviour, etc. It is also useful to examine if the intervention only has an impact on certain people. This makes the decision about what to measure and how to measure it very important, and is an area that AWE professionals identify as a key challenge in the field. The measures chosen need to strongly reflect the content of the intervention. Some examples of validated measures to consider are provided in the Evaluation measures library. If there is an expectation that the intervention is likely to have greater/less impact on some children/young people than others, it is useful to include measures that will capture this differential impact (e.g., age, gender, ethnicity, locality). However, organisations will not be able to collect UK data that could lead to individuals being identified, and any data collected needs to be General Data Protection Regulation (GDPR) compliant; an issue that must be addressed when tracking children in order to assess change. Guide to the General Data Protection Regulation - GOV.UK (www.gov.uk) Outcome/impact evaluations can vary from a single group pre-/post-test design to a more comprehensive approach that compares what happens for those participating in the intervention with a group of children who have not taken part, and examines if any resulting changes have longevity. To enable effective comparison, a ‘control’ group needs to closely represent the group who have participated in the intervention. This is called a ‘matched control’. A ‘waiting list control’ may be used. This is a group that is due to take part in the intervention at a later date. An examination of change over time (i.e., the collection of data at different points following the intervention) requires careful management to ensure each participant’s data are linked. The more comprehensive the evaluation design, the more likely it will be that any observed changes can be attributed to the intervention itself, rather than other things that are happening simultaneously. Other methods can also be used to assess impact on those participating, from brief questionnaires/interviews with participants about the experience of taking part, through to comprehensive case studies that gather evidence relating to the views of those directly involved and the people who are connected to them. Teachers (in the case of school-based interventions) or parents/caregivers in more tailored individualised interventions, are good examples. |Process evaluation A process evaluation is quite different to an outcome evaluation, but undertaken at the same time. It is vital for making improvements to interventions and determining future priorities. It is concerned with how an intervention is working, examines if it is being delivered as planned, and assesses strengths and weaknesses in content and delivery. If a standardised approach has not been followed in certain circumstances, it is important to document and reflect on the reasons for this and any adaptations that have been made. Process evaluation seeks to capture the experiences of those delivering the intervention to identify what is working well as well as aspects that are, or may prove to be, problematic. A range of methods is typically used to assess ways in which an intervention is working and why it might work better in certain settings or with particular groups. This is helpful in enabling optimal targeting of particular interventions, and highlighting which aspects might need to be changed for other groups. A key component of a process evaluation is the construction of a logic model that explains how the intervention is thought to generate outcomes (Public Health England, 2018) – see Step 2 and Facet 6.
