Investing in Establishing Evidence is Investing in Achieving the Best Outcomes for our Children

The importance of evidence when making decisions about the implementation of services is now well understood.  What Works organisations such as the Education Endowment Foundation and the Early Intervention Foundation place evidence at the heart of their work as a means of distinguishing between programmes that work and those that do not. This evidence base is important in ensuring that resources are used to support the implementation of programmes that will make a difference to the target group. This reliance on evidence is going to be vital in Covid-19 recovery plans, as programmes to support all those affected by the pandemic are put in place.

However, while the programmes and services that are the subject of evaluation come at a cost, so too do their evaluations. And it is particularly the most robust forms of evaluation, namely Randomised Controlled Trials (RCTs), which carry a hefty price tag. This is one of the reasons it is important to take a graded approach to establishing evidence, starting with more moderate, small-scale efforts before investing in the most resource intensive evaluations.

This stepwise approach to evaluation is at the heart of the Better Start Bradford Innovation Hub, the evaluation partner of the Better Start Bradford project. The Innovation Hub works closely with service providers to support programme evaluation, balancing practical considerations for how to minimise data collection burdens on service providers with the importance of using the most methodologically rigorous evaluation methods.

As part of this collaboration over the past 5 years, the Innovation Hub has worked with BHT Early Education and Training to establish evidence of promise for their home-based language intervention Talking Together, for children aged 2 years. Talking Together aims to instil parents with the knowledge, skills and confidence to provide a communication rich environment to support their children’s language development. All children aged 2 years in the Better Start Bradford reach area are invited to receive an in-home language screening assessment designed by the service, and based on this screening approximately 55% of families are offered the programme. The six weekly sessions of Talking Together are delivered by BHT’s Language Development Workers, Early Years Practitioners with extensive specialist training in children’s early language and communication development. Prior to the involvement of the Innovation Hub, the programme had been running in another part of Bradford for a number of years, but had not been the subject of any systematic evaluation. With funding from the Nuffield Foundation, we carried out a feasibility study as a first step to establishing a strong evidence base for this programme.

Our findings told us a number of things:

• Firstly, we are confident that we can recruit participants to a trial of this nature, and keep attrition within acceptable levels.
• Secondly, we have established that the programme is well received in the community; parents view the programme positively and are happy to allow Language Development Workers into their homes.
• Thirdly, we have evidence that taking part in the trial was rewarding for the Language Development Workers, which demonstrates that evaluation efforts themselves can have positive impacts.
• Fourthly, we have identified the most appropriate assessments to use to measure the outcome of the intervention.
• Finally, we have established initial evidence that the programme may have a positive treatment effect.

These results are encouraging and represent the first step in establishing a strong evidence base for this programme. They also demonstrate the value of using resources to evaluate existing practice, particularly in communities that may have complex needs not well served by currently available, evaluated programmes.

As part of this Nuffield funded evaluation, we also carried out a series of webinars with practitioners, researchers and policy makers about the importance of evidence in decision-making. We asked participants what the most important factors were for deciding whether to run a programme in their area or setting. Whilst 68% of our 47 respondents selected “cost”, 62% selected “evidence-base”. Many participants felt that not enough emphasis was placed on evidence when decisions were made about service provision, and importantly, that evidence was not very accessible to stakeholders.

In our discussions, we found that funding was a key barrier in facilitating evidence-based interventions. While unsurprising, these concerns about funding further highlight how vital it is to use resources appropriately, and we feel this extends to both the selection of services, as well as efforts to evaluate services.

For the Talking Together programme, we feel the current work has shown that further evaluation is justified and important. We hope that this example of a truly collaborative approach between service providers and researchers is useful to others, particularly within the early education sector, where there is still a need for evidence on how to best support young children’s socioemotional and language development. In these times of considerable need and limited resources, investing in establishing evidence is a vital way of investing in achieving the best outcomes for our children.

Read more of our blogs