Commissioning in context

Commissioners are working in the context of austerity and of increasing financial pressures on commissioning budgets. This influences the use of evidence in a number of ways:

• Evidence of effectiveness and cost-benefit can help to justify spending scarce resources.
• However, many interventions with strong evidence come at a high cost, and commissioners are looking to adapt interventions to fit the available resource.
• The drive for innovation and finding new solutions to save money or improve services is in tension with using interventions with strong evidence, but a broader view of evidence can inform logic models and theories of change, preventing innovation being ‘a blind leap of faith’.

Types of evidence

Participants highlighted different types of evidence used at different point of the commissioning process:

• Evidence of local need, derived from national and local data, surveys and qualitative research with young people and families and feedback from practitioners.
• Evidence about ‘what works’, or what might work here, derived from research produced by academics, voluntary organisations and government and informal intelligence from other local authorities.
• Evidence from the market about what is possible locally, given the skills and capacity of organisations, including the local authority itself, to provide the desired service.

Commissioners identified a tension between listening to children and young people locally and using evidence to inform service design. This tension was partly resolved by talking to young people about the outcomes that they wanted to achieve, and then looking for research evidence to find an appropriate way of achieving those outcomes. This helped commissioners to sift through evidence to find studies that provided evidence for those specific outcomes.

What makes evidence trustworthy, robust and useful?

In deciding what weight to give particular studies or findings, commissioners considered:

• Who produces, publishes and reviews the evidence.
• The context in which the study was done, with studies in English local authorities with similar demographics being preferred.
• The methods used, with mixed method studies providing both hard data and user voice preferred to a single method study.
• Participants varied in their level of confidence that they could judge the methodological merits or pitfalls of a particular study.

Commissioners are not only looking for services or intervention with strong evidence, but are also looking for something that ‘fits’ with the local context and systems. Studies that provide detail about how the intervention is delivered and details of implementation allow commissioners to make more informed choices about local fit.

When making choices, commissioners considered:

• What do we already have here? How will those services be affected by any new service?
• What will the demand for this new service be? What capacity does the service need to meet that demand?
• What capability do we have locally to deliver the service? Are there skills gaps that might be a barrier to recruiting the required professionals?

Accessing research

Commissioners both sought out evidence for particular purposes and tried to keep abreast of new research by signing up to information services and research repositories. When looking for evidence, participants reported making use of organisations that synthesise and quality assure evidence to ensure that the evidence they used was robust. This also helped to mitigate some of the difficulty with paywalls to access academic journals. Some authorities were trying to generate local evidence of effectiveness, by evaluating programmes already in place. Generating local evidence of a similar quality to formal academic evidence was felt to be challenging for a number of reasons:

• Agreeing outcome measures and tools across services to get comparable results requires substantial negotiation and quality assurance.
• Evaluating services operating in a ‘real-world’ environment needs to be able to cope with adaptations that occur in response to local events or changes in the wider system.
• Evidencing counterfactuals for individual children and young people is difficult – i.e. being able to say this child would have come into care without this service.

As well as formal academic research and local evidence, commissioners also drew on the experiences of other local authorities, through informal networking – this was felt to have benefits of being able to get more detailed feedback, and in particular, learning what hadn’t worked as well as what had.

The format of research influenced whether commissioners found the time to ‘dig into the detail’

• Executive summaries with key messages were preferred, in order to be able to make a quick judgement about the relevance of the paper as a whole.
• Charts and diagrams helped commissioners to quickly understand the impact evidence.
• There was some interest in alternative formats for research, such as podcasts and videos, but there were often barriers in accessing these formats at work due to technical limitations.

Recommendations for commissioning teams and their managers

• A mix of skills in reading and critically appraising research is important to make the most of the available evidence. Developing these skills should be part of the professional development of commissioners. This can be done through integrated commissioning teams, a corporate intelligence and insights service or cross-authority collaboration.
• Access to research is a crucial tool for commissioning teams. While there are some free and trusted sources, sometimes this requires a budgetary commitment to get access to current and relevant research.
• Getting out to conferences where research is presented, or meeting with commissioners in other authorities, provides opportunities for professional development as well as increased awareness of the available evidence.
• The organisational culture around use of evidence will shape how individual commissioners work. Giving commissioners a clear steer about the extent to which new services should be ‘evidence based’ will help them to prioritise the most useful research. Are you looking to innovate based on theory, or adapt an existing programme to fit your local system? How should commissioners balance evidence-based services with co-design and user voice?
• Consider the different services you commission and the culture of using evidence within those service areas. Do you use the right language and consider the right research to best influence these decision-makers?

Strategic partnerships with the voluntary sector: Executive Summary

About the project

This report is the output of a Research in Practice Change Project involving five local authorities and their voluntary sector partners. The partnerships were involved in developing innovative services for children and families. This briefing explores the nature of the relationships between public and voluntary sector bodies when working in partnership.

The Change Project sought to explore how local authorities and their voluntary organisation partners were attempting to move away from a transactional approach, in which services are highly specified, closely monitored and delivered in an atmosphere of competition, rather than collaboration. A literature review supported the development of a framework for interviews with the partnerships and key messages were developed and refined at a workshop attended by all participants.

The participants

Detailed case studies of each partnership are provided as appendices to the main report.

Key messages

Shared goals and values are the foundation on which successful partnerships are built. Knowing your partner believes that the same things are important, and is committed to achieving them, helps to move away from a ‘transactional’ approach to a ‘partnership’ approach. The participants in our Change Project chose voluntary sector partners who shared their values:

• A desire to do things differently and to take risks in order to improve outcomes for children and young people.
• A commitment to ongoing learning and development.
• A commitment to making the best use of skills and capacity within the partnership, without being constrained by organisational boundaries.
• A willingness to challenge local and national government and other policymakers to make the innovation possible.

Shared understanding of the problem you are trying to solve requires a conversational approach to defining the problem and potential solutions.

• Before the formal procurement process, the local authorities involved in our Change Project invited potential providers, professionals and children and young people to talk about their experiences and what needed to change. Being willing to be involved in this open dialogue was an important indicator that potential partners wanted to be involved for the right reasons.
• The partnership approach required a different approach to service specifications, with the procurement process identifying outcomes to be achieved, a shared vision and values, and giving the partnership the freedom to develop responses together. This takes time, and as a result contracts were for a much longer period than the traditional one to three year contracts.

Innovation and partnership require trust and mutual respect between partners. Contractual arrangements cannot specify all possible circumstances, and trusting that partners would respond appropriately and constructively was seen as a key criterion for entering into an innovative partnership. One partnership noted that if they got to the point where they were scouring the contract to assign blame or force action then the partnership would be considered a failure.

The partnership culture appears to have stimulated genuine practitioner ownership and engagement in service delivery. The sparks of innovation are ignited at all levels. (IPC, 2017 p4)

Trust between partners develops at different levels, but is based on repeated interactions in which partners demonstrated their values and commitment. The process of resolving disagreements was seen as equally important as the outcome.

• Initially, partners were selected that had features suggesting the organisation could be trusted - organisational trust is derived from governance structures, reputation and prior experience of working together.
• Senior leaders had to develop a trusting working relationship to get past the starting blocks and good communication, honesty and authenticity were all felt to be crucial elements in developing this trust.
• Building a trusting relationship between staff at all levels took longer, and required senior leaders to be visible in promoting the partnership and consistent in how they managed conflict and challenges. Transparency was key.

Embedding the partnership into the wider stakeholder network is crucial and partnerships spent considerable effort in securing the buy-in of elected members and trustees; other Council departments; regulators and government; and other agencies and providers. Partners need to take some time to understand each other’s stakeholder network, and how these stakeholders influence how the partnership operates.

Stakeholder engagement cannot be an after-thought. Taking a new approach to contracts and procurement required upfront engagement with legal and human resources departments, while close collaboration in practice requires IT support, as well as clear referral pathways.

Agreeing an approach evaluation that encourages dialogue and learning supports flexibility and responsiveness. Partners developed monitoring activity and data collections that reflected their shared values, including

• Collaborative definition of outcome measures.
• Formative evaluation that provided recommendations for further development.
• Shared discussion of individual cases to support a shared understanding of ways of working.
• Negotiated problem-solving of operational issues to feed into wider learning about the way the system works.

When defining the partnership and ways of working together, the process is as important as the outcome. Every interaction between the organisations shapes the culture within the partnership. Collaboration and dialogue at every stage was seen as crucial in developing mutual understanding and respect. The Change Project provided senior leaders of the organisations with time and space to reflect on the process itself, and their own role in building trust.

Tips and recommendations

Lessons for the sector

1. “Get senior leadership buy-in from Day One”. It will make everything else easier.
2. “Build a partnership to last a childhood”. The level of investment and system change needed to make a significant impact on outcomes takes time to pay off. Transformative partnerships will struggle to deliver within three year contracts.
3. “Understand your role”. Draw the line between commissioning and delivery and navigate that boundary together. Local authorities need to know when to step back and when to step in.
4. “Be flexible”. Willingness to adapt to changing circumstances is a sign that the organisation is in it for the long haul.
5. “Know the system”. Partnerships don’t operate in isolation, but are overlap with numerous other networks. Draw on their expertise and build wider collaborations.

Tips for building relationships

1. ‘It’s about relationships all the way down’. The quality of the relationship at the top of the partnership is crucial, but relatively easy to develop. The relationships between operational staff, and between staff and managers in each organisation are important too, and require more time and effort to embed.
2. ‘Every interaction matters’. Every time you talk, you are communicating something about the relationship, as well as the content of the conversation, Think about how your words, body language and attitude are influencing how your partner sees you.
3. ‘Model the behaviours you want to see’. Let staff and stakeholders see what effective collaboration looks like, show them how you have constructive, challenging conversations. Choose the right language and use it consistently.
4. ‘Don’t be afraid to get to know your partner’. Talk about things other than work, develop a shared sense of humour, show them you care about their personal as well as professional life.
5. ‘Use the right format for the conversation’ – telephone calls for speed, face-to-face for challenging issues.