• By Dartington SRU
• Posted on Sunday 18^th May, 2014

How important is the organisation?

How much do the characteristics of the organisations that provide evidence-based programmes matter for outcomes for children and young people? It's a question that is rarely asked, as evaluations typically focus on the characteristics of the young people in the intervention, and not on the context around them. A study from 2012 takes a rare look at organisational effects – with some surprising results.

Although the relationship between many organisational characteristics and outcomes for young people was neutral on average, in some cases positive characteristics of organisations were associated with negative effects on young people’s behaviour and functioning, one year after an intervention.

Sonja Schoenwald, from the Medical University of South Carolina, led a team of researchers to investigate how the characteristics of different organisations influenced the long-term success of an evidence-based treatment. They followed up nearly 2,000 young people one year after completing Multisystemic Therapy (MST), an evidence-based treatment for serious antisocial behaviour. In the study, MST was provided through 45 organisations in the US, involving over 400 therapists treating delinquent youths with or without substance use problems.

MST is not intended for substance-using youths by design, but this problem co-occurs highly in the target population of delinquent youths. As this co-occurrence is difficult to treat, and may require additional support, Schoenwald, who serves on the board of MST Services, and her team wondered whether the organisational factors might be even more important in these cases.

Understanding the effect of organisational characteristics is important in the transfer of programmes from research to usual practice settings. In a scientific trial, researchers can more closely control the setting, but in the practice context these environmental factors exist that can exert additional influences on outcomes.

Providers were assessed on organisational climate and structure, both measured by therapists’ own ratings. Organisational climate includes job satisfaction, fairness, role clarity, and emotional exhaustion. Organisational structure covers the degree of therapists’ participation in decision-making, the hierarchy of authority, and explicit rules and procedures governing employee behaviour.

Youth behaviour and functioning problems were assessed with two questionnaires completed by caregivers. The Child Behaviour Checklist measured internalising (withdrawn, anxious and depressive) and externalising (delinquent or aggressive) behaviour. The Vanderbilt Functioning Inventory assessed antisocial behaviour, self-harm and problems at home and with peers.

Results were surprising. In a few cases, positive characteristics of organisations were associated with negative effects on young people’s behaviour and functioning a year after the MST intervention. But most organisational characteristics were associated with a wide range of youth outcomes, and were, on average, neutral.

Some factors had a different influence on substance-using youths, who require greater organisational support, compared to non-users.

Perhaps the most surprising finding was that youths’ externalising behaviour and functioning problems were actually higher when MST was provided by organisations with high levels of job satisfaction, cooperation, growth and advancement, and participation in decision-making.

Similarly, some negative organisational characteristics – role conflict and therapist emotional exhaustion – were associated with improvements in behaviour and functioning problems among substance-using youths.

Other positive organisational aspects, such as clearly defined rules and procedures, led to a less surprising reduction in youth functioning problems. But most of the measures of organisational structure and culture showed no clear effect on youth outcomes.

One of the puzzles from this study is this: why did organisations that provided “growth and advancement” for their therapists tend to have worse outcomes for youths’ externalising behaviour? The researchers offer one potential explanation: a possible mismatch between the characteristics of organisational success and those important for implementing MST with fidelity.

For instance, Schoenwald and colleagues say, growth and advancement of therapists may be linked to increased therapist skill and freedom to decide to adapt certain practices for particular clients. But adaptation in programmes like MST, which are designed to be implemented exactly as specified, may lead to poorer outcomes.

This point is especially apparent when therapist caseloads included youths with substance use problems, who are more difficult to treat. Where success was found alongside greater therapist emotional exhaustion, it may suggest that extra effort was spent on these cases.

If therapist adherence is key, better training and ongoing clinical and organisational support may be needed – perhaps particularly when therapists work in organisations with a positive climate and structure, and where they may feel they need less support. Notably, however, previous research has not supported the idea that positive organisational characteristics combine with poor adherence to affect outcomes.

Schoenwald, S.K., Chapman, J.E., Henry, D.B., & Sheidow, A.J. (2012). Taking effective treatments to scale: Organizational effects on outcomes of Multisystemic Therapy for youths with co-occuring substance use. Journal of Child & Adolescent Substance Abuse, 21, 1, 1-31.