Lies, Damn Lies and Statistics!

Just recently the National Treatment Agency for Substance Misuse put out a press statement - see the NTA website for details - trumpeting the fact that a key government target (doubling the number of people in drug treatment) had been achieved 2 years early.

There than followed a lot of detail about substances used, retention rates, ethnic groups etc. Important as this information is (or might be - if it was ever used for anything!) I then spent a very frustrating 30 minutes looking for anything remotely resembling outcome data. There wasn't any.

I finally went back to the NTA website and found another report released on the same day giving outcomes for the year before. It turns out that for the year 2004/5:

• 160,453 clients received 238,149 episodes of "structured care". (About 1.48 treatment episodes per individual!)
• Opiate dependency accounted for 75% of presenting clients. This was comprised of Heroin (64%) and Methadone (8%) and other (3%).
• 57% of clients remained in treatment for 12 weeks or more. 52% of clients received treatment described as "prescribing modalities" and there was apparently an increased likelihood of these clients being retained in treatment. In other words we are more likely to retain opiate addicts in treatment if we give them drugs! Now there is a suprise!
• Of those individuals "discharged" from treatment during the year 30% had successfully completed treatment. However when one looks at this 30% it turns out that only 6% (5,759 treatment episodes - not individials - are shown as being "Completed Drug Free". (You can be "drug free in the UK and drink 2 bottles of spirits a day - alcohol doesn't count!) 11 out of the 30% "successfully completing treatment" were "referred on" - in other words the individuals had not completed treatment at all! As for the other 13% who "successfully completed treatment" without being drug free I can only assume that they "successfully completed treatment" still using drugs!

A large part of the paper is about "factors associated with retention and succesful discharge". While I agree that the study of such factors is important this report makes comments like: "The proportion of episodes which were successfully discharged was greatest for those with Chinese clients (40%), whereas only 20% of those episodes where the client was Bangladeshi were successfully discharged". The numbers of Chinese and Bangladeshi clients "successfully discharged" were 29 and 143 respectively. I personally believe that making sweeping assertions about treatment outcomes based on samples this small is unjustifiable.

How many agencies (let alone non-english speaking workers) worked with these 2 groups? With numbers this small I would need convincing that the differences do not have more to do with the effectiveness of the agencies concerned, or perhaps even the effectiveness of a few workers, rather than the client's ethnicity! That the UK government bases it's "evidence based approach" to drugs treatment on such dubious logic which resembles an O-Level project rather than any real science is alarming! Where are the controls?

Lest I get so carried away here that I forget the point of all this - in the year concerned £457 million was spent on drug treatment (ignoring the money spent in prisons) and in England at least there is only evidence that 5,759 episodes of treatment ended with the client being discharged drug free.

That's £79,354 each! The total money spent could have bought something like 100,000 individuals for 13 weeks residential treatment in South Africa- yes, South Africa, and I am pretty sure that we could reasonably expect perhaps 50,000 of those individuals to have "Successfully completed treatment drug free".

Why do we let the NTA trumpet such appaling outcomes as some sort of triumph? It's actually more like a cause for despair!