Tag Archives: randomised trials

The end of the hegemony of randomised trials

I accept the argument that randomised trials provide evidence that is not biased by subtle differences between people getting the treatments that are being compared. Risk adjustment does not solve the problem of allocation bias. The problem is that modern medicine has become, to some extent, a victim of its own success.

When I was a medical student, there were a limited number of questions that everybody was asking and the headroom for improvement for most diseases was large. Now, as I try to stave off the living death of retirement, I find that for every question we had 40 years ago, there are dozens of subsidiary questions. Think of the treatment of angina – in those early days we might have wondered whether aspirin was a good thing. Now we have not only aspirin but other antiplatelet medicines; we have other forms of blood thinning; we have stents (of various kinds) and surgery; we have lipid-lowering drugs and anti-inflammatories. All of these may be used in different combinations and in patients with comorbidities, such as diabetes, in the old and in the young and for different durations. The number of permutations is enormous – we are experiencing ‘question inflation.’ And not only are there more questions, but the headroom for improvement gets less as more and more medicines get added to the list.

So we have lots more questions and smaller headroom for further gains with each question. Remember, if you halve the effect size that a trial can detect, you must quadruple the sample size, other things remaining the same. The corollary of all this is that a point must be reached where it becomes difficult, if not impossible, to mount adequately-powered trials – some questions dip below the knowledge horizon for randomised trials.

What is to be done? Firstly, indirect comparisons are going to become one of the main epidemiological methods in overviews of studies. Next, second best is going to have to become best; we’re simply going to have to glean some of our knowledge from large prospective databases. Ironically, the hypothesis-free way in which modern clinical databases are constructed may offer some protection against allocation bias – a point perhaps to be explored by methodological research. Such databases are crucial for detecting unintended effects of treatment and may also help identify topics of greatest priority for randomised trials by detecting larger than expected intended effects.

Lastly, we’re just going to have to get used to living with lots of questions to which there is no perfect answer, or indeed no answer at all. Making decisions under uncertainty is an ineluctable part of all professional practice, including medicine.

A call for an evidence-based criminal justice system

The Government announced this week that it plans to recruit mentors to help people leaving prison make a successful transition to be law-abiding citizens. The scheme is aimed at those who have spent less than a year in jail who currently do not receive any substantive support. The mentors will be ex-prisoners who have not reoffended and understand the challenges faced. The Government thinks this scheme could be provided by social enterprises and that making payments dependent on results achieved – the extent to which recidivism is reduced – will result in better outcomes and value for money for the taxpayer.

This sounds like a good idea but the scheme should be properly and rigorously evaluated. This would enable realistic targets to be set for the performance of the scheme. The most rigorous way to do this would be a randomised trial.

But do I hear you say this would be unethical because it would mean withholding the promising mentoring system from half the potential reoffenders? The answer is no –Radio 4’s Today Programme made it clear that there aren’t enough mentors to go around.

The RCT therefore is not only a scientific tool but also a method to allocate resources in a fair, egalitarian way. The RCT, in other words, would kill two birds with one stone.

The Cabinet Office has recently issued a discussion document entitled Test, Learn, Adapt: Developing Public Policy with Randomised Controlled Trials advocating evidence-based public policy and use of randomised trials to evaluate government policies and programmes. While there is some evidence that mentoring can reduce re-offending, this tends to be based on studies with young people, or in some cases young people at risk of offending rather the general prison population. There are some excellent examples of undertaking randomised control trials on using mentors for new mothers and coping with serious illnesses. Here’s a case where the Government could live up to the high scientific standards it has set for itself.