I accept the argument that randomised trials provide evidence that is not biased by subtle differences between people getting the treatments that are being compared. Risk adjustment does not solve the problem of allocation bias. The problem is that modern medicine has become, to some extent, a victim of its own success.
When I was a medical student, there were a limited number of questions that everybody was asking and the headroom for improvement for most diseases was large. Now, as I try to stave off the living death of retirement, I find that for every question we had 40 years ago, there are dozens of subsidiary questions. Think of the treatment of angina – in those early days we might have wondered whether aspirin was a good thing. Now we have not only aspirin but other antiplatelet medicines; we have other forms of blood thinning; we have stents (of various kinds) and surgery; we have lipid-lowering drugs and anti-inflammatories. All of these may be used in different combinations and in patients with comorbidities, such as diabetes, in the old and in the young and for different durations. The number of permutations is enormous – we are experiencing ‘question inflation.’ And not only are there more questions, but the headroom for improvement gets less as more and more medicines get added to the list.
So we have lots more questions and smaller headroom for further gains with each question. Remember, if you halve the effect size that a trial can detect, you must quadruple the sample size, other things remaining the same. The corollary of all this is that a point must be reached where it becomes difficult, if not impossible, to mount adequately-powered trials – some questions dip below the knowledge horizon for randomised trials.
What is to be done? Firstly, indirect comparisons are going to become one of the main epidemiological methods in overviews of studies. Next, second best is going to have to become best; we’re simply going to have to glean some of our knowledge from large prospective databases. Ironically, the hypothesis-free way in which modern clinical databases are constructed may offer some protection against allocation bias – a point perhaps to be explored by methodological research. Such databases are crucial for detecting unintended effects of treatment and may also help identify topics of greatest priority for randomised trials by detecting larger than expected intended effects.
Lastly, we’re just going to have to get used to living with lots of questions to which there is no perfect answer, or indeed no answer at all. Making decisions under uncertainty is an ineluctable part of all professional practice, including medicine.