At the time of writing this page, this is a collection of things I've written in the past, collected together in a semi-random order.

The Trouble With The Insistence On RCTs

If one insists on things being shown by RCTs, then anything that can't be shown by an RCT won't be observed. It is easy to fall into the trap of believing that everything of importance can and must be established through an RCT. But this is at best a convenient assumption, not a reality established by evidence and reason.

The trouble with demanding that things be shown by RCT is the following:

1. An RCT can only detect a correlation between experimental groups and outcomes, and only in an overall average way.
2. When mind and brain and their inherent complexity cannot be neglected, the number of relevant variables to be controlled for is astronomical, and so real world trials can never hope to control variables sufficiently to produce results that one can put their trust in.

Thought Experiments

I want to consider a few simple hypothetical settings to try to illustrate the problems one faces in trying to apply the 'evidence based medicine' paradigm to psychiatry.

Classifying Pain

Suppose we classify pain based solely on where it occurs in the body: the head, the torso, and the legs. So if you are experiencing pain, you are experiencing either head pain, torso pain, or leg pain, and any two cases of torso pain are believed to be the same thing.

Suppose then we want to trial some treatments for torso pain. Now we select some experimental subjects who are suffering from torso pain. (We do not try to differentiate torn muscles or ligaments, angina, stomach ache: it's all just 'torso pain' so far as we are concerned.) Now anything that specifically treats angina will have no effect on people whose torso pain is something else. Most likely the only 'treatment for torso pain' that gave consistent results is some kind of strong painkiller. Those suffering from angina may well die of heart problems of some kind, but such things will get filtered out by the statistics.

And we can repeat such trails over and again, and get similar results, so we end up with a 'strong evidence base' supporting the idea that 'strong painkillers are an effective treatment for torso pain'.

I'll leave it to you to work through the long laundry list of what's gone wrong here. My fear is that in psychiatry the same thing is going wrong: we trial 'treatments for mania' without considering the possibility that two instances of 'mania' may be similar only in some of their outer symptoms.

Two instances of 'torso pain' may be similar in the sense that pain receptors are involved, and we may regard torso pain as being caused by a chemical imbalance in the pain receptors in the torso, and so see painkillers as an effective remedy for this chemical imbalance.

When it comes to 'mania', no consideration is given either to what caused a particular mania, nor whether different instances of mania may be qualitatively different. Indeed two instances of mania in the same person may be very different in nature and cause.

Randomised controlled trials of treatments for mania can't help much here. I then think it's a similar issue with 'hearing voices', or the many symptoms associated with diagnoses like schizophrenia and psychosis. Moreover, I suspect that, just as the complexity of the brain and the number of qualitatively distinct behaviours it can exhibit are astronomical in their scale, if not beyond astronomical, so the size of a sufficiently precise 'diagnostic manual' cataloguing the various things that can go wrong in the brain and suitable treatments for them will be similarly vast in scale and completely impractical to compile. In short, diagnostic manuals like the DSM are way too short to be accurate.

My Concerns

My fear is that the current RCT-based approach to psychiatry, with a diagnostic manual giving a very small selection of problem categories, and treatments for illnesses meeting those diagnostic categories sought via trials of remedies in RCTs, is the result of researchers implicitly assuming (without proper scrutiny or justification) that the research paradigm that works well with physical medicine necessarily works equally well with psychiatric medicine.

The trouble here is that this effectively sidelines both the role that the myriad individual differences between one person and another have to play, and the potential need for 'treatment' to be interactive, and to draw on more avenues of support than the medical system can provide. By restricting what is measured to what is easily measured, the many significant factors that are hard to quantify are easily ignored, often without conscious thought as to the matter. By not restricting things thus, the task of evaluating treatments becomes intractable. And so we end up with the 'evidence based medicine' paradigm applied naively to mental health, and front-line medical professionals with an unjustified confidence in what they are taught, oblivious to myriad potential foundational issues underlying the reasoning supporting their beliefs and practices. That's my worry. These potential errors in the reasoning process are obvious and natural to someone with a background such as mine, and I have seen no evidence thus far that either typical medical professionals, or even those in research, are fully aware of such issues and their implications.