A Call for Randomized Clinical Trials of Parachutes

In the tight-sphinctered world of academic medicine, it's always delightful to find a journal that still has a sense of humor. The following satirical paper from the British Journal of Medicine made me laugh and laugh.

Smith GCS, Pell JP. (2003). Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials. BMJ, 327(7429), 1459-1461. DOI: 10.1136/bmj.327.7429.1459

In a nutshell, this paper rightfully points out that no one has ever done a randomized, controlled trial (RCT) on the efficacy of parachutes. Furthermore,

Advocates of evidence-based medicine have criticised the adoption of interventions evaluated by using only observational data.

To be funny, satire has to contain enough truth about its subject to properly skewer its underlying fallacies. Smith and Pell's excellent paper carries the concept of parachute efficacy through the usual machinery of evidence-based medicine, all the way to the following hilarious reductio ad absurdum conclusion:

We think that everyone might benefit if the most radical protagonists of evidence-based medicine organised and participated in a double-blind, randomised, placebo-controlled, crossover trial of the parachute.

From skimming through the comments stimulated by this paper, it seems that not all of the BMJ readers recognized it as satire, even though the article concludes with the following contributors' statement:

GCSS had the original idea. JPP tried to talk him out of it. JPP did the first literature search but GCSS lost it. GCSS drafted the manuscript but JPP deleted all the best jokes. GCSS is the guarantor, and JPP says it serves him right.

A Bit More on Evidence-Based Medicine

The formal term "evidence-based medicine" (EBM), is a relative newcomer on the medical research scene, and is based on early publications by Cochrane in 1972 and Sackett and Guyatt in the early 1990's. Their idea is a simple one: make clinical decisions based on a synthesis of the best available evidence about a treatment.

Just a few decades later, there are way many EBM advocates who are way too ready to blindly wield EBM like a mighty sword without understanding its limitations. IMHO, one of its biggest limitations is the current EBM definition of "best available evidence". For example, consider the Oxford Centre for Evidence-based Medicine Levels of Evidence (May 2001):

Level	Therapy/Prevention, Aetiology/Harm
1a	systematic review (with homogeneity*) of RCTs
1b	Individual RCT (with narrow Confidence Interval‡)
1c	All or none§
2a	systematic review (with homogeneity*) of cohort studies
2b	Individual cohort study (including low quality RCT; e.g., <80% follow-up)
2c	“Outcomes” Research; Ecological studies
3a	systematic review (with homogeneity*) of case-control studies
3b	Individual Case-Control Study
4	Case-series (and poor quality cohort and case-control studies§§)
5	Expert opinion without explicit critical appraisal, or based on physiology, bench research or “first principles”

I agree with the idea that all evidence is not created equal. However, there is something fundamentally wrong with the hierarchical list above. For one thing, it puts the principles of physics, chemistry and physiology down at the bottom of the heap in category 5, below even "poor quality case-control studies" in category 4. According to this table, a crappy case-series showing nerve conduction velocities faster than the speed of light would trump Albert Einstein's thoughts on the subject. I find this a bit troubling.


This is only one of the wacky conclusions that can arise from blindly following the precepts of EBM. I highly recommend Kimball Atwood's great Evidence-Based Medicine Primer on the excellent Science-Based Medicine blog. Therein he ably shows some of the limitations of EBM, and why it is not up to the task of evaluating highly implausible claims such as, say, homeopathy.

A Better Alternative: Science-Based Medicine

IMHO, EBM is a decent start, but is still a work in progress. The next logical step would be move to science-based medicine (SBM), wherein basic science is placed a lot higher on the food chain of evidence. This lets us start with this basic concept:

Therapeutic claims require evidence.

and come up with a somewhat more refined version:

Exceptional claims require exceptional evidence.

In other words, when we evaluate some intervention, basic science can help us determine if there is a reasonable scientific mechanism that supports it. If not, why do the study? And, if someone does the study anyway (it happens), the scientific plausibility (or implausibility) of the mechanism can help us decide how credible are the results.

Let's look at two brief examples: homeopathy and parachutes.

Homeopathy

Homeopathy claims therapeutic benefits from extremely dilute solutions of various agents (e.g. Oscillococcinum). However, the solutions usually employed are so dilute there is not a single molecule left of the original agent. Scientific likelihood of any benefit over that of plain water = zip.


Homeopathy counters this with the claim that the diluting water has a "memory" of the original agent, even though it's all gone. However, there is no scientific evidence of such a "memory" effect. Besides, when one considers how much of our planet's water passes through human kidneys, it seems likely that any possible "memory" of Oscillococcinum would be swamped by just the "memory" of used beer.

Bottom line: scientific likelihood of any benefit of homeopathy over that of plain water again = zero/zed/zip.

In this case, the basic science is so overwhelmingly against any benefit that it seems ridiculous to perform an RCT of homeopathic claims. Even if a marginally weak effect were suggested by a clinical trial, the a priori implausibility of homeopathy makes it hard to put much confidence in the result.

Parachutes

The physics of falling bodies, on the other hand, has been understood for a long time, particularly when there is atmospheric drag. The terminal velocity of a falling human in air is about 55 m/s (120 mph). Scientific likelihood of survival = almost 0% (a few fluke cases have been reported of lucky folks surviving parachute failures).

A properly designed parachute can decrease this falling speed to as little as 2.1 m/s, about the speed you'd develop by jumping off a 9 inch stool. Scientific likelihood of survival = virtually 100%.

In this case, the basic science is overwhelmingly in favor of survival. Therefore, I'm willing to take my chances with a parachute during my next plunge from a height -- RCT or no RCT.