Parachutes

Hanno Böck

hboeck.de

@hanno

Horia Varlan, CC by 2.0

Smith, Pell 2003

In 2003 a joke meta analysis on the effects of parachutes was published

Result

We were unable to identify any randomised controlled trials of parachute intervention.

This is an exceptionally well known paper, but what does it actually tell us?

Some things are so obvious that rigorous scientific studies are neither necessary nor ethical

This is almost trivially true, but you can also easily see where this ends

This drug is very effective.

There are no rigorous scientific studies that show any benefit of this drug.

So you also would not use a parachute if you fall out of an airplane? Because they haven't been tested in rigorous studies as well.

So maybe we should read this paper in a different way

When I stronly believe that something is true then rigorous scientific studies are neither necessary nor ethical

Hypothesis

Are people using this parachute trial as an argument for medical interventions that do not work?

Hayes, Kaestner, Mailankody, Prasad 2018, CMAJ Open

The parachute analogy is a poor one for most medical interventions

Falling from an airplane has only 1 causal pathway leading to harm. In contrast, most human diseases have multifactorial etiologies

Parachutes improve survival from nearly 0% to nearly 100%. Empirical analyses show that few medical practices offer so large a magnitude of benefit.

  • The authors looked for scientific citations of the 2003 parachute metaanalysis
  • Checked if there were claims that a medical practice was like a parachute
  • Checked if a randomized conrolled trial was performed in the meantime

822 citations of the 2003 paper

35 with a claim that a practice is akin to parachute

22 with one or more RCTs performed

6 cases: trial confirmed practice

5 cases: trial rejected practice

5 mixed results, 2 halted, 4 ongonig

Refuted claims

  • Empirical deworming therapy of children in endemic areas has clinical benefit
  • Stenting for renal artery stenosis benefits some patients so greatly that it is a parachute
  • Stenting for renovascular hypertension
  • Treating diabetes with agents that lower blood glucose will invariably lead to improved cardiovascular outcomes
  • Intracranial pressure monitoring for severe traumatic brain injury provides useful information that improves outcomes

Criticism of the paper

While I like the overall idea of the paper, the analysis does not seem particularly detailed and there is an obvious error in the supplementary table (duplicate entry for a claim)

Result

When people use the parachute analogy in scientific studies in about half the cases where these claims were later tested they turn out to be wrong

Summary

Although there is widespread interest regarding the BMJ paper arguing that randomized trials are not necessary for practices of clear benefit, there are few analogies in medicine. Most parachute analogies in medicine are inappropriate, incorrect or misused.

We can conclude that people are using the parachute analogy unjustified

The Parachute meta analysis has very successfully used a joke paper to criticize demands for rigorous science

Unfortunately it is a flawed argument, Hayes et al have done their best to scientifically refute that argument

This has not changed the fact that the flawed reasoning of the Parachute paper is still much more popular than the scientific refutation