I hate your stupid paper: Al Roth

[This is a new occasional series in which I tell you that your paper is stupid, you are stupid, and I hate you and your stupid paper. My inaugural paper is by... Judd Kessler and Nobel Prize winner and father of experimental economics, Al Roth!!!! Warning: lengthy, for specialists, contains swearing and rhetorical exaggeration.]

Came across this gem while I was doing the prediction market experiment for replications - a cool idea by the way.

Organ allocation policy and the decision to donate

Abstract

Organ donations from deceased donors provide the majority of transplanted organs in the United States, and one deceased donor can save numerous lives by providing multiple organs.... We study in the laboratory an experimental game modeled on the decision to register as an organ donor and investigate how changes in the management of organ waiting lists might impact donations. 

From the paper:

This paper investigates incentives to donate by means of an experimental game that models the decision to register as an organ donor. The main manipulation is the introduction of a priority rule, inspired by the Singapore and Israeli legislation, that assigns available organs first to those who had also registered to be organ donors. ...

Results from our laboratory study suggest that providing priority on waiting lists for registered donors has a significant positive impact on donation. ...

The instructions to subjects were stated in abstract terms, not in terms of organs. Subjects started each round with one “A unit” (which can be thought of as a brain) and two “B units” (representing kidneys). ...

Whenever a subject’s A unit failed, he lost $1 and the round ended for him (representing brain death)...

At this point, I wished fervently for my A unit to fail, representing brain death.

For any non-specialists out there who don't see the problem... fuck it: for the tiny proportion of non-specialists who aren't already laughing at us like baboons.

Organ donation is a complex and unique decision. It involves the choice to have part of your own body cut out, when you die, in the hope of saving someone else's life.

Now it is perfectly reasonable, though counter-intuitive, to model this as just another cost-benefit decision (perhaps including some "altruistic utility"). The sainted Gary Becker did this for crime and the family - both areas not previously thought of as amenable to cost-benefit analysis - and spawned two whole new fields.

And it is also perfectly reasonable to say "No! Organ donation is different. Cost-benefit analysis just won't apply. I don't trust this economic model."

Here's what is not reasonable: to distrust the economic model; and to try to learn what will really happen, by running a laboratory experiment ... which implements the economic model.

Analogy: suppose I have a simple billiard-ball theory of planetary motion. To predict how planets interact, I build a big billiards table with a lot of billiard balls on strings representing the sun, the earth, Mars and so on. I spin the balls, take measurements and write down my predictions. Now you decide my theory is all wrong. In fact, it doesn't even work for the billiard table! You whack the red ball round on its string: it ends up totally not where my theory predicts! Falsification! Karl Popper's ghost applauds.

"Yes," you tell me, "and now just measure the position of that red ball. I want to know where Mars will be next week."

You see the problem? My billiard-ball theory is wrong. But that theory gave the only reason to think that the billiard table could predict the planets. Without the theory, what are we left with? That's right, Perky: balls. A load of useless balls.

Now there are many lab experiments on decision-making that would be relevant to organ donation. We can test theoretical models of, say, altruism and upstream reciprocity. Then, if we reckoned that the theory had captured all the relevant aspects of behaviour, we could apply it to organ donation; make some predictions; maybe try out a policy experiment. The social science lab is useful for this, because you can get "altruism" and "reprocity" into the lab in a meaningful way. But there is no meaningful way to get "organ donation" into the lab, short of a supply of Romanian orphans and a surprisingly relaxed ethics committee. Just having options with analogous payoffs does not cut it.

The authors of course know this. From the conclusion:

Care must always be taken in extrapolating experimental results to complex envi- ronments outside the lab, and caution is particularly called for when the lab setting abstracts away from important but intangible issues, as we do here.

And perhaps the paper's results can in fact tell us something deep about how institutions can tap upstream reciprocity - but that's not what they talk about. Nor do they deal with this head on. (For example, by adding: "It follows that this very interesting experiment tells us nothing about actual organ donation. We were kidding about the title!")  Instead, the introduction uses that weasel word, "suggest".

Roll up folks, for the new experimental methodology! Finally, unbiased causal identification in the social sciences! Drumroll. Spotlight. "Results suggest..." Parturient montes, nascetur ridiculus mus.* If I want suggestiveness, I'll read ethnography.

Here is why this gets my goat. A graduate student once proposed an experiment on global warming. The next century would be a game with 100 rounds. In each round there was a small chance of a "climate catastrophe" if the players didn't implement "mitigation". Mitigation cost a few cents,  climate catastrophe cost about twenty Euros. From this experiment it was hoped to make behavioural predictions about, uuuuh, the future of the planet. Under different policy regimes.

(And - quickly, in one breath - because it was in the lab, the policy regimes were randomly and exogenously assigned. Yeah, thank God there's no endogeneity! That was such a problem with STUDYING THE REAL WORLD.**)

So I stuck my hand up and said that this was nuts. But now, some other young researcher, planning such an absurdity, can say: "Well, Al Roth did it for brain transplants!"

[S]ubjects started each round with one “A unit” (which can be thought of as a brain) ...

 Seriously, how the fuck can people write this shit with a straight face?

* Translated from the Latin, this means "Fuck you and Google it yourself."

** As our authors put it:

The difficulty of performing comparable experiments or comparisons outside of the lab, however, makes it sensible to look to simple experiments to generate hypotheses about organ donation policies.