Within and between variation

Here is a very interesting post about violent crime and climate change. (There is a paper behind it. Hat tip to Andrew Gelman.)

I think we should be skeptical, for the following reason: within-variation is not between-variation. A helpful analogy is with income. It is widely reported that within a country, richer people are happier, but that above a certain level richer countries are not happier countries. (Which may well be false, but leave that to one side.) The putative explanation is that happiness comes from being richer than others around you, not from just being richer.

In the climate context, it may be that there is a fixed amount of crime in a society, of which more happens on the relatively hotter days. You can tell a physical story to fit this - maybe people get used to the mean annual temperature and react physiologically to deviations. The paper, surprisingly, does not discuss this issue, and indeed touts its array of county-month and county-year fixed effects as a virtue.

Unfortunately, a lot of cross-country regressions in the social sciences involve hoping that variation within a dataset of country-years from maybe the 1950s onwards (why are some countries now richer/more democratic?) will explain variation in a much larger universe of data (why did some countries get rich/democratic in the first place?) Since history is not much like a repeated set of experimental trials, this is just not a very plausible strategy.