Conflict data for October

Here's a quick animation of the Armed Conflict Location Event Database's realtime data for October.

I created this with the code:

acl <- read.csv("/Users/david/ACLEDoct27.csv")
acl$date <- as.Date(acl$EVENT_DATE, "%d-%b-%y")
acl <- acl[acl$date > as.Date("2014-10-01"),]

library(anim.plots)
library(maps)

tmp <- anim.points(LATITUDE ~ LONGITUDE + as.numeric(date),  
      data=acl , speed=1, col=rgb(1,0,0,.7), pch=19,
      cex=log(TOTAL_FATALITIES+1), show=F)

saveGIF(
  replay(tmp,
  before=map('world', fill=T, col=terrain.colors(5)[2:5],
     xlim=c(-20, 60), ylim=c(-40, 40)),
  after=legend("bottomleft", col="red", pt.cex=log(1+c(1,10,100)) ,
      legend=c(1,10,100), pch=19, y.intersp=1.5, x.intersp=1.5, 
      bty="n", title="Fatalities")),
  "conflict.gif")

Easy animations in R using anim.plots

After the last two R posts, here's something a bit more useful.

Creating animations in R was hard. Then Yihui Xie wrote the animation package. animation lets you plot individual frames, then record them.

I've been working on a little R package called anim.plots which adds some syntactic sugar to this. It provides a few commands, similar to the standard R graphics, to create animated plots. Here's a quick demo. (The speed is quite slow: this is because I converted the files to animated GIFs. The original animations go reasonably fast.)

library(anim.plots)
anim.plot(1:5, 1:5, col="green") 

To control what gets plotted when, use the times parameter.

x <- rep(1:100/10, 20)
times <- rep(1:20, each=100) # twenty frames with 100 points each
y <- sin(x*times/4)
waves <- anim.plot(x,y,times, type="l", col="orange", lwd=2, speed=2)

You can make incremental animations using the window parameter. Here's a printout of the first 20 plot symbols:

anim.plot(rep(1:10,2), rep(2:1, each=10), window=1:t, pch=1:20, ylim=c(0,3),cex=2, col=1:5, xlab=paste("Plot symbols", 1:20))

The above code also shows how parameters get recycled where appropriate, either to the number of points, or to the number of frames. For more complex parameters, you might have to use a matrix. Here we zoom in on a distribution of points by changing the xlim and ylim parameters.

 x <- rnorm(4000)
 y <- rnorm(4000)
 x <- rep(x, 40)
 y <- rep(y, 40)
 xlims <- 4*2^(-(1:40/10))
 ylims <- xlims <- rbind(xlims, -xlims)
 anim.plot(x, y, times=40, speed=5, xlim=xlims, ylim=ylims,
       col=rgb(0,0,0,.3), pch=19, bg="white")

The window argument is quite powerful. You can use it to create moving plots over time.

## discoveries 1860-1959
xlim <- rbind(1860:1959,1870:1969)
anim.plot(1860:1959, discoveries, times=1:100, xlim=xlim, col="red", xaxp=rbind(xlim, 10), window=t:(t+10), type="h", lwd=8, speed=5)

There's also a formula interface. Here's a plot of some chicks being fed one of four different diets:

data(ChickWeight)
ChickWeight$chn <- as.numeric(as.factor(ChickWeight$Chick)) 

tmp <- anim.plot(weight ~ chn + Time, data=ChickWeight, col=as.numeric(Diet), pch=as.numeric(Diet), speed=3)

Sometimes you need to run extra plotting commands after each frame. You can do this using the replay command:

replay(tmp, after=legend("topleft", legend=paste("Diet", 1:4), pch=1:4, col=1:4))

 You aren't limited to the standard plot function. Here's a histogram with increasingly fine bins:

anim.hist(rep(rnorm(5000), 7), times=rep(1:7, each=5000), breaks=c(5,10,20,50,100,200, 500, 1000))

 And here's how to animate a plot of a mathematical expression:

anim.curve(x^t, times=10:50/10, n=20)

 You can animate contour plots. Here's the map of the Maunga Whau volcano in Auckland, emerging from a sphere. (Well, I had to come up with something!)

data(volcano)
# create a circle:
tmp <- volcano
tmp[] <- 200 - ((row(tmp) - 43)^2 + (col(tmp) - 30)^2)/20
# an animated contour needs a 3D array:
cplot <- array(NA, dim=c(87,61,20))
cplot[,,1] <- tmp
cplot[,,20] <- volcano
# morph the volcano into a circle:
cplot <- apply(cplot, 1:2, function(x) seq(x[1], x[20], length.out=20))
cplot <- aperm(cplot, c(2,3,1))
anim.contour(z=cplot, times=1:20, speed=3, levels=80 + 1:12*10, lty=c(1,2,2))

Finally, here are last week's earthquakes on the world map.

eq = read.table(
     file="http://earthquake.usgs.gov/earthquakes/catalogs/eqs7day-M1.txt",
     fill=TRUE, sep=",", header=T)
     eq$time <- as.numeric(strptime(eq$Datetime, "%A, %B %d, %Y %X UTC"))
     eq <- eq[-1,]
library(maps)
map('world')
tmp <- anim.points(Lat ~ Lon + time, data=eq, cex=Magnitude, col=rgb(
       1-Depth/200, 0, Depth/200,.7), pch=19, speed=3600*12, show=FALSE)  
replay(tmp, before=map('world', fill=TRUE, col="wheat"))

 

Code is available at http://github.com/hughjonesd/anim.plots.  If you have devtools installed you can get it with:

install_github("hughjonesd/anim.plots")

Screw UKIP

... and Cameron too!

I'm sick of this anti-European bullshit, and I'm going to stand up against it the only way I know how.

par(bg=rgb(0,51/255,153/255))
symbols(sin(1:12/12*2*pi), cos(1:12/12*2*pi),
  stars=matrix(rep(c(0.52,1,0.52, 0.4)*1/7, 10), nrow=12, ncol=20, byrow=T),  
  fg=rgb(255/255,204/255,0), bg=rgb(255/255,204/255,0), inches=F, axes=F, ann=F, crt=90)

The proportions are guesses - couldn't be bothered to do the math for what is described here. The colours are right though. I had to use a 10 pointed star to make sure the stars were the right way up.

Simulating infections

Here's a little R simulation of fashion/disease infection/group identity in a population. There are two groups, red and white. Individuals move around at random. Sometimes, they change group. Whether they change depends on the number of their neighbours who are from a different group. Also, individuals who have just changed are "infectious" (or, trendsetters). Individuals who have not changed for a long time are not very infectious, and are also easy to infect.

It's fun to play with the parameters. It would also be fun to implement this with real people, as e.g. a mobile app, using real time movement.

##############################################
N <- 500
maxdist <- .1 # distance within which "neighbours" are counted
maxt <- 240   # number of repetitions
delay <- 0    # time delay between reps
step <- FALSE # ask after each step?
speed <- 0.01 # speed indivs move
decay <- 1    # speed decaycols <- c("red", "white", "pink", "yellow") # last two for changers
##############################################
 prop2changeprob <- function(props) {
  props^2
}
infectivity <- function(lifespan) 1/lifespan
dist <- function(ind, oths) sqrt((ind$x-oths$x)^2+(ind$y-oths$y)^2)
par(bg="gray")

inds <- data.frame(x=runif(N), y=runif(N), col=sample(1:2, N, replace=TRUE))
inds$oldx <- inds$x
inds$oldy <- inds$y
inds$lifespan <- 0 # since last change

for (t in 1:(maxt)) {
  inds$lifespan <- inds$lifespan + 1
  propother <- sapply(1:nrow(inds), function(i) {
    i <- inds[i,]
    close <- dist(i, inds) < maxdist
    if (any(close)) {
      wt <- infectivity(inds$lifespan)
      wt[!close] <- 0
      sum(wt * (inds$col!=i$col)) / sum(wt) # weighted by
    } else 0
  })
  change <- runif(N) < prop2changeprob(propother)
  displaycol <- inds$col
  inds$col <- ifelse(change, 3-inds$col, inds$col)
  inds$lifespan[change] <- 0
  displaycol <- ifelse(displaycol==inds$col, displaycol, inds$col+2)
  inds$oldx <- inds$x
  inds$oldy <- inds$y
  inds$x <- inds$x + runif(N, -speed,speed) + decay*(inds$x-inds$oldx)
  inds$y <- inds$y + runif(N, -speed,speed)+ decay*(inds$y-inds$oldy)
  #inds$x <- pmax(0, pmin(1, inds$x)) # square
  #inds$y <- pmax(0, pmin(1, inds$y))
  inds$x <- inds$x %% 1 # torus
  inds$y <- inds$y %% 1
 
  plot(inds$x, inds$y, col=cols[displaycol], pch=19, cex=.7, bg="grey",
    xlim=0:1, ylim=0:1)
  Sys.sleep(delay)
  if (step) {
    rl <- readline()
    if (rl=="q") stop("Quitting")
  }
}

The Scottish referendum, the EU and the old new ideas

Here's a reasonable case for Scottish independence.

Ignore all the anxieties about the transition, the tussles over currency and oil and nuclear submarines: plenty of nations smaller than Scotland do fine, and in the long run Scotland can do too. The Union was great in its day, but that day has passed. The future belongs to the small, nimble, united nation-state, competing in a globalized market; not to the lumbering imperialist conglomerate that was put together to build an empire with armed force.

Reasonable, plausible, but wrong. We are no longer living in the optimistic post-Cold War world where free trade was what mattered. Old conflicts have returned in force. Today nations need military and economic heft. The UK is outdated, in some ways, but because it is too small, not too big.

The bigger unit that we need is, of course, a strong European Union. Could Scotland not play an independent role underneath its protective shield? Unfortunately, the politics is all wrong. A Yes in the referendum is bound to weaken the Tories and the establishment, strengthen the insurgency of UKIP, and increase the chance that the rump state of Britain exits the EU. That exit would be a crippling blow for Europe. The future of Scotland would then look much less certain and safe.

10 ideas about genetic differences between human groups

Nicholas Wade's book, A Troublesome Inheritance, has created huge controversy by claiming that race is real and explains human differences in economic performance. Here is a response from many genetic scientists, which also includes links to comment around the web. I don't wish to jump into that debate, but just to give a set of thoughts about the issue of genetic differences between races. (Disclaimer: I am not an expert biologist, just an interested amateur.)

1. The fundamental truth of biology is that humans are basically the same.

Humans make up a single species, which interbreeds with itself. When two people reproduce, a new viable human is created, with DNA that results from combining their DNA, more or less at random. This is like taking parts from two cars and building a new car with it: you can't mix a Maserati and a Ferrari, only two Maseratis of the same model. So, all humans are basically the same.

That is not to deny that the differences between humans have profound social consequences - for example, in deciding who gets rich and poor - and are therefore of great interest to us. But a Martian biologist would probably find two humans as hard to tell apart as we find two sheep. Our similarities vastly outweigh our differences.

2. Races are statistical facts, not buckets.

This also follows from the "single species" idea. To say that I am Caucasian or you are African does not imply that everything about us differs, or that any single gene we have is certain to differ. People interbreed. "Race", if it means anything genetically, is shorthand for having (probably) had many ancestors from a particular area of the world. This may result in someone possessing a particular allele with more likelihood; but that is a matter of probability, not certainty.

Socially, people often treat race and ethnicity as buckets, and that can become self-fulfilling. But this is a fact about society not biology.

3. Within that interpretation, there are real genetic differences between races.

This is now uncontroversial. For example, people from some parts of the world are more likely to have the allele linked to sicke cell disease than others; partly because the allele helps protect against malaria.

4. There are also differences between races in genes that affect behaviour and/or psychology.

For example, different alleles of the MAOA gene exist in different proportions among different racial groups. The MAOA gene is linked to risk-taking and violence. Here's a rather long summary of recent research.

5. These differences do not necessarily add up to "differences between groups in psychology being caused by genes".

Most geneticists think that any given social behaviour is influenced by many genes - there is no neat 1-1 link. So, a group that possesses more of one allele (say MAOA-2R) could possess less of another, undiscovered gene which pushes them the opposite way.

There are reasons this could be true. For example, suppose that over an evolutionary timespan there has been an optimal level of risktaking, and that risktaking is increased by (alleles of) two genes. It is entirely possible for one group to evolve to have the risk-taking allele at one locus, while another group evolves to have the allele at the other locus. The different genes will then not add up to different behaviour, which is instead kept similar by evolutionary pressure.

6. Or, some group differences in behaviour may have genetic causes. We will find out soon either way.

Nevertheless, there is no general reason to think that intergroup genetic differences will cancel out 100% in this way. On the face of it that seems unlikely - as Cochran and Harpending say in The 10,000 Year Explosion, it's like expecting a coin to land on its edge.

Science is progressing fast in this area, with new statistical methods and databases coming online. Whether genetics matters a lot or a little, the debate can and will be settled by empirical data - which is one reason I am writing this.

7. Genetic differences are multi-dimensional.

Think of genetics and most people think of IQ. IQ is politically salient because it is linked to economic outcomes, and because it threatens to array racial groups on a continuum, with higher IQ being better. But real differences will be more complex than that. Think about MAOA: aggression is bad, right? But risk-taking, for example in entrepreneurship, isn't always bad. If there are differences between groups, we will not be able to order them hierarchically.

8. Genes are not destiny, in the large.

OK, here I am disagreeing with Nicholas Wade's position in the speculative second half of his book. Think about Caucasians versus Asians. In the 19th century, British gunboats were knocking on China's door. In 2014, China is the second power in the world, and pushing for first place. These changes happen too fast for genetics to be a useful explanation.

9. Genes are not destiny, in the small.

Genetic does not mean unchangeable. To take a simple example, many genetic diseases have simple cures. More generally, human choices, including human culture, can make up for - or perhaps sometimes amplify! - genetic differences. (Culture itself, by contrast, can be very hard to change.)

10. Liberal equality does not require us all to be identical.

Why don't we deny the vote to stupid people? We are equal not because we are all the same in every way, but because - as Christianity would put it - we are all possessors of a living soul. Similarly, we have a duty to judge individuals as individuals, and not on the basis of belonging to an ethnic group. Doing the former is usually efficient, since ethnicity is at best a weak predictor of someone's behaviour. It is also morally right.

It may be - I think it is quite likely - that genetic differences will account for many of the differences between people. They may also account for some of the differences between groups. Given the rate at which the science is developing, we would be very unwise not to start thinking about this. I do not want to be too sanguine about its consequences: obviously it would be, in some sense, "bad news" for human equality. At the same time, it is not a cause for liberal despair, or for racism and intergroup contempt to triumph. Whatever differences there are will be probabilistic not deterministic, multidimensional and complex; and morally outweighed by our underlying shared humanity - a truth supported equally by religious tradition, and by modern biology.

Non-shock: OKCupid experiments on humans, like every other big website

See this post from their blog. Of course, if this were an academic experiment (i.e. one with the purported aim of finding socially useful knowledge), it would have been highly unethical - experimenting on thousands of humans! Without their consent!

Clearly, society does not have an ethical problem with experimenting on humans: we let companies do that all the time. Since it is only academic experiments that have to pass stringent ethics committees, it must be that what is prima facie unethical is learning useful things from experiments on humans. We have taken our cue from Genesis:

But of the tree of the knowledge of good and evil, thou shalt not eat of it: for in the day that thou eatest thereof thou shalt surely die.

This is how all my experiment designs should be

The standard condition consisted of an undergraduate female stationed by a 1964 Ford Mustang (control car) with a flat left-rear tire. An inflated tire was leaned upon the left side of the auto. The girl, the flat tire, and the inflated tire were conspicuous to the passing traffic....

Find out what happens next.

Linkage

From the Iraqi bloggers I have in my bookmark folder. Many are shut down or have fallen silent.
War in Iraq.
Home is all about being the same.

The paedophilia scandal

 

(Back blogging again, just because I feel the need to write something for the non-academic audience, no matter how small!)

There is a furore over paedophilia. Every part of the establishment seems to be under suspicion: the church, all three political parties, the NHS....

Let's think about what happens if, as seems quite likely, many cases are uncovered where paedophiles were protected because they were important people. I fear there will be a serious disproportion between the policy response and the public's response. The policy response will essentially be more safeguards for children, more criminal record checks, more whistleblowing laws. What else could it be? One makes policy to fix problems.

The political consequences, though, would surely be a final, wholesale collapse of trust in the political system. Duck houses on the taxpayer are one thing, but enabling child abuse.... If that happens, the best – best! – outcome I see is that UKIP sweep the board at general election time, bringing Neil Hamilton into office alongside a slew of fishy populist coves. 

But a worse, and just as likely, outcome is: nothing. The duopoly nature of first-past-the-post elections means that no outside party can coordinate the voters to break through. Labour and Conservatives continue to dominate Parliament, with an absolute majority of the electorate not bothering to turn up. Democracy becomes a cynical game of mobilizing the parties' core supporters. There is a vacuum not of power but of authority. Politicians simply lack the credibility to impose change against any group's will. Through the first half of the new century, Britain flies on autopilot.

This foreboding makes me think of Macaulay on Machiavelli:

... condemned to watch the lingering agony of an exhausted country, to tend it during the alternate fits of stupefaction and raving which precede its dissolution, and to see the symptoms of vitality disappear one by one, till nothing is left but coldness, darkness, and corruption.

Nudges enter public consciousness

http://penny-arcade.com/comic/2014/03/31/volt-hunters

Linkage

Interesting perspective on Mariana Mazzucato's book The Entrepreneurial State.

I called it right

Time for some credit-claiming. In 2011 I predicted that immigration would become a live issue in relation to the EU: "there will be increasing pressure to pull up the drawbridges". Yup.

In the interests of balance I should report a couple of verbal predictions that didn't go so well; earlier this year, "don't buy bitcoin until it falls to $20". At the time bitcoin was $100. Sorry, bruv. And, some time in 1996, "this internet thing isn't going to be that important".

To celebrate

To celebrate this week's important Hull news, a special repost:

Interesting ngram

Here's a rather interesting ngram.

Dear literati, please finish Middlemarch

"She was by way of being terrified of him - he was so fearfully clever, and the first night when she had sat by him, and he talked about George Eliot, she had been really frightened, for she had left the third volume of Middlemarch in the train and she never knew what happened in the end...."
-- Virginia Woolf, To The Lighthouse

"Middlemarch/Still lying triumphantly closed..."
-- Don Paterson imagines his deathbed

Linkage

Dammit.

A financier recalls his time at Lehman. Funny and scathing: '... the part of Wall Street that I worked in was simply transferring wealth from the less sophisticated investors... to the more sophisticated.... Of course, the traders had all sorts of excuses and jargon to deal with this truth. “Oh no,” they would say, “We are important providers of liquidity that create stable financial markets. We’re a crucial part of a system. And besides, if we don’t do it, someone else will.” These are the lies that people tell themselves so that they can buy larger homes.'

A common reaction by those introduced to the Public Choice perspective is: what's the point of ever giving policy advice if you assume all politicians are rogues? Acemoglu and Robinson suggest, however, that politics must be factored into policy advice.

Meritocracy, the experience. Also, postmodernism the experience, and drugs, the experience.

¡Steve Bong salutes Lady Thatcher!

Social science statistics: of muscle cars and macro-level statistics.

AMC Rebel 1970, a typical American muscle car

Trying to think through another worry about social science practice, and in particular how we do macro-level empirics - comparing countries, polities or regions. Warning: I strongly suspect that this will mostly prove that I should read more Barbara Geddes. (And who shouldn't?)

Suppose you were a Chrysler executive in 1970, and were evaluating the potential threat from Japanese cars.

Here is an approach you could take. Divide cars into American-made and Japanese-made. Examine the average performance of each. Find that American cars are superior to Japanese cars. Go to sleep contentedly. Inferior cars will never take over our market.

First generation Toyota Corolla, 1966

This approach is obviously flawed. "American cars" and "Japanese cars" are not natural kinds, like "killer whale" and "grey whale". They refer to the deliberate creations of human intelligence. The average Japanese car of the 1960s was not a useful predictor for the average Japanese car of the 1970s. The best car would probably have been a more useful predictor, because Japanese car industry executives were capable of learning.

Now consider how political scientists evaluate the relative performance of democracies and authoritarian regimes. (Or parliamentary versus presidential regimes, or whatever else.) Assume away all the problems of causal identification, unobserved heterogeneity and so on. Even beside all of this: they are basically taking the same approach as above. They look at the average performance of democracies versus dictatorships in the past. To get a big sample, they might go quite far back - to 1950, or even beyond to the 19th century.

But political regimes are also the products of human learning and intelligence. "Democracy" and "dictatorship" are not natural kinds either. They are systems put in place and altered by people. For example, British democracy of 2013 -- with its legal checks on the executive, its relationship to the EU, its devolved governments in the nations, and its quangos and bureaucrats -- is quite different from British democracy in the 1970s -- with its corporatist structure, industrial policy and so forth. Bluntly, both democratic and non-democratic regimes learn, and are constantly rebuilding themselves over time.

For this reason, average past democratic or authoritarian performance may well not be the quantity of interest. Democrats (dictators) should worry about the best, most successful authoritarian (democratic) regimes. Data from the 1950s are likely to have limited relevance.

Toyota Corolla 2011

What's the counter-Thatchual?

The world needs more Margaret Thatcher policy punditry and by God I will provide it!

My colleague Tom Scotto asked a simple question, that I think has not got enough attention in many of the very thoughtful comments from economists and others: "What's the counterfactual?"

In other words: although it is interesting to consider whether

• it was a mistake not to start a Norwegian-style sovereign wealth fund for North Sea oil,
• the monetarist policy of the early 1980s made the recession unnecessarily hard,
• supply-side changes weakened the unions and modernized our economy, or
• her European policy was foresighted, or counter-productive,

a more fundamental question is: what would have happened if Margaret Thatcher had not won power?

There are two natural ways to cut this. If Thatcher had not won power in 1979, then Callaghan would have stayed on. Or, you could look at 1983, where a random shock (named Galtieri) kept her in power. I'm too young to remember this stuff, but I'll make some guesses.

Surely the result in 1979 is easy to call. Callaghan was a failure; he would have continued to be a failure. Evidence from the 1979 Labour manifesto shows that these guys planned to continue down the failed path of the 'seventies.

Now we set ourselves the task of bringing inflation down to 5 per cent in three years. It is an ambitious target. We need the assistance of everyone. 

three-way talks between ministers, management and unions to consider the best way forward for our country's economy...

Industrial democracy - giving working men and women a voice in the decisions which affect their jobs - is an idea whose time has come...

Labour will strengthen the Price Commission, giving it greater powers to initiate investigations and reduce prices ....

We reaffirm the policy that we have pursued that wherever we give direct aid to a company out of public funds, we shall reserve the right to take a proportionate share of the ownership of the company....

1983 on the face of it seems just as easy. Labour ran under a manifesto which has been called "the longest suicide note in history", including withdrawal from the EEC, unilateral nuclear disarmament, renationalisation....

But then Labour was not the only game in town. What if the SDP/Liberal alliance had won? Might we then have had a moderate, responsible centre-Left government, making some necessary reforms but without engendering the division and bitterness that Thatcher left?

I doubt it. My feeling is that politics like those of Schroeder, Blair, even Lula were only possible after Thatcher. The centre-Left had to swallow the bitter pill of accepting (some) New Right ideas. Even the 1983 SDP counterfactual would have been something like France at the same period (or now): half-hearted acceptance that you cannot actually get more Left-wing; but no real reforms.

In any case, the main point is: Thatcherism is best evaluated against real alternatives that might have come about, rather than against the analyst's ideal policy.