Just recovering from a couple of nights' hard partying in Harrogate. (Don't laugh.) Chris Sorsby is off to Canada so me, Dave and Chris Baehrend have been up to see him.
I'm off to New York, then Ann Arbor on Thursday, where I'll be attending the University of Michigan Summer School - course on advanced game theory. Perhaps I'll finally develop some half-decent formal modelling abilities.... Stranger things have happened. There are many cool things at UMich, as well as Robert Axelrod (the Evolution of Cooperation guy) and Leda Cosmides and John Tooby (progenitors of evolutionary psychology) they have Juan Cole, a historian with a great blog about the Iraq war. (Hmm, scribbling this on Chris' PowerBook which although very beautiful does not seem to have a web browser that lets me put links into blogger. Juan Cole is at http://www.juancole.com .)
Sunday, 17 July 2005
Wednesday, 13 July 2005
SPSS is a crock of crap
The people who wrote the user interface for SPSS should be burned to death with matches. That is all.
Tuesday, 12 July 2005
A little sample from my current paper
(The context is a discussion of cues and heuristics in voting - ie how voters can make decisions without being hugely knowledgeable about politics. But I think this is interesting anyway. It is partly inspired by the economics literature on "informational cascades". This is just a very simple example.)
If I trust that you know better than me how to vote, I may vote correctly. But I am certainly not helping make the democratic majority decision any more accurate. Under certain conditions, I may even make it less accurate.
Here's a simple formal example. Suppose that there are three voters on a simple Yes-No issue. One of these outcomes is unequivocally the right one. The voters each receive some information (a signal) about whether Yes or No is the right outcome. The accuracy of those signals varies: the voters have probabilities 0.7, 0.75 and 0.8 respectively of getting accurate information which suggests they should vote in the correct way. Also, these probabilities are known to all the voters in common, although the signals themselves are private. Suppose that the voters vote independently, and try to vote the right way. They all then follow their signals, which are more likely to be right than wrong. The chance of a correct decision is the chance of all three voting right, plus the probability of three different majorities of two voting correctly: 0.7*0.75*0.8 + 0.3*0.75*0.8 + 0.7*0.25*0.8 + 0.7*0.75*0.2 = 0.845.
Now suppose that the best informed voter, Mrs Point Eight, announces what her signal is (which way her information points) before the vote. The other two voters are then each in one of two situations. Either their signal agrees with Mrs Point Eight: if so they vote the same way anyway. Or their signal disagrees. If so, they now have two conflicting signals. But Mrs Point Eight's signal is more accurate than theirs, so they sensibly choose to ignore their own signal. In either case, they vote with Mrs Point Eight. Unfortunately, this means that the chance of reaching a correct decision has gone down to 0.8.1 By trusting a better-informed person, the individual voters have become more accurate (their chance of being right is 0.8 instead of 0.7 or 0.6) but have made the collective outcome less accurate.
Saturday, 9 July 2005
Monday, 4 July 2005
Cambridge at the weekend
Down to Jesus college at the weekend for a 10 year anniversary dinner. There were only about 20 of us so I suppose I am now officially qualified as a Cambridge old fart. Yikes.
Anyway, it was a psychedelic evening. Thanks to Juliet Banfield for putting it on, and a big shout out to old friends and also new, or rather recently remade - hello Holly, Mike, Susie, et al. A night when I make a spectacular ass of myself, my dears, can still give me such intense happiness that I spend most of the drive back home actually singing. (Perhaps this is bad in some way?)
As usual, there are photographs, some of which involve Dave Padua and are sexually ambiguous. Watch this space.
Anyway, it was a psychedelic evening. Thanks to Juliet Banfield for putting it on, and a big shout out to old friends and also new, or rather recently remade - hello Holly, Mike, Susie, et al. A night when I make a spectacular ass of myself, my dears, can still give me such intense happiness that I spend most of the drive back home actually singing. (Perhaps this is bad in some way?)
As usual, there are photographs, some of which involve Dave Padua and are sexually ambiguous. Watch this space.
Friday, 1 July 2005
Tuesday, 28 June 2005
Google scholar monitor
If you have the Firefox web browser, and regularly use Google Scholar, then check out this wonderful little bit of tech. It basically adds a "saved searches" box to Google scholar, which lets you see if new articles have been added. Very useful for monitoring research. You'll need the Greasemonkey extension for Firefox. I love this idea of consumers customizing web pages.
turned on by car insurance policy documents?
Then maybe you should read this red hot little number. (Just something I'm looking at at the moment. No, really, you do not have any reason to read this document.)
Thursday, 9 June 2005
Game theory of killer initiatives and legislative responses to initiatives
(Just something I jotted down - a sample of the kind of work I do.)
A common institution in the context of direct legislation is to allow the legislature to respond to a proposed initiative. I put forward a game-theoretic rationale for having this institution, but point out some potential problems. The problems also apply to cases where interest groups outside the legislature sponsor “killer initiatives” which are drafted so as to affect other initiatives on the ballot box.
The rationale is as follows. We consider policy on a single dimension. The status quo is SQ and voter median ideal point is at MV. All utility functions are given by distance:
utility = |outcome – ideal point|
We assume that proposing an initiative costs C, and that the potential proposer is extreme, with ideal point marked PIP. MV' marks the point [MV + (MV -SQ)], i.e. the median voter is indifferent between MV' and SQ. To simplify, we assume that indifferent voters vote “yes” on all proposals.
SQ--------------MV-------------MV'----------------PIP--------
If the legislature may not respond to the initiative, and if MV' – SQ > C, then the proposer proposes an initiative at MV', which the voters accept. Policy is now as far from the median voter as it was before, but in the opposite direction.
Now suppose that the legislature can respond to proposed initiatives by bringing forward its own counter-proposal. If it does so, then the counter-proposal, rather than the status quo, will be the result if the ballot proposal fails. The legislature incurs no cost in bringing forward a counter-proposal: this seems reasonable, as it has no signature requirement to fulfil. Most likely the legislature's ideal point LIP will be at SQ: however, we consider all alternatives.
SQ--------LIP-----MV-------------Mv'----------------PIP--------
If LIP < MV: any proposal to the right of MV can be countered by a proposal to the left of MV, but closer to MV. Therefore, if MV – SQ > C, the proposer proposes MV; otherwise the proposer does nothing.
If MV' > LIP > MV, any proposal to the right of LIP will be met by a counter-proposal at LIP. Therefore, if LIP – SQ > C, the proposer proposes LIP; otherwise the proposer does nothing.
If PIP > LIP > MV' (ie legislature and proposer are both extreme), and if LIP – SQ > C, again the proposer proposes LIP or any proposal to the right. The legislature then responds with LIP and offers the public the unappealing choice between two proposals which are both less preferred to the median voter than the status quo. The result is LIP.
If LIP > PIP > MV' then proposer proposes PIP and legislature responds with PIP or any proposal to the right; here again, proposer and legislature collude to offer the public a Hobson's choice. The result is PIP.
In the latter two cases, a rule like the one outlined makes outcomes worse from the point of view of the median voter and the associated majority in his or her lea. However, these seem rather implausible: why shouldn't the legislature legislate directly, rather than take the roundabout initiative route? In the more plausible first two cases, the result is always better for the median voter. Allowing legislative counter-proposals essentially turns the contest into a classic Downsian two-horse race.
Potential problems
The institution as proposed above is rather simple. However, it doesn't always work like that. Switzerland allows the legislature to bring forward proposals in response to an initiative, but rather than replacing the status quo, they are voted on separately. If both the original initiative and the legislature's response pass, the legislature's proposal overrides the original initiative. (I think. Hey, this is a blog entry.) In a less institutionalised, but broadly similar way, “killer initiatives” are sometimes seen in the US. These are drafted so as to override the effect of another proposed initiative: again, if both pass, the killer initiative will specify limitations to the effects of its target.
We can model both these situations similarly, as proposers 1 and 2 decide in turn to propose initiatives. If both proposers introduce initiatives which pass, we assume that 2's proposal is implemented, as it contains clauses designed to nullify the effect of 1's proposal.
The choice situation facing the voter now becomes complicated and may require sophisticated voting – something that, it is often believed, voters in a direct democracy find difficult (e.g. Lacy and Niou 2000). That paper points out a similar result when preferences over two initiatives are non-separable. In this case, preferences over outcomes are single-peaked and the space of outcomes is one-dimensional: the problems arise because voters don't know what the effect of their vote will be. As an example, consider the following choice situation where two initiatives have been proposed:
---SQ--------------P2-------------P1---
Given single-peakedness, there are four possible preference profiles for voters:
SQ > P2 > P1
P1 > P2 > SQ
P2 > SQ > P1
P2 > P1 > SQ
The last two groups of voters have no problem: they can vote yes on P2, and yes or no on P1. A voter who happened to be decisive on either outcome could not possibly experience regret, as they would always have brought about an outcome which they preferred to the alternative – either by achieving the best possible result, P2, or by preventing P1 from supplanting the status quo, or by possibly allowing P1 to win (assuming P2 was not implemented).
The “extremists” in the first two groups are not so lucky. Consider first those who wish to preserve the status quo. A voter (or group of voters) who happens to be decisive on P2 faces the following set of choices:
| P1 outcome | Vote | Result |
| Passes | Yes | P1 and P2 pass: outcome P2 |
|
| No | P1 passes: outcome P1 |
| Fails | Yes | P2 passes: outcome P2 |
|
| No | neither pass: outcome SQ |
Clearly, if P1 is going to fail, it's better to vote no on P2 and preserve the status quo. Equally, if P1 will pass, you should vote for P2 in order to salvage something.
Partisans of P1 face a similar problem. They should vote for P2 if P1 is going to fail, but against P2 if P1 is going to pass. The problem in terms of votes is analogous to that mentioned in Lacy and Niou (2000). For example, a member of the first preference group has the following preference ordering for outcomes of the votes:
| P1 | P2 | Result | Ranking |
| N | N | SQ | 1 |
| N | Y | P2 | 2= |
| Y | Y | P2 | 2= |
| Y | N | P1 | 3 |
Thus, despite single-peakedness over the actual issue, we end up with non-separable preferences over the outcomes of votes.
Wednesday, 8 June 2005
R notes
These are for myself but also public.
R project for statistical computing, in case you didn't know.
Adding a straight regression line to a plot
model = lm(y ~ x)
plot(y ~ x)
abline(model)
Adding a wiggly line to a plot
plot(foo ~ bar)
lines(lowess(bar, foo))
Tuesday, 7 June 2005
Monday, 6 June 2005
Friday, 3 June 2005
A very large cock
This is at Hoxton City Farm, which is 100 yards from Brick lane, but like a different planet.
New York
No lobsters were harmed during the making of this picture.
The Guggenheim staff told me off for this.
The Guggenheim staff told me off for this.
Fitter, happier /
more productive /
( NOT like a cat / tied to a stick / that's driven into / frozen winter shit...)
The French and Dutch No's.
For a few months I was a volunteer in the Simon Community, which deals with rough sleepers in London. There was a resident there who was a lot of trouble. He was very shrewd and insightful, and he could be extremely aggressive. Anyway, if he thought he was being messed around somehow , he would look straight at you and say: "I'm not a prick."
Brussels tried to get the European electorate to vote for a long, turgid, incomprehensible constitution - oops, sorry, constitutional treaty. (It was a constitution because everyone had to vote for it, but it was long and boring because it was a treaty. Got that?) If they didn't, the sky would fall in, there was no plan B, et cetera. To which the reply came: "I'm not a prick."
One more thing while I'm ranting. Various important Brussels people have been publicly puzzling, agonizing, over what the two No's can possibly "mean". Here's a clue: they mean No, you numbwits.
Thursday, 2 June 2005
france0002.thumb
Like an existentialist philosopher, you say? Hmm... I suppose I would call myself kind of deep.
Sunday, 29 May 2005
Exteriors after Mondrian
I went to the Barbican today and saw an exhibition called Colour after Klein. The bottom floor was famous contemporary artists doing their thing, very formal and conceptual. The top floor showed me a revelation, a video documentary by this European artist called Anri Sala about an Eastern European city (Tirana, I think) where the artist-mayor had had the buildings painted in these bright primary colours, abstract designs almost like Mondrian in places on a city-wide scale. So what? Well the mayor was very interesting. He was trying to revive and give some sense of civic identity to this very poor, very brutalised place that had suffered from 70 years of tyranny. He made a comment about trying to turn this from a place where people had to live, to a place where people chose to live. He described the colours as a way of creating the first steps towards a democracy. In fact, he used the phrase “an avant-garde of democracy”.
To go back to my previous entry about culture, the idea of culture as not just a brute fact about people, but as something with a moral purpose, has been discovered before. As I mentioned the 18th century was very articulate about this, but another time which I had forgotten about was the start of the 20th when artists developed the concept of the avant-garde. The avant-garde led and others would eventually follow. This could be applied within, but also beyond, the sphere of art. So strangely enough here was this fellow on the very outskirts of European culture – I mean, Albania, FFS – and the ideas that made sense to him were those of the daring arrogant modernists. In fact, his conception of democracy was explicitly artistic. He raised, and dismissed, the idea that the way to colour a street was for each person privately to choose his own favourite colour and then to have “some kind of golden mean”, which would of course turn out to be “grey”. Now that is just the idea of democracy that one strand of modern political science has embraced – a way of aggregating people's preferences. And it does indeed, in the best case, lead to a kind of grey middle-of-the-road moderate median voter party, or so the theory runs.
The other strand of thinking about democracy that is currently fashionable is “deliberative democracy”, the idea that coming together in a debate can change people's preferences. I am a little sceptical of deliberative democracy, but that is for another day. The important thing for the moment is to note that the deliberative democrats are usually very egalitarian, usually explicitly egalitarian. It's all about debate in which everyone gets a say and everyone must be listened to, sometimes even people who don't actually have logical arguments must be listened to because otherwise it would be oppressive. (Okay, I am compressing a fairly intelligent argument into a caricature.) But if we look at the shape of current democratic polities, realistically the debate is going to be vertical not horizontal, between political leaders in the pulpit and the public on the ground, and perhaps even more stratified than that – between political leaders and cultural elites (avant-gardes) and the masses. As a way to think about this, the idea of the leader as artist has something to recommend it.
Of course that idea has a long and rather questionable history. It tends to be linked to futurism and to Fascism, maybe Communism too, with forebears in Machiavelli and Rousseau. (Leo Strauss wrote on this topic, I think.) But you don't need to go that far to see the potential dangers of thinking in this way. Another interesting comment by this mayor was that the painting was a way of undoing not only the brutalising effect of communism, but also the “brutal” way in which individuals in the apartment blocks in these streets had added bits on to their dwelling places – balconies and so forth – without respect for their neighbours. This seemed to evince a lot of contempt for ordinary people and their selfish little desires – it reminded me of Le Corbusier designing his “machines for living”, and then the inhabitants doing shocking things like installing window boxes, or hanging out their washing. And indeed the colours merrily spilled from one house and flat to another, which you could see as bringing people democratically together, or you could see as violating their desire for a space they could call their own. We did not hear during this documentary from any of the people in the coloured flats.
But then the mayor may well be right: after all political science and politics exist because, in the presence of externalities, individuals cannot always work things out satisfactorily on their own. I am tempted to say that culture and cultural leadership is a specific strategy which is suited to some times but not others. At the same time I am aware that the original debate, at the end of the 18th century, between “institutionalists” and “culturalists” was one in which “culture” was developed at least partly as an excuse by incumbent elites who did not like the challenge of Enlightenment to reform their institutions in order to make them more rational. This is much the same debate that I believe is now going on between the World Bank and IMF and their critics, and indeed between the American hot gospellers of the democratic way and those canny country specialists in the State Department and the Foreign Office (and probably their equivalents in every government) who make much play of their knowledge of the special unique facets of each particular people. So to make the case that culture matters, and cultural leadership needs to be nurtured, seems like an interesting research topic.
Friday, 27 May 2005
Tuesday, 24 May 2005
How social science is different from natural science
Some people believe that social science is fundamentally a different kind of enterprise from natural science: in fact, they may even avoid the science word and talk about “social theory” or similar. Others see no difference at all. There is a conventional story which the former group tell to justify their position, and it goes like this. Studying humans is fundamentally different from studying natural objects because human actions have meaning. Therefore, the methods of natural science are inappropriate.
As it stands, this clearly needs a lot of filling out. In itself it has no more prima facie validity than the following claim: studying fish is fundamentally different from studying other natural objects because they live in water. Therefore, the methods, etc. What's so special about meaning? Different kinds of answers are given.
The first answer would be: “to understand human actions, we need to understand their meaning. Understanding is different from explanation and is the appropriate goal for the human sciences: we need to see the internal logic of people's actions. Natural science can't help us here.” This argument is weak. Suppose that, by an application of the laws of supply and demand, plus a certain amount of practical measurement, I can predict what people will pay for corn this season. This does not seem to involve understanding, but it is still very useful knowledge. So, I deny the “appropriate goal” claim. There are lots of cases where we could sensibly wish to know what humans will do, whether or not we understand the why of it.
A better answer: “predicting what humans will do is a matter of finding out what they think they ought to do. But the best way to do that is to work out what they really ought to do. This is not an empirical matter.” There are different ways of parsing “ought” here: in terms of simple goal-oriented reason, or in more complex moral (but perhaps culturally specific) terms. The first approach – if you want X, Y is the best way to get it – is taken by economic theory. Economists of the Austrian school, like Hayek and von Mises, thought that this made economics (the fundamental social science, they thought) into something completely different from natural science. Their thought on this matter is far from pellucid, so this may be a misinterpretation. The second approach involves working out how people think about a subject in a more nuanced and sensitive way, perhaps believing that the structure of ideas which motivate action is different in different cultures, and attempting to trace this structure. This is the approach taken by the notorious cultural studies. There are as many theorists of cultural studies as there are practitioners, but the practical approach can be summed up like this: to understand a society, apply the methods you learnt in secondary school for understanding a novel. Figure out people's ideas – their logic, perhaps the flaws or concealments in that logic, their rhetoric and linguistic tricks.
Both these approaches – yes, cultural studies too – should be taken seriously. But there are better descriptions of what practitioners in the field are doing. For economics (and rational choice theory generally), Daniel Dennett has put it very convincingly as follows. We could predict what humans will do using the methods of natural science, but it would require a very detailed knowledge of the physical state of our test subjects, and might be completely wrong given only small errors in that knowledge. Working out subjects' goals, and predicting their actions as optimal given those goals, is an immensely powerful shortcut. Note the implication. If rational choice theory is a shortcut, then it may in certain fields be wrong. In fact it would be astounding if people always and everywhere behaved with complete optimality. If so, then we need to test our theories, and that means we are well within the fold of normal experimental science. And in fact, most economists try to back up their theories with empirical evidence, and have devoted great effort to working out how to do this – hence the importance of econometrics. With regard to the second school of thought, which prefers sensitive contextual analysis to mathematical simplification, we can give the same story and make the same demand. If you think that understanding is a shortcut to prediction, that's fine, but in any given situation you will need to prove it by doing empirical tests. This need is even stronger than for economists. Most economics is derivable from a hypothesis that humans behave optimally (a less contentious description than “rationally”), plus some descriptions of human goals. And a lot of economics assumes the same basic set of goals for humans: maximize your wealth or your well-being as a risk-averse function of wealth. The hypothesis plus the basic goals have been tested in a wide variety of situations and have performed very well – pick up any journal of empirical economics for examples. So in many areas we could reasonably have a presumption in their favour. The cultural studies approach, by its nature, is less general and more culture- and context- specific. Any specific claims about how humans will act in a situation are going to come with less prior support, because they are less closely related to other previous work. So it is even more important to test them.
One more explanation of how social science is different is worth mentioning. It was put forward by Habermas. We could do social science like natural science. But science is about learning to control and manipulate objects. Learning to control and manipulate humans is ethically dubious. The social science that we ought to do will have “an interest in liberation”. It will therefore not simply aim at explanation and prediction of human behaviour: instead it will help humans understand their situation better. This argument is ethical rather than epistemic. We could aim at explanation, but we shouldn't.
Habermas is right to be worried about the dangerous potential of science, but he is drawing the border in the wrong place. There is physical science, like research into chemical weapons, which is dangerous and ought not to be done (although there may be tragic dilemmas where it is better for “us” to do this science rather than wait for “them” to do it). This science may in predictable ways make human lives worse. Specifically there is physical science that may help the cause of human oppression – the development, say, of torture techniques or truth drugs. On the other hand, explanatory social science, despite making human behaviour more predictable, can aid the cause of human liberation – normally because it may help us to exert collective control over individual actions which harm human freedom. If we can predict military coups, perhaps we can stop them; if we can explain crime, perhaps we can prevent it.
In other words, there is not much in Habermas' argument beyond the old idea that science has huge power for good or ill. I wish scientists were more aware of that and more thoughtful about the implications of their research, but it is physical scientists who are most in need of this awareness. Perhaps childlike curiosity is just the way good science has to be done, and we will always need other people to play the role of worried parent. (We certainly have no shortage of candidates for this role, most of whom, it must be said, are visibly not up to the job.) In any case, there is no reason to divide social science from physical science on the basis of this argument.
So far I have dismissed various types of claim that social science is fundamentally different from natural science. In fact, I don't believe that they are fundamentally different. If you still do, I would invite you carefully to consider the following areas of scientific endeavour, and decide which side of the dividing line to put them, or come up with some explanation of how they can involve work on both sides of this supposedly impassable chasm:
epidemiology
the study of flows of motor traffic
neuropsychology
Having said all that, I do think that there is something rather different about social science – not a dividing wall, but some unusual characteristics. My idea does better at explaining what is unique and different about studying human beings, while paying respect to the underlying unity of science. It also seems to fit better with the kind of worries people have about social science.
The Bayesian image of knowledge is this: a subject has a prior belief expressed as a probability of statement X being true. He or she then receives some new information – perhaps as a result of his or her own actions, such as performing an experiment or analysing some statistics – and updates probabilities accordingly. The Popperian image of science is based on the idea that scientific laws are general statements, that is, claims about an infinite number of scenarios. We cannot therefore prove them: we can only disprove them, and science is just those disprovable statements that have not yet been disproved. A scientist must actively try to disprove his or her own theories by means of experiment. How the Bayesian and Popperian ideas fit together is not completely clear to me, but the important point is the image they have of the subject. This is a unitary image. News from the outside world is instantly integrated into a subject's belief system as priors are updated or laws rejected.
As an explanation of how we ought to find things out, I have no problem with either of these. As an explanation of how people do in fact come by what they call “knowledge”, the Bayesian idea is rather like the idea of rational choice. It is an immensely powerful and useful shorthand for a much more complicated biological process, the mysterious process whereby the human brain acquires beliefs as a result of things happening in the external world and then inside the brain itself. This shorthand is essential. But it has a drawback. It gives a very misleading impression of how human brains really handle knowledge, and this has consequences for how we can realistically expect to do social science.
I invite you to evaluate for yourselves the following claim: you know a great deal about the world. In particular, you know far more about the social world than could possibly be deduced from any theory you explicitly, consciously hold. Here are some examples of what you know, if you are a reasonably well-informed person:
Britons are great animal lovers
The mobile phone has made social interaction much more fluid
When taking a late night minicab, it's wise to agree a price beforehand
Nixon was a crook
Et cetera. We know much more than is in our minds at any one time. Human beings are not like CPUs receiving inputs of facts and updating a central belief system: they are more like sponges of memory, impressions and information which is stored away willy-nilly, sometimes easily accessible, sometimes nearly forgotten.
So what? Well, for physical science, so not very much. Physical science more or less starts where our common knowledge of how physical objects work leaves off: Galileo pointed out in his dialogues that, counter-intuitively, objects dropped from a ship's mast in motion did not fall backwards relative to the ship. In other words, very little of our existing lay knowledge is much use when we are doing physical science. The important knowledge is in the existing corpus of physical science, which has already been carefully digested and squared with scientific theories. (Although even here, it is often true that a new theory appeals because it provides an elegant explanation for an already-known fact.)
The situation is very different for much social science. Our explicit knowledge of physical objects is rather limited; our implicit knowledge, that displayed by a footballer or a hunter, is perhaps much greater, but it cannot be put into statements and made available to test our theories. But humans are experts on society. We know how people work in general, and what people have done in particular. And we are able to put that knowledge into words, in order to pass it on to others. We are shrewd observers, and fabulous gossips. (How could we not be, when evolution demanded it?) So, when somebody puts forward a theory, or when we put one forward ourselves, we already have a huge bank of existing knowledge to draw upon in order to test it. What is more, in my experience as a social scientist, the most illuminating theories have been those which sum up my existing knowledge in a new and elegant pattern. That is how I felt when I discovered the idea of cognitive cascades, for example, or Olsen's logic of collective action.
This does not make the fundamental epistemic structure of social science any different from that of natural science. It is still a matter of testing our beliefs against the facts. But it does mean that many of the facts are there for us already. We do not in the first instance need to go out into the world and do experiments because we can falsify many theories off the top of our heads.
A straightforward consequence of this is that much conventional social science does indeed have the fault, that lay observers often ascribe to it, of being an attempt to prove the bleeding obvious. Examples are too common to need multiplication: the research that confirms that men do less washing up than women; that American community leaders are better educated than average. These are things we already knew. An equally straightforward consequence is that a lot of the most interesting social science does not busy itself too much with evidence-gathering, but just lays its claims out and leaves them to be backed up by what economists call casual empiricism.
At this point objectors will be queuing up. Let me dispose of one objection and accommodate some others.
A strong objection is as follows. “You say that we have this vast bank of existing knowledge. But unless it has been through the scientific process, it isn't knowledge at all, but just belief. For example, you claim that Nixon was a crook/that Britons are great animal lovers. But this is a highly contentious claim. I, a social/political scientist, have rigorously evaluated the evidence and discovered that Britons love animals no more than other nations/Nixon was the paragon of high-minded patriotism.” As an aside, it must be said that social scientists like nothing better than to claim to possess knowledge unavailable to the well-informed lay observer; this esoteric knowledge, all too often, turns out to be a contentious or overblown reinterpretation of the existing public facts, but it serves its purpose of making us feel important and knowledgeable. Never mind. Isn't the objection valid? How can we really be sure that our beliefs about the world aren't fundamentally mistaken, and isn't that the point of doing social science?
My answer is given originally, I believe, by Donald Davidson (though I may well be bowdlerizing him terribly). It is perfectly true that some of our beliefs can be mistaken, and we would be arrogant to expect anything else. The mistake comes in generalizing from that and thinking that all, or even the majority, of our beliefs can be mistaken. The great majority of our beliefs must be true: otherwise, they could not be identified as beliefs at all. Somebody who is systematically mistaken about everything would be unintelligible to us. We only come to understand other people's beliefs on the basis of some shared truths which we can assume they are referring to. This is true both for the anthropologist struggling to understand a foreign language, and for the ordinary conversation in which a shared grammar and vocabulary are not enough to guarantee correct communication of meaning. (Suppose grandma starts to talk about how Churchill fought the strikers, and won the war in the Falklands. At first you mildly remonstrate that it was just a battle in the Falklands. Then you realize she's confusing Churchill with Thatcher.)
Of course this doesn't mean that humans cannot be systematically wrong about quite large areas of belief. Science in general would be pointless in that case. For example, people routinely forget about the knock-on effects of taxes: most people instinctively assume that an increase in the top rate of tax will bring in more money, but it isn't necessarily so. And in general, much of economics is built upon working through the collective consequences of individual behaviour more rigorously than we are used to do. Similarly, there are quite well-known and predictable theoretical mistakes and practical misjudgments that people make – they take account of sunk costs, or of the framing of a particular problem. (Tversky won his Nobel prize discovering some of these.) I am merely saying that in many areas of social science, humans (and particularly the well-educated social scientist) have the facts already.
This first objection, then, is overblown. But some others would be quite valid. So, I am not suggesting that we should never explicitly test our theories. In fact, everything should be tested – I am just suggesting that we have an existing mechanism already in place for doing it. But even so, there are plenty of claims which do need explicit testing, where our intuitions are not enough or do not exist at all. And no claim can possibly suffer from being tested. My argument is modest. We are working with an image of science that does not quite fit what we actually, inevitably do. Social scientists don't think up theories out of the blue, by random inspiration, then test them. 9 out of 10 random thoughts can be rejected at once, and the remaining good thought often makes such intuitive sense – fits so well with our existing, highly accurate and nuanced knowledge about the world – that the testing is a formality. I wish that social scientists would give more priority to the generation of what is insightful, and less to the rigorous testing of what is obvious or nearly obvious.
There is a further point which comes from considering what social science is for. Rather often, we are not in fact making claims about an infinite universe. Instead, we are examining how a particular process actually turned out at a particular time in history. For example, we can make scientific claims about “elections” as a general phenomenon, claims that need to be validated every time people vote on a course of action. But just as usefully, we can try to understand British elections or American elections – particular finite sets of events under specific historical conditions. When we do the latter, we are not talking for the benefit of eternity, but trying to help people understand what is going on right now. Here again, rigorous testing may be less important than finding the non-obvious. Intelligent public opinion thinks that Iraq damaged Blair and Labour in the last UK general election. Public opinion might be completely mistaken. But, if you are looking to expand the public mind, there may be better avenues to explore than testing this hypothesis to destruction. I hope I am not slaughtering anybody's baby out there.
The mainstream of social science has rejected the arguments of those who want to cut off the study of society from the methodology and ideals of science as a whole. I rejoice at that. It's a wonderful feeling to be part of such a grand collective endeavour – much more fun than being a big fish in a small pond. And the empirical sciences of man have many interesting results, and are getting more interesting by the day with our deeper understanding of human nature. But this fundamental unity shouldn't force us to think of what we do in ways borrowed from particle physics. Like all the sciences we have our own particular ways of working, and our own place in the wider universe of useful speech acts. That's really worth celebrating.
Back home
I'm back and recovering from jetlag in Sally Bradshaw's very nice Highbury flat, where I am cat-sitting. I missed the departmental meeting. Bad GTA rep. I did mean to go up, but I just lay down at 10 o'clock and basically woke up at 3.
New York, more thoughts.
Everywhere I went, the food was awesome, just in little random places. In fact the only bad food was at grand locations like on 5th Avenue and at the Met. I had the best bagel of my life, the best sushi of my life, a spectacular Greek salad. The best Greek food ever – fabulous octopus – pity about the singing though. (Some cultural forms don't deserve to survive.)
Which urbane sophisticated hipster failed to go to the MoMA because he confused it with the Met? Even after actually arriving at the Met and spotting the signs for exhibitions of Etruscan and Roman art?
The Guggenheim is beautiful and had a small but perfectly formed display of really excellent early C20. My favourite period.
Things that will make Londoners feel at home in New York: lots of Japanese tourists, cheeky sparrows and randy pigeons, foreign languages audible everywhere. Things that will make you feel foreign: politeness, the squirrels (they have stronger forearms and look more like Disney cartoon squirrels), all the different American accents.
The real difference you notice with London is the civic pride that New Yorkers have. The city really has a sense of identity. Whereas in London, we just grumble.
Sunday, 15 May 2005
more NY
well, some of my stereotypes are holding up better than others.... I walked down to the Empire State building today, from Tzvika's v swish flat on Central Park West. (Does he actually own this place? Please god, tell me he doesn't.)
I know it was sunday and the park and everything but I have to say it doesn't so far seem like a fast paced city. It's positively chilled compared to London. People are taking their kids for a stroll everywhere. It's very clean and feels really safe. I guess if you walked through Hyde Park or Regent's Park you might feel the same. But even down in Times Square it is fairly relaxed and friendly.
Exciting too though. I saw a man trying to fish in Central Park with a little green mechanical fish on the end of his line.
I went up the Empire State building. It seems like a well-run fool-money separation organization (note to Americans: kind of like all of England, right?) but the view at the top is wow. It was extremely peaceful.
As for the women and investment bankers... yeah, that still seems bang on the money. I would add one other category: joggers. These beings are mostly found in the parks. Joggers are not really in the whole mating game: they reproduce asexually. A lot of joggers together is called a demonstration.
tasers
A friend of a friend has been trying to sell taser guns to the UK police. The guns deliver a strong electric shock via a couple of implants shot into a suspect's skin.
Now Amnesty International say about 100 people have died in incidents linked to the taser in the US, and it turns out that one member of a supposedly neutral panel to evaluate taser safety was a paid consultant for the company.
The US police can already shoot people with real guns, so perhaps tasers are a good solution for them. We should oppose them in the UK.
Apropos of my previous post, the current Prospect has a lot of good stuff about this: an article by two home office policy wonks giving, as far as possible, just the facts; an article on (the mythical?) Middle England; Bhiku Parekh and David Goodhart on racism.
Anyway, I've arrived now. The flight in was spectacular. Contrails drifting by past a deep red sunset, lighter and darker blue cloudbanks, then the New York lights. London looks like a big ugly sodium monster from the air: it literally looks evil. LA looks flashy but square – there are swimming pools everywhere. New York looks fragile like fairyland. The streetlights throw soft glowing circles. Then you drop down over the water and down towards it and bam, there comes the Christmas-tree coloured runway. I should buy one of the I HEART NY T-shirts because I'm smitten already.
Given what I said about the world's knowledge of America, here is a summary of what I already know about New York. The population of New York is made up of 2 groups: frustrated single women and investment bankers. There are many more of the first group than the second, about 9 to 1 in fact. Sometimes the two get together and become Smug Marrieds. Smug New York Marrieds are, obviously, twice as Smug. However the women are still frustrated as their husbands are always out investment banking.
There are also some artists, who live in Greenwich Village and have blue hair. However artists are all also women (and hence single and frustrated) so this is not really a separate group. There is also the bird lady as seen in Home Alone. Her marital status is unclear.
Everyone in New York talks very fast and is rude but with a heart of gold. New Yorkers drink coffee and stage art happenings or flashmobs. This is when a bunch of artists get together and point at an investment banker, perhaps so as to enable one of their number to sneak up and marry him.
Two Chekhov short stories which the young and ambitious and idealistic ought to read: Ward Six and The Black Monk. And two if you're in love: Lady with Lapdog and About Love. People think Chekhov is an ironist, which he is, but being suspicious of beliefs does not preclude you from having them, and in fact, in his unguarded moments, he has a very clear and urgent agenda: in his less powerful works (like About Love) the message is more straightforward.
Blogging At O'Hare
So the conference is over. It was a bit of an emotional rollercoaster[*]. To the casual observer it might look like just a bunch of poli sci wonks in a room at a conference centre, but it was my first proper conference and my first “proper”
presentation. (No offence to my Essex friends who came to my PhD
colloquium.) Like a debutante's coming-out ball.
-
My presentation was good
But my paper wasn't
Anyway, people were extremely nice – nice enough to lift me out of the gloom of not having anything interesting to tell people after two years (or 29, depending on your POV). Big shout out to the Columbia students, all those who came out
and got drunk last night, and all the older people who were kind enough to chat to. Also a big shout out to Tiffany, my brother's friend who is in MSU & the army and who helpfully drove me away from the conference centre so I could vomit my unwisely eaten breakfast into a municipal rubbish bin, rather than, say, in full view of my academic peers. Tiffany apparently wants to be in the CIA
and also an entertainment lawyer, so she can sue people and then kill
them. She also owns a gun, no, several guns and has promised to take
me to the shooting range when I come back to Michigan in August.
Screw “advanced game theory”, I want to learn some really
useful skills: next year the PhD colloquium will have an
excitingly unpredictable element.
I'm going to braindump now. In New York in two hours. Kind of frazzled but not yet ready to let my feet hit the floor.
Everything in America is efficient.
It's kind of scary. It makes me feel rather defensive. This society
is close to mine, but ten or twenty years ahead. (Not always in a
good way, they are fatter for example.)
MSU has 55000 students. Check that out. These universities are vast. I can't comprehend how things get organized on that scale. They have a green fountain there. Green is the MSU colour. I have a T-shirt that says Michigan State and Tiffany laughed when I said in the UK they wouldn't realise it meant MichiganState University.
The work I saw was more professional than mine. I need to raise my game. I've started thinking about what Albert and Hugh think I ought to do, which is write something theoretical. People seemed to like the theoretical bit of my paper...
better than the other bits. I am not sure how well the ideal of direct democracy sits with the idea of democratic leadership. On the other hand, I suspect governors are less weakened by the initiative than legislatures.
Nobody had clever econometric suggestions.
Americans are very good at making presentations. I guess they start them at an early age. They are confident, to the point, and articulate. On the other hand, sometimesit seems a bit workaday.
After listening to these guys, I get the feeling that PoliSci is a huge discipline, about which I basically know next to nothing.
I saw a prom. I've seen tens of proms already of course – in the movies. Everyone in the world gets a guided tour of the American imaginary. It makes the actual society all the more surprising, when you go there. (I've been to LA loads but for obvious reasons that doesn't exactly count in this context.)
The prom was just like the movies, though – except that lots of the guys were wearing pink diamonds, pimp hats... loads of bling.
Cultural studies people tend to think of culture as a sea that you swim in, something
you can't escape. I see culture as a set of tools. But they are tools
like money rather than tools like an axe – they rely on other
people, they can't be used on their own. (Co-ordination games.)
If everything is just institutions, then political science has a simple task: find the
best institutions and put them in place across all states, perhaps
with the aid of powerful actors. (Making appropriate allowances for
differences, of course, institutions are not one size fits all.)
There's clearly a lot of mileage in this idea – who wouldn't
want an independent central bank? But what it forgets is that if
peoples don't figure things out for themselves, they won't have
self-respect.
Why do people in the US care so much about democracy? Because it's theirs.
Conservatives tend to loathe and mistrust the rational choice approach to social reality – Kenneth Minogue is on record grumbling about it – and maybe the feeling is mutual. But I suspect there are very clear explications of some conservative arguments in terms of people's interests and motivations – if you delve deep enough.
How do you get status? What counts as behaviour worthy of respect? Within certain
limits, that's a coordination game.
People think of culture as falling into two categories: low culture (eating beetroot
for dinner) and high culture (Cezanne). Low culture is everyday, and
everyone has it equally. High culture is rarefied and eclectic, and
only a few people have (an interest in) it. In fact, “high
culture” almost refers to the same things as “art”.
Perhaps we should think back to the eighteenth century: the ladder of
commerce and the empire des modes. Culture is not possessed
equally by everyone. It's possessed by elites. (A question asked at
the conference: why did Arkansas voters not pay attention to Arkansas
political elites? But is that bunch of local politicians an “elite”?
Is Colchester town council an elite?) But culture is everyday. Or at
least it isn't just about that prestigious hobby, art. Culture is the
way your elites behave: fashion, the French king planting potatoes
under armed guard, the English middle classes trooping to church on
Sunday (observed by Mr Eliot). That can shape a society.
* so I apologize for what I said about young ambitious people in my previous post. Everyone at Lansing was super nice.
Friday, 13 May 2005
in Lansing
so it is, by my laptop's BST clock, 3.40. I've been travelling for a while and just been to the reception. I'm fairly whacked.
Being in America is hard at first because it is just different enough to be uncomfortable. India or China would maybe be overwhelming. But this is... just a bit off what you expect. In addition, I guess I have a bit of an inferiority complex about America. They have - and this is obvious to anyone with any sense from the moment you get off the plane - they have a lot of stuff right that we just have not yet cottoned on to. They are a rich and powerful and basically contented nation, with a massive sense of purpose. So when I feel uncomfortable with thing X or thing Y, I never know - is it real, or is it just me trying to pick holes? But in any case, I get vibes that make me need to play English music and use sarcasm. Ah well, wherever you go you are never satisfied.
The reception was interesting. People who have successful careers are generally much nicer and more relaxed than people who are still trying to build them. William Jacoby and Saundra Schneider said hello. (They are the guys who gave me my dependent variable, a couple, v. nice.) I went straight through the jetlag and drank beer until I needed to go here, post and sleep. Met some nice people. What I don't like so much is the intellectual conversations you have - where everyone is trying to show off. It seems like an almost inevitable part of academia. I also don't like worrying about what people will think of me, although it seems that as someone who has to do social life very deliberately rather than naturally, there's always going to be a certain amount of artificiality showing through in how I relate to people. So that's just tough, people.
Spent ages trying to find a wireless connection at the airport. Fairly easy at the hotel though.
Manana. Dan's friend Tiffany is going to meet me and we will take her dog for a run. Oh dear, it is a dreadful cliche but it does seem that 20 year olds and people who are still young and struggling with life have got some sort of honesty that we just kind of lose later - until we become carefree old farts....
Friday, 6 May 2005
Thursday, 5 May 2005
Friday, 29 April 2005
beat that suckaz
Your score is...
+67.9
That means that you're...
* better informed than 92% of people who've already taken the quiz
* better informed than 96% of Labour supporters who've already taken the quiz
* better informed than 91% of Conservative supporters who've already taken the quiz
* better informed than 90% of Liberal Democrat supporters who've already taken the quiz
W00t!
Thursday, 28 April 2005
Wednesday, 27 April 2005
The I word
I've been having a few arguments about the immigration issue. It is, as they say, an emotive issue. What follows are just some thoughts.
First, some commentators in the media - Roy Hattersley in yesterday's Guardian, for example - are angry with the Conservatives for playing on people's fears, and turning this into an ugly election. I find this curious. The way the Tories have raised the issue is about as unexceptionable as you could ask for. Their posters have made two claims: "It's not racist to set limits to immigration" and "It's time to set limits to immigration". The first of these is surely right and nobody seems to be disputing it. What about the second? I don't know whether it is time to set limits to immigration: but the Tory advert is simply a statement of their platform. It's not racist or xenophobic in any way. As for the manifesto, it's hardly inflammatory: "Britain has benefited from immigration. We all gain from the social diversity, economic vibrancy and cultural richness that immigration brings."
So I don't see what's wrong with how the Tories are raising the issue. In fact, I rather suspect that what is causing the anger is that they raised it at all. They broke an old taboo in British politics, like the one on the death penalty. Now it is fine to think that democracy is not always a very good way to settle issues. I am a liberal first and a democrat second - as we all should be. But surely if there is any issue that has to be decided collectively, it is that of immigration: and surely if nations do have any rights, they have the right to decide on their immigration policy.
Second, many people in the media have attacked the Tories for confusing the issue, and specifically, for confusing asylum and immigration. Asylum is a matter of our legal and moral obligations; immigration is a matter of economic interest. The Tories are using worries about the asylum system to stir up feeling against immigration.
Is this really fair? Surely the reason people are worried about asylum is that they think the asylum system is being abused by economic migrants. No matter how important the legal distinction between asylum-seekers and immigrants may be in the legal system, in the communities which receive them it is a distinction without a difference.
The attentive reader will notice that so far, I haven't said anything about the actual issues. Is immigration out of control? Is it time to limit immigration? Is the asylum system being abused?
To use the most important words in the language: I don't know.
Some things about immigration are unequivocally good, some are equally clearly bad, and most of the important things are in between. The goods: immigration brings smart, enterprising people into our society. It provides more working-age people to help pay for pensions (although perhaps we should be thinking about why we don't have enough people working as it is....)
The bads: immigration makes a small country fuller of people. It puts pressure on public services like the NHS which find it hard to meet with increased demand (although perhaps we should be thinking about why our public services aren't more flexible....)
As I said the real kicker is what is in between. Culturally, we gain from the diversity; but we can also lose, in a quite straightforward way, as when schools have to deal with children who speak many different first languages. Economically, Britain gains from having more workers; but perhaps employers use migrant labour to drive down wages or even to avoid minimum-wage laws - so that workers in some parts of the economy may lose out. How do we balance these competing judgments? I don't know. What I am sure of is that the way to decide is by looking for proper social scientific evidence.
How high is immigration? A good place to start is the Home Office statistics. In particular, take a look at the chart on page 8 of this document. It shows "grants of settlement" - ie, people being allowed to stay permanently in the UK - for 1983-2003. Over the past few years - since 1997, say - there has certainly been a substantial rise. Whether you think the figures (about 120,000 per year for the past couple of years) are too high or too low depends on your views on the points above. You should also bear in mind that other people have left the UK, so these are not net figures. That matters if you worry about the effects of population pressure, but less if you worry about the difficulties of getting along between different cultures. Some people might also worry about people staying in the UK unofficially: I don't know whether they are right.
Is immigration out of control? There are several points here. First of all, is the asylum system working as it should? There seems to be a widespread consensus that it wasn't - that large backlogs were unfair to all concerned and also allowed people to drop out of the system. On the other hand, the government is now trying to tighten this up and has had some success (asylum applications are substantially down on a few years ago). Second, there is a rather gloomy possibility. Perhaps the asylum system itself - even if it works as intended - intrinsically prevents us from controlling immigration, because it means that we have duties to take in refugees fleeing persecution, and there are potentially millions of such people in the world. It would be grim indeed to be faced with a choice between controlling our immigration policy and fulfilling a basic humanitarian goal. Nevertheless, I can't be sure that this isn't the case. Perhaps a proactive ethical foreign policy would mitigate this problem by lessening persecution abroad. Third, there is the issue of Europe. The Tories want more national control of immigration policy. What the situation is as it stands, I don't know: this EU policy document seems like a good starting point.
Is the asylum system being abused? By definition, reliable statistics about this are hard to come by. The Home Office rejects a lot of asylum applications, but that may be because of applicants failing to fulfil bureaucratic procedures, rather than because they are not genuinely fleeing persecution. My view is based on my own judgments about human behaviour. If I lived in a poor and dangerous country, I would like to come to the UK, and I might well be prepared to lie to do it. So I would be astounded if some asylum-seekers were not, as the tabloids say, "bogus". On the other hand, I strongly suspect that Home Office immigration officials are under pressure to turn down applications, even if they didn't want to themselves - after all, what kind of person works in this kind of job? - so I would also be surprised if some genuine refugees fleeing persecution were not turned down and sent back to really dangerous situations. How many are there of each? I have no idea.
Finally, a general point. Until a few years ago, multiculturalism was, according to all decent upstanding opinion, unequivocally a good thing, and anyone who said the opposite was probably a racist and certainly dangerous - like that dreadful Norman Tebbit with his cricket test. Then, in a surprisingly short time, this consensus broke down. Doubts were expressed. Important people like Trevor Phillips started saying that integration into a common culture was needed. And now, hardly anyone says that multiculturalism is simply a good thing.
From this I learnt that opinions based on what other people will think of you are likely to change very fast: in fact, speaking for myself, I don't think they are worth having.
Tuesday, 26 April 2005
Wednesday, 20 April 2005
They're not all bad.
I'm extremely glad to see that Frank Field voted against the Prevention of Terrorism Act. Good for him.
Who you gonna vote for? NONE OF THE ABOVE!!
This is very cool and perhaps the shape of things to come.
Wednesday, 13 April 2005
Just had to quote a bit more ...
52. The form controls Firefox draws in Web pages are not just inconsistent with those in the rest of the operating system, they are quite possibly the ugliest interactive controls seen in any graphical interface since AmigaDOS 2.04. For example, text fields have borders that, by default, are visible on only two sides out of four. <select> elements are rendered not as normal option menus, but as much less efficient drop-down scrolling listboxes. And radio buttons look like they were drawn in the dark with a broken pencil.
Other than that, Firefox is a nice browser. Just for the record.
I actually find this stuff really encouraging. The open source desktop is now only about as bad as Windows.
I like it
A snarky look at Linux/Gnome usability. It's supposedly about the cool new Ubuntu distro, but in fact a lot is relevant to all of Gnome. Some of it's a bit wacky:
18. Ubuntu supports the annoying and near-useless Insert and Caps Lock keys, with no obvious way to deactivate them. (Windows and Mac OS also make the mistake of supporting Caps Lock.
But some of it is bang on and extremely funny:
20. A lot of technical gibberish is displayed when the computer starts up, and when it shuts down.
25. The dialog for choosing what language to use lists languages in a control that does not look clickable. Among other options, the list includes â??Lastâ?? without telling me which language that was, â??System Defaultâ?? without telling me which language that is, and â??POSIX/C Englishâ?? which is an oxymoron.
It's good stuff. Better still, this guy is apparently an Ubuntu employee, so hopefully he will have some leverage to get things fixed...
Currying in R
I'm deep in the R language as I attempt to make sense of my data. Here's a nice trick I've learnt for when you want to customize one of R's panel plots.
Suppose you want to produce a set of plots for some of your variables. You fire up the pairs function:
pairs(~ AGE + LIBERALISM + EDUCATION)
and you get a nice set of 6 little plots showing AGE against LIBERALISM, AGE against EDUCATION, etc. etc. As it is hard to take all the information in, you decide to add a smoothing line in there:
pairs(~ AGE + LIBERALISM + EDUCATION, panel=panel.smooth)
But suppose you want to change the way panel.smooth operates. You could just pass in extra parameters, but that isn't always the case, and perhaps in any case you want to reuse the same parameters over again elsewhere - but without setting them globally via par.set().
We'll use a little computer science trick known as currying. Currying means creating a new function, based on the old one but with some arguments "pre-filled in".
First, make a copy of your target function:
mysmooth = panel.smooth
Now have a look at its arguments:
formals(mysmooth)
You should see a list of named parameters. There's one called col.smooth, which chooses the colour of the smoothing line. Let's edit it.
formals(mysmooth)$col.smooth = "turquoise"
And we'll change the plotting symbol to something smaller:
formals(mysmooth)$pch = "."
Now just pass this new function into pairs():
pairs(~ AGE + LIBERALISM + EDUCATION, panel=mysmooth)
and there you go. You can keep your mysmooth function around for whenever you want a particular "look and feel", but without remembering all the individual graphics parameters to set.
Tuesday, 12 April 2005
Tuesday, 5 April 2005
HOWTO deal with panel data in R, part one
These are really just notes for myself, but they may be useful to others. I am learning as I go.
There aren't really any easy introductions online, as far as I can tell. A useful resource appears to be the S-plus users' guide to nlme.
First of all, load the "nlme" library.
>library(nlme)
Now you can create your panel data object. Assume your data is loaded in a data frame called "df"; the time variable is called "year"; the panel id variable is called, say, "country". You also have a dependent variable called "GDP".
> panel.data <- groupedData(GDP ~ year | country, df)
Note that nlme doesn't specifically know that this is time data. It is just grouped data. It knows that variables with the same score on "year", have something in common; and so do variables with the same value of "country".
You might also have multiple "grouping factors" - for example, if you have data for male and female test subjects, from different cities, tested at different time periods, you could do
> panel.data <- groupedData( dependent.variable ~ time.period | city/sex , data.frame )
Your cases may also have some other things in common. For example, population may vary with time in your observations, but presumably the country's physical area does not. You can tell R that as follows:
> panel.data <- groupedData(GDP ~ year | country, df, outer = ~ area)
I am not yet sure exactly how that information gets used, but it seems important. I am also not yet sure how you specify multiple "outer" variables; or when exactly a variable should be "outer" and when it should be a "grouping factor". (But often the latter is intuitive - countries are groups but country area is not a group.)
Monday, 4 April 2005
Mark Twain quote
"Next the statesmen will invent cheap lies, putting the blame upon the nation that is attacked, and every man will be glad of those conscience-soothing falsities, and will diligently study them, and refuse to examine any refutations of them; and thus he will by and by convince himself that the war is just, and will thank God for the better sleep he enjoys after this process of grotesque self-deception." -- Mark Twain
Subscribe to:
Posts (Atom)