Honore-style fixed effects estimators for censored data in R

Below is some R code to implement fixed effects Tobit, a la Honore (1992) "Trimmed LAD and Least Squares Estimation of Truncated and Censored Regression Models with Fixed Effects", and also fixed effects double-censored Tobit a la Alan, Honore and Petersen (2008).

# FE tobit a la Honore (1992). See Honore (2002) Port Econ J for a simple guide

# y1 partner y2 other

honore <- function (b, x1, x2, y1, y2) {

dxb <- (x1 - x2) %*% b

sum(

(pmax(y1, dxb) - pmax(y2, - dxb) - dxb)^2 +

2*(y1 < dxb)*(dxb-y1)*y2 +

2*(y1 < -dxb)* (-dxb-y2)*y1

)

}

fetobit <- function (x1, x2, y1, y2) {

# x <- model.matrix(form, dataset)

initvals <- lm.fit(x1 - x2, y1 - y2)$coefficients

res <- optim(initvals, fn=honore, x1=x1, x2=x2, y1=y1, y2=y2,

method="BFGS", control=list(reltol=1e-8, maxit=1000))

if (res$convergence != 0) warning("Didn't converge")

res$par

}

# estimate the standard error, asymptotically.

fetstderr <- function(x1, x2, y1, y2, bhat) {

N <- nrow(x1)

x <- x1 - x2

xb <- x %*% bhat

G <- matrix(0, nrow=length(bhat), ncol=length(bhat))

V <- G

# this should be done faster...

for (i in 1:N) {

xtx <- outer(x[i,], x[i,])

G <- G + ( - y2[i] < xb[i] & xb[i] < y1[i] ) * xtx

V <- V + ((y2[i]^2) * (y1[i] <= xb[i]) +

(y1[i]^2) * (xb[i] <= - y2[i]) +

(y1[i] - y2[i] - xb[i])^2 * (- y2[i] < xb[i] & xb[i] < y1[i])

) * xtx

}

G <- G / N

V <- V / N

Ginv <- solve(G)

sqrt(diag(Ginv %*% V %*% Ginv / N)) # right?

}

# two-sided censoring

# y's must be censored at 0 and 1

alan <- function (b, x1, x2, y1, y2) {

d <- pmax(-1, pmin(as.matrix(x1-x2) %*% as.matrix(b), 1))

# sum of R(y1, y2, max(-1, min(,(x1-x2) %*% b, 1)))

r1 <- r2 <- rep(NA, length(d))

r1 <- ifelse(d <= y1 -1, d + d^2/2 +(y1-1)^2/2,

ifelse(y1-1 < d & d <= 0, d*y1,

ifelse(0 < d & d <= y1, d*y1 - d^2/2,

ifelse(y1 < d, 2,<="" div="" y1^2="">

NA))))

r2 <- ifelse(d <= -y2, -y2^2/2,

ifelse(-y2 < d & d <= 0, d^2/2 + d*y2,

ifelse(0 < d & d <= 1-y2, d*y2,

ifelse(1-y2 < d, (y2-1)^2="" -="" 2,<="" 2="" d="" d^2="" div="">

NA))))

- sum(r1 - r2) # Alan et al say minimize, but I say maximize, ie minimize the minus sum

}

fe2tobit <- function (x1, x2, y1, y2) {

initvals <- lm.fit(x1 - x2, y1 - y2)$coefficients

res <- optim(initvals, fn=alan, x1=x1, x2=x2, y1=y1, y2=y2,

method="BFGS", control=list(reltol=1e-8, maxit=1000))

if (res$convergence != 0) warning("Didn't converge")

res$par

}

fet2stderr <- function (x1, x2, y1, y2, bhat) {

N <- nrow(x1)

x <- x1 - x2

xb <- x %*% bhat

V <- G <- matrix(0, nrow=length(bhat), ncol=length(bhat))

u1 <- ifelse(xb > 0, pmax(y1-xb,0), pmin(y1,1+xb))

u2 <- ifelse(xb > 0, pmin(y2, 1-xb), pmax(y2+xb, 0))

u <- u1 - u2

gwt <- ifelse(xb < -1 | xb > 1, 0,

(-1 < xb & xb < y1-1) - (0 < xb & xb < y1) -

(-y1 < xb & xb < 0) + (1-y2 < xb & xb < 1))

for (i in 1:N) {

xtx <- outer(x[i,], x[i,])

G <- G + gwt[i] * xtx

if (-1 < xb[i] & xb[i] < 1) {

v <- u[i]*x[i,]/2

V <- V + outer(v, v)

}

}

G <- G / N

V <- V / N

Ginv <- solve(G)

sqrt(diag(Ginv %*% V %*% Ginv / N))

}

Points to note:

• fetobit expects data to be censored at 0; fe2tobit expects censoring at 0 and 1. There should be no NAs. 
• Warning! I have done only a very quick test on these. They seem to work but I am not a proper econometrician. Use at your own risk. 
• The standard errors may be inaccurate for low N. Consider bootstrap estimation. 
• Warning! Google searches for "Trimmed LAD" may lead to unexpected results.

Good riddance to bad rubbish

So, last week was a good time to write about media prurience.

I am vindictively glad about the News of the World being cancelled. The NotW was not a good paper that went wrong. It was always a bottom-feeder. Its privacy-invading habits aren't new either. Ony the methods are new.

The NotW was not unique. The Sun is not much better. Nor is the Mirror. The Star is probably worse. All of these papers feed off and feed the worst in human nature: sexual prurience, the vicarious enjoyment of cruelty, other people's misery as entertainment, phony outrage, simplistic politics, and demonizing people who can't answer back.

A strong tabloid press can be a great advantage for a democracy. I hope this country gets one. The British people deserve it.

In the round, the British media are not much to be proud of, but Rupert Murdoch had a special, particular role in bringing this about. He took the Sun and the NotW downmarket. He also turned one of the great papers of the world, the Times, which Trollope satirized as the all-powerful Thunderer, into a competitor with the Daily Mail. On its own terms, the Times is a great product. It has the mix of news its readers want. It's well written and full of content. But there is a useful way to draw the line between quality paper and others: do the writers assume that their readers are as intelligent as they are? I think the Times crossed that line a long time ago. Great product. Lousy paper. So my dream story for next week is “Murdoch arrested. Photos page 2, 3, 4, 5, 6.”

Meanwhile... I have not heard much about what economists would call the demand side. There will always be a market for gossip, sensationalism, celebrity and trivia. A healthy society accepts that, but it also knows that these are not really things to be proud of. Perhaps now would be a good time to reconsider our collective reading habits.

Mothers-in-law

So, I avoided the “future mother-in-law sends rude email” story. It looked very funny and juicy, but it seemed not nice of the email recipient to forward it to all his friends. Now it turns out that maybe the whole thing was a publicity stunt for the groom's wedding business.

There's a fine line here between funny and shameful. Firmly on the shameful side are all the tweeters spreading gossip about footballer's private lives. Some parts of the press were trying to make them into heroes of free speech. Not really. Worse still is the self-publicizing Lib Dem MP who raised the footballer's name in the House, using parliamentary privilege to get round the privacy injunction. Anyone should think very hard before using that privilege. It may be very important in remedying injustices. It is not for retweeting prurient gossip which a judge has decided ought to be private.

Wanting to know about other people's private lives is dirty. Of course we all want to do it. But it is dirty, the industry that exists around it is dirty too, and if you do it, then worthless people will use it against you.

There's a relevance to the DSK affair. This is more complex. The French public has an interest in knowing the character of candidates for high office; a lot of evidence came out suggesting Strauss-Kahn did not have a respectful attitude to women. But he did not deserve to be tried by the media, and found guilty of rape by public opinion because he “seemed like the type”. That kind of logic is strictly banned from the courtroom, for the good reason that it would expose defendants to the sort of exploitation that may have been planned here. Banning it from our thoughts would also be good. Partly because we would be less vulnerable to spin, manipulation and free PR for wedding catering businesses, but mostly because it is bad in itself.