Monday, 29 December 2014
Wednesday, 17 December 2014
Tuesday, 16 December 2014
A Christmas R parcel: refset
I have a new R package on github, called refset. refset's tagline is "subsets with reference semantics". In English, that means: a refset is a subset which automatically stays in sync with its parent data. When you change the parent, the refset is updated, and when you change the refset, you are changing the parent.
The README has full details, but here's a brief taster:
... continued on github. Enjoy!
The README has full details, but here's a brief taster:
Installation
library(devtools)
install_github("hughjonesd/refset")
Creating a refset
library(refset)
employees <- data.frame(
id=1:4,
name=c("James", "Sylvia", "Meng Qi", "Luis"),
age=c(28,44,38, 23),
gender=factor(c("M", "F", "F", "M")),
stringsAsFactors=FALSE)
refset(rs, employees[1:2,])
Refsets refer to the original
rs
## id name age gender
## 1 1 James 28 M
## 2 2 Sylvia 44 F
employees$name[1] <- "Jimmy"
rs
## id name age gender
## 1 1 Jimmy 28 M
## 2 2 Sylvia 44 F
Refsets change the original
rs$age <- c(29, 45) employees$age## [1] 29 45 38 23
Tuesday, 9 December 2014
Bits from the report on the CIA's enhanced interrogation techniques
Original document.
The waterboarding technique was physically harmful, inducing convulsions and vomiting. Abu Zubaydah, for example, became "completely unresponsive, with bubbles rising through his open, full mouth.'" Internal CIA records describe the waterboarding of Khalid Shaykh Mohammad as evolving into a "series of near drownings."
Sleep deprivation involved keeping detainees awake for up to 180 hours, usually standing or in stress positions, at times with their hands shackled above their heads. At least five detainees experienced disturbing hallucinations during prolonged sleep deprivation and, in at least two of those cases, the CIA nonetheless continued the sleep deprivation. Contrary to CIA representations to the Department of Justice, the CIA instructed personnel that the interrogation of Abu Zubaydah would take "precedence" over his medical care, resulting in the deterioration of a bullet wound Abu Zubaydah incurred during his capture....
At least five CIA detainees were subjected to "rectal rehydration" or rectal feeding without documented medical necessity. The CIA placed detainees in ice water "baths." The CIA led several detainees to believe they would never be allowed to leave CIA custody alive, suggesting to one detainee that he would only leave in a coffin-shaped box. One interrogator told another detainee that he would never go to court, because "we can never let the world know what I have done to you." CIA officers also threatened at least three detainees with harm to their families— to include threats to harm the children of a detainee, threats to sexually abuse the mother of a detainee, and a threat to "cut [a detainee's] mother's throat."
...
Conditions at CIA detention sites were poor, and were especially bleak early in the program. CIA detainees at the COBALT detention facility were kept in complete darkness and constantly shackled in isolated cells with loud noise or music and only a bucket to use for human waste. Lack of heat at the facility likely contributed to the death of a detainee. The chief of interrogations described COBALT as a "dungeon." Another senior CIA officer stated that COBALT was itself an enhanced interrogation technique."
At times, the detainees at COBALT were walked around naked or were shackled with their hands above their heads for extended periods of time. Other times, the detainees at COBALT were subjected to what was described as a "rough takedown," in which approximately five CIA officers would scream at a detainee, drag him outside of his cell, cut his clothes off, and secure him with Mylar tape. The detainee would then be hooded and dragged up and down a long corridor while being slapped and punched.
Even after the conditions of confinement improved with the construction of new detention facilities, detainees were held in total isolation except when being interrogated or debriefed by CIA personnel. Throughout the program, multiple CIA detainees who were subjected to the CIA's enhanced interrogation techniques and extended isolation exhibited psychological and behavioral issues, including hallucinations, paranoia, insomnia, and attempts at self-harm and self-mutilation. Multiple psychologists identified the lack of human contact experienced by detainees as a cause of psychiatric problems.
...
The CIA's COBALT detention facility in Country -- began operations in September 2002 and ultimately housed more than half of the 119 CIA detainees identified in this Study. The CIA kept few formal records of the detainees in its custody at COBALT. Untrained CIA officers at the facility conducted frequent, unauthorized, and unsupervised interrogations of detainees using harsh physical interrogation techniques that were not—and never became—part of the CIA's formal "enhanced" interrogation program. The CIA placed a junior officer with no relevant experience in charge of COBALT. On November --, 2002, a detainee who had been held partially nude and chained to a concrete floor died from suspected hypothermia at the facility.
...
The CIA contracted with two psychologists to develop, operate, and assess its interrogation operations....
On the CIA's behalf, the contract psychologists developed theories of interrogation based on "learned helplessness,"and developed the list of enhanced interrogation techniques that was approved for use against Abu Zubaydah and subsequent CIA detainees. The psychologists personally conducted interrogations of some of the CIA's most significant detainees using these techniques. They also evaluated whether detainees' psychological state allowed for the continued use of the CIA's enhanced interrogation techniques, including some detainees whom they were themselves interrogating or had interrogated. The psychologists carried out inherently governmental functions, such as acting as liaison between the CIA and foreign intelligence services, assessing the effectiveness of the interrogation program, and participating in the interrogation of detainees in held in foreign government custody.
In 2005, the psychologists formed a company specifically for the purpose of conducting their work with the CIA. Shortly thereafter, the CIA outsourced virtually all aspects of the program.
In 2006, the value of the CIA's base contract with the company formed by the psychologists with all options exercised was in excess of $180 million...
...
[Individuals detained by the CIA included] Abu Hudhaifa, who was subjected to ice water baths and 66 hours of standing sleep deprivation before being released because the CIA discovered he was likely not the person he was believed to be ... Muhammad Khan, who, like Zarmein, was among detainees about whom the CIA acknowledged knowing "very little"... Haji Ghalgi, who was detained as "useful leverage" against a family member... Nazar Ali, an "intellectually challenged" individual whose taped crying was used as leverage against his family member...
...
a cable described Abu Zubaydah's cell as white with no natural lighting or windows, but with four halogen lights pointed into the cell." An air conditioner was also in the room. A white curtain separated the interrogation room from the cell. The interrogation cell had three padlocks. Abu Zubaydah was also provided with one of two chairs that were rotated based on his level of cooperation (one described as more comfortable tlian the other). Security officers wore all black uniforms, including boots, gloves, balaclavas, and goggles to keep Abu Zubaydah from identifying the officers, as well as to prevent Abu Zubaydah "from seeing the security guards as individuals who he may attempt to establish a relationship or dialogue with."... In addition, either loud
rock music was played or noise generators were used to enhance Abu Zubaydah's "sense of hopelessness."' Abu Zubaydah was typically kept naked and sleep deprived.
...
The "aggressive phase of interrogation" continued until August 23,
2002. Over the course ofthe entire 20 day "aggressive phase of interrogation," Abu Zubaydah spent a total of 266 hours (11 days, 2 hours) in the large (coffin size) confinement box and 29 hours in a small confinement box, which had a width of 21 inches, a depth of 2.5 feet, and a height of 2.5 feet. The CIA interrogators told Abu Zubaydah that the only way he would leave
the facility was in the coffin-shaped confinement box.
According to the daily cables from DETENTION SITE GREEN,
Abu Zubaydah frequently "cried," "begged," "pleaded," and "whimpered," but continued to deny that he had any additional information on current threats to, or operatives in, the United States.
...
After approximately three weeks, the CIA developed a more aggressive treatment regimen "without unnecessary conversation." Majid Khan was then subjected to involuntary rectal feeding and rectal hydration, which included two bottles of Ensure. Later that same day, Majid Khan's "lunch tray," consisting of hummus, pasta with sauce, nuts, and raisins, was "pureed" and rectally infused. Additional sessions ofrectal feeding and hydration followed. In addition to his hunger strikes, Majid Klian engaged in acts of self-harm that included attempting to cut his wrist on two occasions, an attempt to chew into his arm at the inner elbow, an attempt to cut a vein in the top of his foot, and an attempt to cut into his skin at the elbow joint using a filed toothbrush.
The waterboarding technique was physically harmful, inducing convulsions and vomiting. Abu Zubaydah, for example, became "completely unresponsive, with bubbles rising through his open, full mouth.'" Internal CIA records describe the waterboarding of Khalid Shaykh Mohammad as evolving into a "series of near drownings."
Sleep deprivation involved keeping detainees awake for up to 180 hours, usually standing or in stress positions, at times with their hands shackled above their heads. At least five detainees experienced disturbing hallucinations during prolonged sleep deprivation and, in at least two of those cases, the CIA nonetheless continued the sleep deprivation. Contrary to CIA representations to the Department of Justice, the CIA instructed personnel that the interrogation of Abu Zubaydah would take "precedence" over his medical care, resulting in the deterioration of a bullet wound Abu Zubaydah incurred during his capture....
At least five CIA detainees were subjected to "rectal rehydration" or rectal feeding without documented medical necessity. The CIA placed detainees in ice water "baths." The CIA led several detainees to believe they would never be allowed to leave CIA custody alive, suggesting to one detainee that he would only leave in a coffin-shaped box. One interrogator told another detainee that he would never go to court, because "we can never let the world know what I have done to you." CIA officers also threatened at least three detainees with harm to their families— to include threats to harm the children of a detainee, threats to sexually abuse the mother of a detainee, and a threat to "cut [a detainee's] mother's throat."
...
Conditions at CIA detention sites were poor, and were especially bleak early in the program. CIA detainees at the COBALT detention facility were kept in complete darkness and constantly shackled in isolated cells with loud noise or music and only a bucket to use for human waste. Lack of heat at the facility likely contributed to the death of a detainee. The chief of interrogations described COBALT as a "dungeon." Another senior CIA officer stated that COBALT was itself an enhanced interrogation technique."
At times, the detainees at COBALT were walked around naked or were shackled with their hands above their heads for extended periods of time. Other times, the detainees at COBALT were subjected to what was described as a "rough takedown," in which approximately five CIA officers would scream at a detainee, drag him outside of his cell, cut his clothes off, and secure him with Mylar tape. The detainee would then be hooded and dragged up and down a long corridor while being slapped and punched.
Even after the conditions of confinement improved with the construction of new detention facilities, detainees were held in total isolation except when being interrogated or debriefed by CIA personnel. Throughout the program, multiple CIA detainees who were subjected to the CIA's enhanced interrogation techniques and extended isolation exhibited psychological and behavioral issues, including hallucinations, paranoia, insomnia, and attempts at self-harm and self-mutilation. Multiple psychologists identified the lack of human contact experienced by detainees as a cause of psychiatric problems.
...
The CIA's COBALT detention facility in Country -- began operations in September 2002 and ultimately housed more than half of the 119 CIA detainees identified in this Study. The CIA kept few formal records of the detainees in its custody at COBALT. Untrained CIA officers at the facility conducted frequent, unauthorized, and unsupervised interrogations of detainees using harsh physical interrogation techniques that were not—and never became—part of the CIA's formal "enhanced" interrogation program. The CIA placed a junior officer with no relevant experience in charge of COBALT. On November --, 2002, a detainee who had been held partially nude and chained to a concrete floor died from suspected hypothermia at the facility.
...
The CIA contracted with two psychologists to develop, operate, and assess its interrogation operations....
On the CIA's behalf, the contract psychologists developed theories of interrogation based on "learned helplessness,"and developed the list of enhanced interrogation techniques that was approved for use against Abu Zubaydah and subsequent CIA detainees. The psychologists personally conducted interrogations of some of the CIA's most significant detainees using these techniques. They also evaluated whether detainees' psychological state allowed for the continued use of the CIA's enhanced interrogation techniques, including some detainees whom they were themselves interrogating or had interrogated. The psychologists carried out inherently governmental functions, such as acting as liaison between the CIA and foreign intelligence services, assessing the effectiveness of the interrogation program, and participating in the interrogation of detainees in held in foreign government custody.
In 2005, the psychologists formed a company specifically for the purpose of conducting their work with the CIA. Shortly thereafter, the CIA outsourced virtually all aspects of the program.
In 2006, the value of the CIA's base contract with the company formed by the psychologists with all options exercised was in excess of $180 million...
...
[Individuals detained by the CIA included] Abu Hudhaifa, who was subjected to ice water baths and 66 hours of standing sleep deprivation before being released because the CIA discovered he was likely not the person he was believed to be ... Muhammad Khan, who, like Zarmein, was among detainees about whom the CIA acknowledged knowing "very little"... Haji Ghalgi, who was detained as "useful leverage" against a family member... Nazar Ali, an "intellectually challenged" individual whose taped crying was used as leverage against his family member...
...
a cable described Abu Zubaydah's cell as white with no natural lighting or windows, but with four halogen lights pointed into the cell." An air conditioner was also in the room. A white curtain separated the interrogation room from the cell. The interrogation cell had three padlocks. Abu Zubaydah was also provided with one of two chairs that were rotated based on his level of cooperation (one described as more comfortable tlian the other). Security officers wore all black uniforms, including boots, gloves, balaclavas, and goggles to keep Abu Zubaydah from identifying the officers, as well as to prevent Abu Zubaydah "from seeing the security guards as individuals who he may attempt to establish a relationship or dialogue with."... In addition, either loud
rock music was played or noise generators were used to enhance Abu Zubaydah's "sense of hopelessness."' Abu Zubaydah was typically kept naked and sleep deprived.
...
The "aggressive phase of interrogation" continued until August 23,
2002. Over the course ofthe entire 20 day "aggressive phase of interrogation," Abu Zubaydah spent a total of 266 hours (11 days, 2 hours) in the large (coffin size) confinement box and 29 hours in a small confinement box, which had a width of 21 inches, a depth of 2.5 feet, and a height of 2.5 feet. The CIA interrogators told Abu Zubaydah that the only way he would leave
the facility was in the coffin-shaped confinement box.
According to the daily cables from DETENTION SITE GREEN,
Abu Zubaydah frequently "cried," "begged," "pleaded," and "whimpered," but continued to deny that he had any additional information on current threats to, or operatives in, the United States.
...
After approximately three weeks, the CIA developed a more aggressive treatment regimen "without unnecessary conversation." Majid Khan was then subjected to involuntary rectal feeding and rectal hydration, which included two bottles of Ensure. Later that same day, Majid Khan's "lunch tray," consisting of hummus, pasta with sauce, nuts, and raisins, was "pureed" and rectally infused. Additional sessions ofrectal feeding and hydration followed. In addition to his hunger strikes, Majid Klian engaged in acts of self-harm that included attempting to cut his wrist on two occasions, an attempt to chew into his arm at the inner elbow, an attempt to cut a vein in the top of his foot, and an attempt to cut into his skin at the elbow joint using a filed toothbrush.
Sunday, 16 November 2014
Charlie Plott on ESA
Reposting a comment by Charlie Plott on the ESA mailing list, regarding ethical approval, which is so sensible that I would like it framed on the wall of every IRB in the land. My emphases.
Dear Juan-Camilo:
I think that your assessment might be adjusted a bit. First, the currently evolving criteria regarding the protection of human subjects reflect evidence of harm as opposed to imagined harms. What people might imagine as sources of harm cannot be accepted as rules governing research and the tendency to confuse the two has been identified as a source of regulatory creep. The concept of "psychological harm" is a good case in point that brings "harms" that some minds might imagine (the mind of anyone who might be on an IRB) into the discussion as if it was demonstrated fact. I think that there is absolutely no evidence of harm in laboratory experimental economics experiments. Field experiments might be different but even there the examples typically deal with exposing people to police or social sanctions (e.g. publicly revealed as a homosexual) . Secondly, rules that some researchers might use to guide their own science are not rules that should be generalized as rules to impose for human subjects protection by and IRB. In particular, your list seems to confuse the two- especially with regard to deception, information about procedures, treatments and randomness.
1. A rule against deception is a black hole. The NRC devoted many pages to this and concluded that the subject suffered no harm if the discomfort did not last longer than the experiment. If the deception was related to self evaluation or self capacities then debriefing should be done because such deceptions can have lingering effects.
2. There is no need for a signed consent form if there is no exposure to harm beyond those experienced as part of daily life. Subject agreement to a database is enough. The requirement for written consent is also a black hole and prevents experiments with remote subject pools. In addition it invites long reviews of wording, legal review, etc. Current NRC thinking trashed the idea.
3. Subjects need to know what they will do in the experiment as a demonstration that they will not be exposed to harm. Information regarding treatments, interactions, etc. ... need not be part of information given to subjects unless it is related to risk. Such information can compromise the scientific integrity of the research. I never go over such details unless it is part of the controls I need.
4. Similarly, random procedures explanations might be something needed for experimental control in some types of experiments but that is not an issue for the protection of human subjects. The question to pose is whether or not the subject is at risk and what is the historical evidence of harm? Do you really want to exclude experiments in which subjects are surprised? Does surprise expose them to harm? I think not. Where is the evidence of harm?
5. Why must losses be avoided if the subject is informed about the possibilities? This would mean that serious gambling experiments are out of bounds.
Basically, there is a confusion between what might constitute procedures appropriate for scientific control and procedures needed for the protection of human subjects. The former can differ from experiment to experiment and should not be imposed as part of the latter.
Charlie
Friday, 14 November 2014
Saturday, 8 November 2014
Friday, 7 November 2014
Tuesday, 4 November 2014
Gapminder in one line
Gapminder has received a lot of attention for its cool animated graphics. I downloaded their data and ran:
palette(adjustcolor(rainbow(23), 1, .6, .6, .6,
offset=c(0,0,0,-0.1)))
library(anim.plots)
gm <- anim.plot(lifex ~ GDP + year, data=gm_data, log="x",
cex=sqrt(pop)*0.0004, pch=19, col=region, xlab="GDP",
ylab="Life expectancy", speed=5, subset=year>1900)
ani.options(interval=0.2)
anim.save(gm, file="gapminder.mp4", type="Video")
Holograms and talking Hans Rosling will be included in the 2.0 release ;-)
Friday, 31 October 2014
Conflict data for October
Here's a quick animation of the Armed Conflict Location Event Database's realtime data for October.
I created this with the code:
acl <- read.csv("/Users/david/ACLEDoct27.csv")
acl$date <- as.Date(acl$EVENT_DATE, "%d-%b-%y")
acl <- acl[acl$date > as.Date("2014-10-01"),]
library(anim.plots)
library(maps)
tmp <- anim.points(LATITUDE ~ LONGITUDE + as.numeric(date),
data=acl , speed=1, col=rgb(1,0,0,.7), pch=19,
cex=log(TOTAL_FATALITIES+1), show=F)
saveGIF(
replay(tmp,
before=map('world', fill=T, col=terrain.colors(5)[2:5],
xlim=c(-20, 60), ylim=c(-40, 40)),
after=legend("bottomleft", col="red", pt.cex=log(1+c(1,10,100)) ,
legend=c(1,10,100), pch=19, y.intersp=1.5, x.intersp=1.5,
bty="n", title="Fatalities")),
"conflict.gif")
acl <- read.csv("/Users/david/ACLEDoct27.csv")
acl$date <- as.Date(acl$EVENT_DATE, "%d-%b-%y")
acl <- acl[acl$date > as.Date("2014-10-01"),]
library(anim.plots)
library(maps)
tmp <- anim.points(LATITUDE ~ LONGITUDE + as.numeric(date),
data=acl , speed=1, col=rgb(1,0,0,.7), pch=19,
cex=log(TOTAL_FATALITIES+1), show=F)
saveGIF(
replay(tmp,
before=map('world', fill=T, col=terrain.colors(5)[2:5],
xlim=c(-20, 60), ylim=c(-40, 40)),
after=legend("bottomleft", col="red", pt.cex=log(1+c(1,10,100)) ,
legend=c(1,10,100), pch=19, y.intersp=1.5, x.intersp=1.5,
bty="n", title="Fatalities")),
"conflict.gif")
Easy animations in R using anim.plots
After the last two R posts, here's something a bit more useful.
Creating animations in R was hard. Then Yihui Xie wrote the animation package. animation lets you plot individual frames, then record them.
I've been working on a little R package called anim.plots which adds some syntactic sugar to this. It provides a few commands, similar to the standard R graphics, to create animated plots. Here's a quick demo. (The speed is quite slow: this is because I converted the files to animated GIFs. The original animations go reasonably fast.)
library(anim.plots)
anim.plot(1:5, 1:5, col="green")
x <- rep(1:100/10, 20)
times <- rep(1:20, each=100) # twenty frames with 100 points each
y <- sin(x*times/4)
waves <- anim.plot(x,y,times, type="l", col="orange", lwd=2, speed=2)
anim.plot(rep(1:10,2), rep(2:1, each=10), window=1:t, pch=1:20, ylim=c(0,3),cex=2, col=1:5, xlab=paste("Plot symbols", 1:20))
The above code also shows how parameters get recycled where appropriate, either to the number of points, or to the number of frames. For more complex parameters, you might have to use a matrix. Here we zoom in on a distribution of points by changing the xlim and ylim parameters.
x <- rnorm(4000)
y <- rnorm(4000)
x <- rep(x, 40)
y <- rep(y, 40)
xlims <- 4*2^(-(1:40/10))
ylims <- xlims <- rbind(xlims, -xlims)
anim.plot(x, y, times=40, speed=5, xlim=xlims, ylim=ylims,
col=rgb(0,0,0,.3), pch=19, bg="white")
## discoveries 1860-1959
xlim <- rbind(1860:1959,1870:1969)
anim.plot(1860:1959, discoveries, times=1:100, xlim=xlim, col="red", xaxp=rbind(xlim, 10), window=t:(t+10), type="h", lwd=8, speed=5)
There's also a formula interface. Here's a plot of some chicks being fed one of four different diets:
data(ChickWeight)
ChickWeight$chn <- as.numeric(as.factor(ChickWeight$Chick))
tmp <- anim.plot(weight ~ chn + Time, data=ChickWeight, col=as.numeric(Diet), pch=as.numeric(Diet), speed=3)
Sometimes you need to run extra plotting commands after each frame. You can do this using the replay command:
replay(tmp, after=legend("topleft", legend=paste("Diet", 1:4), pch=1:4, col=1:4))
anim.hist(rep(rnorm(5000), 7), times=rep(1:7, each=5000), breaks=c(5,10,20,50,100,200, 500, 1000))
anim.curve(x^t, times=10:50/10, n=20)
data(volcano)
# create a circle:
tmp <- volcano
tmp[] <- 200 - ((row(tmp) - 43)^2 + (col(tmp) - 30)^2)/20
# an animated contour needs a 3D array:
cplot <- array(NA, dim=c(87,61,20))
cplot[,,1] <- tmp
cplot[,,20] <- volcano
# morph the volcano into a circle:
cplot <- apply(cplot, 1:2, function(x) seq(x[1], x[20], length.out=20))
cplot <- aperm(cplot, c(2,3,1))
anim.contour(z=cplot, times=1:20, speed=3, levels=80 + 1:12*10, lty=c(1,2,2))
eq = read.table(
file="http://earthquake.usgs.gov/earthquakes/catalogs/eqs7day-M1.txt",
fill=TRUE, sep=",", header=T)
eq$time <- as.numeric(strptime(eq$Datetime, "%A, %B %d, %Y %X UTC"))
eq <- eq[-1,]
library(maps)
map('world')
tmp <- anim.points(Lat ~ Lon + time, data=eq, cex=Magnitude, col=rgb(
1-Depth/200, 0, Depth/200,.7), pch=19, speed=3600*12, show=FALSE)
replay(tmp, before=map('world', fill=TRUE, col="wheat"))
Code is available at http://github.com/hughjonesd/anim.plots. If you have devtools installed you can get it with:
install_github("hughjonesd/anim.plots")
Saturday, 25 October 2014
Screw UKIP
... and Cameron too!
I'm sick of this anti-European bullshit, and I'm going to stand up against it the only way I know how.
The proportions are guesses - couldn't be bothered to do the math for what is described here. The colours are right though. I had to use a 10 pointed star to make sure the stars were the right way up.
I'm sick of this anti-European bullshit, and I'm going to stand up against it the only way I know how.
par(bg=rgb(0,51/255,153/255))
symbols(sin(1:12/12*2*pi), cos(1:12/12*2*pi),
stars=matrix(rep(c(0.52,1,0.52, 0.4)*1/7, 10), nrow=12, ncol=20, byrow=T),
fg=rgb(255/255,204/255,0), bg=rgb(255/255,204/255,0), inches=F, axes=F, ann=F, crt=90)
The proportions are guesses - couldn't be bothered to do the math for what is described here. The colours are right though. I had to use a 10 pointed star to make sure the stars were the right way up.
Tuesday, 21 October 2014
Simulating infections
Here's a little R simulation of fashion/disease infection/group identity in a population. There are two groups, red and white. Individuals move around at random. Sometimes, they change group. Whether they change depends on the number of their neighbours who are from a different group. Also, individuals who have just changed are "infectious" (or, trendsetters). Individuals who have not changed for a long time are not very infectious, and are also easy to infect.
It's fun to play with the parameters. It would also be fun to implement this with real people, as e.g. a mobile app, using real time movement.
##############################################
N <- 500
maxdist <- .1 # distance within which "neighbours" are counted
maxt <- 240 # number of repetitions
delay <- 0 # time delay between reps
step <- FALSE # ask after each step?
speed <- 0.01 # speed indivs move
decay <- 1 # speed decaycols <- c("red", "white", "pink", "yellow") # last two for changers
##############################################
prop2changeprob <- function(props) {
props^2
}
infectivity <- function(lifespan) 1/lifespan
dist <- function(ind, oths) sqrt((ind$x-oths$x)^2+(ind$y-oths$y)^2)
par(bg="gray")
inds <- data.frame(x=runif(N), y=runif(N), col=sample(1:2, N, replace=TRUE))
inds$oldx <- inds$x
inds$oldy <- inds$y
inds$lifespan <- 0 # since last change
for (t in 1:(maxt)) {
inds$lifespan <- inds$lifespan + 1
propother <- sapply(1:nrow(inds), function(i) {
i <- inds[i,]
close <- dist(i, inds) < maxdist
if (any(close)) {
wt <- infectivity(inds$lifespan)
wt[!close] <- 0
sum(wt * (inds$col!=i$col)) / sum(wt) # weighted by
} else 0
})
change <- runif(N) < prop2changeprob(propother)
displaycol <- inds$col
inds$col <- ifelse(change, 3-inds$col, inds$col)
inds$lifespan[change] <- 0
displaycol <- ifelse(displaycol==inds$col, displaycol, inds$col+2)
inds$oldx <- inds$x
inds$oldy <- inds$y
inds$x <- inds$x + runif(N, -speed,speed) + decay*(inds$x-inds$oldx)
inds$y <- inds$y + runif(N, -speed,speed)+ decay*(inds$y-inds$oldy)
#inds$x <- pmax(0, pmin(1, inds$x)) # square
#inds$y <- pmax(0, pmin(1, inds$y))
inds$x <- inds$x %% 1 # torus
inds$y <- inds$y %% 1
plot(inds$x, inds$y, col=cols[displaycol], pch=19, cex=.7, bg="grey",
xlim=0:1, ylim=0:1)
Sys.sleep(delay)
if (step) {
rl <- readline()
if (rl=="q") stop("Quitting")
}
}
Wednesday, 17 September 2014
The Scottish referendum, the EU and the old new ideas
Here's a reasonable case for Scottish independence.
Ignore all the anxieties about the transition, the tussles over currency and oil and nuclear submarines: plenty of nations smaller than Scotland do fine, and in the long run Scotland can do too. The Union was great in its day, but that day has passed. The future belongs to the small, nimble, united nation-state, competing in a globalized market; not to the lumbering imperialist conglomerate that was put together to build an empire with armed force.
Reasonable, plausible, but wrong. We are no longer living in the optimistic post-Cold War world where free trade was what mattered. Old conflicts have returned in force. Today nations need military and economic heft. The UK is outdated, in some ways, but because it is too small, not too big.
The bigger unit that we need is, of course, a strong European Union. Could Scotland not play an independent role underneath its protective shield? Unfortunately, the politics is all wrong. A Yes in the referendum is bound to weaken the Tories and the establishment, strengthen the insurgency of UKIP, and increase the chance that the rump state of Britain exits the EU. That exit would be a crippling blow for Europe. The future of Scotland would then look much less certain and safe.
Sunday, 14 September 2014
10 ideas about genetic differences between human groups
Nicholas Wade's book, A Troublesome Inheritance, has created huge controversy by claiming that race is real and explains human differences in economic performance. Here is a response from many genetic scientists, which also includes links to comment around the web. I don't wish to jump into that debate, but just to give a set of thoughts about the issue of genetic differences between races. (Disclaimer: I am not an expert biologist, just an interested amateur.)
1. The fundamental truth of biology is that humans are basically the same.
Humans make up a single species, which interbreeds with itself. When two people reproduce, a new viable human is created, with DNA that results from combining their DNA, more or less at random. This is like taking parts from two cars and building a new car with it: you can't mix a Maserati and a Ferrari, only two Maseratis of the same model. So, all humans are basically the same.
That is not to deny that the differences between humans have profound social consequences - for example, in deciding who gets rich and poor - and are therefore of great interest to us. But a Martian biologist would probably find two humans as hard to tell apart as we find two sheep. Our similarities vastly outweigh our differences.
2. Races are statistical facts, not buckets.
This also follows from the "single species" idea. To say that I am Caucasian or you are African does not imply that everything about us differs, or that any single gene we have is certain to differ. People interbreed. "Race", if it means anything genetically, is shorthand for having (probably) had many ancestors from a particular area of the world. This may result in someone possessing a particular allele with more likelihood; but that is a matter of probability, not certainty.
Socially, people often treat race and ethnicity as buckets, and that can become self-fulfilling. But this is a fact about society not biology.
3. Within that interpretation, there are real genetic differences between races.
This is now uncontroversial. For example, people from some parts of the world are more likely to have the allele linked to sicke cell disease than others; partly because the allele helps protect against malaria.
4. There are also differences between races in genes that affect behaviour and/or psychology.
For example, different alleles of the MAOA gene exist in different proportions among different racial groups. The MAOA gene is linked to risk-taking and violence. Here's a rather long summary of recent research.
5. These differences do not necessarily add up to "differences between groups in psychology being caused by genes".
Most geneticists think that any given social behaviour is influenced by many genes - there is no neat 1-1 link. So, a group that possesses more of one allele (say MAOA-2R) could possess less of another, undiscovered gene which pushes them the opposite way.
There are reasons this could be true. For example, suppose that over an evolutionary timespan there has been an optimal level of risktaking, and that risktaking is increased by (alleles of) two genes. It is entirely possible for one group to evolve to have the risk-taking allele at one locus, while another group evolves to have the allele at the other locus. The different genes will then not add up to different behaviour, which is instead kept similar by evolutionary pressure.
6. Or, some group differences in behaviour may have genetic causes. We will find out soon either way.
Nevertheless, there is no general reason to think that intergroup genetic differences will cancel out 100% in this way. On the face of it that seems unlikely - as Cochran and Harpending say in The 10,000 Year Explosion, it's like expecting a coin to land on its edge.
Science is progressing fast in this area, with new statistical methods and databases coming online. Whether genetics matters a lot or a little, the debate can and will be settled by empirical data - which is one reason I am writing this.
7. Genetic differences are multi-dimensional.
Think of genetics and most people think of IQ. IQ is politically salient because it is linked to economic outcomes, and because it threatens to array racial groups on a continuum, with higher IQ being better. But real differences will be more complex than that. Think about MAOA: aggression is bad, right? But risk-taking, for example in entrepreneurship, isn't always bad. If there are differences between groups, we will not be able to order them hierarchically.
8. Genes are not destiny, in the large.
OK, here I am disagreeing with Nicholas Wade's position in the speculative second half of his book. Think about Caucasians versus Asians. In the 19th century, British gunboats were knocking on China's door. In 2014, China is the second power in the world, and pushing for first place. These changes happen too fast for genetics to be a useful explanation.
9. Genes are not destiny, in the small.
Genetic does not mean unchangeable. To take a simple example, many genetic diseases have simple cures. More generally, human choices, including human culture, can make up for - or perhaps sometimes amplify! - genetic differences. (Culture itself, by contrast, can be very hard to change.)
10. Liberal equality does not require us all to be identical.
Why don't we deny the vote to stupid people? We are equal not because we are all the same in every way, but because - as Christianity would put it - we are all possessors of a living soul. Similarly, we have a duty to judge individuals as individuals, and not on the basis of belonging to an ethnic group. Doing the former is usually efficient, since ethnicity is at best a weak predictor of someone's behaviour. It is also morally right.
It may be - I think it is quite likely - that genetic differences will account for many of the differences between people. They may also account for some of the differences between groups. Given the rate at which the science is developing, we would be very unwise not to start thinking about this. I do not want to be too sanguine about its consequences: obviously it would be, in some sense, "bad news" for human equality. At the same time, it is not a cause for liberal despair, or for racism and intergroup contempt to triumph. Whatever differences there are will be probabilistic not deterministic, multidimensional and complex; and morally outweighed by our underlying shared humanity - a truth supported equally by religious tradition, and by modern biology.
1. The fundamental truth of biology is that humans are basically the same.
Humans make up a single species, which interbreeds with itself. When two people reproduce, a new viable human is created, with DNA that results from combining their DNA, more or less at random. This is like taking parts from two cars and building a new car with it: you can't mix a Maserati and a Ferrari, only two Maseratis of the same model. So, all humans are basically the same.
That is not to deny that the differences between humans have profound social consequences - for example, in deciding who gets rich and poor - and are therefore of great interest to us. But a Martian biologist would probably find two humans as hard to tell apart as we find two sheep. Our similarities vastly outweigh our differences.
2. Races are statistical facts, not buckets.
This also follows from the "single species" idea. To say that I am Caucasian or you are African does not imply that everything about us differs, or that any single gene we have is certain to differ. People interbreed. "Race", if it means anything genetically, is shorthand for having (probably) had many ancestors from a particular area of the world. This may result in someone possessing a particular allele with more likelihood; but that is a matter of probability, not certainty.
Socially, people often treat race and ethnicity as buckets, and that can become self-fulfilling. But this is a fact about society not biology.
3. Within that interpretation, there are real genetic differences between races.
This is now uncontroversial. For example, people from some parts of the world are more likely to have the allele linked to sicke cell disease than others; partly because the allele helps protect against malaria.
4. There are also differences between races in genes that affect behaviour and/or psychology.
For example, different alleles of the MAOA gene exist in different proportions among different racial groups. The MAOA gene is linked to risk-taking and violence. Here's a rather long summary of recent research.
5. These differences do not necessarily add up to "differences between groups in psychology being caused by genes".
Most geneticists think that any given social behaviour is influenced by many genes - there is no neat 1-1 link. So, a group that possesses more of one allele (say MAOA-2R) could possess less of another, undiscovered gene which pushes them the opposite way.
There are reasons this could be true. For example, suppose that over an evolutionary timespan there has been an optimal level of risktaking, and that risktaking is increased by (alleles of) two genes. It is entirely possible for one group to evolve to have the risk-taking allele at one locus, while another group evolves to have the allele at the other locus. The different genes will then not add up to different behaviour, which is instead kept similar by evolutionary pressure.
6. Or, some group differences in behaviour may have genetic causes. We will find out soon either way.
Nevertheless, there is no general reason to think that intergroup genetic differences will cancel out 100% in this way. On the face of it that seems unlikely - as Cochran and Harpending say in The 10,000 Year Explosion, it's like expecting a coin to land on its edge.
Science is progressing fast in this area, with new statistical methods and databases coming online. Whether genetics matters a lot or a little, the debate can and will be settled by empirical data - which is one reason I am writing this.
7. Genetic differences are multi-dimensional.
Think of genetics and most people think of IQ. IQ is politically salient because it is linked to economic outcomes, and because it threatens to array racial groups on a continuum, with higher IQ being better. But real differences will be more complex than that. Think about MAOA: aggression is bad, right? But risk-taking, for example in entrepreneurship, isn't always bad. If there are differences between groups, we will not be able to order them hierarchically.
8. Genes are not destiny, in the large.
OK, here I am disagreeing with Nicholas Wade's position in the speculative second half of his book. Think about Caucasians versus Asians. In the 19th century, British gunboats were knocking on China's door. In 2014, China is the second power in the world, and pushing for first place. These changes happen too fast for genetics to be a useful explanation.
9. Genes are not destiny, in the small.
Genetic does not mean unchangeable. To take a simple example, many genetic diseases have simple cures. More generally, human choices, including human culture, can make up for - or perhaps sometimes amplify! - genetic differences. (Culture itself, by contrast, can be very hard to change.)
10. Liberal equality does not require us all to be identical.
Why don't we deny the vote to stupid people? We are equal not because we are all the same in every way, but because - as Christianity would put it - we are all possessors of a living soul. Similarly, we have a duty to judge individuals as individuals, and not on the basis of belonging to an ethnic group. Doing the former is usually efficient, since ethnicity is at best a weak predictor of someone's behaviour. It is also morally right.
It may be - I think it is quite likely - that genetic differences will account for many of the differences between people. They may also account for some of the differences between groups. Given the rate at which the science is developing, we would be very unwise not to start thinking about this. I do not want to be too sanguine about its consequences: obviously it would be, in some sense, "bad news" for human equality. At the same time, it is not a cause for liberal despair, or for racism and intergroup contempt to triumph. Whatever differences there are will be probabilistic not deterministic, multidimensional and complex; and morally outweighed by our underlying shared humanity - a truth supported equally by religious tradition, and by modern biology.
Tuesday, 29 July 2014
Non-shock: OKCupid experiments on humans, like every other big website
See this post from their blog. Of course, if this were an academic experiment (i.e. one with the purported aim of finding socially useful knowledge), it would have been highly unethical - experimenting on thousands of humans! Without their consent!
Clearly, society does not have an ethical problem with experimenting on humans: we let companies do that all the time. Since it is only academic experiments that have to pass stringent ethics committees, it must be that what is prima facie unethical is learning useful things from experiments on humans. We have taken our cue from Genesis:
Clearly, society does not have an ethical problem with experimenting on humans: we let companies do that all the time. Since it is only academic experiments that have to pass stringent ethics committees, it must be that what is prima facie unethical is learning useful things from experiments on humans. We have taken our cue from Genesis:
But of the tree of the knowledge of good and evil, thou shalt not eat of it: for in the day that thou eatest thereof thou shalt surely die.
Tuesday, 22 July 2014
This is how all my experiment designs should be
The standard condition consisted of an undergraduate female stationed by a 1964 Ford Mustang (control car) with a flat left-rear tire. An inflated tire was leaned upon the left side of the auto. The girl, the flat tire, and the inflated tire were conspicuous to the passing traffic....Find out what happens next.
Wednesday, 16 July 2014
Linkage
From the Iraqi bloggers I have in my bookmark folder. Many are shut down or have fallen silent.
War in Iraq.
Home is all about being the same.
War in Iraq.
Home is all about being the same.
Monday, 14 July 2014
The paedophilia scandal
(Back blogging again, just because I feel the need to write something for the non-academic audience, no matter how small!)
The political consequences, though, would surely be a final, wholesale collapse of trust in the political system. Duck houses on the taxpayer are one thing, but enabling child abuse.... If that happens, the best – best! – outcome I see is that UKIP sweep the board at general election time, bringing Neil Hamilton into office alongside a slew of fishy populist coves.
But a worse, and just as likely, outcome is: nothing. The duopoly nature of first-past-the-post elections means that no outside party can coordinate the voters to break through. Labour and Conservatives continue to dominate Parliament, with an absolute majority of the electorate not bothering to turn up. Democracy becomes a cynical game of mobilizing the parties' core supporters. There is a vacuum not of power but of authority. Politicians simply lack the credibility to impose change against any group's will. Through the first half of the new century, Britain flies on autopilot.
... condemned to watch the lingering agony of an exhausted country, to tend it during the alternate fits of stupefaction and raving which precede its dissolution, and to see the symptoms of vitality disappear one by one, till nothing is left but coldness, darkness, and corruption.
Sunday, 13 July 2014
Subscribe to:
Posts (Atom)