Tuesday, 17 November 2009

lme false convergence

If you are using the R package lme4, and get error messages about "false convergence", then use the option verbose=TRUE in your call to lmer, and examine the output, which shows how the estimates of your betas change as the estimation proceeds. If you see any values for your betas are rather small, then divide that variable by 10 or 100 or 1000. This seems to help lmer to converge correctly.

10 comments:

  1. Thanks David! I was scouring the many R listservs for a solution to this issue, and this was the only one that worked.

    ReplyDelete
  2. Anonymous1:35 pm

    This should probably be mentioned in the lme4 help. If anyone is still following up here, why exactly does this work so well? I mean is there a logical/numerical/statistical reason that I'm not grasping?

    For now it's a mystery to me but it sure does work wonders. I've literally spent days both recently and in the past worrying about overfitting and whatnot, never being able to pinpoint the problem, and all along the solution was this simple.

    So thank you Dr. Hugh-Jones!

    ReplyDelete
  3. I think the reason is that the numerical algorithm doesn't cope well if numbers get very large/small.

    ReplyDelete
  4. Marie Galligan11:06 pm

    Thanks for that tip, it was really helpful and worked wonders for me too. Attempted to fit quadratic and cubic terms for age and encountered the same problem. I was including them using poly() but it was making the model coefficients difficult to interpret.

    ReplyDelete
  5. Yael Kisel4:33 pm

    Thanks for this tip! It worked for me too, with the added bonus that it makes sense to me!

    ReplyDelete
  6. I had already spent quite a while with the same problem on lme and this trick worked :)...saved my day and my weekend, many thanks!

    ReplyDelete
  7. Anonymous12:00 am

    Will this solution work for lmer with binomial data with a yes/no response? If so, I don't see how I can just divide the variable by 10, 100, etc. Please help.

    ReplyDelete
  8. Almost surely your dependent variable will not be the problem - it will be one of your indep vars that is causing the problem.

    ReplyDelete
  9. Anonymous9:46 pm

    Oh, I apologize. Let me clarify. The independent variables are categorical factors (treatment, sex, breeding condition). How can I correct for this problem? Thanks!

    ReplyDelete
  10. Hmm, well in that case I wonder if this is the right problem. But it might be; if so, then I guess you need to create appropriate dummy variables and multiply or divide by some amount. Awkward but it should still work.

    ReplyDelete