Discussion of Richly Parameterized Linear Models: Additive, Time Series, and Spatial Models Using Random Effects.
For the foreseeable future, this page will be organized by chapter in the book.
My thanks to students who gave permission to post class projects they did in my course. (Go to this link for more info about my course, which I teach out of this book.) Remember these were done for a class, with a short hard deadline and no chance to revise, so you can't expect them to meet the same standard as a journal submission.
Chapter 3: Penalized splines as mixed linear models
Using the Lasso, or another method that selects variables and shrinks, to pick knot locations for a penalized spline (Open Question #1, p. 99). This exercise suggested replacing the normal distribution for the random effect, in the mixed-linear-model formulation of a penalized spline, with a different distribution, e.g., the double-exponential distribution that leads to the Lasso. The idea was that if you used, say, a truncated polynomial basis with a knot at every distinct predictor value, this approach would select knots and shrink changes at the retained knots, and thus provide a possible alternative to analysis methods that choose knot locations as part of the analysis (free-knot splines are one such method). Two students have tried this for class projects and it hasn't worked out so well. The students were Yunzhang Zhu in 2012 (his project is here) and Quan Zhang in 2014 (his project is here). It's possible this idea is workable but the right regularization hasn't been tried yet.
Point-wise confidence intervals for penalized spline fits (Open Question #2, p. 100). Among other things, Section 3.3.1 discusses point-wise confidence intervals for penalized spline fits as presented in Section 6.4 of Ruppert, Wand, and Carroll's (RWC's) Semiparametric Regression (2003). On p. 97, I argued that the problem was bias, not inadequately wide intervals, so that point-wise confidence intervals could be improved not by widening the confidence interval, as RWC and others suggest, but by bias-correcting its center. Kristen Cunanan tested this idea in her 2014 class project. Briefly, bias-correcting the center of the interval had little effect on coverage, at least for datasets of the size Kristen considered. My only consolation is that the competition (the intervals suggested by RWC and by Sun and Loader) were just as bad. The problem is that all three of these competitors do a lousy job of compensating for bias in the fit at corners, peaks, etc. My suggestion did a lousy job because although the mixed-linear-model machinery provides an estimate of the spline fit's bias at each predictor value, that estimated bias was far too small for the situations Kristen considered, which I had agreed were reasonable before Kristen computed the results. I've seen other problems, e.g., clustered data problems with decent-sized clusters, in which the bias can be estimated rather better than in Kristen's paper, so this weakness in the bias estimate appears to be problem-specific.
November 2019: Ning Dai, who was a PhD student in the U of MN's School of Statistics, has done something novel with this problem which I think is extremely cool because (a) it works and (b) it is easy to generalize to just about anything that includes smoothing. This work began as a project in my class but developed in a direction quite different from the class project. She stopped trying to get this work published because she changed direction in her work life; you can find the last version of the manuscript on arXiv.
Chapter 13: Random effects old and new
A different take on one kind of new-style random effect. While teaching this material recently, I realized that the discussion of the third variety of new-style random effect ("A sample has been drawn ...") wasn't quite right. I re-worked that for a talk; here are the slides for the talk. Note in particular pages 9 and 20-22, especially p. 22.
Chapter 19: Multiple Maxima in the Restricted Likelihood and Posterior
Another example of a restricted likelihood with two local maxima. Philip Reiss (Department of Statistics, University of Haifa, formerly Division of Biostatistics, NYU School of Medicine) sent me a second naturally-occurring example of a restricted likelihood with two maxima. (I say "naturally-occurring" because Lisa Henn's example in Chapter 19 was manufactured, i.e., not from a real dataset.) Philip's paper reporting this example (among other things) came out in March 2014 and is available online by clicking here; the example of two local maxima appears on page 243 of this article.