Nav: Back

Shrinkage Regression REFERENCES

Brown, L. (1975). "Estimation with incompletely specified loss functions (the case of several location parameters.)" Journal American Statistical Association 70, 417-427.

Breiman, L. (1995). "Better subset regression using the non-negative garrote." Technometrics 37, 373-384.

Bunke, 0. (1975). "Least squares estimators as robust and minimax estimators." Math. Operations forsch u. Statist. 6, 687-688.

Burr, TL and Fry, HA.  (2005).  "Biased regression: the case for cautious application."  Technometrics  47, 284-296.  [Includes references to material on this website.]

Casella, G. (1980). "Minimax ridge regression estimation." Annals of Statistics 8, 1036-1056.

Casella, G. (1985). "Condition numbers and minimax ridge-regression estimators." Journal American Statistical Association 80, 753-758.

Chatterjee, S and Hadi, A. S. (1988). Sensitivity Analysis in Regression. New York: John Wiley.

Cook, R. D. and Weisberg, S. (1994). Introduction to Regression Graphics. New York: John Wiley.

Cook, R. D. (1977). "Detection of influential observations in linear regression." Technometrics 19, 15-18.

Croghan, T.W., Obenchain. R.L. and Crown, W.E. (1998). "What does treatment of depression really cost?" Health Affairs 17(4); 198-208. View/Download Paper

Computation of marginal effects from an ill-conditioned model of log(cost.) View/Download Appendix Technical details of shrinkage and smearing methods.

Efron B. and Morris, C. N. (1976). "Discussion" (of Dempster, Schatzoff and Wermuth.) Journal American Statistical Association 72, 91-93.

Efron B, Hastie T, Johnstone I, Tibshirani R.  (2004). “Least Angle Regression.”  Annals of Statistics 32: 407-499 (with discussion.)

Frank, I. E. and Freidman, J. H. (1993). "A statistical view of some chemometrics regression tools." (with discussion) Technometrics 35, 109-148.

Fu, W. J. (1997). "Penalized Regressions: the Bridge versus the Lasso." One of 4 winners in the 1997 student paper competition sponsored by the ASA Statistical Computing Section.

Wenjiang's thesis under Rob Tibshani at the University of Toronto is entitled "A Statistical Shrinkage Model and Its Applications."

Gibbons, D. G. (1981). "A simulation study of some ridge estimators." Journal of the American Statistical Association 76, 131-139.

Goldstein M. and Smith, A. F. M (1974). "Ridge-type estimators for regression analysis." Journal of the Royal Statistical Society B 36, 284-291.

Golub, G. H., Heath, M., and Wahba, G. (1979). "Generalized cross- validation as a method for choosing a good ridge parameter." Technometrics 21, 215-223.

Gruber, M. H. J. (1998). Improving Efficiency by Shrinkage: The James-Stein and Ridge Regression Estimators.  New York: Marcel Dekker.

Hoerl, A. E. and Kennard, R. W. (1970a). "Ridge regression: biased estimation for nonorthogonal problems." Technometrics 12, 55-67.

Hoerl, A. E. and Kennard, R. W. (1970b). "Ridge regression: applications to nonorthogonal problems." Technometrics 12, 69-82.

LeBlanc, M. and Tibshirani, R. (1998).  "Monotone shrinkage of trees."   Journal of Computational and Graphical Statistics 7, 417-433.

Longley, J. W. (1967). "An appraisal of least squares programs for the electronic computer from the point of view of the user." Journal of the American Statistical Association 62, 819-841.

Mallows, C. L. (1973). "Some comments on Cp." Technometrics 15, 661-677.

Mallows, C. L. (1995). "More comments on Cp." Technometrics 37, 362-372.

Marquardt, D. W. (1970). "Generalized inverses, ridge regression, biased linear estimation, and nonlinear estimation." Technometrics 12, 591-612.

Marquardt, D. W. (1980). "Comment: You should standardize the predictor variables in your regression models." Journal of the American Statistical Association 75, 87-91.

Massy, W. F. (1965). "Principal components regression in exploratory statistical research." Journal American Statistical Association 60, 234-256.

Nash, J. C. (1992). "Statistical shareware: illustrations from regression techniques." The American Statistician 46, 312-318.

Nash reviewed PC software that included softRX freeware stand-alone (DOS) applications.

Obenchain, R. L. and Vinod, H. (1974). "Estimates of partial derivatives from ridge regression on ill-conditioned data." NBER-NSF Seminar on Bayesian Inference in Econometrics, Ann-Arbor, Michigan.

Obenchain, R. L. (1975a). "Residual optimality: ordinary vs. weighted vs. biased least squares." Journal of the American Statistical Association 70, 375-379.

Ordinary least squares and diagonally-weighted least squares always produce residuals that are optimal estimators of lack-of-fit ...even when the expectation model and/or dispersion model you have specified are/is wrong!

Obenchain, R. L. (1975b). "Ridge analysis following a preliminary test of the shrunken hypothesis" (with discussion.) Technometrics 17, 431-445.

Classical, normal-theory monitoring of the likelihood-of-MSE-optimality along ANY shrinkage path.

Obenchain, R. L. (1977). "Classical F-tests and confidence regions for ridge regression." Technometrics 19, 429-439.

See this paper and/or my Frequently-Asked-Questions page for information on confidence intervals in shrinkage regression.

Obenchain, R. L. (1978). "Good and optimal ridge estimators." Annals of Statistics 6, 1111-1121.

The "ridge-function" theorem; maximum likelihood estimation of the "inferior direction" and of scaled MSE risk along any direction in p-dimensional regression coefficient space; MSE risk optimality of 0 or X'y along directions orthogonal to or parallel to beta, respectively.

Obenchain, R. L. (1980). Comment on "A critique of some ridge regression methods." Journal of the American Statistical Association 75, 95-96.

Obenchain, R. L. (1981). "Maximum likelihood ridge regression and the shrinkage pattern alternatives." I.M.S. Bulletin 10, 37; Absract 81t-23. (67 page review article.)

Obenchain, R. L. (1984). "Maximum likelihood ridge displays." Communications in Statistics A 13, 227-240.

Proceedings of the Fordham Ridge Symposium, ed. H. D. Vinod; illustrations of usage of RXridge SAS/IML freeware.

Obenchain, R. L. (1995). "Maximum likelihood ridge regression." Stata Technical Bulletin 28, 22-36.

Introduction to basic shrinkage/ridge concepts; illustrations of usage of RXridge .ADO freeware.

Obenchain, R. L. (1997a). "Maximum likelihood shrinkage in regression."

Closed form expressions for classical, fixed coefficient, maximum likelihood estimation within the 2-parameter shrinkage family, and some simulated MSE risk profiles.

Obenchain, R. L. (1997b). "Influential observations in ridge regression."

Exposition of basic "Visual Re-Regression" concepts.

Obenchain, R. L. (1997c). Shrinkage Regression: ridge, BLUP, Bayes and Stein. Preliminary draft (200+ pages.)

Obenchain, R. L. (2005). RXshrink: an R package for maximum likelihood shrinkage in generalized (2-parameter) ridge regression or least angle regression (LAR).  Or... Download Windows package from this site

Obenchain RL.  PDF File containing copies of all the Shrinkage Regression Tutorial pages from this web site. [16 pages.] 2008.  View/Download PDF copy

Shumway, R. H. (1982). "Maximum likelihood estimation of the ridge parameter in linear regression." Technical Report, Department of Statistics, University of California at Davis.

Stein, C. M. (1956). "Inadmissibility of the usual estimator of the mean of a multivariate normal distribution." Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Pobability 1, 197-206. University of California Press.

Strawderman, W. E. (1978). "Minimax adaptive generalized ridge regression estimators." Journal American Statistical Association 73, 623-627.

Theil, H. (1963). "On the use of incomplete prior information in regression analysis." Journal of the American Statistical Association 58, 401-414.

Thisted, R. (1976). "Ridge regression, minimax estimation, and empirical bayes methods." Technical Report No. 28, Division of Biostatistics, Stanford University.

Tibshirani, R. (1996). "Regression shrinkage and selection via the lasso." Journal of the Royal Statistical Society B 58, 267-288.

Tierney, Luke. (1990). LISP-STAT: An Object-Oriented Environment for Statistical Computing and Dynamic Graphics. New York: John Wiley and Sons.

Tukey, J. W. (1975). "Instead of Gauss-Markov Least Squares; What?" Applied Statistics, ed. R. P. Gupta. Amsterdam-New York: North Holland Publishing Company.

Vinod, H. D. and Ullah, A. (1981). Recent Advances in Regression Methods. New York: Marcel Dekker.

Vinod, H. D. (1995). "Double bootstrap for shrinkage estimators." Journal of Econometrics 68, 287-302.

Walter, Bernhard (1994). "XLisp-Stat code for shrinkage/ridge regression." Techniche Universitad Munich.

Nav: Back