Document Detail

What you see may not be what you get: a brief, nontechnical introduction to overfitting in regression-type models.
MedLine Citation:
PMID:  15184705     Owner:  NLM     Status:  MEDLINE    
Statistical models, such as linear or logistic regression or survival analysis, are frequently used as a means to answer scientific questions in psychosomatic research. Many who use these techniques, however, apparently fail to appreciate fully the problem of overfitting, ie, capitalizing on the idiosyncrasies of the sample at hand. Overfitted models will fail to replicate in future samples, thus creating considerable uncertainty about the scientific merit of the finding. The present article is a nontechnical discussion of the concept of overfitting and is intended to be accessible to readers with varying levels of statistical expertise. The notion of overfitting is presented in terms of asking too much from the available data. Given a certain number of observations in a data set, there is an upper limit to the complexity of the model that can be derived with any acceptable degree of uncertainty. Complexity arises as a function of the number of degrees of freedom expended (the number of predictors including complex terms such as interactions and nonlinear terms) against the same data set during any stage of the data analysis. Theoretical and empirical evidence--with a special focus on the results of computer simulation studies--is presented to demonstrate the practical consequences of overfitting with respect to scientific inference. Three common practices--automated variable selection, pretesting of candidate predictors, and dichotomization of continuous variables--are shown to pose a considerable risk for spurious findings in models. The dilemma between overfitting and exploring candidate confounders is also discussed. Alternative means of guarding against overfitting are discussed, including variable aggregation and the fixing of coefficients a priori. Techniques that account and correct for complexity, including shrinkage and penalization, also are introduced.
Michael A Babyak
Related Documents :
11993755 - Evaluation of a novel structural model to describe the endogenous release of lead from ...
12631985 - Real-time modelling of influenza outbreaks--a linear regression analysis.
18852855 - Assessment of bias with emphasis on method comparison.
16100735 - Ranked set sampling for efficient estimation of a population proportion.
21807665 - Temporal variation of erythemally effective uvb/uva ratio at chilton, uk.
19578575 - Factors associated with the incidence of urban visceral leishmaniasis: an ecological st...
Publication Detail:
Type:  Journal Article    
Journal Detail:
Title:  Psychosomatic medicine     Volume:  66     ISSN:  1534-7796     ISO Abbreviation:  Psychosom Med     Publication Date:    2004 May-Jun
Date Detail:
Created Date:  2004-06-08     Completed Date:  2004-07-19     Revised Date:  2013-05-20    
Medline Journal Info:
Nlm Unique ID:  0376505     Medline TA:  Psychosom Med     Country:  United States    
Other Details:
Languages:  eng     Pagination:  411-21     Citation Subset:  IM    
Duke University Medical Center, Durham, NC 27710, USA.
Export Citation:
APA/MLA Format     Download EndNote     Download BibTex
MeSH Terms
Computer Simulation
Data Interpretation, Statistical
Models, Statistical
Psychophysiology / statistics & numerical data
Psychosomatic Medicine / statistics & numerical data
Regression Analysis
Research Design
Statistics as Topic / education,  methods*

From MEDLINE®/PubMed®, a database of the U.S. National Library of Medicine

Previous Document:  Historical sexual abuse and current thyroid axis profiles in women with premenstrual dysphoric disor...
Next Document:  Patients with carcinoid syndrome exhibit symptoms of aggressive impulse dysregulation.