| General conditions for predictivity in learning theory. | |
| | |
MedLine Citation:
|
PMID: 15042089 Owner: NLM Status: MEDLINE |
Abstract/OtherAbstract:
|
Developing theoretical foundations for learning is a key step towards understanding intelligence. 'Learning from examples' is a paradigm in which systems (natural or artificial) learn a functional relationship from a training set of examples. Within this paradigm, a learning algorithm is a map from the space of training sets to the hypothesis space of possible functional solutions. A central question for the theory is to determine conditions under which a learning algorithm will generalize from its finite training set to novel examples. A milestone in learning theory was a characterization of conditions on the hypothesis space that ensure generalization for the natural class of empirical risk minimization (ERM) learning algorithms that are based on minimizing the error on the training set. Here we provide conditions for generalization in terms of a precise stability property of the learning process: when the training set is perturbed by deleting one example, the learned hypothesis does not change much. This stability property stipulates conditions on the learning map rather than on the hypothesis space, subsumes the classical theory for ERM algorithms, and is applicable to more general algorithms. The surprising connection between stability and predictivity has implications for the foundations of learning theory and for the design of novel algorithms, and provides insights into problems as diverse as language learning and inverse problems in physics and engineering. |
| | |
Authors:
|
Tomaso Poggio; Ryan Rifkin; Sayan Mukherjee; Partha Niyogi |
Related Documents
:
|
16689919 - Organisational learning and self-adaptation in dynamic disaster environments. 16856649 - A statistical property of multiagent learning based on markov decision process. 480269 - Frustration-instigated behavior and learned helplessness. 19172169 - Metric learning using iwasawa decomposition. 22665429 - What is the optimum duration of an asynchronous distance learning course? 17343839 - Learning non-local dependencies. 8594109 - Effectiveness of standardized patient instruction. 3390869 - Stretching, strengthening, and conditioning for the competitive tennis player. 8214839 - Beliefs and practices of emergency medicine faculty and residents regarding professiona... |
Publication Detail:
|
Type: Journal Article; Research Support, Non-U.S. Gov't; Research Support, U.S. Gov't, Non-P.H.S. |
Journal Detail:
|
Title: Nature Volume: 428 ISSN: 1476-4687 ISO Abbreviation: Nature Publication Date: 2004 Mar |
Date Detail:
|
Created Date: 2004-03-25 Completed Date: 2004-04-12 Revised Date: 2006-11-15 |
Medline Journal Info:
|
Nlm Unique ID: 0410462 Medline TA: Nature Country: England |
Other Details:
|
Languages: eng Pagination: 419-22 Citation Subset: IM |
Affiliation:
|
Center for Biological and Computational Learning, McGovern Institute Computer Science Artificial Intelligence Laboratory, Brain Sciences Department, MIT, Cambridge, Massachusetts 02139, USA. tp@ai.mit.edu |
Export Citation:
|
APA/MLA Format Download EndNote Download BibTex |
| MeSH Terms | |
Descriptor/Qualifier:
|
Algorithms* Intelligence Language Learning / physiology* Models, Theoretical Probability Research Design |
| Comments/Corrections | |
Comment In:
|
Nature. 2004 Mar 25;428(6981):378
[PMID:
15042073
]
|
From MEDLINE®/PubMed®, a database of the U.S. National Library of Medicine
Previous Document: Myosin gene mutation correlates with anatomical changes in the human lineage.
Next Document: Imaging cortical correlates of illusion in early visual cortex.