Document Detail

Partial BFGS update and efficient step-length calculation for three-layer neural networks.
MedLine Citation:
PMID:  9117895     Owner:  NLM     Status:  MEDLINE    
Second-order learning algorithms based on quasi-Newton methods have two problems. First, standard quasi-Newton methods are impractical for large-scale problems because they require N2 storage space to maintain an approximation to an inverse Hessian matrix (N is the number of weights). Second, a line search to calculate a reasonably accurate step length is indispensable for these algorithms. In order to provide desirable performance, an efficient and reasonably accurate line search is needed. To overcome these problems, we propose a new second-order learning algorithm. Descent direction is calculated on the basis of a partial Broydon-Fletcher-Goldfarb-Shanno (BFGS) update with 2Ns memory space (s < < N), and a reasonably accurate step length is efficiently calculated as the minimal point of a second-order approximation to the objective function with respect to the step length. Our experiments, which use a parity problem and a speech synthesis problem, have shown that the proposed algorithm outperformed major learning algorithms. Moreover, it turned out that an efficient and accurate step-length calculation plays an important role for the convergence of quasi-Newton algorithms, and a partial BFGS update greatly saves storage space without losing the convergence performance.
K Saito; R Nakano
Related Documents :
20125455 - Refining algorithm for the design of multilayer filters.
19206795 - Speech enhancement in discontinuous transmission systems using the constrained-stabilit...
11487835 - Staff dosimetry protocols in interventional radiology.
17521285 - Feature selection via coalitional game theory.
15506995 - Efficient two-sample designs for microarray experiments with biological replications.
23005495 - Detecting alternative graph clusterings.
Publication Detail:
Type:  Journal Article    
Journal Detail:
Title:  Neural computation     Volume:  9     ISSN:  0899-7667     ISO Abbreviation:  Neural Comput     Publication Date:  1997 Jan 
Date Detail:
Created Date:  1997-04-24     Completed Date:  1997-04-24     Revised Date:  2000-12-18    
Medline Journal Info:
Nlm Unique ID:  9426182     Medline TA:  Neural Comput     Country:  UNITED STATES    
Other Details:
Languages:  eng     Pagination:  123-41     Citation Subset:  IM    
NTT Communication Science Laboratories, Kyoto, Japan.
Export Citation:
APA/MLA Format     Download EndNote     Download BibTex
MeSH Terms
Models, Statistical
Neural Networks (Computer)*
Reproducibility of Results

From MEDLINE®/PubMed®, a database of the U.S. National Library of Medicine

Previous Document:  Flat minima.
Next Document:  Neural networks for functional approximation and system identification.