Document Detail


Partial BFGS update and efficient step-length calculation for three-layer neural networks.
MedLine Citation:
PMID:  9117895     Owner:  NLM     Status:  MEDLINE    
Abstract/OtherAbstract:
Second-order learning algorithms based on quasi-Newton methods have two problems. First, standard quasi-Newton methods are impractical for large-scale problems because they require N2 storage space to maintain an approximation to an inverse Hessian matrix (N is the number of weights). Second, a line search to calculate a reasonably accurate step length is indispensable for these algorithms. In order to provide desirable performance, an efficient and reasonably accurate line search is needed. To overcome these problems, we propose a new second-order learning algorithm. Descent direction is calculated on the basis of a partial Broydon-Fletcher-Goldfarb-Shanno (BFGS) update with 2Ns memory space (s < < N), and a reasonably accurate step length is efficiently calculated as the minimal point of a second-order approximation to the objective function with respect to the step length. Our experiments, which use a parity problem and a speech synthesis problem, have shown that the proposed algorithm outperformed major learning algorithms. Moreover, it turned out that an efficient and accurate step-length calculation plays an important role for the convergence of quasi-Newton algorithms, and a partial BFGS update greatly saves storage space without losing the convergence performance.
Authors:
K Saito; R Nakano
Related Documents :
18252635 - Exploring constructive cascade networks.
15551585 - Evaluation of task-oriented performance of several fully 3d pet reconstruction algorithms.
11752765 - Algorithms for deriving crystallographic space-group information. ii. treatment of spec...
17271815 - Detection and segmentation of changes of dynamics in biological time series.
23451315 - A new tool for testing and training ophthalmoscopic skills.
8338505 - New technologies for in-flight pasteless bioelectrodes.
Publication Detail:
Type:  Journal Article    
Journal Detail:
Title:  Neural computation     Volume:  9     ISSN:  0899-7667     ISO Abbreviation:  Neural Comput     Publication Date:  1997 Jan 
Date Detail:
Created Date:  1997-04-24     Completed Date:  1997-04-24     Revised Date:  2000-12-18    
Medline Journal Info:
Nlm Unique ID:  9426182     Medline TA:  Neural Comput     Country:  UNITED STATES    
Other Details:
Languages:  eng     Pagination:  123-41     Citation Subset:  IM    
Affiliation:
NTT Communication Science Laboratories, Kyoto, Japan.
Export Citation:
APA/MLA Format     Download EndNote     Download BibTex
MeSH Terms
Descriptor/Qualifier:
Algorithms*
Learning*
Memory*
Models, Statistical
Neural Networks (Computer)*
Reproducibility of Results

From MEDLINE®/PubMed®, a database of the U.S. National Library of Medicine


Previous Document:  Flat minima.
Next Document:  Neural networks for functional approximation and system identification.