Improving neural network training based on Jacobian rank deficiency

Guian Zhou, Jennie Si

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Analysis and experimental results obtained in [1] have revealed that many network training problems are ill-conditioned and may not be solved efficiently by the Gauss-Newton method. The Levenberg-Marquardt algorithm has been used successfully in solving nonlinear least squares problems, however only for reasonable size problems due to its significant computation and memory complexities within each iteration. In the present paper we develop a new algorithm in the form of a modified Gauss-Newton which on one hand takes advantage of the Jacobian rank deficiency to reduce computation and memory complexities, and on the other hand, still has similar features to the Levenberg-Marquardt algorithm with better convergence properties than first order methods.

Original languageEnglish (US)
Title of host publicationArtificial Neural Networks, ICANN 1996 - 1996 International Conference, Proceedings
PublisherSpringer Verlag
Pages531-536
Number of pages6
ISBN (Print)3540615105, 9783540615101
DOIs
StatePublished - Jan 1 1996
Event1996 International Conference on Artificial Neural Networks, ICANN 1996 - Bochum, Germany
Duration: Jul 16 1996Jul 19 1996

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume1112 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other1996 International Conference on Artificial Neural Networks, ICANN 1996
CountryGermany
CityBochum
Period7/16/967/19/96

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Improving neural network training based on Jacobian rank deficiency'. Together they form a unique fingerprint.

  • Cite this

    Zhou, G., & Si, J. (1996). Improving neural network training based on Jacobian rank deficiency. In Artificial Neural Networks, ICANN 1996 - 1996 International Conference, Proceedings (pp. 531-536). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 1112 LNCS). Springer Verlag. https://doi.org/10.1007/3-540-61510-5_91