Subset-based training and pruning of sigmoid neural networks

Guian Zhou, Jennie Si

Research output: Contribution to journalArticle

8 Scopus citations

Abstract

In the present paper we develop two algorithms, subset-based training (SBT) and subset-based training and pruning (SBTP), using the fact that the Jacobian matrices in sigmoid network training problems are usually rank deficient. The weight vectors are divided into two parts during training, according to the Jacobian rank sizes. Both SBT and SBTP are trust-region methods. Compared with the standard Levenberg-Marquardt (LM) method, these two algorithms can achieve similar convergence properties as the LM but with fewer memory requirements. Furthermore the SBTP combines training and pruning of a network into one comprehensive procedure. The effectiveness of the two algorithms is evaluated using three examples. Comparisons are made with some existing algorithms. Some convergence properties of the two algorithms are given to qualitatively evaluate the performance of the algorithms.

Original languageEnglish (US)
Pages (from-to)79-89
Number of pages11
JournalNeural Networks
Volume12
Issue number1
DOIs
StatePublished - Jan 1999

Keywords

  • Jacobian matrix
  • Neural networks
  • Nonlinear least squares
  • Pruning
  • Rank deficiency
  • Subset-based updating
  • Training
  • Trust regions

ASJC Scopus subject areas

  • Cognitive Neuroscience
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'Subset-based training and pruning of sigmoid neural networks'. Together they form a unique fingerprint.

  • Cite this