Subset based training and pruning of sigmoid neural networks

Guian Zhou, Jennie Si

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In the present paper we develop two algorithms, Subset Based Training (SBT) and Subset Based Training and Pruning (SBTP), using the fact that the Jacobian matrices in sigmoid network training problems are usually rank deficient. The weight vectors are divided into two parts during training, according to the Jacobian rank sizes. Both SBT and SBTP are trust region methods. Comparing to the standard Levenberg-Marquardt (LM) method, these two algorithms can achieve similar convergence properties as the LM but with less memory requirements. Furthermore the SBTP combines training and pruning of a network into one comprehensive procedure. Some convergence properties of the two algorithms are given to qualitatively evaluate the performance of the algorithms.

Original languageEnglish (US)
Title of host publicationProceedings of the American Control Conference
Pages58-62
Number of pages5
Volume1
DOIs
StatePublished - 1998
Event1998 American Control Conference, ACC 1998 - Philadelphia, PA, United States
Duration: Jun 24 1998Jun 26 1998

Other

Other1998 American Control Conference, ACC 1998
CountryUnited States
CityPhiladelphia, PA
Period6/24/986/26/98

Fingerprint

Neural networks
Set theory
Jacobian matrices
Data storage equipment

ASJC Scopus subject areas

  • Electrical and Electronic Engineering

Cite this

Zhou, G., & Si, J. (1998). Subset based training and pruning of sigmoid neural networks. In Proceedings of the American Control Conference (Vol. 1, pp. 58-62). [694628] https://doi.org/10.1109/ACC.1998.694628

Subset based training and pruning of sigmoid neural networks. / Zhou, Guian; Si, Jennie.

Proceedings of the American Control Conference. Vol. 1 1998. p. 58-62 694628.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Zhou, G & Si, J 1998, Subset based training and pruning of sigmoid neural networks. in Proceedings of the American Control Conference. vol. 1, 694628, pp. 58-62, 1998 American Control Conference, ACC 1998, Philadelphia, PA, United States, 6/24/98. https://doi.org/10.1109/ACC.1998.694628
Zhou G, Si J. Subset based training and pruning of sigmoid neural networks. In Proceedings of the American Control Conference. Vol. 1. 1998. p. 58-62. 694628 https://doi.org/10.1109/ACC.1998.694628
Zhou, Guian ; Si, Jennie. / Subset based training and pruning of sigmoid neural networks. Proceedings of the American Control Conference. Vol. 1 1998. pp. 58-62
@inproceedings{80671a93f2ed4fd287c0ff2beac206d3,
title = "Subset based training and pruning of sigmoid neural networks",
abstract = "In the present paper we develop two algorithms, Subset Based Training (SBT) and Subset Based Training and Pruning (SBTP), using the fact that the Jacobian matrices in sigmoid network training problems are usually rank deficient. The weight vectors are divided into two parts during training, according to the Jacobian rank sizes. Both SBT and SBTP are trust region methods. Comparing to the standard Levenberg-Marquardt (LM) method, these two algorithms can achieve similar convergence properties as the LM but with less memory requirements. Furthermore the SBTP combines training and pruning of a network into one comprehensive procedure. Some convergence properties of the two algorithms are given to qualitatively evaluate the performance of the algorithms.",
author = "Guian Zhou and Jennie Si",
year = "1998",
doi = "10.1109/ACC.1998.694628",
language = "English (US)",
isbn = "0780345304",
volume = "1",
pages = "58--62",
booktitle = "Proceedings of the American Control Conference",

}

TY - GEN

T1 - Subset based training and pruning of sigmoid neural networks

AU - Zhou, Guian

AU - Si, Jennie

PY - 1998

Y1 - 1998

N2 - In the present paper we develop two algorithms, Subset Based Training (SBT) and Subset Based Training and Pruning (SBTP), using the fact that the Jacobian matrices in sigmoid network training problems are usually rank deficient. The weight vectors are divided into two parts during training, according to the Jacobian rank sizes. Both SBT and SBTP are trust region methods. Comparing to the standard Levenberg-Marquardt (LM) method, these two algorithms can achieve similar convergence properties as the LM but with less memory requirements. Furthermore the SBTP combines training and pruning of a network into one comprehensive procedure. Some convergence properties of the two algorithms are given to qualitatively evaluate the performance of the algorithms.

AB - In the present paper we develop two algorithms, Subset Based Training (SBT) and Subset Based Training and Pruning (SBTP), using the fact that the Jacobian matrices in sigmoid network training problems are usually rank deficient. The weight vectors are divided into two parts during training, according to the Jacobian rank sizes. Both SBT and SBTP are trust region methods. Comparing to the standard Levenberg-Marquardt (LM) method, these two algorithms can achieve similar convergence properties as the LM but with less memory requirements. Furthermore the SBTP combines training and pruning of a network into one comprehensive procedure. Some convergence properties of the two algorithms are given to qualitatively evaluate the performance of the algorithms.

UR - http://www.scopus.com/inward/record.url?scp=84881337044&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84881337044&partnerID=8YFLogxK

U2 - 10.1109/ACC.1998.694628

DO - 10.1109/ACC.1998.694628

M3 - Conference contribution

AN - SCOPUS:84881337044

SN - 0780345304

SN - 9780780345300

VL - 1

SP - 58

EP - 62

BT - Proceedings of the American Control Conference

ER -