Subset based training and pruning of sigmoid neural networks

Guian Zhou; Jennie Si

doi:10.1109/ACC.1998.694628

Subset based training and pruning of sigmoid neural networks

Guian Zhou, Jennie Si

Electrical Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

In the present paper we develop two algorithms, Subset Based Training (SBT) and Subset Based Training and Pruning (SBTP), using the fact that the Jacobian matrices in sigmoid network training problems are usually rank deficient. The weight vectors are divided into two parts during training, according to the Jacobian rank sizes. Both SBT and SBTP are trust region methods. Comparing to the standard Levenberg-Marquardt (LM) method, these two algorithms can achieve similar convergence properties as the LM but with less memory requirements. Furthermore the SBTP combines training and pruning of a network into one comprehensive procedure. Some convergence properties of the two algorithms are given to qualitatively evaluate the performance of the algorithms.

Original language	English (US)
Title of host publication	Proceedings of the 1998 American Control Conference, ACC 1998
Pages	58-62
Number of pages	5
DOIs	https://doi.org/10.1109/ACC.1998.694628
State	Published - 1998
Event	1998 American Control Conference, ACC 1998 - Philadelphia, PA, United States Duration: Jun 24 1998 → Jun 26 1998

Publication series

Name	Proceedings of the American Control Conference
Volume	1
ISSN (Print)	0743-1619

Other

Other	1998 American Control Conference, ACC 1998
Country/Territory	United States
City	Philadelphia, PA
Period	6/24/98 → 6/26/98

ASJC Scopus subject areas

Electrical and Electronic Engineering

Access to Document

10.1109/ACC.1998.694628

Cite this

@inproceedings{80671a93f2ed4fd287c0ff2beac206d3,

title = "Subset based training and pruning of sigmoid neural networks",

abstract = "In the present paper we develop two algorithms, Subset Based Training (SBT) and Subset Based Training and Pruning (SBTP), using the fact that the Jacobian matrices in sigmoid network training problems are usually rank deficient. The weight vectors are divided into two parts during training, according to the Jacobian rank sizes. Both SBT and SBTP are trust region methods. Comparing to the standard Levenberg-Marquardt (LM) method, these two algorithms can achieve similar convergence properties as the LM but with less memory requirements. Furthermore the SBTP combines training and pruning of a network into one comprehensive procedure. Some convergence properties of the two algorithms are given to qualitatively evaluate the performance of the algorithms.",

author = "Guian Zhou and Jennie Si",

note = "Funding Information: Supported in part by the NSF under grant ECS-9553202, by the EPRI under grant RP8015-03, and by Motorola. ; 1998 American Control Conference, ACC 1998 ; Conference date: 24-06-1998 Through 26-06-1998",

year = "1998",

doi = "10.1109/ACC.1998.694628",

language = "English (US)",

isbn = "0780345304",

series = "Proceedings of the American Control Conference",

pages = "58--62",

booktitle = "Proceedings of the 1998 American Control Conference, ACC 1998",

}

TY - GEN

T1 - Subset based training and pruning of sigmoid neural networks

AU - Zhou, Guian

AU - Si, Jennie

N1 - Funding Information: Supported in part by the NSF under grant ECS-9553202, by the EPRI under grant RP8015-03, and by Motorola.

PY - 1998

Y1 - 1998

N2 - In the present paper we develop two algorithms, Subset Based Training (SBT) and Subset Based Training and Pruning (SBTP), using the fact that the Jacobian matrices in sigmoid network training problems are usually rank deficient. The weight vectors are divided into two parts during training, according to the Jacobian rank sizes. Both SBT and SBTP are trust region methods. Comparing to the standard Levenberg-Marquardt (LM) method, these two algorithms can achieve similar convergence properties as the LM but with less memory requirements. Furthermore the SBTP combines training and pruning of a network into one comprehensive procedure. Some convergence properties of the two algorithms are given to qualitatively evaluate the performance of the algorithms.

AB - In the present paper we develop two algorithms, Subset Based Training (SBT) and Subset Based Training and Pruning (SBTP), using the fact that the Jacobian matrices in sigmoid network training problems are usually rank deficient. The weight vectors are divided into two parts during training, according to the Jacobian rank sizes. Both SBT and SBTP are trust region methods. Comparing to the standard Levenberg-Marquardt (LM) method, these two algorithms can achieve similar convergence properties as the LM but with less memory requirements. Furthermore the SBTP combines training and pruning of a network into one comprehensive procedure. Some convergence properties of the two algorithms are given to qualitatively evaluate the performance of the algorithms.

UR - http://www.scopus.com/inward/record.url?scp=84881337044&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84881337044&partnerID=8YFLogxK

U2 - 10.1109/ACC.1998.694628

DO - 10.1109/ACC.1998.694628

M3 - Conference contribution

AN - SCOPUS:84881337044

SN - 0780345304

SN - 9780780345300

T3 - Proceedings of the American Control Conference

SP - 58

EP - 62

BT - Proceedings of the 1998 American Control Conference, ACC 1998

T2 - 1998 American Control Conference, ACC 1998

Y2 - 24 June 1998 through 26 June 1998

ER -

Subset based training and pruning of sigmoid neural networks

Abstract

Publication series

Other

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this