Abstract
In this paper we introduce an advanced supervised training method for neural networks. It is based on Jacobian rank deficiency and it is formulated, in some sense, in the spirit of the Gauss-Newton algorithm. The Levenberg-Marquardt algorithm, as a modified Gauss-Newton, has been used successfully in solving nonlinear least squares problems including neural-network training. It outperforms (in terms of training accuracy, convergence properties, overall training time, etc.) the basic backpropagation and its variations with variable learning rate significantly, however, with higher computation and memory complexities within each iteration. The new method developed in this paper is aiming at improving convergence properties, while reducing the memory and computation complexities in supervised training of neural networks. Extensive simulation results are provided to demonstrate the superior performance of the new algorithm over the Levenberg-Marquardt algorithm.
Original language | English (US) |
---|---|
Pages (from-to) | 448-453 |
Number of pages | 6 |
Journal | IEEE Transactions on Neural Networks |
Volume | 9 |
Issue number | 3 |
DOIs | |
State | Published - Dec 1 1998 |
Keywords
- Gauss-Newton method
- Jacobian rank deficiency
- Neural-network training
- Subset updating
- Trust region algorithms
ASJC Scopus subject areas
- Software
- Computer Science Applications
- Computer Networks and Communications
- Artificial Intelligence