Design of Pattern Classifiers with the Updating Property Using Stochastic Approximation Techniques

A nonparametric training procedure for finding the optimal weights of the discriminant functions of a pattern classifier in any optimization criterion, expressible as a convex function from an arbitrary sequence of sample patterns, is proposed. This design procedure is based on the stochastic approximation technique, and has the updating property because it processes the sample patterns whenever they become available. This procedure is used to find the optimal weights for the least-mean-square error criterion, and is shown to require very simple computation which leads to simple im-plementation. Both two-category and multi-category cases are considered, and an acceleration scheme to increase the rate of convergence for the training procedure is also presented. These results are demonstrated by examples.