How Do You Rate This Publication?       

Title:	Fast Back-Propagation Learning Using Steep Activation Functions and Automatic Weight
Author(s):	Cho, Tai-Hoon; Conners, Richard W.; Araman, Philip A.
Date:	1992
Source:	Proceedings, 1991 IEEE International Conference on Systems, Man, and Cybernetics. pp. 1587-1592.
Description:	In this paper, several back-propagation (BP) learning speed-up algorithms that employ the ãgainä parameter, i.e., steepness of the activation function, are examined. Simulations will show that increasing the gain seemingly increases the speed of convergence and that these algorithms can converge faster than the standard BP learning algorithm on some problems. However, these algorithms may also suffer from increased instability, i.e., they frequently fail to converge within a finite time. One potential cause for the instability is an inappropriate choice for the initial weights. To overcome the instability resulting from this cause it is proposed that automatic weight reinitialization be used whenever the convergence speed becomes ãvery slowä due to a local minimum or premature saturation. On the simulations performed BP algorithms with larger initial gain (around 2 or 3) and automatic weight reinitialization converged much faster and were more stable than algorithms employing the same gain but not employing automatic weight reinitialization. The simulations performed involved a diverse set of problems including exclusive-or (XOR), encoder, and parity problems.
	View and Print this Publication (65 KB)
Publication Notes:	We recommend that you also print this page and attach it to the printout of the article, to retain the full citation information. This article was written and prepared by U.S. Government employees on official time, and is therefore in the public domain. Our on-line publications are scanned and captured using Adobe Acrobat. During the capture process some typographical errors may occur. Please contact the SRS Webmaster, srswebmaster@fs.fed.us if you notice any errors which make this publication unuseable.
	Get the latest version of the Adobe Acrobat reader or Acrobat Reader for Windows with Search and Accessibility