Improving Convergence of Back-Propagation by Handling Flat-Spots in the Output Layer

摘要: Abstract Back-propagation (BP) [9, 5] is one of the most widely used procedures for training multi-layer artificial neural networks with sigmoid units. Though successful in a number applications, its convergence to set desired weights can be excruciatingly slow. Several modifications have been proposed improving learning speed [2, 4, 8, 1, 6]. The phenomenon flat-spots known play significant role slow BP [2]. formulation Learning rule prevents network from effectively presence flat-spots. In this paper we propose new approach minimize error such that occurring output layer are appropriately handled, thereby permitting learn even improvement provided by technique demonstrated on standard benchmark data-sets. More importantly, speedup obtained little or no increase computational requirements each iteration.