Regression modeling in back-propagation and projection pursuit learning

摘要: We study and compare two types of connectionist learning methods for model-free regression problems: 1) the backpropagation (BPL); 2) projection pursuit (PPL) emerged in recent years statistical estimation literature. Both BPL PPL are based on projections data directions determined from interconnection weights. However, unlike use fixed nonlinear activations (usually sigmoidal) hidden neurons BPL, systematically approximates unknown activations. Moreover, estimates all weights simultaneously at each iteration, while cyclically (neuron-by-neuron layer-by-layer) iteration. Although have comparable training speed when a Gauss-Newton optimization algorithm, proves more parsimonious that requires fewer to approximate true function. To further improve performance PPL, an orthogonal polynomial approximation is used place supersmoother method originally proposed activation PPL. >