Techniques in Neural Learning

This paper takes ideas developed in a theoretical framework by Maass 8] and adapts them for a practical learning algorithm for feedforward sigmoid neural networks. A number of di erent techniques are presented which are based loosely around the common theme of taking advantage of the linearity of the net input to a neuron, or in other words the fact that there is only a single non-linearity per neuron. Some experimental results are included, though many of the ideas are as yet untested. The paper can therefore be viewed as a tool box o ering a selection of possible techniques for incorporation in practical, heuristic learning algorithms for multi-layer perceptrons.