Neural network optimization for good generalization performance

Generalization is one of the most important abilities of artificial neural networks. In order to improve the generalization performance, we have to choose a network which will not overfit the input data. There are however very few known theoretical ways of making this choice. We usually choose a network with an excess parameters and then try to simplify the network during the training.