Sample sizes for sigmoidal neural networks

This paper applies the theory of Probably Approximately Correct(PAC) learning to feedforward neural networks with sigmoidal activation functions. Despite the best known upper bound on the VC dimension of such networks being 0 ((WlV) 2), for W parameters and N computational nodes, it is shown that the asymptotic bound on the sample size required for learning with increasing accuracy 1–c and decreasing probabllty of failure d is o ((l/e)(wlog (l/c)+(WN) 2+ log (l/6)), For practical values of. sand d the formula obtained for the sample sizes is a factor 2 log (2e/c) smaller than a naive use of the VC dimension result would give. Similar results are obtained for learning where the hypothesis is only guaranteed to correctly classify a given proportion of the training sample. The results are formulated in general terms and show that for many learning classes defined by smooth functions thresholded at the output, the …