Probabilistic bit stream neural chip: Theory

A design for a neural network chip is proposed using probabilistic bit streams to represent real values. This paper analyzes the performance of the proposed neurons in this design and demonstrates that very simple operations can be used to obtain the desired functionality. It is also shown that a suitable ‘activation function’ for neurons of this type can be obtained using the interaction of two probability distributions. Finally, the paper introduces a variant of the back-propagation learning algorithm which involves computing the derivatives of the output with respect to individual weights in a network of such units.