A recurrent network with stochastic weights

Stochastic neural networks for global optimization are usually built by introducing random fluctuations into the network. A natural method is to use stochastic weights rather than stochastic activation functions. We propose a new model in which each neuron has very simple functionality but all the weights are stochastic. It is shown that the stationary distribution of the network uniquely exists and it is approximately a Boltzmann-Gibbs distribution when the size of the network is not too small. A new technique to implement simulated annealing is proposed. Simulation results on the graph bisection problem show that the power of the network is comparable with that of a Boltzmann machine.