Optimization with the Hopfield network based on correlated noises: an empirical approach

This paper presents two simple optimization techniques based on combining the Langevin Equation with the Hopfield Model. Proposed models - referred as Stochastic Model (SM) and Pulsed Noise Model (PNM) - can be viewed as straightforward stochastic extensions of the Hopfield optimization network. Optimization with SM, unlike in previous related models, in which delta-correlated Gaussian noises were considered, is based on Gaussian noises with positive autocorrelation times. This is a reasonable assumption from a hardware implementation point of view. In the other model - PNM, Gaussian noises are injected to the system only at certain time instances, as opposite to continuously maintained delta-correlated noises used in the previous related works. In both models (SM and PNM), intensities of noises added to the model are independent of neurons' potentials. Moreover, instead of impractically long inverse logarithmic cooling schedules, linear cooling is tested. With the above strong simplifications neither SM nor PNM is expected to rigorously maintain Thermal Equilibrium (TE). However, approximate numerical tests based on the canonical Gibbs-Boltzmann distribution show, that differences between rigorous and estimated values of TE parameters are relatively low (within a few percent). In this sense both models are said to perform Quasi Thermal Equilibrium. Optimization performance and Quasi Thermal Equilibrium properties of both models are tested on the Traveling Salesman Problem.

ICSI Technical Report TR-97-019

Technical Report