Title:Self-organized annealing in laterally inhibited neural networks shows power law decay

Abstract: In this paper we present a method which assigns to each layer of a multilayer neural network, whose network dynamics is governed by a noisy winner-take-all mechanism, an approximated temperature $\beta^{-1}$. This approximated temperature is obtained by comparison of a softmax mechanism where a temperature is well defined with the noisy winner-take-all mechanism. We apply this method to a multilayer neural network during learning the XOR-problem with a Hebb-like learning rule and show that after a transient the approximated temperature decreases in each layer according to a power law $\beta^{-1}\sim t^{-\gamma}$. This indicates a self-organized annealing behaviour induced by the learning rule itself instead of an external adjustment of a control parameter as in physically motivated optimization methods e.g. simulated annealing. As a consequence of this method one can connect the functional learning bebaviour of a neural network, objectively observable by the network error, to the neuronal level using the approximated temperature which serves as a kind of order parameter for the network performance.