Title:Reinforcement Learning Without Backpropagation or a Clock

Abstract:In this paper we introduce a reinforcement learning (RL) approach for training policies, including artificial neural network policies, that is both \emph{backpropagation-free} and \emph{clock-free}. It is \emph{backpropagation-free} in that it does not propagate any information backwards through the network. It is \emph{clock-free} in that no signal is given to each node in the network to specify when it should compute its output and when it should update its weights. We contend that these two properties increase the biological plausibility of our algorithms and facilitate distributed implementations. Additionally, our approach eliminates the need for customized learning rules for hierarchical RL algorithms like the option-critic.