Nonlinear Sciences > Adaptation and Self-Organizing Systems
[Submitted on 6 Jun 2020 (v1), last revised 2 Mar 2021 (this version, v2)]
Title:Multiple-timescale Neural Networks: Generation of Context-dependent Sequences and Inference through Autonomous Bifurcations
View PDFAbstract:Sequential transitions between metastable states are ubiquitously observed in the neural system and underlie various cognitive functions. Although a number of studies with asymmetric Hebbian connectivity have investigated how such sequences are generated, the focused sequences are simple Markov ones. On the other hand, supervised machine learning methods can generate complex non-Markov sequences, but these sequences are vulnerable against perturbations. Further, concatenation of newly learned sequence to the already learned one is difficult due to catastrophe forgetting, although concatenation is essential for cognitive functions such as inference. How stable complex sequences are generated still remains unclear. We have developed a neural network with fast and slow dynamics, which are inspired by the experiments. The slow dynamics store history of inputs and outputs and affect the fast dynamics depending on the stored history. We show the learning rule that requires only local information can form the network generating the complex and robust sequences in the fast dynamics. The slow dynamics work as bifurcation parameters for the fast one, wherein they stabilize the next pattern of the sequence before the current pattern is destabilized. This co-existence period leads to the stable transition between the current and the next pattern in the sequence. We further find that timescale balance is critical to this period. Our study provides a novel mechanism generating the robust complex sequences with multiple timescales in neural dynamics. Considering the multiple timescales are widely observed, the mechanism advances our understanding of temporal processing in the neural system.
Submission history
From: Tomoki Kurikawa [view email][v1] Sat, 6 Jun 2020 15:09:21 UTC (3,334 KB)
[v2] Tue, 2 Mar 2021 04:25:44 UTC (2,226 KB)
Current browse context:
nlin.AO
Change to browse by:
References & Citations
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
Connected Papers (What is Connected Papers?)
CORE Recommender (What is CORE?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.