Quantitative Biology > Neurons and Cognition
[Submitted on 8 Dec 2021]
Title:Suboptimal and trait-like reinforcement learning strategies correlate with midbrain encoding of prediction errors
View PDFAbstract:During probabilistic learning organisms often apply a sub-optimal "probability-matching" strategy, where selection rates match reward probabilities, rather than engaging in the optimal "maximization" strategy, where the option with the highest reward probability is always selected. Despite decades of research, the mechanisms contributing to probability-matching are still under debate, and particularly noteworthy is that no differences between probability-matching and maximization strategies have been reported at the level of the brain. Here, we provide theoretical proof for a computational model that explains the complete range of behaviors between pure maximization and pure probability-matching. Fitting this model to behavior of 60 participants performing a probabilistic reinforcement learning task during fMRI scanning confirmed the model-derived prediction that probability-matching relates to an increased integration of negative outcomes during learning, as indicated by a stronger coupling between midbrain BOLD signal and negative prediction errors. Because the degree of probability-matching was consistent within an individual across nine different conditions, our results further suggest that the tendency to express a particular learning strategy is a trait-like feature of an individual.
References & Citations
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
Connected Papers (What is Connected Papers?)
CORE Recommender (What is CORE?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.