Computer Science > Machine Learning
[Submitted on 2 Oct 2018 (v1), last revised 13 Jun 2019 (this version, v6)]
Title:EMI: Exploration with Mutual Information
View PDFAbstract:Reinforcement learning algorithms struggle when the reward signal is very sparse. In these cases, naive random exploration methods essentially rely on a random walk to stumble onto a rewarding state. Recent works utilize intrinsic motivation to guide the exploration via generative models, predictive forward models, or discriminative modeling of novelty. We propose EMI, which is an exploration method that constructs embedding representation of states and actions that does not rely on generative decoding of the full observation but extracts predictive signals that can be used to guide exploration based on forward prediction in the representation space. Our experiments show competitive results on challenging locomotion tasks with continuous control and on image-based exploration tasks with discrete actions on Atari. The source code is available at this https URL .
Submission history
From: Hyoungseok Kim [view email][v1] Tue, 2 Oct 2018 11:33:57 UTC (7,715 KB)
[v2] Thu, 4 Oct 2018 15:26:16 UTC (7,715 KB)
[v3] Tue, 27 Nov 2018 13:50:56 UTC (7,520 KB)
[v4] Thu, 24 Jan 2019 01:07:50 UTC (4,670 KB)
[v5] Tue, 14 May 2019 07:06:05 UTC (8,680 KB)
[v6] Thu, 13 Jun 2019 05:41:38 UTC (8,699 KB)
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Connected Papers (What is Connected Papers?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
alphaXiv (What is alphaXiv?)
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Hugging Face (What is Huggingface?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
CORE Recommender (What is CORE?)
IArxiv Recommender
(What is IArxiv?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.