Computer Science > Emerging Technologies
[Submitted on 23 Aug 2018 (this version), latest version 11 Sep 2020 (v5)]
Title:Insect cyborgs: Biological feature generators improve machine learning accuracy on limited data
View PDFAbstract:Despite many successes, machine learning (ML) methods such as neural nets often struggle to learn given small training sets. In contrast, biological neural nets (BNNs) excel at fast learning. We can thus look to BNNs for tools to improve performance of ML methods in this low-data regime.
The insect olfactory network, though simple, can learn new odors very rapidly. Its two key structures are a layer with competitive inhibition (the Antennal Lobe, AL), followed by a high dimensional sparse plastic layer (the Mushroom Body, MB). This AL-MB network can rapidly learn not only odors but also handwritten digits, better in fact than standard ML methods in the few-shot regime.
In this work, we deploy the AL-MB network as an automatic feature generator, using its Readout Neurons as additional features for standard ML classifiers. We hypothesize that the AL-MB structure has a strong intrinsic clustering ability; and that its Readout Neurons, used as input features, will boost the performance of ML methods.
We find that these "insect cyborgs", ie classifiers that are part-moth and part-ML method, deliver significantly better performance than baseline ML methods alone on a generic (non-spatial) 85-feature, 10-class task derived from the MNIST dataset. Accuracy improves by an average of 6% to 33% for N < 15 training samples per class, and by 6% to 10% for N > 15. Remarkably, these moth-generated features increase ML accuracy even when the ML method's baseline accuracy already exceeds the AL-MB's own limited capacity.
The two structures in the AL-MB, a competitive inhibition layer and a high-dimensional sparse layer with Hebbian plasticity, are novel in the context of artificial NNs but endemic in BNNs. We believe they can be deployed either prepended as feature generators or inserted as layers into deep NNs, to potentially improve ML performance.
Submission history
From: Charles Delahunt [view email][v1] Thu, 23 Aug 2018 05:01:05 UTC (346 KB)
[v2] Sat, 26 Jan 2019 22:11:48 UTC (328 KB)
[v3] Tue, 16 Apr 2019 18:21:02 UTC (271 KB)
[v4] Fri, 6 Dec 2019 00:49:17 UTC (308 KB)
[v5] Fri, 11 Sep 2020 22:19:39 UTC (353 KB)
Current browse context:
cs.ET
References & Citations
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
Connected Papers (What is Connected Papers?)
CORE Recommender (What is CORE?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.