Neuromorphic Computing Resources

What do you need to know as background for neuromorphic research?

• Machine learning concepts

  • What is a deep neural network?
  • How do methods like stochastic gradient descent work?

• Spiking neural networks - how do they differ from DNNs?

Undergraduate students in this area have suggested the following resources as being helpful for these topics:

• DeepLizard Backpropagation and Deep Learning Videos - provides an in-depth but approachable introduction to deep learning concepts
• The How to Build a Brain Book by Chris Eliasmith

Suggested Background for Georgia Tech Undergrads

• A background in data structures and machine learning concepts (CS 1332 and 4641)
• Special topics courses like “Computation and the Brain” (CS8803-CAB) when offered
• Courses from the neuroscience undergraduate track like “Principles in Neuroscience” (NEURO 2001) would likely also be helpful but may require more of a biomedical background.

Additional Background and Related Topics

Dr. Sam Shapero (GTRI researcher working with LCA and related signal processing topics) suggests understanding the following topics for neuromorphic computing research:

• Porting DNN applications to SNNs

  • See this paper as an example for Loihi Massa, et al., An Efficient SNN for Recognizing Gestures…,IJCNN2020

• Non-linear dynamics and their attractors

  • (YouTube video by Dr. Liz Bradley)

• Basic neuroscience concepts like feed-forward processing, lateral inhibition, etc.

• Recurrent structures (SNN) to implement systems of non-linear equations

  • LCA is one example of implementing a Hopfield network
  • See papers like this one: An Overview of Neuromorphic Computing for AI-Enabled HW-Based Hopfield Neural Network

• Gradient descent applied to spiking circuits

  • See papers like this one, Huh, et al.,Gradient Descent for Spiking Neural Networks, NeurIPS18

• Basins of attraction; chaotic attractors

  • Blackmore, Nonlinear Dynamics, Chaos and Strange Attractors with Applications, 2016 slide overview

Videos

• Nengo and the Neural Engineering Framework (playlist)
• Spiking Neural Networks for More Efficient AI Algorithms
• Building a Computer Like Your Brain
• Neuromorphic computing with emerging memory devices
• Realizing the Promise of Spiking Neuromorphic Hardware - Loihi overview

Books

• How to build a brain by Chris Eliasmith - a book by one of the current leaders in neuromorphic computing research
• Neuronal Dynamics - an online book by EPFL
• Principles of Neural Science - well-regarded book that focuses on the neuroscience aspect of brain functions. Note this does not strictly talk about neuromorphic computing but it provides a good background as to how many neuromorphic systems are designed to mimic the brain.

Courses

• Computation and the Brain, CS8803 at Georgia Tech - a special topics course that covers the basics of neuroscience, relation to deep learning networks, and theoretical approaches to neuromorphic computing
• Harvard’s Fundamentals of Neuroscience EdX

Survey Papers

• A Survey of Neuromorphic Computing and Neural Networks in Hardware - a very comprehensive overview of neuromorphic computing from 2017
• Towards spike-based machine intelligence with neuromorphic computing - not a survey but a nice Nature article overview of neuromorphic computing and related challenges
• Spiking Neural Networks Hardware Implementations and Challenges: A Survey - a hardware focused survey from 2019

Reports on Neuromorphic Trends

These reports published in the US provide insight into the state-of-the-art and perspective on challenges and future research areas for neuromorphic computing.

1. Neuromorphic Computing – Architectures, Models, and Applications Workshop, Report
2. Neuromorphic Computing – From Materials Research to Systems Architecture Roundtable, Report
3. A Federal Vision for Future Computing: A Nanotechnology-Inspired Grand Challenge
4. Neuro-Inspired Computational Elements Workshop Report

Other Resources

• The Computer and the Brain by John von Neumann - an older book from one of the pioneers of computing. Note that neuroscience and computing have both evolved since this was written in 1958!