Learning in neural networks

Introduction to class: No Backprop please; Background in neuroscience and hardware.

First Lecture: Hebbian learning for PCA

Hebbian learning for ICA

The problem of independent component analysis (ICA) is introduced and its relation to PCA (wee last week) is discussed. We will show that nonlinear Hebbian learning gives rise to ICA. To implement ICA, data normally needs to be centered and prewhitened. All concepts are discussed step by step.

This week we apply Hebbian learning rules in a network with lateral interactions. We will see that neurons can then extract several principal components or independent components. We will also see that such a network can perform k-means clustering. With strong inhibitory interactions one also talks about 'competitive dynamics' or competitive learning.

This lecture ends the overview of 2-factor rules. Next week we continue with Reinforcement learning and 3-factor rules.

A preliminary introduction to Reinforcement Learning (RL0):

We look at bandit problems which I call 1-step-horizon problems: you take an action and immediately the trial ends with a reward of variable magnitude.

RL 1 (continued). The discussion  on Reinforcement Learning in the multistep horizon is continued.

Bellman equation, SARSA, and variations of SARSA. Backup diagram. Eligibility traces.

RL2 - The full framework of TD learning, including deep reinforcement learning.

Policy gradient methods in Reinforcement Learning.

Final session on the Foundation of Reinforcement Learning.

We discuss the Actor-Critic architecture and the Actor-Critic-Algorithm in the narrow sense, sometimes also called advantage actor critic.

Reinforcement Learning and the Brain:  Surprise and Novelty are alternatives to Reward as a third factor.

Detailed Models of Plasticity with Spiking Neurons.

Representation learning in deep networks without Backprop.