Learning is Complicated

Humans ability to learn new things is astounding! Just look at all the things we've created and use today. Now we're trying to get machines to learn like us. Unfortunately, we haven't figured out how to do this very well yet. We've created interesting AI systems, like the large language models (LLMs) we see running a lot of AI tools. None of them can learn like we do, though. They really can't even learn like animals do. Instead, today's AI is trained on a huge amount of data and learns statistical patterns that determine the AI's output, depending on its current input. Once it is trained, it cannot learn new information without potentially forgetting a lot of other information. AI cannot continuously learn like we do. But why not?

How Machines Learn

AI typically uses deep learning neural network algorithms. These neural networks are made of multiple layers of nodes that are connected to nodes in neighboring layers with weighted connections. The weighted connections are what adapt during learning. This adaptation is performed through an algorithm invented in 1986 called backpropagation (Rumelhart D. et al.). Describing it simply, information is passed through the network, and its outputs are compared to the desired outputs. The difference between them is the error of the neural network. This error is passed backwards through the network using the backpropagation algorithm to adjust weights so that the network's outputs come closer to the desired outputs. This is repeated thousands of times to get the network to learn to map inputs to outputs. This is not how the brain learns.

One thing the backpropagation seems to have correct is that the connections between neurons are strengthened or weakened during learning. One other similarity between backpropagation and how neurons learn is that neurons backpropagate their action potential (spike) from the soma (the cell body) back through the dendrites. This is where the similarity between backpropagation and the brain ends.

The Brain's "Learning Algorithm" is Very Complex

From what we understand about the brain so far, neurons alter synaptic connection strengths through multiple means. First, there is a short-term plasticity that synapses have based on the spike input frequency they receive. Synapses can undergo synaptic facilitation, where the synapse temporarily gets stronger, or synaptic depression, where the synapse temporarily gets weaker (Tsodyks, M. et al.). Depending on the synapse, it may undergo both based on the spike frequency. This allows a synapse to be a temporal information filter. The change in the synapse's strength decays quickly over a matter of milliseconds to seconds, hence being short-term.

Synapses also experience long-term plasticity through long-term potentiation and long-term depression (Maffei, A). Long-term potentiation (LTP) typically occurs when a higher level of calcium has entered the synapse.  LTP strengthens a synapse and this change can last from minutes to hours. Long-term depression (LTD) typically occurs when a lower level of calcium (some but not none) enters the synapse. LTD weakens a synapse and, like LTP, the change can last from minutes to hours. It gets even more complicated, thought.

The timing of input spikes to a synapse and the output spike produced by the synapse's neuron is crucial. LTP and LTD are a part of a more complex system called spike-timing dependent plasticity (STDP). STDP is where learning occurs within a specific time window between when the synapse receives an input spike and the synapse receives a backpropagating output spike from its neuron (Shouval, H. Z. et al.). If the spikes do not occur close enough in time to each other, no learning occurs. Generally, it is presumed that if an input occurs before the output, LTP occurs at the synapse, and if the input happens after the output, LTD occurs. Sometimes, this can be reversed. However, the full truth is much more complicated. Depending on the neuron and its morphology (shape) and the current inputs flowing through the neuron, there are various different time window shapes for STDP learning. But it gets even more complex.

The brain is full of neurons that modulate other neurons and/or synapses through neuromodulators. Also, the brain contains glial cells, called astrocytes, that wrap around and influence synapses. Both of these are capable of altering the shape of a synapse's STDP learning time window (Foncelle, A. et al.). This creates an incredibly complex dynamic, especially when a neuromodulator, like dopamine, can switch the change in a synapse from LTD to LTP. Some neuromodulators can cause a synapse to only undergo LTD or LTP only. With so many things being able to influence or change the learning that a synapse experiences, it is obvious that there is no single "learning algorithm" in the brain.

So Can We Get Machines To Learn Like Us?

We believe we can get machines to learn like us, but it will take new, deep research and experimentation on much more realistic neuron models. This is what we are currently developing with our Neuron Modeler project. We want a neuron that models inputs from neuromodulators, astrocytes, and various other types of synapses. We want to model short-term and long-term plasticity and the timing systems involved with them. In fact, our current plan is to model a dynamic Hebbian learning algorithm that allows for various STDP time windows created through the local inputs at the synapse, both from the input neuron and the synapse's neuron (Scheper, T. V. olde, et al.). This model simplifies the computations involved in modeling STDP while allowing it to be highly flexible, and we believe it matches how neurons behave by only having the synapse's local information to work with. 

We feel that with more biologically accurate and plausible models, we can learn how, when, and why certain STDP shapes are needed to perform proper learning. Once we discover this information, we will be able to construct much more powerful neural networks that actually learn like us!