Jascha Achterberg
@achterbrain.bsky.social
Neuroscience & AI at University of Oxford and University of Cambridge | Principles of efficient computations + learning in brains, AI, and silicon 🧠 NeuroAI | Gates Cambridge Scholar
www.jachterberg.com
www.jachterberg.com
We also find that while complex regions are needed to learn complex tasks, these tasks are eventually moved toward simpler regions, similar to how you may struggle the first time when learning a new skill, but slowly get better with practice.
November 21, 2025 at 12:01 PM
We also find that while complex regions are needed to learn complex tasks, these tasks are eventually moved toward simpler regions, similar to how you may struggle the first time when learning a new skill, but slowly get better with practice.
Furthermore, we find that these pathways mirror our expected behavior of pathways in the brain! We find that difficult tasks need to be learned in more complex regions, similar to how you need to think “harder” when learning how to solve a difficult math problem.
November 21, 2025 at 12:01 PM
Furthermore, we find that these pathways mirror our expected behavior of pathways in the brain! We find that difficult tasks need to be learned in more complex regions, similar to how you need to think “harder” when learning how to solve a difficult math problem.
With these three features in place, we find that our third criterion of distinct pathways is also met. While baseline models exhibit largely random expert usage patterns, our models exhibit highly structured pathways between regions that reliably emerge during learning.
November 21, 2025 at 12:01 PM
With these three features in place, we find that our third criterion of distinct pathways is also met. While baseline models exhibit largely random expert usage patterns, our models exhibit highly structured pathways between regions that reliably emerge during learning.
Our third contribution is expert dropout. Without this feature, we find models suffer large performance deficits when experts outside of the active pathway are disabled. However, we would want models to be primarily dependent on the experts that are most being used.
November 21, 2025 at 12:01 PM
Our third contribution is expert dropout. Without this feature, we find models suffer large performance deficits when experts outside of the active pathway are disabled. However, we would want models to be primarily dependent on the experts that are most being used.
When put together, these two contributions resulted in remarkable pathway consistency in our model, which we measured by correlating the routing patterns across 10 different models trained on the same tasks.
November 21, 2025 at 12:01 PM
When put together, these two contributions resulted in remarkable pathway consistency in our model, which we measured by correlating the routing patterns across 10 different models trained on the same tasks.
We then identify three inductive biases that yield pathways that meet each of these criteria.
The first of these is a routing loss that penalizes the use of more complex experts during training, and the second scales this loss by the model’s performance on the task being solved.
The first of these is a routing loss that penalizes the use of more complex experts during training, and the second scales this loss by the model’s performance on the task being solved.
November 21, 2025 at 12:01 PM
We then identify three inductive biases that yield pathways that meet each of these criteria.
The first of these is a routing loss that penalizes the use of more complex experts during training, and the second scales this loss by the model’s performance on the task being solved.
The first of these is a routing loss that penalizes the use of more complex experts during training, and the second scales this loss by the model’s performance on the task being solved.
We first needed to create a model in which we could study pathway formation. We chose a Heterogeneous Mixture-of-Experts architecture, in which information can be dynamically routed to computational experts, or regions, of varying sizes.
We train model on 82 tasks of varying complexity (ModCog)!
We train model on 82 tasks of varying complexity (ModCog)!
November 21, 2025 at 12:01 PM
We first needed to create a model in which we could study pathway formation. We chose a Heterogeneous Mixture-of-Experts architecture, in which information can be dynamically routed to computational experts, or regions, of varying sizes.
We train model on 82 tasks of varying complexity (ModCog)!
We train model on 82 tasks of varying complexity (ModCog)!
Brains have many pathways / subnetworks but which principles underlie their formation?
In our #NeurIPS paper lead by Jack Cook we identify biologically relevant inductive biases that create pathways in brain-like Mixture-of-Experts models🧵
#neuroskyence #compneuro #neuroAI
arxiv.org/abs/2506.02813
In our #NeurIPS paper lead by Jack Cook we identify biologically relevant inductive biases that create pathways in brain-like Mixture-of-Experts models🧵
#neuroskyence #compneuro #neuroAI
arxiv.org/abs/2506.02813
November 21, 2025 at 12:01 PM
Brains have many pathways / subnetworks but which principles underlie their formation?
In our #NeurIPS paper lead by Jack Cook we identify biologically relevant inductive biases that create pathways in brain-like Mixture-of-Experts models🧵
#neuroskyence #compneuro #neuroAI
arxiv.org/abs/2506.02813
In our #NeurIPS paper lead by Jack Cook we identify biologically relevant inductive biases that create pathways in brain-like Mixture-of-Experts models🧵
#neuroskyence #compneuro #neuroAI
arxiv.org/abs/2506.02813
*Hierarchical choice code*
Temporal cross-correlation analysis revealed hierarchical coding of problem structure across all regions, with most regions being driven by temporal similarity of time windows across choices, with vlPFC also responding for the repeated order of operations across choices.
Temporal cross-correlation analysis revealed hierarchical coding of problem structure across all regions, with most regions being driven by temporal similarity of time windows across choices, with vlPFC also responding for the repeated order of operations across choices.
May 29, 2025 at 9:55 AM
*Hierarchical choice code*
Temporal cross-correlation analysis revealed hierarchical coding of problem structure across all regions, with most regions being driven by temporal similarity of time windows across choices, with vlPFC also responding for the repeated order of operations across choices.
Temporal cross-correlation analysis revealed hierarchical coding of problem structure across all regions, with most regions being driven by temporal similarity of time windows across choices, with vlPFC also responding for the repeated order of operations across choices.
*Move codes*
Next, analysing dynamics in the "Move space", we found: Move coding develops first in vlPFC before reaching dPM; other regions showed weaker move coding. "Move space" generally orthogonal to "Goal space" (except in dmPFC).
Next, analysing dynamics in the "Move space", we found: Move coding develops first in vlPFC before reaching dPM; other regions showed weaker move coding. "Move space" generally orthogonal to "Goal space" (except in dmPFC).
May 29, 2025 at 9:55 AM
*Move codes*
Next, analysing dynamics in the "Move space", we found: Move coding develops first in vlPFC before reaching dPM; other regions showed weaker move coding. "Move space" generally orthogonal to "Goal space" (except in dmPFC).
Next, analysing dynamics in the "Move space", we found: Move coding develops first in vlPFC before reaching dPM; other regions showed weaker move coding. "Move space" generally orthogonal to "Goal space" (except in dmPFC).
*Goal and location codes*
We projected neural activity into the "Goal space" & measured distances between projections grouped by current position vs. goal. We saw regional specialization: vlPFC driven by location; dmPFC more driven by goal (maintained throughout trial); dPM & I/O with mixed code.
We projected neural activity into the "Goal space" & measured distances between projections grouped by current position vs. goal. We saw regional specialization: vlPFC driven by location; dmPFC more driven by goal (maintained throughout trial); dPM & I/O with mixed code.
May 29, 2025 at 9:55 AM
*Goal and location codes*
We projected neural activity into the "Goal space" & measured distances between projections grouped by current position vs. goal. We saw regional specialization: vlPFC driven by location; dmPFC more driven by goal (maintained throughout trial); dPM & I/O with mixed code.
We projected neural activity into the "Goal space" & measured distances between projections grouped by current position vs. goal. We saw regional specialization: vlPFC driven by location; dmPFC more driven by goal (maintained throughout trial); dPM & I/O with mixed code.
*Analysing neural subspaces*
To understand frontal cortex computations, we identified neural subspaces in relation to key variables and studied the population dynamics over the duration of the trial. For each region, we asked: Which variables drive the shape & dynamics of projections?
To understand frontal cortex computations, we identified neural subspaces in relation to key variables and studied the population dynamics over the duration of the trial. For each region, we asked: Which variables drive the shape & dynamics of projections?
May 29, 2025 at 9:55 AM
*Analysing neural subspaces*
To understand frontal cortex computations, we identified neural subspaces in relation to key variables and studied the population dynamics over the duration of the trial. For each region, we asked: Which variables drive the shape & dynamics of projections?
To understand frontal cortex computations, we identified neural subspaces in relation to key variables and studied the population dynamics over the duration of the trial. For each region, we asked: Which variables drive the shape & dynamics of projections?
*Recordings*
We recorded 1374 neurons from four frontal regions with semi-chronic microelectrode arrays: ventrolateral, dorsal premotor, dorsomedial, insula/orbitofrontal. This wide coverage across frontal cortex in the same complex task makes this a unique dataset in monkey electrophysiology.
We recorded 1374 neurons from four frontal regions with semi-chronic microelectrode arrays: ventrolateral, dorsal premotor, dorsomedial, insula/orbitofrontal. This wide coverage across frontal cortex in the same complex task makes this a unique dataset in monkey electrophysiology.
May 29, 2025 at 9:55 AM
*Recordings*
We recorded 1374 neurons from four frontal regions with semi-chronic microelectrode arrays: ventrolateral, dorsal premotor, dorsomedial, insula/orbitofrontal. This wide coverage across frontal cortex in the same complex task makes this a unique dataset in monkey electrophysiology.
We recorded 1374 neurons from four frontal regions with semi-chronic microelectrode arrays: ventrolateral, dorsal premotor, dorsomedial, insula/orbitofrontal. This wide coverage across frontal cortex in the same complex task makes this a unique dataset in monkey electrophysiology.
*Routes through the maze*
Depending on available choice options, goal locations can be reached via 2-step or 4-step routes.
Depending on available choice options, goal locations can be reached via 2-step or 4-step routes.
May 29, 2025 at 9:55 AM
*Routes through the maze*
Depending on available choice options, goal locations can be reached via 2-step or 4-step routes.
Depending on available choice options, goal locations can be reached via 2-step or 4-step routes.
*Task*
Monkeys solve a multi-step maze task which requires them to navigate a 2D grid from the start location to 1 of 4 goal locations, using saccades. Monkeys start at the center, need to remember the goal location (presented at the start of the trial), and navigate using presented choice options.
Monkeys solve a multi-step maze task which requires them to navigate a 2D grid from the start location to 1 of 4 goal locations, using saccades. Monkeys start at the center, need to remember the goal location (presented at the start of the trial), and navigate using presented choice options.
May 29, 2025 at 9:55 AM
*Task*
Monkeys solve a multi-step maze task which requires them to navigate a 2D grid from the start location to 1 of 4 goal locations, using saccades. Monkeys start at the center, need to remember the goal location (presented at the start of the trial), and navigate using presented choice options.
Monkeys solve a multi-step maze task which requires them to navigate a 2D grid from the start location to 1 of 4 goal locations, using saccades. Monkeys start at the center, need to remember the goal location (presented at the start of the trial), and navigate using presented choice options.
Plenty of training programs now put a focus on Open Science. We find that our respondents already show strong experience with open source and open sharing of science!
November 8, 2024 at 8:31 AM
Plenty of training programs now put a focus on Open Science. We find that our respondents already show strong experience with open source and open sharing of science!
We do find that the training needs of respondents are extremely heterogenous, with both methods and theory across both neuro and AI being the gap that some group of respondents wanted to close in their own education.
November 8, 2024 at 8:31 AM
We do find that the training needs of respondents are extremely heterogenous, with both methods and theory across both neuro and AI being the gap that some group of respondents wanted to close in their own education.
What surprised us was that most respondents hoped to find a position that allows to combine academia with industry but only few thought they were likely to achieve their goal. This might inform both how we plan courses but also how we think about hiring in the future!
November 8, 2024 at 8:30 AM
What surprised us was that most respondents hoped to find a position that allows to combine academia with industry but only few thought they were likely to achieve their goal. This might inform both how we plan courses but also how we think about hiring in the future!
There’s been some discussion in the community whether NeuroAI should be about “Neuro to AI”-transfer or vice versa. We find that both are topics of great interest!
November 8, 2024 at 8:30 AM
There’s been some discussion in the community whether NeuroAI should be about “Neuro to AI”-transfer or vice versa. We find that both are topics of great interest!
First of all, why are students motivated to work on the intersection of neuroscience and AI in the first place? Is it just a career move, based on AI being a hot topic? No! We find curiosity is much more important than career-focused drivers.
November 8, 2024 at 8:29 AM
First of all, why are students motivated to work on the intersection of neuroscience and AI in the first place? Is it just a career move, based on AI being a hot topic? No! We find curiosity is much more important than career-focused drivers.
Here Andrea Luppi and I + lots of collaborators (see below) survey current trainees on the intersection of AI / neuro to learn how trainees think they could be best supported! Some highlighted findings in thread, with lots more in the paper:
November 8, 2024 at 8:29 AM
Here Andrea Luppi and I + lots of collaborators (see below) survey current trainees on the intersection of AI / neuro to learn how trainees think they could be best supported! Some highlighted findings in thread, with lots more in the paper:
Last year Zador et al called to train a new generation of researchers, equally at home in CompSci & #neuroscience to accelerate our understanding of the nature of intelligence, but how can we get this interdisciplinary training right? 🧵
#MLsky #NeuroAI #CompNeuro
www.nature.com/articles/s41...
#MLsky #NeuroAI #CompNeuro
www.nature.com/articles/s41...
November 8, 2024 at 8:26 AM
Last year Zador et al called to train a new generation of researchers, equally at home in CompSci & #neuroscience to accelerate our understanding of the nature of intelligence, but how can we get this interdisciplinary training right? 🧵
#MLsky #NeuroAI #CompNeuro
www.nature.com/articles/s41...
#MLsky #NeuroAI #CompNeuro
www.nature.com/articles/s41...
Join our ARIA-funded project as a postdoc on brain-inspired computing 🤖🧠, at Imperial College London! Super exciting opportunity connecting both fundamental research and the creation of cutting-edge technologies!
#neuroscience #MLsky #NeuroAI #CompNeuro
www.imperial.ac.uk/jobs/search-...
#neuroscience #MLsky #NeuroAI #CompNeuro
www.imperial.ac.uk/jobs/search-...
September 9, 2024 at 11:25 AM
Join our ARIA-funded project as a postdoc on brain-inspired computing 🤖🧠, at Imperial College London! Super exciting opportunity connecting both fundamental research and the creation of cutting-edge technologies!
#neuroscience #MLsky #NeuroAI #CompNeuro
www.imperial.ac.uk/jobs/search-...
#neuroscience #MLsky #NeuroAI #CompNeuro
www.imperial.ac.uk/jobs/search-...
Starting next week I will be in the US 🇺🇸, until mid AUG. First in New York, later SF & Boston! If you want to meet to discuss (or know of a meet-up on) efficient AI, dynamic representations, NeuroAI, hardware accelerators, & neuromorphic, please reach out!
#neuroscience #MLsky #NeuroAI #CompNeuro
#neuroscience #MLsky #NeuroAI #CompNeuro
June 3, 2024 at 2:00 PM
Starting next week I will be in the US 🇺🇸, until mid AUG. First in New York, later SF & Boston! If you want to meet to discuss (or know of a meet-up on) efficient AI, dynamic representations, NeuroAI, hardware accelerators, & neuromorphic, please reach out!
#neuroscience #MLsky #NeuroAI #CompNeuro
#neuroscience #MLsky #NeuroAI #CompNeuro
PyTorch user guides offering advice for some very specific life situations --
nothing like a little hidden side comment to remind us all that these documentations and guides are all written by the community!
via pytorch.org/tutorials/ad...
#MLSky #compneuro #neuroAI #deeplearning
nothing like a little hidden side comment to remind us all that these documentations and guides are all written by the community!
via pytorch.org/tutorials/ad...
#MLSky #compneuro #neuroAI #deeplearning
May 2, 2024 at 9:14 AM
PyTorch user guides offering advice for some very specific life situations --
nothing like a little hidden side comment to remind us all that these documentations and guides are all written by the community!
via pytorch.org/tutorials/ad...
#MLSky #compneuro #neuroAI #deeplearning
nothing like a little hidden side comment to remind us all that these documentations and guides are all written by the community!
via pytorch.org/tutorials/ad...
#MLSky #compneuro #neuroAI #deeplearning