6/ 💭We draw a metaphor to a dynamic mixture of expert models, reflecting changing neural preferences in task and temporal integration across time, and suggest that an ideal single model would require task-independent training and an architecture enabling dynamic switching.

5/ ⚠️ Overall, our results challenge the conventional view of a temporal processing hierarchy progressing from low- to high-level representations, as typically observed in image perception.

4/ Additionally, we find state space models show superior alignment to intermediate posterior activity through mid-level action features, in which self-supervised pretraining is also beneficial.

3/ In contrast, responses in frontal electrodes best align with high-level static action representations and show no temporal correspondence to the video.

2/ We find responses in posterior electrodes, after initial alignment to hierarchical static object processing, best align to mid-level temporally-integrative representations of actions and closely match the unfolding video content. 🎥

1/ To do so, we propose Cross-Temporal Representational Similarity Analysis (CT-RSA) 📈💡, which matches the best time-unfolded model features to dynamically evolving brain responses, revealing novel insights on how continuous visual input is integrated in the brain.

📢 New preprint, together with @sargechris.bsky.social!

Building on @sargechris.bsky.social's previous work, we benchmark 100+ image and video models 🤖 on brain representational alignment, this time to EEG data of humans 🧠 watching videos! 🧵⬇️

www.biorxiv.org/content/10.1...

6/ 💭 We draw a metaphor to a dynamic mixture of expert models, reflecting changing neural preference in task and temporal integration across time, and suggest that an ideal single model would require task-independent training and an architecture enabling dynamic switching.

5/ ⚠️ Overall, our results challenge the conventional view of a temporal processing hierarchy progressing from low- to high-level representations, as typically observed in image perception.

4/ Additionally, we find state space models show superior alignment to intermediate posterior activity through mid-level action features, in which self-supervised pretraining is also beneficial.

3/ In contrast, responses in frontal electrodes best align with high-level static action representations and show no temporal correspondence to the video.

2/ We find responses in posterior electrodes, after initial alignment to hierarchical static object processing, best align to mid-level representations of temporally-integrative actions and closely match the unfolding video content. 🎥

1/ To do so we propose Cross-Temporal Representational Similarity Analysis (CT-RSA) 📈💡, which matches the best time-unfolded model features to dynamically evolving brain responses, revealing novel insights on how continuous visual input is integrated in the brain.

🚨PhD position alert!🚨
Joint PhD position with @fedemar.bsky.social @predictivebrain.bsky.social and Sonja Kotz, to investigate the neurobiological underpinnings of perceptual decisions
vacancies.maastrichtuniversity.nl/job/Maastric...
Please share and if you have questions, send me a message!

There are 2 PhD positions in my lab in Amsterdam (collaboration with Sander Bohte, @tessamdekker.bsky.social and Ingmar Visser) on NeuroAI of developmental vision.

academicpositions.nl/ad/centrum-w...

(1/4) The Algonauts Project 2025 challenge is now live!

Participate and build computational models that best predict how the human brain responds to multimodal movies!

Submission deadline: 13th of July.

#algonauts2025 #NeuroAI #CompNeuro #neuroscience #AI

algonautsproject.com