@lucaschubu.bsky.social
Check out our updated pre-print here: arxiv.org/abs/2502.15678
Testing the Limits of Fine-Tuning for Improving Visual Cognition in Vision Language Models
Pre-trained vision language models still fall short of human visual cognition. In an effort to improve visual cognition and align models with human behavior, we introduce visual stimuli and human judg...
arxiv.org
June 2, 2025 at 7:45 AM
Check out our updated pre-print here: arxiv.org/abs/2502.15678
Check out our pre-print here: arxiv.org/abs/2502.15678. This is joint work with @kozzyvoudouris.bsky.social, @elifakata.bsky.social, Matthias Bethge, Josh Tenenbaum, and @ericschulz.bsky.social.
Testing the limits of fine-tuning to improve reasoning in vision language models
Pre-trained vision language models still fall short of human visual cognition. In an effort to improve visual cognition and align models with human behavior, we introduce visual stimuli and human judg...
arxiv.org
February 25, 2025 at 11:31 AM
Check out our pre-print here: arxiv.org/abs/2502.15678. This is joint work with @kozzyvoudouris.bsky.social, @elifakata.bsky.social, Matthias Bethge, Josh Tenenbaum, and @ericschulz.bsky.social.
Finally, we fine-tuning a model on human responses for the synthetic intuitive physics dataset. We find that this model not only shows a higher agreement with human observers, but that it also generalizes better to the real block towers.
February 25, 2025 at 10:45 AM
Finally, we fine-tuning a model on human responses for the synthetic intuitive physics dataset. We find that this model not only shows a higher agreement with human observers, but that it also generalizes better to the real block towers.
Models fine-tuned on intuitive physics also do not robustly generalize to an almost identical but visually different dataset (Lerer columns below). They are fine-tuned on synthetic block towers, while the dataset by Lerer et al. features pictures of real block towers.
February 25, 2025 at 10:45 AM
Models fine-tuned on intuitive physics also do not robustly generalize to an almost identical but visually different dataset (Lerer columns below). They are fine-tuned on synthetic block towers, while the dataset by Lerer et al. features pictures of real block towers.
We fine-tuned models on tasks from intuitive physics and causal reasoning. Models fine-tuned on intuitive physics (first two rows) do not perform well on causal reasoning and vice versa. Models fine-tuned on both perform well in either domain, showing models can learn both.
February 25, 2025 at 10:45 AM
We fine-tuned models on tasks from intuitive physics and causal reasoning. Models fine-tuned on intuitive physics (first two rows) do not perform well on causal reasoning and vice versa. Models fine-tuned on both perform well in either domain, showing models can learn both.