The pleasure was mine! Thanks, for such an interesting conversation

🎧 Hear Dr. Hupkes discuss her work on GenBench and how consistency, generalization, and reasoning shape our understanding of LLMs.
🎬 YouTube: www.youtube.com/watch?v=CuTW...
🎙️ Apple Podcasts: podcasts.apple.com/ca/podcast/w...
🎧 Spotify: open.spotify.com/show/51RJNlZ...
#WiAIR #NLP #WomenInAI

Sounds like the way back to the closet..

Results from evaluating 15 models on INCLUDE reveal stark performance variations among languages, emphasizing the need for equitable AI tools.
Public release of INCLUDE encourages further research on fair and inclusive AI.
Dataset: huggingface.co/datasets/Coh...
/4

INCLUDE is the largest multilingual benchmark of its kind, containing 197,243 MCQA pairs from 1,926 examinations across 44 languages and 15 scripts coming from 52 countries.
/3

LLMs hold immense potential, but performance disparities across languages limit their global impact. INCLUDE is a large multilingual language understanding
benchmark that includes regional educational, professional, and practical tests collected by native speakers.
/2

⏳Submission deadline: Feb 17, 2025
🗓️Workshop date: April 26-May 1, 2025 (TBD)
📍 Join us in Yokohama, Japan (also hybrid)

Submit your work and help shape the future of LLMs!