Lightnews — Scholar-powered news

Women in AI Research - WiAIR @wiair.bsky.social · 1d

✈️🤖 AI Safety Like Aviation: Too Ambitious or Absolutely Necessary?

Can AI ever be as safely regulated as aviation?
Ana Marasović shares her vision for the future of AI governance — where safety principles and regulation become the default, not an afterthought.

www.youtube.com/@WomeninAIRe...

1

Women in AI Research - WiAIR @wiair.bsky.social · 4d

🎧 Tune in to hear Ana and her co-authors💫
🎬 YouTube: www.youtube.com/@WomeninAIRe...
🎙️ Spotify: open.spotify.com/show/51RJNlZ...
🍎 Apple: podcasts.apple.com/ca/podcast/w...
📄 Paper: arxiv.org/abs/2407.03545 (8/8🧵)
#ExplainableAI #TrustworthyAI #HumanAICollaboration #NLP #WomenInAI #WiAIR

Women in AI Research WiAIR

Women in AI Research (WiAIR) is a podcast dedicated to celebrating the remarkable contributions of female AI researchers from around the globe. Our mission is to challenge the prevailing perception th...

www.youtube.com

Women in AI Research - WiAIR @wiair.bsky.social · 4d

💭 Takeaway: True explainability isn’t about opening the black box — it’s about building systems that know when to ask for help and let humans lead when it matters most. (7/8🧵)

1

Women in AI Research - WiAIR @wiair.bsky.social · 4d

📊 Finetuning a stronger model (Flan-T5-3B) boosted performance by +22–24 F1 points — reminding us that reliable collaboration starts with capable models. (6/8🧵)

1

Women in AI Research - WiAIR @wiair.bsky.social · 4d

🤝 The paper proposes a smarter path forward: let models defer to humans when uncertain, rather than explaining every prediction — boosting both efficiency and trust. (5/8🧵)

1

Women in AI Research - WiAIR @wiair.bsky.social · 4d

⚖️ Even when models were correct, human–AI teams often underperformed compared to AI alone. People hesitated or over-relied — showing that explanations don’t always improve judgment. (4/8🧵)

1

Women in AI Research - WiAIR @wiair.bsky.social · 4d

🔍 From 50 + explainability datasets, only 4 — ContractNLI, SciFact-Open, EvidenceInference v2, and ILDC — were suitable for studying explanation utility in realistic contexts. Most datasets miss how humans actually use AI explanations. (3/8🧵)

1

Women in AI Research - WiAIR @wiair.bsky.social · 4d

Ana and her co-authors dive deep in “On Evaluating Explanation Utility for Human-AI Decision Making in NLP” (Findings of #EMNLP2024) 🧠 — asking whether explanations truly help humans make better decisions, or just make us feel more confident. (2/8🧵)

1

Women in AI Research - WiAIR @wiair.bsky.social · 4d

How do we really know when and how much to trust large language models? 🤔
In this week’s #WiAIRpodcast, we talk with Ana Marasović (Asst Prof @ University of Utah; ex @ Allen AI, UWNLP) about explainability, trust, and human–AI collaboration. (1/8🧵)

1 1

Women in AI Research - WiAIR @wiair.bsky.social · 6d

We dive into how to make AI systems that truly earn our trust - not just appear trustworthy.

🎬 Full episode now on YouTube → youtu.be/xYb6uokKKOo
Also on Spotify: open.spotify.com/show/51RJNlZ...
Apple: podcasts.apple.com/ca/podcast/w...

1

Women in AI Research - WiAIR @wiair.bsky.social · 6d

💡 Key takeaways from our conversation:
• Real AI research is messy, nonlinear, and full of surprises.
• Trust in AI comes in two forms: intrinsic (how it reasons) and extrinsic (proven reliability).
• Sometimes, human-AI collaboration makes things… worse.

1

Women in AI Research - WiAIR @wiair.bsky.social · 6d

🎙️ New Women in AI Research episode out now!
This time, we sit down with @anamarasovic.bsky.social to unpack some of the toughest questions in AI explainability and trust.

🔗 Watch here → youtu.be/xYb6uokKKOo

youtu.be

1 1 1

Women in AI Research - WiAIR @wiair.bsky.social · 8d

🎙️ New #WiAIR episode coming soon!

We sat down with Ana Marasović to talk about the uncomfortable truths behind AI trust.
When can we really trust AI explanations?

Watch the trailer youtu.be/GBghj6S6cic
Then subscribe on YouTube to catch the full episode when it drops.

Women in AI Research - WiAIR @wiair.bsky.social · 11d

Our new guest at #WiAIRpodcast is @anamarasovic.bsky.social
(Asst prof @ University of Utah , Ex @ Allen AI). We'll talk with her about faithfulness, trust and robustness in AI.
The episode is coming soon, don't miss:
www.youtube.com/@WomeninAIRe...

#WiAIR #NLProc

1 2

Women in AI Research - WiAIR @wiair.bsky.social · 13d

Don't miss the full episode:
🎬 YouTube: www.youtube.com/watch?v=DPhq...
🎙 Spotify: open.spotify.com/episode/7aHP...

#WiAIRpodcast #WiAIR

Open Science and LLMs, with Dr. Valentina Pyatkin

Can open-source large language models really outperform closed ones like Claude 3.5? 🤔 In this episode of the Women in AI Research podcast, Jekaterina Novikova and Malikeh Ehghaghi engage with…

www.youtube.com

Women in AI Research - WiAIR @wiair.bsky.social · 13d

"Inclusivity is about saying: Come sit with us!" 💡

Valentina Pyatkin reminds us that AI research isn’t just about models and benchmarks - it’s about building a community where everyone feels welcome.

#AI #Inclusivity #WomenInAI

1 1

Women in AI Research - WiAIR @wiair.bsky.social · 15d

📄 Paper: arxiv.org/abs/2506.01937
📦 Dataset: huggingface.co/datasets/all...
🛠 Code: github.com/allenai/rewa...
🎬 YouTube: www.youtube.com/watch?v=DPhq...
🎧 Spotify: open.spotify.com/episode/7aHP...
🍏 Apple Podcasts: podcasts.apple.com/ca/podcast/o...
#WiAIR #OpenScience (8/8🧵)

RewardBench 2: Advancing Reward Model Evaluation

Reward models are used throughout the post-training of language models to capture nuanced signals from preference data and provide a training target for optimization across instruction following, reas...

arxiv.org

Women in AI Research - WiAIR @wiair.bsky.social · 15d

🧭 Takeaway: If you use reward models for RLHF, Best-of-N, or data filtering, RB2 gives you a harder, fairer yardstick—plus open evidence to guide choices. (7/8🧵)

1

Women in AI Research - WiAIR @wiair.bsky.social · 15d

🔬 The team trained & evaluated 100+ reward models (fully open). Key lessons: training >1 epoch can help; RMs often show lineage bias, preferring completions from their own model family. (6/8🧵)

1

Women in AI Research - WiAIR @wiair.bsky.social · 15d

⚖️ But for PPO/RLHF, correlation is more nuanced. Reward–policy lineage and training setup matter. A top RB2 score doesn’t always equal best PPO gains. (5/8🧵)

1

Women in AI Research - WiAIR @wiair.bsky.social · 15d

🎯 RB2 accuracy strongly correlates with Best-of-N sampling (Pearson r≈0.87). Good RB2 scores → better inference-time performance. (4/8🧵)

1

Women in AI Research - WiAIR @wiair.bsky.social · 15d

📉 Results: models that scored high on the original RewardBench often fall ~20 points lower on RB2. A clear sign that earlier benchmarks overstated reward model quality. (3/8🧵)

1

Women in AI Research - WiAIR @wiair.bsky.social · 15d

📑 RB2 uses unseen human prompts (held-out WildChat) to avoid leakage. Each prompt: 1 chosen + 3 rejected responses. Domains: factuality, precise instruction following, math, safety, focus, & ties (multiple valid answers). (2/8🧵)

1

Women in AI Research - WiAIR @wiair.bsky.social · 15d

🤔 How do we know if a reward model is truly good? In our last #WiAIR episode, Valentina Pyatkin (AI2 & University of Washington) introduced RewardBench 2—a harder, cleaner benchmark for reward models in post-training. (1/8🧵)

1

Women in AI Research - WiAIR @wiair.bsky.social · 18d

💥 Behind every success is a story of rejection.
Persistence, curiosity, and resilience are what truly drive AI careers. 🚀

Don't miss the full episode:
🎬 YouTube: youtube.com/watch?v=DPhq...
🎙 Spotify: open.spotify.com/episode/7aHP...

1