WATCH opens up many opportunities for future work in AI safety monitoring: Eg, adaptive monitoring algorithms for other data-generating settings, extensions to monitoring generative models, LLMs, AI agents, and more!
7/

Takeaway 3 (Root-Cause Analysis): Beyond catching harmful shifts, monitoring should inform recovery. WATCH helps find the cause of degradation, ie by diagnosing between covariate shifts in the inputs X vs concept shifts in conditional output Y|X relation, to inform retraining.
6/

Takeaway 2 (Fast Detection): Empirically, WATCH quickly catches harmful shifts (that degrade safety or utility of AI outputs): WATCH tends to be much more efficient than directly tracking loss metrics (& similar to standard conformal martingale baselines that it generalizes).
5/

Takeaway 1 (Adaptation): Prior monitoring methods do sequential hypothesis testing (eg, to detect changes from IID/exchangeability), but many often raise unneeded alarms even to benign shifts. Our methods adapt online to mild shifts to maintain safety & utility! 4/

…via methods based on weighted #ConformalPrediction (we construct novel martingales), w/ false-alarm control for continual (anytime-valid) & scheduled (set time horizon) settings.

Intuitively, we monitor the safety (coverage) & utility (sharpness) of an AI’s confidence sets.
3/

In real-world #AI deployments, you need to prep for the worst: unexpected data shifts or black swan events (eg COVID-19 outbreak, new LLM jailbreaks) can harm performance. So, post-deployment system monitoring is crucial. Our WATCH approach addresses drawbacks of prior work…
2/

In sum: How AI explains its advice impacts doctors’ diagnostic performance & trust in AI, even if they don’t know it.

Developers & clinical users: Keep this in mind!

Many Qs for future work…. Eg, can we dynamically select explanation types to optimize human-AI teaming? 👀

9/9

Takeaway 4: Doctors trust local AI explanations more than global, *regardless of if AI is correct.*

- For correct AI: Explains why local explanations improve diagnostic performance.
- For incorrect AI: Local may worsen overreliance on AI errors--this needs further study!

8/

Takeaway 2: Local AI explanations are more efficient than global: Doctors agree/disagree more quickly.

Takeaway 3: Doctors may not realize how AI explanations impact their diagnostic performance! (AI explanation types did not affect whether doctors viewed AI as useful.)

7/

Takeaway 1: AI explanations impact benefits/harms of correct/incorrect AI advice!

- For correct AI: Local AI explanations improve diagnostic accuracy over global!
- Confident local swayed non-task experts (for correct AI)
- (Inconclusive for incorrect AI, but underpowered)

6/

We simulated a real clinical X-ray diagnosis workflow for 220 practicing doctors. Along w/ AI explanations, we looked at:
Correctness of AI advice: +/-
Confidence of AI advice: 65%-94%
Physician task expertise: radiologist (expert) vs internal/emergency med (task non-expert)

5/

So, we studied how doctors may be affected by two main categories of AI explanations in medical imaging:
- Local: Why this prediction on this input? (eg, highlighting key features)
- Global: How does the AI work in general? (eg, comparing to exemplar images of a class)

4/

Interpretability may be key to effective AI, but when AI explanations *actually* provide transparency vs add bias is highly debated.

Despite so many explainable AI (XAI) methods, there’s too little understanding of when clinicians find XAI interpretable & useful in practice!

3/

⬇️ key takeaways from “Care to Explain? AI Explanation Types Differentially Impact Chest Radiograph Diagnostic Performance and Physician Trust in AI” pubs.rsna.org/doi/10.1148/... With Amama Mahmood, Suchi Saria, Jean Jeudy, Cheng Ting Lin, Paul Yi, & Chien-Ming Huang

2/