9/n 📖 For more details, check out our updated manuscript: biorxiv.org/content/10.1...

#ProteinLM #AI4Science #ProteinDesign #MachineLearning #Bioinformatics

8/n 🔬 Interesting result: proteins with high likelihoods from MSA-based estimates tend to have more deleterious mutations — a pattern not observed for ESM2.

7/n 🔬 Interesting result: random seeds also impact predicted sequence likelihoods.

6/n 🔬 Interesting result: when scale up model size, predicted sequence likelihoods vary across proteins — some show no improvement, while others show substantial gains.⚡

5/n 🎯 pLMs align best with MSA-based per-residue likelihoods at moderate levels of sequence likelihood, which explains the bell-shaped relationship we observed in fitness prediction performance.

4/n 🔍 Interestingly, while overall sequence likelihoods differ, per-residue predicted likelihoods are correlated.

The better a pLM’s per-residue likelihoods align with MSA-based estimates, the better its performance on fitness prediction. 📈

3/n 🧬 Predicted likelihoods from MSA-based methods directly reflect evolutionary constraints.

Even though pLMs are also trained to learn evolutionary information, their predicted whole sequence likelihoods show no correlation with MSA-based methods. ⚡

2/n 📊 We found that general pLMs trained on large datasets show a bell-shaped relationship between fitness prediction performance and wild-type sequence likelihood.

Interestingly, MSA-based models do not show this trend.

1/n 🧬 These models estimate sequence likelihood from different inputs — sequence, structure, and homologs.

To infer mutation effects, the log-likelihood ratio (LLR) between the mutated and wild-type sequences is used. ⚖️

read our preprint here: www.biorxiv.org/content/10.1...

4/n We also compared SeqDance's attention with ESM2-35M.

3/n here is the comparison on viral proteins in ProteinGYM, (our models have 35M parameters)

2/n on the mega-scale protein stability dataset, it's clear that ESM2's performance is correlated with the number of homologs in its training set. but our models show robust performance for proteins without homologs in training set.

1/n to perform zero-shot fitness prediction, we use our models SeqDance/ESMDance to predict dynamic properties of both wild-type and mutated sequences. the relative changes bettween them are used to infer mutation effects.