Lightnews — Scholar-powered news

David Kelley @drkbio.bsky.social · 1d

Will this recipe work for other organisms? We think it depends on genome size and proportion of nucleotides under selection, which drives the value of the self-supervised stage and training data scale. An exciting question for future work!

2

David Kelley @drkbio.bsky.social · 1d

This was a massive effort, driven by the incredible work of Calico intern Kuan-Hao Chao (@kuanhaochao.bsky.social
). Huge thanks to him, Majed Mohamed Magzoub, and Johannes Linder!

1

David Kelley @drkbio.bsky.social · 1d

My take: While MPRAs are powerful, they lose vital genomic context like local chromatin and post-transcriptional regulation. For modeling complex gene regulation in vivo, models trained on endogenous sequences are essential.

1

David Kelley @drkbio.bsky.social · 1d

Each wins on its “home field”:
* MPRA-trained models excel at predicting MPRA data, including variant sequences.
* Shorkie excels at predicting expression from promoters in their natural genomic context and eQTLs.

1

David Kelley @drkbio.bsky.social · 1d

How does Shorkie compare to models trained on massively parallel reporter assays (MPRAs)?

1

David Kelley @drkbio.bsky.social · 1d

This translates to variant effect prediction where Shorkie accurately predicts the impact of cis-eQTLs, outperforming alternative models at classifying influential regulatory variants.

1

David Kelley @drkbio.bsky.social · 1d

Shorkie also captures dynamic regulatory changes. Using new time-course RNA-seq data from TF inductions, we showed Shorkie can track how the importance of specific TF motifs changes over time.

1

David Kelley @drkbio.bsky.social · 1d

This pre-training strategy makes a huge difference. Shorkie substantially outperforms the same model trained from scratch, boosting gene-level expression prediction from a Pearson's R of 0.74 to 0.88.

1 1

David Kelley @drkbio.bsky.social · 1d

But which genomes work best? We trained on different phylogenetic levels, from close S. cerevisiae strains to the fungal kingdom. The Saccharomycetales order was the sweet spot, providing the right balance of diversity and conserved regulatory grammar for the model to learn from.

1

David Kelley @drkbio.bsky.social · 1d

Our hypothesis: Jumpstart supervised learning with self-supervision--before predicting chromatin and expression, we first asked our model to predict masked-out nucleotides across many related genomes, so it learns conserved elements like genes and their promoters.

1

David Kelley @drkbio.bsky.social · 1d

However, yeast's small genome provides limited data, making it tough for deep learning models to learn complex regulatory rules from scratch.

1

David Kelley @drkbio.bsky.social · 1d

At Calico, we've been studying S. cerevisiae for years to understand replicative aging. Along the way, we've generated rich datasets to probe its regulatory networks, which helped make this work possible.

1

David Kelley @drkbio.bsky.social · 1d

Excited to share our new paper on predicting gene expression in yeast! We introduce "Shorkie," a supervised ML model that builds off a self-supervised foundation to interpret regulatory DNA.
Preprint: www.biorxiv.org/content/10.1...

Predicting dynamic expression patterns in budding yeast with a fungal DNA language model

Predicting gene expression from DNA sequence remains challenging due to complex regulatory codes. We introduce a masked DNA language model pretrained on 165 fungal genomes closely related to budding y...

www.biorxiv.org

2 6 8

David Kelley @drkbio.bsky.social · Aug 4

The poster abstract deadline for the @keystonesymposia.bsky.social AI in Molecular Biology meeting in Santa Fe is coming up on August 21st, so get your submissions in!

www.keystonesymposia.org/conferences/...

AI in Molecular Biology | Keystone Symposia

Join us at the Keystone Symposia on AI in Molecular Biology, September 2025, in Santa Fe, with field leaders!

www.keystonesymposia.org

1 2

David Kelley @drkbio.bsky.social · Jul 23

We’ve done some experiments, but the metrics aren’t conclusive, so choose your own adventure! We’ve released these models open source, open weight for all to use. github.com/calico/borzo...

borzoi-paper/extensions/prime at main · calico/borzoi-paper

github.com

2

David Kelley @drkbio.bsky.social · Jul 23

We hypothesized that training with cell-type-specific and 3' data might make these models particularly effective for transfer to datasets with similar properties.

1 1

David Kelley @drkbio.bsky.social · Jul 23

Transfer learning has emerged as a key application for multitask sequence models like these. For more, check out another recent paper from Han Yuan, whose analysis explores various transfer strategies and shows how powerful this approach can be. www.biorxiv.org/content/10.1...

Parameter-Efficient Fine-Tuning of a Supervised Regulatory Sequence Model

DNA sequence deep learning models accurately predict epigenetic and transcriptional profiles, enabling analysis of gene regulation and genetic variant effects. While large-scale training models like E...

www.biorxiv.org

1

David Kelley @drkbio.bsky.social · Jul 23

Hence the name: Borzoi Prime to emphasize their 3’ expertise!

1

David Kelley @drkbio.bsky.social · Jul 23

Indeed, he discovered the new models better predict alternative polyadenylation and QTL variants that affect where transcripts get cleaved and polyadenylated. This key regulatory layer influences cell type-specific protein production.

1

David Kelley @drkbio.bsky.social · Jul 23

Drawing on his expertise and interest in isoform regulation, Johannes hypothesized that single-cell RNA-seq’s 3’ sequencing protocols might reveal additional capabilities in these models.

1

David Kelley @drkbio.bsky.social · Jul 23

Using single cell eQTL studies, he evaluated the cell type specific variant effect predictions and found good concordance.

1

David Kelley @drkbio.bsky.social · Jul 23

As cell-type-specific applications emerged, Johannes Linder took a fresh look.

1

David Kelley @drkbio.bsky.social · Jul 23

We trained these models in early 2023 (which is why they’re algorithmically similar to the originals), but initial metrics were underwhelming, so we shelved them.

1

David Kelley @drkbio.bsky.social · Jul 23

Side note—want your amazing data included in future training runs of open source, open weight models? Make and release BigWig tracks!

1

David Kelley @drkbio.bsky.social · Jul 23

We curated several cell atlas collections to produce pseudobulk coverage tracks. Thank you to the CZI Tabula projects and the BICCN Brain Cell Atlas for making this possible!

1