GAME was designed in consultation with many functional genomics and ML researchers. Many enthusiastically contributed their models and datasets in GAME modules. Thanks to all the coauthors and colleagues for your support in developing GAME! 🙏

6) Better estimates of real-world performance (since tasks and models are cleanly separated and uniformly applied)
7) GAME enables continual benchmarking; we anticipate a yearly “state of the field” to be easily produced. Here's an example for gene expression prediction using existing GAME modules.

GAME has many key advantages for genomics model benchmarking:
1) Via the API, all models are inherently compatible with all benchmarks
2) GAME will work across platforms – just get Apptainer installed, and away you go 💻↔️🖥️

The “Matcher” enables pairing up of the benchmark tasks and things the model can predict. Evaluator wants predictions in heart cells? The Matcher can tell you that your cardiomyocyte model is your best bet. The Matcher and Predictors also communicate via an API!

Benchmarks are containerized as “Evaluators”. We created several, including chromatin conformation, MPRA, and synthetic cis-regulatory variant effects. As new datasets come out, they can be added to GAME to see whether models can predict them.

Models are containerized in “Predictors”. We created DREAM-RNN (K562), Enformer, Orca, DeepBICCN2, and Borzoi Predictors. We anticipate as new models are created, model builders will encapsulate their models, referencing the examples we provide.

GAME uses Application Programming Interfaces, a concept introduced to the genomics model world with Kipoi, to enable uniform application of any benchmark to any model.

⏰Abstract deadline for the Cascadia Advanced Genomic Technologies Meeting is April 30!⚠️ We're hoping this will be the first of many, catalyzing collaboration and leveraging our regional strength in advanced genomics technologies! See you there!
de-boer-lab.github.io/CAGT_meeting/

Reminder: abstract deadline for the Cascadia Advanced Genomic Technologies Meeting is April 30! We're hoping this will be the first of many, catalyzing collaboration and leveraging our regional strength in advanced genomics technologies! Please spread the word!
de-boer-lab.github.io/CAGT_meeting/

Last call for a postdoc position in single cell analysis/perturb-seq with @nikashakiba.bsky.social at UBC in beautiful Vancouver Canada!
ubc.wd10.myworkdayjobs.com/en-US/ubcfac...

I think this approach is generally misguided because it assumes you have adequately controlled the variables at play in your system. With 1000 TFs floating around, gook luck. See de Boer et al nat biotech 2020 for a great example of this contolled approach failing.