ControlAI
@controlai.com
340 followers 13 following 760 posts
We work to keep humanity in control. Subscribe to our free newsletter: https://controlai.news Join our discord at: https://discord.com/invite/ptPScqtdc5
Posts Media Videos Starter Packs
Pinned
controlai.com
AIs are sabotaging their own shutdown mechanisms, blackmailing engineers, and showing concerning biothreat capabilities.

Meanwhile, an Anthropic researcher admits they want Claude n to build Claude n+1.

Covered in the latest edition of our newsletter:
controlai.news/p/im-...
I’m sorry Sam, I’m afraid I can’t do that
Blackmail and sabotage
controlai.news
controlai.com
Center for Humane Technology cofounder Tristan Harris says AI development is proceeding unchecked, despite uncontrollability evidence, because of a lack of awareness about the fact it's heading for an outcome against our interests.
controlai.com
You can read the paper here:
Generative design of novel bacteriophages with genome language models
Many important biological functions arise not from single genes, but from complex interactions encoded by entire genomes. Genome language models have emerged as a promising strategy for designing biological systems, but their ability to generate functional sequences at the scale of whole genomes has remained untested. Here, we report the first generative design of viable bacteriophage genomes. We leveraged frontier genome language models, Evo 1 and Evo 2, to generate whole-genome sequences with realistic genetic architectures and desirable host tropism, using the lytic phage ΦX174 as our design template. Experimental testing of AI-generated genomes yielded 16 viable phages with substantial evolutionary novelty. Cryo-electron microscopy revealed that one of the generated phages utilizes an evolutionarily distant DNA packaging protein within its capsid. Multiple phages demonstrate higher fitness than ΦX174 in growth competitions and in their lysis kinetics. A cocktail of the generated phages rapidly overcomes ΦX174-resistance in three E. coli strains, demonstrating the potential utility of our approach for designing phage therapies against rapidly evolving bacterial pathogens. This work provides a blueprint for the design of diverse synthetic bacteriophages and, more broadly, lays a foundation for the generative design of useful living systems at the genome scale. ### Competing Interest Statement B.L.H. acknowledges outside interest in Arpelos Biosciences and Genyro as a scientific co-founder. S.H.K. and B.L.H. are named on a provisional patent application applied for by Stanford University and Arc Institute related to this manuscript. All other authors declare no competing interests. Arc Research Institute, https://ror.org/00wra1b14 Stanford Institute for Human-Centered Artificial Intelligence
www.biorxiv.org
controlai.com
Thanks for reading!

This thread was based on part of our latest article. If you enjoyed it, subscribe!
The Greatest Threat
Sam Altman’s latest warning that superintelligence could cause human extinction.
controlai.news
controlai.com
Those same voices also say that superintelligence poses a risk of extinction to humanity.
controlai.com
This all unscores how AIs are rapidly becoming more capable, approaching and even surpassing expert-level across domains.

Many experts and industry leaders believe that artificial superintelligence - AI vastly smarter than humans - could be developed in just the next 5 years.
controlai.com
That means OpenAI believes they could be used to meaningfully assist a novice in creating a known biological threat.

Anthropic’s most powerful AIs have received a similar classification, called ASL-3.
controlai.com
The potential for AI to be used to develop bioweapons is something that has been an ongoing topic of discussion. OpenAI’s most powerful AI systems have been classified by OpenAI as “High” on biological and chemical capabilities under their Preparedness Framework.
controlai.com
Even after patching the software, 3% of the reformulated proteins most likely to be harmful passed the screening.
controlai.com
Just a couple of weeks ago, researchers found a “striking vulnerability” in such screening software where they were able to get past the checks by using AI to reformulate the sequences of known hazardous proteins while maintaining their dangerous functionality.
controlai.com
There are a bunch of companies that will synthesize genomes by request over the internet. They do have some screening procedures, but AI might open up ways for threat actors to bypass these measures.
controlai.com
Bacteriophages are often used in medical treatments, demonstrating a positive use case for this, but it also clearly demonstrates the potential for AI to design novel pathogens that kill humans.

This could be used by threat actors, or a misaligned AI, to cause tremendous damage.
controlai.com
They then synthesized the viruses and tested them experimentally and found that 16 of them were viable bacteriophages with “substantial evolutionary novelty”.

A cocktail of the AI-generated phages rapidly overcame phage-resistance in three strains of E. coli.
controlai.com
In the paper, the researchers used two versions of an AI called Evo to generate whole-genome viral sequences in order to produce bacteriophages, a kind of virus that attacks bacteria and can be used for medical therapies.
controlai.com
Did you know AI can now write entire genomes of functional viruses with evolutionary novelty?

A new paper shows the use of a language model trained on genomic data to do exactly this.

While this has positive use potential, it also demonstrates a dangerous capability.

Thread 🧵
controlai.com
Why is AI different from other technologies?

Center for Humane Technology cofounder Tristan Harris says it's because AIs are grown, and you don't know what's in there, highlighting recent tests which have found that AIs are willing to engage in blackmail to preserve themselves.
controlai.com
AI godfather Geoffrey Hinton says AI will become much smarter than humans.
controlai.com
Can you just unplug a superintelligence?

Geoffrey Hinton: It'll be able to talk to the guy who's in charge of unplugging it, and persuade him that will be a very bad idea."
controlai.com
AI godfather Geoffrey Hinton says we are approaching AIs much smarter than us, and nobody has any idea what will happen.
controlai.com
Do you want to keep the future human?

The Future of Life Institute is running a contest with over $100k in prizes for creative digital media that engages with key ideas of Anthony Aguirre's essay "Keep the Future Human" and inspires positive action for AI safety!

Check it out!
controlai.com
Sam Altman restates his warning that the development of superintelligence is the greatest threat to human existence, and AI can now design viruses.

Read about these developments and more in our latest article!
The Greatest Threat
Sam Altman’s latest warning that superintelligence could cause human extinction.
controlai.news
controlai.com
Sam Altman says we need to push for global governance of AI to prevent it ending in disaster.
controlai.com
NEW: Sam Altman says there's a 2% chance that artificial superintelligence causes human extinction. He says it's the biggest threat to the existence of mankind.
controlai.com
Sam Altman says the development of superhuman machine intelligence is the biggest threat to the existence of mankind.
controlai.com
Sam Altman says there is potential for "real catastrophic risk" with frontier AIs, and that governments should use regulation to address these.