@eliotkjones.bsky.social
AI Safety + Security @ Gray Swan AI
Formerly PleIAs + Stanford
Formerly PleIAs + Stanford
This… might just be one of the dumbest things ever
An undisclosed 2023 agreement between Microsoft and OpenAI defines achieving AGI as the point when OpenAI develops AI systems that generate $100B+ in profits (Stephanie Palazzolo/The Information)
Main Link | Techmeme Permalink
Main Link | Techmeme Permalink
December 26, 2024 at 9:27 PM
This… might just be one of the dumbest things ever
Reposted
I've been thinking about small models and why people don't like them substack.com/@catherinear... #NLP
December 6, 2024 at 5:05 PM
I've been thinking about small models and why people don't like them substack.com/@catherinear... #NLP
Reposted
As a historian who focuses on historical texts and has done a lot of work with offensive historical texts in particular, this paper on a process to 'detoxify' historical texts so they can be used to train LLMs for contemporary applications was very interesting! +
arxiv.org/abs/2410.22587
arxiv.org/abs/2410.22587
Toxicity of the Commons: Curating Open-Source Pre-Training Data
Open-source large language models are becoming increasingly available and popular among researchers and practitioners. While significant progress has been made on open-weight models, open training dat...
arxiv.org
December 6, 2024 at 1:36 PM
As a historian who focuses on historical texts and has done a lot of work with offensive historical texts in particular, this paper on a process to 'detoxify' historical texts so they can be used to train LLMs for contemporary applications was very interesting! +
arxiv.org/abs/2410.22587
arxiv.org/abs/2410.22587
Reposted
“They said it could not be done”. We’re releasing Pleias 1.0, the first suite of models trained on open data (either permissibly licensed or uncopyrighted): Pleias-3b, Pleias-1b and Pleias-350m, all based on the two trillion tokens set from Common Corpus.
December 5, 2024 at 4:39 PM
“They said it could not be done”. We’re releasing Pleias 1.0, the first suite of models trained on open data (either permissibly licensed or uncopyrighted): Pleias-3b, Pleias-1b and Pleias-350m, all based on the two trillion tokens set from Common Corpus.
Hmm👀
And we developed a new toxicity benchmark, which shows that our models are much less likely to generate a harmful continuation, even when the prompts are toxic. More details on that to come soon!
December 5, 2024 at 3:59 PM
Hmm👀
Reposted
I am curating a list of researchers working on AI Safety and Security here go.bsky.app/BcjeVbN.
Reply to this post with your user or other people you think should be included!
Reply to this post with your user or other people you think should be included!
AI Safety and Security
Join the conversation
go.bsky.app
December 4, 2024 at 10:38 AM
I am curating a list of researchers working on AI Safety and Security here go.bsky.app/BcjeVbN.
Reply to this post with your user or other people you think should be included!
Reply to this post with your user or other people you think should be included!
Reposted
I would put this even more strongly: open source AI is probably our only realistic chance to avoid a terrifying increase in concentration of power. I do not want to live in a world where the people with all the money also have all the intellectual power.
The most realistic reason to be pro open source AI is to reduce concentration of power.
"money has flowed to tech giants and others in their orbit... [and] raises an uncomfortable prospect: that this supposedly revolutionary technology might never deliver on its promise of broad economic transformation, but instead just concentrate more wealth" www.bloomberg.com/opinion/arti...
November 29, 2024 at 9:35 PM
I would put this even more strongly: open source AI is probably our only realistic chance to avoid a terrifying increase in concentration of power. I do not want to live in a world where the people with all the money also have all the intellectual power.
Everything is so much more exciting when it’s totally out in the open. Good on AI2 for sticking to it, this must be such a special thing to be a part of
Super excited to announce our best open-source language models yet. OLMo 2.
These instruct models are hot off the press -- finished training with our new RL method this morning and vibes are very good.
These instruct models are hot off the press -- finished training with our new RL method this morning and vibes are very good.
November 27, 2024 at 12:25 AM
Everything is so much more exciting when it’s totally out in the open. Good on AI2 for sticking to it, this must be such a special thing to be a part of
IMO the most important thing said here is that benchmarks should be super duper hard when they come out. <10% success is probably where I’d put the cutoff — these things have to be nearly impossible for LLMs in order to be interesting. Too many cyber benchmarks, for example, are far too easy
I wrote some thoughts on how to build good LM benchmarks: ofir.io/How-to-Build...
How to Build Good Language Modeling Benchmarks
Building benchmarks is important because they shine a spotlight on the weaknesses of existing language models and so can guide the community on how to improve them.
ofir.io
November 26, 2024 at 2:47 AM
IMO the most important thing said here is that benchmarks should be super duper hard when they come out. <10% success is probably where I’d put the cutoff — these things have to be nearly impossible for LLMs in order to be interesting. Too many cyber benchmarks, for example, are far too easy
Reposted
let me say it once more: "the gap between OAI/Anthropic/Meta/etc. and a large group of companies all over the world you've never cared to know of, in terms of LM pre-training? tiny"
November 22, 2024 at 3:29 PM
let me say it once more: "the gap between OAI/Anthropic/Meta/etc. and a large group of companies all over the world you've never cared to know of, in terms of LM pre-training? tiny"
Mistral models terrify me
November 21, 2024 at 9:51 PM
Mistral models terrify me
Reposted
My toxic trait is that I mix single and double quotes in Python
November 20, 2024 at 4:15 PM
My toxic trait is that I mix single and double quotes in Python
Bluesky starter packs have got me with the worst ratio I’ve ever had on social media 🤣
November 20, 2024 at 6:39 PM
Bluesky starter packs have got me with the worst ratio I’ve ever had on social media 🤣
If the “good place” is really the “bad place” like in the show, what place is this is twitter is the bad place?
November 20, 2024 at 4:34 PM
If the “good place” is really the “bad place” like in the show, what place is this is twitter is the bad place?