Lightnews — Scholar-powered news

⚡️🌙 @dystopiabreaker.xyz · 1d

andymasley.substack.com/p/the-ai-wat...

The AI water issue is fake

On the national, local, and personal level

andymasley.substack.com

5 34 140

⚡️🌙 @dystopiabreaker.xyz · 1d

who would win, thousands of math/neuro/compsci/cogsci phds, or f(x)=max(0,x)

1 1 8

⚡️🌙 @dystopiabreaker.xyz · 1d

one of those slight, out-of-distribution tweaks that looks totally obvious in retrospect

2 4

⚡️🌙 @dystopiabreaker.xyz · 1d

it is baffling to me that mcculloch and pitts didn't discover relu

1 5

⚡️🌙 @dystopiabreaker.xyz · 1d

but fr i think that Werbos, Kunihiko Fukushima, and Vapnik + Chervonenkis are criminally unrecognized in defining modern AI or at least basically describing it

1 6

⚡️🌙 @dystopiabreaker.xyz · 1d

and then plato and the classics as well

1 8

⚡️🌙 @dystopiabreaker.xyz · 1d

the first thing i did when i got access to gpt3 in 2020 was generate socratic dialogs between von neumann, turing, gödel, nash, and newton

1 1 10

⚡️🌙 @dystopiabreaker.xyz · 1d

and this hides the reply _for everyone else_!

4

⚡️🌙 @dystopiabreaker.xyz · 1d

too easy to abuse imo and not actually effective at preventing pile-ons

1 16

⚡️🌙 @dystopiabreaker.xyz · 1d

and erodes the public record

1 35

⚡️🌙 @dystopiabreaker.xyz · 1d

this sets up unfavorable incentives across the entire platform in addition to the problem of interfering with sensemaking

2 38

⚡️🌙 @dystopiabreaker.xyz · 1d

the fact that blocking a user hides their reply for everyone, not just the user who blocked them, seems like an extremely toxic pattern for collective sense-making.

1. user makes [xyz claim]
2. another user corrects them
3. blocked, hidden

it should be explicitly shown as hidden instead

16 20 140

⚡️🌙 @dystopiabreaker.xyz · 1d

it's a lot easier with claude's help :)

3

⚡️🌙 @dystopiabreaker.xyz · 1d

among other feats such as winning gold medals at IOI and IMO

2

⚡️🌙 @dystopiabreaker.xyz · 1d

one of whom is warning everyone about imminent extinction and the other who is leading a company which is using LLMs to find and patch critical security vulnerabilities at and increasing pace

1 4

⚡️🌙 @dystopiabreaker.xyz · 1d

except, uh, the two AI experts who won the physics and chemistry nobel prizes last year

1 3

⚡️🌙 @dystopiabreaker.xyz · 1d

one of the funny things about posting with an anime pfp pseudonymously is that sometimes people make weird assumptions about you, for example that you’re posting about AI because you’re a NEET grok gooner who thinks their AI gf is conscious

7 6 100

⚡️🌙 @dystopiabreaker.xyz · 1d

sgd.fyi hopefully this helps

Built, or grown?

No, it wasn't… we designed the learning algorithm… But we don't really understand exactly how they do those things.

sgd.fyi

1

⚡️🌙 @dystopiabreaker.xyz · 1d

in which claude and i implement naive gradient descent that runs in javascript in your webbed browser

23

⚡️🌙 @dystopiabreaker.xyz · 1d

i made a thing that hopefully makes this clearer or is at least fun to play with: sgd.fyi

6 20 100

⚡️🌙 @dystopiabreaker.xyz · 1d

but i think there’s actually two kinds of determinism here,

- the strict definition (exactly the same prompt, greedy sampling should predict the same logits)
- the informal meaning, i.e. that there should be some deterministic relationship between different prompts

1 4

⚡️🌙 @dystopiabreaker.xyz · 1d

you need all 3! deterministic sampling, stable numerics, and batch invariance

1 2

⚡️🌙 @dystopiabreaker.xyz · 1d

i don’t think you need this for there to be labor problems but it’s very helpful for scaling RL and various other use cases

1 2

⚡️🌙 @dystopiabreaker.xyz · 1d

i.e given sequence T, token T+1 will always predict the same output distribution and select the same token from that distribution

2 4

⚡️🌙 @dystopiabreaker.xyz · 1d

during inference, sampling of the logits

1 4