However, there's some things the model has trouble figuring out from the screenshot:
1. What's clickable and their exact bounds
2. What's scrollable and in which direction

So it's sometimes necessary to augment your screenshot with distilled metadata from the DOM/a11y tree

1. DOM code is messy, sometimes incredibly convoluted to position/hide things
2. Devs suck at accessibility: unlabeled images and icon buttons mean there's info in the pixels that's not in the DOM text
3. Image: 200-300 tokens. DOM JSON: 1K-10K tokens

Indeed. I saw a proposal that the Library of Congress should run a public cloud that citizens can use for these kinds of public-good networks and services. It would be protected by the first amendment. However, we're seeing that even constitutional protections don't matter under authoritarianism

All the most expensive houses in my area are far away from public transit. Y'all don't understand: having to drive isn't a luxury, it's a curse

Waze reports your driving to your insurance? How do they know who your insurance is? Source?

Just say yuppies, it's cleaner

How would we ensure the EU then doesn't turn around and start surveilling everyone on their public network? I'm not too trusting of a social media from the same government trying to push Chat Control

I tried to put him to bed an hour earlier (so, the same time he's used to), but somehow he knows. He can detect the change even though he can't read clocks, and insists on jumping around the house an hour longer

me. it's my kid.