What I don't really understand are those base64/gzip tricks. Maybe PDS shouldn't be so restrictive about blobs. For now, we're pretending everyone provides infinite space, but I think that will change...

"It’s time to create new structures, new institutions, that can lead us to the future we want to live in, the one that we deserve, that my kids will grow up to see."

May more people be as brave as you.

It's your server, your infrastructure, you can do whatever you want, and it would be fairer to state your views and that you don't want people with opposing political views there — rather than pretending to be neutral.

Judging "sole purpose" and "dehumanization" means endless edge cases, accusations of bias, and pressure to expand restrictions.

Nice! By the way, it seems that this problem is also related to your implementation..
main.atproto-resolver-benchmark.pages.dev

hmm, such differences remotely will be minimal, that would require proper stress tests and benchmarks locally

What is the goal of your implementation anyway? I originally thought it was about performance, but this goes a bit against that

It feels the same

Through my home connection (I have a similar ping to your instance/plcbundle, a difference of about 1ms, in your favor) I get very similar numbers ~62.5/did/s

Interesting! Creating a filtered bundles is one of the side quests I have around plcbundle. According to my basic rules, we can filter out at least 41.5 GB, which will bring raw export size to ~33 GB

just few bubbles

I don't know. 25 years ago, I started with FreeBSD because its excellent stability couldn't be compared to Linux back then. But today, we're somewhere else, and it seems to me that it's not really true anymore, or only minimally. (for servers)

FreeBSD is amazing, and it's a great loss that it's declining rather than gaining more adoption. On one hand, I understand, because Linux is also great and has really matured over the years... but still...

Yeah, it could have been much worse, and I think we're playing with fire a bit, because it could blow up at any time.

note: raw data is 75 GB, you're talking about the size of the postgres database, right?

I think some kind of protection will inevitably have to be built into PLC in the future, and I really hope it won't be solved by centralization and people who will be approving things.

In the end, it doesn't matter at all whether they are valid or not, because even if they were, those people would still find a way. There are plenty of options.

PLC is dump because it has no solid protection against misuse, and validation can never solve that.

I think you're confusing two things here. There are very few factually invalid documents according to the specification – the problem is fully valid but malicious (spam) documents.

and it's only going to get worse

Nobody tests anything, this is intentional spam. Most likely produced by bitter fans of rival networks

And this is just one type of many; according to my initial estimates, at least 60% of the directory is made up of similar spam.