Little script to see for yourself gist.github.com/jordanisaacs...

Well looks like they didn’t read up on the history of using io-uring safely in rust. They have the main io-uring issue of sharing buffers with the kernel. github.com/Azure/kimoji...
The classic from without boats (2020) without.boats/blog/io-uring/

Omg they are using my rustix-uring library, looks like I got to get back on the maintenance grind 😅 github.com/jordanisaacs...

Ah bad link, here it is github.com/Azure/kimoji...

And page servers are sharded (one copy stored on object storage) allowing for easy scaling up/down of page servers. The standard `GetPage @ LSN` API.

They have automatic failovers of primary to read replicas (and if old primary comes back online it will come back as a read replica). Since compute nodes are stateless this is easy for them to do. A mention of ringfencing but no details on it. Video has a nice animation of this.

WAL writes are majority quorum from the primary. They are using rust & thread per-core io-uring on nvme. A new open source runtime, kimojio! github.com/Azure/kimoji.... Excited to dig into that.

Some interesting stuff in the talk. It’s neon style durable WAL log & async WAL replay to create pages. All backed by object storage. There is SSD cache on all nodes.

Adam prout gave a talk on it at ignite today. No recording yet ignite.microsoft.com/en-US/sessio...

So after a day of work I had my 3 line fix :) Add 'SHF_GNU_RETAIN' flag to the inline asm for the main flag list. And a dummy variable in the macro that uses the view sections to force a GC reference:
`__attribute__((retain,used,section("view-section")))
static const char dummy[0];`

Of course this is where I ended up at maskray’s blog doing deep reads of linker GC (maskray.me/blog/2021-02... ) and metadata sections (maskray.me/blog/2021-01...). I realized the main flag list should always exist and the view should only be kept if in use.

There has actually been a ton of consideration to handling these sort of “meta” sections that have no direct access and thus do not participate in general GC references. But upon further looking the dynamic flag didn’t use any of the special flags. So I found my bug, the linker was GCing it

Luckily I had two build flavors with one passing and the other failing. I manually went through every linker flag difference to guess which one was causing issues. And '—gc-sections' looked suspicious. The linker has some crazy complexity to how it decides to GC sections.

So dynamic flag automatically generates sections using inline assembly. One section has the list of all flags. Then there are arbitrarily many other sections that are a “view” into that list of flags for more efficient lookups. There is no access of any sections beyond using '__start' and '__stop'

But the sections existed in the object files leaving me flummoxed. Why wasn’t the linker using the sections? So I went down one of the most interesting systems rabbit holes I’ve had recently.

Google blacklisted the iCloud private relay ip addresses. Been broken for ages while now. On iOS safari you need to press the “show ip address” option and then try searching again and it works.

Everyone complains about fighting the compiler, but the true boss battle is the linker. I’d rather have a thousand line c++ template error than an utterly unhelpful 5 line linker error.