Highlighting some of my team's recent work: We've changed Materialize to use swap instead of memory-mapped files, with nice performance and efficiency improvements.

Based on Timely 0.24, we released Differential Dataflow 0.17. Compatibility changes plus removing the Filter and Freeze wrappers because they weren't used and difficult to maintain. github.com/TimelyDatafl...

We just released Timely Dataflow 0.24! Here are some exciting changes from @frankmcsherry.bsky.social and myself.
The container abstractions got a complete rework, and we introduce a new pattern to distribute data. Details below.
github.com/TimelyDatafl...

A lot of performance improvements went into this one!

The Materialize engineering team uncovered a rare concurrency bug 🪲in Rust’s 🦀 unbounded channels that could lead to double-free memory errors. After thorough debugging and working with the Rust and crossbeam communities, the fix is now part of @rust-lang.org 1.87.0.
🔗 bit.ly/3Fan1Om

Agents generate more data and place more demand on systems than ever before—and standards like MCP will only accelerate this trend. Learn how Delphi is rethinking how they build data-intensive applications—from the db to the UI: bit.ly/42BZdf9

I'm very excited to share that @materialize.com now has a Self-Managed option, as well as a Community Edition license that lets you play around with the exact same bits but for free.

materialize.com/blog/materia...

For interested folks, the RustNYC presentation I gave about the `columnar` crate is now available in the repo: github.com/frankmcsherr...

It's not as good without the talk track, but it's got the benchmark results, you can admire the stylistic watercolors (so therapeutic to make; strong recommend).

We just landed the first use of the columnar crate in Materialize! Stay tuned for more.

github.com/MaterializeI...

I wrote a bit about efficient maintenance of temporal filters in @materialize.com. By periodically restarting replicas, we can limit resource requirements (RAM and CPU), all without any other user-visible impact.

materialize.com/blog/replica...

An ocelot with an odd shape at the Museum für Naturkunde in Berlin.

I was recently coveted a 10 year journal on jetpens, because that would be cool to do, and realized that I already had one! The blog turns 10 this month, and I thought it might be worth taking a tour through each of the years and seeing what I can jot down about each.

github.com/frankmcsherr...

Origami tree at the @natural-history.bsky.social

Sonne additional details by @frankmcsherry.bsky.social bsky.app/profile/fran...

Sometimes it's easier to mitigate a problem than solve it: a feature I recently implemented reduces the memory and CPU @materialize.com requires to maintain future updates. Some details why solving the problem is hard and the mitigation easy-ish.
materialize.com/changelog/20...

Enjoying the call of the coquí in Puerto Rico

Just published lgalloc 0.4.0. The only change is that stats don't print errors anymore but forward to the caller. Docker on MacOS doesn't present numa information! crates.io/crates/lgalloc

Interesting article on the future of aircraft design: hackaday.com/2024/11/21/b...