rmoff 🏃♂️🫖🥓
@rmoff.net
Pinned
rmoff 🏃♂️🫖🥓
@rmoff.net
· Oct 28
Travelling for Work, with Kids at Home
rmoff.net
My most popular blog post is one I wrote about Kafka listener configuration. But the one that I am most proud of when people says it's been useful for them is when I wrote about trying to get the balance right when travel is part of your job: rmoff.net/2019/02/09/t....
good stuff from @vanlightly.bsky.social: How Would You Like Your Iceberg Sir? Stream or Batch Ordered?
jack-vanlightly.com/blog/2025/11...
jack-vanlightly.com/blog/2025/11...
November 12, 2025 at 5:03 PM
good stuff from @vanlightly.bsky.social: How Would You Like Your Iceberg Sir? Stream or Batch Ordered?
jack-vanlightly.com/blog/2025/11...
jack-vanlightly.com/blog/2025/11...
One for my DBA friends! A very good talk from @andypavlo.bsky.social, looking at how LLMs can _actually_ be used to help improve performance. It's measured and reasonable, with plenty of caveats and real-world considerations from someone who _really_ gets RDBMS.
www.p99conf.io/session/chat...
www.p99conf.io/session/chat...
November 12, 2025 at 4:04 PM
One for my DBA friends! A very good talk from @andypavlo.bsky.social, looking at how LLMs can _actually_ be used to help improve performance. It's measured and reasonable, with plenty of caveats and real-world considerations from someone who _really_ gets RDBMS.
www.p99conf.io/session/chat...
www.p99conf.io/session/chat...
Reposted by rmoff 🏃♂️🫖🥓
🚀 We released DuckDB v1.4.2, the second patch release of our LTS edition.
🔎 We are shipping new Iceberg features, improved logger/profiler integration and several bugfixes. The new DuckDB version can also read and write Vortex files.
📖 For more details, read
duckdb.org/2025/11/12/a...
🔎 We are shipping new Iceberg features, improved logger/profiler integration and several bugfixes. The new DuckDB version can also read and write Vortex files.
📖 For more details, read
duckdb.org/2025/11/12/a...
November 12, 2025 at 1:22 PM
🚀 We released DuckDB v1.4.2, the second patch release of our LTS edition.
🔎 We are shipping new Iceberg features, improved logger/profiler integration and several bugfixes. The new DuckDB version can also read and write Vortex files.
📖 For more details, read
duckdb.org/2025/11/12/a...
🔎 We are shipping new Iceberg features, improved logger/profiler integration and several bugfixes. The new DuckDB version can also read and write Vortex files.
📖 For more details, read
duckdb.org/2025/11/12/a...
Protobuf - use it faster, or don't use it at all? Couple of interesting talk from P99 CONF this year:
* www.p99conf.io/session/8x-b...
* www.p99conf.io/session/pars...
* www.p99conf.io/session/8x-b...
* www.p99conf.io/session/pars...
November 11, 2025 at 4:05 PM
Protobuf - use it faster, or don't use it at all? Couple of interesting talk from P99 CONF this year:
* www.p99conf.io/session/8x-b...
* www.p99conf.io/session/pars...
* www.p99conf.io/session/8x-b...
* www.p99conf.io/session/pars...
I missed this release last week - #ApacheFluss now supports writing to both #ApacheIceberg and Lance (as well as the original #ApachePaimon.
fluss.apache.org/blog/release...
fluss.apache.org/blog/release...
Blog | Apache Fluss™ (Incubating)
Blog
fluss.apache.org
November 11, 2025 at 2:20 PM
I missed this release last week - #ApacheFluss now supports writing to both #ApacheIceberg and Lance (as well as the original #ApachePaimon.
fluss.apache.org/blog/release...
fluss.apache.org/blog/release...
Three Flink-related talks at P99 CONF this year - here are the recordings and slides:
* www.p99conf.io/session/apac...
* www.p99conf.io/session/buil...
* www.p99conf.io/session/rivi...
* www.p99conf.io/session/apac...
* www.p99conf.io/session/buil...
* www.p99conf.io/session/rivi...
November 11, 2025 at 11:10 AM
Three Flink-related talks at P99 CONF this year - here are the recordings and slides:
* www.p99conf.io/session/apac...
* www.p99conf.io/session/buil...
* www.p99conf.io/session/rivi...
* www.p99conf.io/session/apac...
* www.p99conf.io/session/buil...
* www.p99conf.io/session/rivi...
Alex Jacobs: The Case Against pgvector alex-jacobs.com/posts/the-ca...
The Case Against pgvector | Alex Jacobs
What happens when you try to run pgvector in production and discover all the things the blog posts conveniently forgot to mention
alex-jacobs.com
November 10, 2025 at 4:05 PM
Alex Jacobs: The Case Against pgvector alex-jacobs.com/posts/the-ca...
P99 CONF nails it in terms of both content, *and* UX for attending. Super-simple registration. Free registration. Videos available on-demand afterwards with no gating or games. Clear and usable website. These folk know how to do a developer conference. A+++
🎥 Videos & slides: www.p99conf.io
🎥 Videos & slides: www.p99conf.io
P99 CONF – All Things Performance On-Demand
P99 CONF is a cross-industry virtual event for engineers and by engineers, centered around low-latency, high-performance design.
www.p99conf.io
November 10, 2025 at 11:15 AM
P99 CONF nails it in terms of both content, *and* UX for attending. Super-simple registration. Free registration. Videos available on-demand afterwards with no gating or games. Clear and usable website. These folk know how to do a developer conference. A+++
🎥 Videos & slides: www.p99conf.io
🎥 Videos & slides: www.p99conf.io
You Should Write An Agent fly.io/blog/everyon...
You Should Write An Agent
They're like riding a bike: easy, and you don't get it until you try.
fly.io
November 9, 2025 at 4:05 PM
You Should Write An Agent fly.io/blog/everyon...
How DataDog built a low-latency, multi-tenant data replication platform with Kafka, Kafka Connect, Debezium, and Temporal www.datadoghq.com/blog/enginee...
Replication redefined: How we built a low-latency, multi-tenant data replication platform | Datadog
Discover how Datadog engineered a scalable Change Data Capture (CDC) platform to replicate data across systems in near real time—reducing search latency by 87%, increasing availability, and powering…
www.datadoghq.com
November 8, 2025 at 4:05 PM
How DataDog built a low-latency, multi-tenant data replication platform with Kafka, Kafka Connect, Debezium, and Temporal www.datadoghq.com/blog/enginee...
Blogged: How we built the demo for the Current NOLA Day 2 keynote using Kafka, Flink, and AI
🔗 rmoff.net/2025/11/06/h...
🔗 rmoff.net/2025/11/06/h...
How we built the demo for the Current NOLA Day 2 keynote using Flink and AI
rmoff.net
November 7, 2025 at 5:28 PM
Blogged: How we built the demo for the Current NOLA Day 2 keynote using Kafka, Flink, and AI
🔗 rmoff.net/2025/11/06/h...
🔗 rmoff.net/2025/11/06/h...
How Faire use Cursor as their analytics IDE craft.faire.com/we-moved-ana...
We moved analytics into an IDE — and haven’t looked back
Faire’s journey to AI-native analysis with Cursor
craft.faire.com
November 7, 2025 at 4:05 PM
How Faire use Cursor as their analytics IDE craft.faire.com/we-moved-ana...
Good post from Sam Harris about public speaking, including some good lessons for conference speakers specifically samharris.substack.com/p/the-silent...
The Silent Crowd
Overcoming Your Fear of Public Speaking
samharris.substack.com
November 6, 2025 at 4:05 PM
Good post from Sam Harris about public speaking, including some good lessons for conference speakers specifically samharris.substack.com/p/the-silent...
#duckDB question - is there a smart way to expose all the `value` string fields without manually unpacking them by name?
e.g. this is tedious:
SELECT value.animalName.string, value.deviceID.string
Sample record: gist.github.com/rmoff/38aff1...
e.g. this is tedious:
SELECT value.animalName.string, value.deviceID.string
Sample record: gist.github.com/rmoff/38aff1...
November 6, 2025 at 11:55 AM
#duckDB question - is there a smart way to expose all the `value` string fields without manually unpacking them by name?
e.g. this is tedious:
SELECT value.animalName.string, value.deviceID.string
Sample record: gist.github.com/rmoff/38aff1...
e.g. this is tedious:
SELECT value.animalName.string, value.deviceID.string
Sample record: gist.github.com/rmoff/38aff1...
Quick post about the data blips on the latest @thoughtworks.com tech radar:
rmoff.net/2025/11/05/t...
rmoff.net/2025/11/05/t...
Tech Radar (Nov 2025) - data blips
rmoff.net
November 5, 2025 at 5:16 PM
Quick post about the data blips on the latest @thoughtworks.com tech radar:
rmoff.net/2025/11/05/t...
rmoff.net/2025/11/05/t...
Reposted by rmoff 🏃♂️🫖🥓
Stream-order vs batch-order in Iceberg:
* Flink wants temporal locality.
* Spark wants value locality.
Same table, conflicting physics.
New post: jack-vanlightly.com/blog/2025/11...
* Flink wants temporal locality.
* Spark wants value locality.
Same table, conflicting physics.
New post: jack-vanlightly.com/blog/2025/11...
How Would You Like Your Iceberg Sir? Stream or Batch Ordered? — Jack Vanlightly
Today I want to talk about stream analytics, batch analytics and Apache Iceberg. Stream and batch analytics work differently but both can be built on top of Iceberg, but due to their differences there...
jack-vanlightly.com
November 5, 2025 at 2:52 PM
Stream-order vs batch-order in Iceberg:
* Flink wants temporal locality.
* Spark wants value locality.
Same table, conflicting physics.
New post: jack-vanlightly.com/blog/2025/11...
* Flink wants temporal locality.
* Spark wants value locality.
Same table, conflicting physics.
New post: jack-vanlightly.com/blog/2025/11...
Reposted by rmoff 🏃♂️🫖🥓
💯 we had an entire "logs" database at WePay and it was a nightmare. It's convenient until it isn't. And backing out is brutal.
📝 Blogged: "'You Don't Need Kafka, Just Use Postgres' Considered Harmful"
In which I'm arguing that both Postgres and Kafka are great tools for their respective purposes. But don't create your custom implementation of one on top of the other.
👉 www.morling.dev/blog/you-don...
In which I'm arguing that both Postgres and Kafka are great tools for their respective purposes. But don't create your custom implementation of one on top of the other.
👉 www.morling.dev/blog/you-don...
November 3, 2025 at 4:06 PM
💯 we had an entire "logs" database at WePay and it was a nightmare. It's convenient until it isn't. And backing out is brutal.
Check it out - I was on a podcast :) @gamussa.dev was kind enough to spend some time chatting with me about my time working with Oracle, Kafka, and more.
🎧 confluent.buzzsprout.com/186154/episo...
🎥 www.youtube.com/watch?v=U0t5...
🎧 confluent.buzzsprout.com/186154/episo...
🎥 www.youtube.com/watch?v=U0t5...
November 3, 2025 at 4:05 PM
Check it out - I was on a podcast :) @gamussa.dev was kind enough to spend some time chatting with me about my time working with Oracle, Kafka, and more.
🎧 confluent.buzzsprout.com/186154/episo...
🎥 www.youtube.com/watch?v=U0t5...
🎧 confluent.buzzsprout.com/186154/episo...
🎥 www.youtube.com/watch?v=U0t5...
ICYMI: Interesting Links in the data world for October: rmoff.net/2025/10/31/i...
November 3, 2025 at 10:24 AM
ICYMI: Interesting Links in the data world for October: rmoff.net/2025/10/31/i...
With nary 8.5 hours to spare (GMT) before the end of the month, herewith a whole lotta links about things in the data engineering world that I found interesting this month.
👉 rmoff.net/2025/10/31/i...
👉 rmoff.net/2025/10/31/i...
Interesting links - October 2025
rmoff.net
October 31, 2025 at 3:35 PM
With nary 8.5 hours to spare (GMT) before the end of the month, herewith a whole lotta links about things in the data engineering world that I found interesting this month.
👉 rmoff.net/2025/10/31/i...
👉 rmoff.net/2025/10/31/i...
Reposted by rmoff 🏃♂️🫖🥓
If you were affected by recent changes to MinIO, check out the @chainguard.dev MinIO image. We didn't want people to be stuck on an unmaintained image, so we've added the our version to the free tier.
Secure and Free MinIO Chainguard Containers
MinIO pulled its free images—but Chainguard has you covered. Get zero-CVE, continuously built MinIO and MinIO Client containers, free and secure from Chainguard.
www.chainguard.dev
October 28, 2025 at 11:05 AM
If you were affected by recent changes to MinIO, check out the @chainguard.dev MinIO image. We didn't want people to be stuck on an unmaintained image, so we've added the our version to the free tier.
🐿️ My new favourite laptop sticker 😍😆
October 28, 2025 at 9:45 AM
🐿️ My new favourite laptop sticker 😍😆
One of the things I love about @vanlightly.bsky.social's posts is that he is both opinionated *and* highly informed and articulate in his argument. This post is a great example: Why I’m not a fan of zero-copy Apache Kafka-Apache Iceberg jack-vanlightly.com/blog/2025/10...
Why I’m not a fan of zero-copy Apache Kafka-Apache Iceberg — Jack Vanlightly
Over the past few months, I’ve seen a growing number of posts on social media promoting the idea of a “zero-copy” integration between Apache Kafka and Apache Iceberg. The idea is that Kafka topics…
jack-vanlightly.com
October 24, 2025 at 3:08 PM
One of the things I love about @vanlightly.bsky.social's posts is that he is both opinionated *and* highly informed and articulate in his argument. This post is a great example: Why I’m not a fan of zero-copy Apache Kafka-Apache Iceberg jack-vanlightly.com/blog/2025/10...
Excellent analysis from @vanlightly.bsky.social: A Fork in the Road: Deciding #ApacheKafka’s Diskless Future jack-vanlightly.com/blog/2025/10...
October 23, 2025 at 3:07 PM
Excellent analysis from @vanlightly.bsky.social: A Fork in the Road: Deciding #ApacheKafka’s Diskless Future jack-vanlightly.com/blog/2025/10...
Ended up going with asciidoc + revealJS.
Love the low friction and focus on writing.
Check out the results: rmoff.net/talk/blog-wr... and the source: raw.githubusercontent.com/rmoff/rmoff-...
Love the low friction and focus on writing.
Check out the results: rmoff.net/talk/blog-wr... and the source: raw.githubusercontent.com/rmoff/rmoff-...
October 23, 2025 at 10:53 AM
Ended up going with asciidoc + revealJS.
Love the low friction and focus on writing.
Check out the results: rmoff.net/talk/blog-wr... and the source: raw.githubusercontent.com/rmoff/rmoff-...
Love the low friction and focus on writing.
Check out the results: rmoff.net/talk/blog-wr... and the source: raw.githubusercontent.com/rmoff/rmoff-...