Tom Cooper
banner
tomcooper.dev
Tom Cooper
@tomcooper.dev
Senior Principal Engineer at IBM, Ex-Red Hat, Ex-Reddit, Ex-Twitter, Distributed Systems PhD, interested in all things stream processing, also cycling
It was a really great talk, you covered a lot in 15mins! I am keen to work on an OSS library for embeddings wrapping Langchain4j so may ping you for opinions/reviews if you have time?
May 21, 2025 at 8:35 PM
Luckily we have 2 people on the Strimzi team who literally wrote the book on Kafka Connect and Strimiz could do something (TBD) to help.
May 21, 2025 at 4:11 PM
Mostly about how everyone wants DR, try MM2, bounce off it and then end up with some very unique setups and trade offs. TL;DR MM2 (and connect more generally) is hard to get started with and has some sharp edges.
May 21, 2025 at 4:09 PM
"Process" not "Polymorphic" 🤦‍♂️
May 21, 2025 at 2:15 PM
So nice to get a deep technical deep dive on a subject. They did a great job of explaining the migration process. If you want more detail we have blog posts:
May 21, 2025 at 12:42 PM
Ok so the main thrust is actually about removing Kafka from the mix & what you need to do in Iceberg to emulate topic compaction. A really interesting custom solution due to missing features in the Flink Iceberg connector. The headline, it works, it doesn't need Kafka but it's not as fast #Current25
May 21, 2025 at 9:36 AM
Wooop Open Source Datastack FTW! Pinterest moving from an in-house to OSS based architecture! #Current25
May 20, 2025 at 4:49 PM
Great to hear that OpenAI have identified and fixed several issues with PyFlink and intend to push those fixes upstream 👏👏👏
May 20, 2025 at 3:50 PM
It's PyFlink for all the things and OpenAI uses the upstream Flink Kubernetes Operator
May 20, 2025 at 3:41 PM
This was a very interesting talk full of intriguing ideas. However, there was a lot of "you _could_ do <complicated thing> and then do <even more complicated thing> based on that". I would love to see some real life demonstrations of all these agents linked together doing what we expect.
May 20, 2025 at 3:11 PM