Delta Lake
LightNews LightNews
banner
deltalakeoss.bsky.social
Delta Lake
@deltalakeoss.bsky.social
41 followers 6 following 130 posts
Delta Lake is an open-source storage framework that enables building a Lakehouse architecture for Spark, Flink, Trino, Hive, Scala, Java, Rust, Python, & more.
Posts Replies Media Videos
deltalakeoss.bsky.social
Did you know you can easily enable Variant Type in existing Delta tables? 💡 Just run an ALTER TABLE command to activate the Delta feature: `variantType`.

Watch the full webinar ➡️ www.youtube.com/live/0BfUG2P...

#DeltaLake #OpenSource #OSS
October 16, 2025 at 6:48 PM
deltalakeoss.bsky.social
Open Lakehouse + AI spotlight: @hannes.muehleisen.org shares how anyone can contribute to DuckDB & DuckLake—both MIT-licensed, open source, & welcoming PRs and community engagement on GitHub.

Check out this clip to hear how easy it is to get involved and start contributing today. 🚀

#openlakehouse
October 15, 2025 at 7:13 PM
deltalakeoss.bsky.social
One of the most powerful new capabilities in Delta Lake 4.0 is support for collations, giving you much finer control over how text is compared and sorted. ✅

In this clip, Youssef Mrini explains how to enable collations.

🎥 Watch the full webinar: youtube.com/live/0BfUG2PZyNs?si=nzcalgk1gvOEFBrY
October 8, 2025 at 7:25 PM
deltalakeoss.bsky.social
Delta Connect is a plugin atop Spark Connect, introduced in Spark 3.4, and enables gRPC communication using Protocol Buffers. ✅ This allows client implementations in languages such as Rust and Go to interact with Spark outside the JVM, provided they support the protocol.

#opensource #oss #deltalake
October 3, 2025 at 4:46 PM
deltalakeoss.bsky.social
At Open Lakehouse + AI Amsterdam last month, Sammy Sidhu—co-creator of the Daft multi-modal query engine and CEO of Eventual—shared how his open source journey began. 🚀

Ready to shape the future of data, open source, and AI together? Join us at our next Open Lakehouse + AI events! ⬇️

#opensource
September 30, 2025 at 2:30 PM
deltalakeoss.bsky.social
“Open source is giving back, because it means everyone who supports our work can participate and benefit from the software we are making." — @hannes.muehleisen.org (@duckdb.org) at Open Lakehouse + AI Amsterdam 🇳🇱

Ready to shape the future of data, open source, and AI together? ⬇️

#opensource #oss
September 22, 2025 at 2:40 PM
deltalakeoss.bsky.social
Dropping features in #DeltaLake used to break table compatibility and history. Delta 4.0 fixes this by protecting checkpoints and rewriting recent history—keeping tables safe and stable. 💪

🎥 Watch the full webinar replay: www.youtube.com/watch?v=0BfU...

#opensource #oss
September 19, 2025 at 7:01 PM
deltalakeoss.bsky.social
Upcoming Webinar ➡️ Exploring Delta’s Rust Kernel: a look behind the scenes at what powers delta-rs!

Join R. Tyler Croy, Robert Pack & Scott Haines to learn how kernel changes deliver consistency, new features & faster performance for Delta engines.

📅 Sept 30 @ 9AM PT
🔗 RSVP: luma.com/delta-0930
September 10, 2025 at 4:07 PM
deltalakeoss.bsky.social
📣 Mark your calendar for Tuesday, October 7 at 9AM PT for the next Open Lakehouse + AI webinar: “From Functions to AI Agents: Reimagining the Lakehouse for an Agentic Future!” 🚀

🔗 Register here: luma.com/OLAI-107

#opensource #deltalake #agentic #aiagents #oss
September 9, 2025 at 5:45 PM
deltalakeoss.bsky.social
Excited to announce Delta Live! Live Rust Hacking and AMA — streaming tomorrow, Sept 9 on Twitch. 👏

Join R. Tyler Croy and Robert Pack for a hands-on Rust session and live Q&A!

📅 Sept 9
⏰ 7AM PT / 10AM ET
🔗 twitch.tv/agentdero

Bring your questions and gain insights from #DeltaLake maintainers. 🦀
September 8, 2025 at 2:33 PM
deltalakeoss.bsky.social
The first-ever Open Lakehouse + AI Meetup in Amsterdam was a huge success! 🙌🇳🇱

The evening brought together the open source and data engineering community to explore the latest advancements in open lakehouse and AI architectures. 🚀
August 28, 2025 at 1:06 PM
deltalakeoss.bsky.social
🚨 Happening TODAY, August 26 at 7AM PT!

Join R. Tyler Croy (Buoyant Data) & Robert Pack (Databricks) for live Rust hacking and get answers to all your #DeltaLake questions — streaming LIVE on Twitch!

🎥 Tune in: www.twitch.tv/agentdero

#rustlang #rust #kernel
August 26, 2025 at 11:56 AM
deltalakeoss.bsky.social
🚨 This Week: Open Lakehouse Meetup – Amsterdam!

Last chance to join Amsterdam’s open source and data engineering community for an evening of hands-on learning and fresh ideas about the future of open lakehouse systems. 🙌

🗓 Wednesday, August 27
⏰ 5-9PM
📍 Amsterdam

🔗 Register now: lu.ma/OLM-827
August 25, 2025 at 10:15 AM
deltalakeoss.bsky.social
Join R. Tyler Croy, Delta Lake maintainer and founder of Buoyant Data, for “Delta Live! Live Rust hacking and AMA” 🦀 on Twitch.

📅 Tuesday, August 19
⏰ 7:00AM PT / 10:00AM ET

Tune in here ➡️ twitch.tv/agentdero

#opensource #deltalake #oss #rust #rustlang
August 18, 2025 at 10:12 PM
deltalakeoss.bsky.social
🇳🇱 Join us in Amsterdam on August 27 for the Open Lakehouse Meetup!

Discover how your #lakehouse is ready for production-scale multimodal ML pipelines in "Your Lakehouse Has Everything You Need" with Sammy Sidhu, CEO & Co-Founder of Eventual!

🔗 Register now to save your spot: lu.ma/OLM-827

#oss
August 18, 2025 at 1:31 PM
deltalakeoss.bsky.social
Exciting update in Delta Kernel Java! 🎉

Engines can now write file statistics to the Delta log, enabling smarter data skipping with read‑time filters. The result: faster queries, better performance, and improved resource use.

🔗 Explore all the new features in Delta 4.0: github.com/delta-io/del...
August 14, 2025 at 8:45 PM
deltalakeoss.bsky.social
📢 Join us in Amsterdam for the 𝗢𝗽𝗲𝗻 𝗟𝗮𝗸𝗲𝗵𝗼𝘂𝘀𝗲 𝗠𝗲𝗲𝘁𝘂𝗽! 🌟

Learn from Robert Pack & Ion Koutsouris about making #ApacheIceberg, #DeltaLake & more fully composable to improve interoperability across query engines.

📅 Wednesday, August 27
🕝 5-9PM
📍 Amsterdam

🔗 Register: lu.ma/OLM-827

#opensource #oss
August 11, 2025 at 2:06 PM
deltalakeoss.bsky.social
@hannes.muehleisen.org, Co-Creator of @duckdb.org, will be talking about the DuckLake project at the Open Lakehouse Meetup in Amsterdam on August 27th! Don't miss it. 🦆🦀

Sign up here ➡️ lu.ma/OLM-827

#duckdb #opensource #oss #deltalake #openlakehouse
August 6, 2025 at 5:40 PM
deltalakeoss.bsky.social
Curious about how Delta Lake structures its tables and achieves both scalability and reliability?

In our recent webinar, Scott Haines breaks down the building blocks of a #DeltaLake table.

▶️ Watch the full webinar to learn more: www.youtube.com/live/O8_82Cu...

#opensource #oss #metadata #parquet
July 24, 2025 at 5:10 PM
deltalakeoss.bsky.social
In this clip, Ion Koutsouris explains how lakeFS garbage collection policy integrates with #DeltaLake to manage unreferenced files and automate cleanups.

By leveraging #Spark jobs, #lakeFS ensures your Delta Lake storage remains lean and organized. ✅

🎥 Learn more: www.youtube.com/live/OH1vH19...
July 18, 2025 at 7:34 PM
deltalakeoss.bsky.social
“We cut our streaming ingestion costs over 90% by adopting kafka-delta-ingest, which means we can invest those savings in really interesting large language model products or innovative data processes.” — R. Tyler Croy, Principal Engineer at Scribd & Maintainer of delta-rs 💬
July 18, 2025 at 1:36 PM
deltalakeoss.bsky.social
🤔 How fast can you process a 10M-row CSV into Delta Lake?

Daniel Beach put Daft to the test by reading a 1.1GB CSV (10 million lines) and writing it to Delta Lake—all in just 3 minutes on AWS Lambda. 🚀

🎥 Watch the full webinar: www.youtube.com/watch?v=BR9o...

#opensource #oss #deltalake #daft
July 17, 2025 at 7:18 PM
deltalakeoss.bsky.social
With Liquid Clustering, you don’t need to worry about cardinality. Whether your data has high or low cardinality, #LiquidClustering dynamically handles it—no need for manual partition tuning. ✅

🎥 Learn more: www.youtube.com/live/l8CEyXg...

(1/2)

#opensource #oss #deltalake #linuxfoundation
July 16, 2025 at 6:11 PM
deltalakeoss.bsky.social
"With over three petabytes of processed data and more than 1,200 active users, our Lakehouse platform powered by Delta Lake is at the core of how we drive insights at scale." - Satya Mandavilli

👉 Watch the full video: www.youtube.com/live/1Bp8VUW...

#opensource #oss #deltalake #lakehouse
July 15, 2025 at 5:59 PM
deltalakeoss.bsky.social
In this clip, Youssef Mrini explains how deletion vectors in #DeltaLake help you avoid rewriting #Parquet files for every update or delete. Instead, a bitmap marks which rows are deleted, so files are only rewritten when necessary. ✅

🎥 Watch the full video for more: www.youtube.com/live/O8_82Cu...
July 14, 2025 at 9:26 PM