Buf
LightNews LightNews
banner
buf.build
Buf
@buf.build
170 followers 9 following 71 posts
Kafka and gRPC for the modern age. | buf.build | connectrpc.com
Posts Replies Media Videos
buf.build
If you’re at #Current25 this week, stop by the Buf booth for…
- The solution to streaming data’s data problem
- Cool tarot card stickers
- Free O’Reilly book Data Engineering Design Patterns
- Trick or treat candy
- Chance to win 1 of 6 Halloween Lego sets
October 29, 2025 at 1:55 PM
buf.build
Join Buf engineer Miguel Young de la Sota tomorrow when he breaks down hyperpb: a fully-dynamic Protobuf parser that's 10x faster than dynamicpb, the standard Go solution for dynamic Protobuf.
Thursday 10/23 @ 10:25am PT at
@P99CONF
Register here: www.p99conf.io
P99 CONF Event 2025 – All Things Performance On-Demand
P99 CONF is a cross-industry virtual event for _engineers_ and by engineers, centered around low-latency, high-performance design.
www.p99conf.io
October 22, 2025 at 5:29 PM
buf.build
For streaming data the gap between "hello world" tutorials and bulletproof production systems is filled with hard-learned lessons. Join Bartosz Konieczny and our own Scott Haines Oct 22 at 9am PT for Linux Foundation webinar “Streaming Data Design Patterns"
www.linuxfoundation.org/webinars/str...
Streaming Data Design Patterns: Battle-Tested Tips and Tricks for Mission-Critical Systems
Get insights from the best open source projects and people. View one of our upcoming or on-demand webinars on topics from Kubernetes to security.
www.linuxfoundation.org
October 21, 2025 at 4:58 PM
buf.build
Excited to co-sponsor the Apache Iceberg Bay Area Meetup, Oct 1!
Our own Scott Haines joins a great lineup of speakers covering real-world production stories & tech deep dives.
📍 San Francisco 🗓️ Oct 1, 2025 (afternoon)
Register: luma.com/hbt7ahud?tk=...
Apache Iceberg™ Bay Area Meetup: Oct Edition · Luma
🔹 Apache Iceberg™ Bay Area: Oct Edition 📍 Contemporary Jewish Museum, San Francisco 🗓️ October 1, 2025 | ⏰ Half-day, afternoon event 💥 Free to attend –…
luma.com
September 25, 2025 at 4:07 PM
buf.build
🚀 Here’s a winning combo: Bufstream + @tigrisdata.com = unlimited Kafka message retention with zero egress fees!

Drop-in Kafka replacement that scales globally without breaking the bank. Check out this step-by-step guide to get started 👇

buf.build/blog/bufstre...

#Kafka #DataLakehouse
buf.build
September 17, 2025 at 3:05 PM
buf.build
📅 TODAY: Scott Haines from Buf Devrel & Youssef Mrini dive into #DeltaLake 4.0 at 12pm ET/9am PT ⏰

Featuring #DeltaConnect for remote Delta Lake table access via #gRPC + a spark-connect deep dive!

Join: www.linkedin.com/events/73690...
LinkedIn Login, Sign in | LinkedIn
Login to LinkedIn to keep in touch with people you know, share ideas, and build your career.
www.linkedin.com
September 16, 2025 at 1:06 PM
buf.build
Big milestone! Protovalidate just hit v1.0

It's big news for the Protobuf semantic validation library trusted by Microsoft, GitLab, Nike and 1000's more to:
✅ Define validation rules only once
✅ Enforce everywhere (Go, Java, Python, C++, TypeScript)

Details in our blog: buf.build/blog/protova...
Protovalidate is now v1.0
After two years of development, we're proud to announce that Protovalidate has reached v1.0. Protovalidate is the semantic validation library for Protobuf. Protobuf gives you the structure of your dat...
buf.build
September 12, 2025 at 8:23 PM
buf.build
PSA: Buf is giving away "Data Engineering Design Patterns" (O'Reilly) for FREE 🎉

Tired of data pipeline problems that someone else has already cracked? This book has the patterns you actually need, from data ingestion to data observability.

Includes open source examples.

buf.build/resources/da...
Data Engineering Design Patterns
This hands-on guide from O’Reilly Media focuses on critical data engineering design patterns, including data ingestion, data quality, and idempotency.
buf.build
September 3, 2025 at 4:43 PM
buf.build
LinkedIn's Kafka origin story reveals 3 gaps: Schema evolution, plug-and-play integration, and governance.

Buf complete that vision:

- CLI breaking change detection
- Bufstream broker-side schema awareness
- BSR organization-wide governance

Finish what Kafka started. ⚡

buf.build/blog/finishi...
Finishing Kafka: LinkedIn proved it works, Buf makes it simple
Stanislav Kozlovski's deep dive into the Kafka creation story appeared last weekend. It detailed how LinkedIn solved data problems with Kafka and also revealed three remaining challenges—schema evolut...
buf.build
August 26, 2025 at 3:38 PM
buf.build
🚨 With streaming data, any record flowing through your pipelines is a potential point of failure. When schemas drift or malformed data sneaks through, your downstream systems don't just slow down—they break.

🛠️ Read about how semantic validation solves it: buf.build/blog/semanti...
buf.build
August 21, 2025 at 6:47 PM
buf.build
Working with Protobuf? We’ve got the online workshop for you! Our engineers are hosting a 1-hour session that shows how to make depending on Protobuf schemas as easy as using go get, npm, mvn, or pip.

Thurs Aug 21 9 AM PDT / 12 PM EDT / 5 PM BST

Sign-up today at: buf.build/events/bsr-w...
Getting Started with the Buf Schema Registry
Our interactive workshop is the best way to learn the ins and outs of our centralized registry for tracking and evolving Protobuf APIs.
buf.build
August 7, 2025 at 9:10 PM
buf.build
Announcing hyperpb, a fully-dynamic Protobuf parser that achieves 3x performance over generated Go code! Our hyperspeed dynamic parsing turns once-unscalable products into ordinary, even essential components of your stack.
Check out our benchmarks! ⚡

buf.build/blog/hyperpb
Introducing hyperpb: 10x faster dynamic Protobuf parsing that’s even 3x faster than generated code
Today we’re announcing public availability of hyperpb, a fully-dynamic Protobuf parser that is 10x faster than dynamicpb, the standard Go solution for dynamic Protobuf. In fact, it’s so efficient that...
buf.build
July 17, 2025 at 1:04 PM
buf.build
Streaming data got a data problem? Don’t miss our workshop on schema-driven governance for Kafka. Our engineering and product teams will show what you can do when brokers understand the shape of your streaming data. Q&A throughout!

Thursday, July 10 9 AM PT / 12 PM ET / 5 PM BST
How to bring schema-driven governance to streaming data
Our interactive workshop is the best way to learn the ins and outs of Bufstream, our Kafka-compatible, schema-first streaming data platform.
buf.build
July 8, 2025 at 4:57 PM
buf.build
"If you're aiming to build robust, modern data pipelines without Kafka’s complexity, schema-first is the way forward." Couldn't have said it better ourselves.

www.linkedin.com/posts/brijpa...
Why Schema-Driven Streaming is the Future of Kafka | Brij kishore Pandey
Why Schema-Driven Streaming is the Future of Kafka Anyone who’s managed Kafka at scale knows the pain: -> Managing multiple clusters -> Dealing with frequent schema mismatches -> Wasting ...
www.linkedin.com
July 3, 2025 at 5:35 PM
buf.build
Don’t miss our interactive workshop on bringing schema-driven governance to Kafka. Our engineering and product teams will deep dive into how it’s done. Plus, we’ll answer your questions throughout.

📷 Thursday, July 10 📷 9 AM PDT/12 PM EDT/5 PM BST

buf.build/events/works...
How to bring schema-driven governance to streaming data
Our interactive workshop is the best way to learn the ins and outs of Bufstream, our Kafka-compatible, schema-first streaming data platform.
buf.build
July 2, 2025 at 8:09 PM
buf.build
Another day, another great video on the data quality issues that plague Apache Kafka and other Kafka-compatible platforms. Abhishek Veeramalla goes in depth on schema validation, plus Kafka’s high storage and networking costs. Spoiler alert: Bufstream is the answer.

www.youtube.com/watch?v=oR4F...
Data Quality Concerns & Storage Cost challenges with Apache Kafka
YouTube video by Abhishek.Veeramalla
www.youtube.com
July 1, 2025 at 1:03 PM
buf.build
See how Clinician Nexus governs data quality with the Buf CLI + Buf Schema Registry and keeps costs low with Kafka-compatible Bufstream. Results: consistent validation, quick updates & complete audit trail. #DAIS #Kafka

www.youtube.com/watch?v=qZLG...
Unifying Human-Curated Data Ingestion and Real-Time Updates with Databricks DLT, Protobuf and BSR
YouTube video by Databricks
www.youtube.com
June 24, 2025 at 12:58 PM
buf.build
TL;DR: The first 15 field numbers are special: most runtimes will decode them much faster than the other field numbers. When designing a message type for decoding performance, it’s good to use these field numbers on fields that are almost always present.

buf.build/blog/totw-9-...
Tip of the week #9: Some numbers are more equal than others
The first 15 field numbers are special: most runtimes will decode them much faster than the other field numbers. When designing a message type for decoding performance, it’s good to use these field nu...
buf.build
June 17, 2025 at 12:10 PM
buf.build
Oneofs are a disaster. So we fixed them. buf.build/blog/fixing-...
Oneofs are a disaster. Protovalidate has fixed them.
Instead of using oneofs, you can now use the new (buf.validate.message).oneof Protovalidate annotation. As long as you're validating your messages with Protovalidate, (buf.validate.message).oneof does...
buf.build
June 13, 2025 at 6:06 PM
buf.build
Big news from #DataAISummit! Bufstream now supports Databricks Managed Iceberg Tables in private preview, bringing together Buf's schema-first approach with Databricks' industry-leading data governance and optimization capabilities. Read all about it!

buf.build/blog/buf-ann...
Buf Announces Support for Databricks Managed Iceberg Tables
We’re announcing a major step toward our vision of a single schema definition in Protobuf that governs your entire data flow — from API contracts to streaming topics to analytical tables. Bufstream no...
buf.build
June 12, 2025 at 6:11 PM
buf.build
TL;DR Don’t use required, no matter how tempting. You won’t be able to get rid of it later when you realize it was a bad idea.

buf.build/blog/totw-8-...
Tip of the week #8: Never use required
Don’t use 'required' to modify fields—you won’t be able to get rid of it later when you realize it was a bad idea.
buf.build
June 9, 2025 at 7:06 PM
buf.build
TL;DR: buf convert is a powerful tool for examining wire format dumps, by converting them to JSON and using existing JSON analysis tooling. protoscope can be used for lower-level analysis, such debugging messages that have been corrupted.

buf.build/blog/totw-7-...
Tip of the week #7: Scoping it out
buf convert and protoscope are powerful tools for examining and debugging wire format dumps.
buf.build
June 2, 2025 at 7:56 PM
buf.build
Don’t miss our interactive workshop on bringing schema-driven governance to Kafka. Our engineering and product teams will deep dive into how it’s done. Plus, we’ll answer your questions throughout.

Thursday, May 29 9 AM PDT / 12 PM EDT / 5 PM BST
How to bring schema-driven governance to streaming data
Our interactive workshop is the best way to learn the ins and outs of Bufstream, our Kafka-compatible, schema-first streaming data platform.
buf.build
May 27, 2025 at 5:54 PM
buf.build
TL;DR: Enum values can have aliases. This feature is poorly designed and shouldn’t be used. The ENUM_NO_ALLOW_ALIAS Buf lint rule prevents you from using them by default.

buf.build/blog/totw-6-...
Tip of the week #6: The subtle dangers of enum aliases
Enum values are a poorly designed feature and shouldn’t be used.
buf.build
May 19, 2025 at 6:32 PM
buf.build
We’re excited to announce that Bufstream is now available on the AWS Marketplace. Enterprise customers can purchase through their AWS account and accelerate their deployment of Bufstream.

buf.build/blog/bufstre...
Bufstream is now on the AWS Marketplace
We're excited to announce that Bufstream is now available on the AWS Marketplace. Enterprise customers can purchase through their AWS account and accelerate their deployment of Bufstream.
buf.build
May 16, 2025 at 4:09 PM