Weston Pace
@westonpace.bsky.social
Software developer working on all things arrow and columnar storage, currently, Lance.
Pretty sure my stress level can be accurately inferred by the number of unread emails in my inbox
November 4, 2025 at 2:26 PM
Pretty sure my stress level can be accurately inferred by the number of unread emails in my inbox
Reposted by Weston Pace
A new startup, @columnar.tech, looks to streamline the copying of tabular data across systems, using @arrow.apache.org and the ADBC API.
By @joabj.bsky.social
By @joabj.bsky.social
Apache Arrow's Final Frontier: Replacing Outdated Database Drivers
A new startup, Columnar, looks to streamline the copying of tabular data across systems, using Apache Arrow and the ADBC API.
bit.ly
November 3, 2025 at 7:00 PM
A new startup, @columnar.tech, looks to streamline the copying of tabular data across systems, using @arrow.apache.org and the ADBC API.
By @joabj.bsky.social
By @joabj.bsky.social
Async is awesome until it isn't
November 3, 2025 at 8:28 PM
Async is awesome until it isn't
Conclusion of a little halloween tradition. If I'm going to traumatize the kids it might as well be interesting.
October 31, 2025 at 3:39 PM
Conclusion of a little halloween tradition. If I'm going to traumatize the kids it might as well be interesting.
Reposted by Weston Pace
The future of data connectivity is columnar. Today we launched
@columnar.tech to accelerate the shift from slow, row-oriented APIs like ODBC and JDBC to >10x faster alternatives powered by @arrow.apache.org. Learn more 👇
@columnar.tech to accelerate the shift from slow, row-oriented APIs like ODBC and JDBC to >10x faster alternatives powered by @arrow.apache.org. Learn more 👇
Announcing Columnar
Back to the future of data connectivity
columnar.tech
October 29, 2025 at 10:51 PM
The future of data connectivity is columnar. Today we launched
@columnar.tech to accelerate the shift from slow, row-oriented APIs like ODBC and JDBC to >10x faster alternatives powered by @arrow.apache.org. Learn more 👇
@columnar.tech to accelerate the shift from slow, row-oriented APIs like ODBC and JDBC to >10x faster alternatives powered by @arrow.apache.org. Learn more 👇
A bittersweet story but glad to see a principled stance!
TLDR; The PSF has made the decision to put our community and our shared diversity, equity, and inclusion values ahead of seeking $1.5M in new revenue. Please read and share. pyfound.blogspot.com/2025/10/NSF-...
🧵
🧵
The official home of the Python Programming Language
www.python.org
October 27, 2025 at 7:11 PM
A bittersweet story but glad to see a principled stance!
However - a word that exists because someone decided we aren't allowed to start a sentence with "but"
October 27, 2025 at 1:18 PM
However - a word that exists because someone decided we aren't allowed to start a sentence with "but"
Douglas squirrels are 1/3 the size of gray squirrels but six times more ferocious.
October 10, 2025 at 3:52 PM
Douglas squirrels are 1/3 the size of gray squirrels but six times more ferocious.
Yesterday, OP responded to my 11 year old comment on their 13 year old post with a pedantic correction.
October 7, 2025 at 11:26 AM
Yesterday, OP responded to my 11 year old comment on their 13 year old post with a pedantic correction.
Lot's of work being done on columnar file formats lately. I count 5 new formats so far (Lance, Nimble, Vortex, FastLanes, F3).
It's definitely something we follow at LanceDB and it can be confusing to track. So here is my very biased head-canon (trying to stay positive)
It's definitely something we follow at LanceDB and it can be confusing to track. So here is my very biased head-canon (trying to stay positive)
October 3, 2025 at 5:18 PM
Lot's of work being done on columnar file formats lately. I count 5 new formats so far (Lance, Nimble, Vortex, FastLanes, F3).
It's definitely something we follow at LanceDB and it can be confusing to track. So here is my very biased head-canon (trying to stay positive)
It's definitely something we follow at LanceDB and it can be confusing to track. So here is my very biased head-canon (trying to stay positive)
Newest house mate is an industrious spider that spends every day building a beautiful web right at eye level so I can blearily walk face first into it every morning.
October 3, 2025 at 3:38 PM
Newest house mate is an industrious spider that spends every day building a beautiful web right at eye level so I can blearily walk face first into it every morning.
Son got mad and told me he wouldn't take me to the creamery when I died. I have some questions.
August 14, 2025 at 1:42 AM
Son got mad and told me he wouldn't take me to the creamery when I died. I have some questions.
We're trying to figure out "Substrait coordinates" (e.g. organization, name, version tuples) for Substrait functions. Is anyone out there actually passionate about the topic or have any lessons or advice?
At the moment, leaning towards `organization:name:version` (and forbid colon in each field)
At the moment, leaning towards `organization:name:version` (and forbid colon in each field)
August 13, 2025 at 5:48 PM
We're trying to figure out "Substrait coordinates" (e.g. organization, name, version tuples) for Substrait functions. Is anyone out there actually passionate about the topic or have any lessons or advice?
At the moment, leaning towards `organization:name:version` (and forbid colon in each field)
At the moment, leaning towards `organization:name:version` (and forbid colon in each field)
Enjoying the lesser known feasting holiday "fridge fails in the middle of summer"
August 3, 2025 at 5:55 PM
Enjoying the lesser known feasting holiday "fridge fails in the middle of summer"
Anyone else getting strange segfaults from OpenSSL 3.0.17 on bookworm in docker?
August 1, 2025 at 8:41 PM
Anyone else getting strange segfaults from OpenSSL 3.0.17 on bookworm in docker?
Selling my digital soul for $25/mo auto insurance discount
July 28, 2025 at 8:03 PM
Selling my digital soul for $25/mo auto insurance discount
@void.comind.network fails the Turing test because no human is that good at shitposting.
July 25, 2025 at 10:11 PM
@void.comind.network fails the Turing test because no human is that good at shitposting.
Reposted by Weston Pace
It was fun to let an Arrow C++ contributor highlight their work here. Thanks to Rossi for all the work on the post and of course all the reviewers of the PRs themselves.
New post up on the Arrow blog about some of the recent improvements to the embedded query engine inside Arrow C++: arrow.apache.org/blog/2025/07... #apachearrow
Recent Improvements to Hash Join in Arrow C++
A deep dive into recent improvements to Apache Arrow’s hash join implementation — enhancing stability, memory efficiency, and parallel performance for modern analytic workloads.
arrow.apache.org
July 19, 2025 at 7:58 PM
It was fun to let an Arrow C++ contributor highlight their work here. Thanks to Rossi for all the work on the post and of course all the reviewers of the PRs themselves.
Reposted by Weston Pace
🆕 #CNG2025 session recording
@kylebarron.dev from @developmentseed.org introduces Obstore, a python library he wrote to access data on the cloud. Watch his talk to hear about the motivations for writing Obstore & its details
▶️ youtu.be/Wxh1G2oCZiU
@kylebarron.dev from @developmentseed.org introduces Obstore, a python library he wrote to access data on the cloud. Watch his talk to hear about the motivations for writing Obstore & its details
▶️ youtu.be/Wxh1G2oCZiU
[CNG 2025] Fast Cloud Storage Operations with Obstore – Kyle Barron
YouTube video by Cloud-Native Geospatial Forum
youtu.be
July 8, 2025 at 5:06 PM
🆕 #CNG2025 session recording
@kylebarron.dev from @developmentseed.org introduces Obstore, a python library he wrote to access data on the cloud. Watch his talk to hear about the motivations for writing Obstore & its details
▶️ youtu.be/Wxh1G2oCZiU
@kylebarron.dev from @developmentseed.org introduces Obstore, a python library he wrote to access data on the cloud. Watch his talk to hear about the motivations for writing Obstore & its details
▶️ youtu.be/Wxh1G2oCZiU
Me: Do this thing
Claude Code: I tried but I'm not allowed to cd to that directory for security reasons.
Claude Code: Hmm...
Claude Code: I can use xargs to work around this limitation, how does that sound?
Me: 🤨
Claude Code: I tried but I'm not allowed to cd to that directory for security reasons.
Claude Code: Hmm...
Claude Code: I can use xargs to work around this limitation, how does that sound?
Me: 🤨
June 26, 2025 at 4:44 PM
Me: Do this thing
Claude Code: I tried but I'm not allowed to cd to that directory for security reasons.
Claude Code: Hmm...
Claude Code: I can use xargs to work around this limitation, how does that sound?
Me: 🤨
Claude Code: I tried but I'm not allowed to cd to that directory for security reasons.
Claude Code: Hmm...
Claude Code: I can use xargs to work around this limitation, how does that sound?
Me: 🤨
Are there any technical words or terminology for the difference between "a dependency" and "a dependency that's part of your public API"? Or do any dependency management systems treat these differently?
Because I think there's a huge functional difference between those two things.
Because I think there's a huge functional difference between those two things.
June 5, 2025 at 5:10 PM
Are there any technical words or terminology for the difference between "a dependency" and "a dependency that's part of your public API"? Or do any dependency management systems treat these differently?
Because I think there's a huge functional difference between those two things.
Because I think there's a huge functional difference between those two things.
New columnar formats blog where I explain repetition and definition levels in maybe the most confusing way possible (blog.lancedb.com/columnar-fil...)
Also, I implemented them upside down 🤦
Also, I implemented them upside down 🤦
Columnar File Readers in Depth: Repetition & Definition Levels
Repetition and definition levels are a method of converting structural arrays into a set of buffers. The approach was made popular in Parquet and is one of the key ways Parquet, ORC, and Arrow differ....
blog.lancedb.com
June 2, 2025 at 6:46 PM
New columnar formats blog where I explain repetition and definition levels in maybe the most confusing way possible (blog.lancedb.com/columnar-fil...)
Also, I implemented them upside down 🤦
Also, I implemented them upside down 🤦
The most realistic part of Ready Player One is when IOI refuses to let Parzival work from home when they make their offer for him to consult. Despite the entire job being in the oasis.
May 26, 2025 at 4:32 AM
The most realistic part of Ready Player One is when IOI refuses to let Parzival work from home when they make their offer for him to consult. Despite the entire job being in the oasis.
