mdsumner
@mdsumner.bsky.social
300 followers 380 following 530 posts
maxing the pixels to glow meshes, R, Python, GDAL, virtual Zarr, coordinate systems, southern ocean research
Posts Media Videos Starter Packs
Reposted by mdsumner
zacklabe.com
In contrast to the other side of the #Arctic, sea ice along the Atlantic front is a record low for this time of year. This relates to the recent record warmth and southerly winds/waves pushing the marginal ice zone closer to the North Pole.

Data from @nsidc.bsky.social (nsidc.org/data/seaice_...).
Graphic showing a line graph time series of daily Arctic sea ice extent in the Atlantic sector of the Arctic (Greenland + Barents + Kara Seas) for each year from 1979 to 2025. This year is shown in red. Purple to white color shading is used for each other year's line. There is a long-term decreasing trend for every day of the year.
Reposted by mdsumner
afredston.bsky.social
niche #rstats tweet: does anyone have a nice snippet of code they would share to download, import, and reshape GLORYS12 ocean data?
mdsumner.bsky.social
Tell us more, looks like opendap and netcdf, is there Zarr?

I've been (Zarr) virtualizing the Bluelink reanalysis from NCI thredds, and have a good handle on the options available to R 🙏
Reposted by mdsumner
kylewalker.bsky.social
On the blog: how to map hundreds of thousands of US Census blocks performantly with R and PMTiles.

This is the first in a three-post series on production mapping workflows for big spatial datasets with PMTiles.

Read the post: walker-data.com/post...
Reposted by mdsumner
mackaszechno.bsky.social
In honor of spooky month, share a 4 word horror story that only someone in your profession would understand

"We've rounded the coordinates"
Reposted by mdsumner
3blue1brown.com
Ever since I made a video about Fourier Transforms, one of the most requested topics on the channel has been its close cousin, the Laplace Transform.

I've been having a lot of fun animating a mini-series about this topic, and the main part is now out.

youtu.be/j0wJBEZdwLs
But what is a Laplace Transform?
YouTube video by 3Blue1Brown
youtu.be
Reposted by mdsumner
mikemahoney218.com
my partner finished his journey of visiting every blue bike station today (please clap)

unfortunately the app does NOT throw confetti when you hit 100%
A screenshot of the blue bikes app with 100% of stations visited
mdsumner.bsky.social
GDAL

You get the full API (vector, raster, mdim, osr, directly all the VSI facilities, ...) and call points for the cli lib, not just a watered down wrapper based on an outdated interpretation
Reposted by mdsumner
paleolimbot.bsky.social
This in turn was due to @yutannihilation-en.bsky.social's fantastic work on Savvy (lightweight R bindings for Rust)!
yutannihilation-en.bsky.social
Thanks to @paleolimbot.bsky.social's great work, the sedonadb R package is now also available to Windows via R-universe 🎉 I'm excited particularly because I believe sedonadb is the easiest tool for R users to access GeoParquet data. #rstats

apache.r-universe.dev/sedonadb
sedonadb: Bindings for Apache SedonaDB
apache.r-universe.dev
Reposted by mdsumner
yutannihilation-en.bsky.social
Thanks to @paleolimbot.bsky.social's great work, the sedonadb R package is now also available to Windows via R-universe 🎉 I'm excited particularly because I believe sedonadb is the easiest tool for R users to access GeoParquet data. #rstats

apache.r-universe.dev/sedonadb
sedonadb: Bindings for Apache SedonaDB
apache.r-universe.dev
mdsumner.bsky.social
HIGHLY recommended, I have some reading to do
Yellow book cover of On Alexis Wright, title is interleaved text as WRITERS GEORDIE WILLIAMSON ON ALEXIS WRIGHT WRITERS. Includes little inset circular pics of Geordie Williamson and Alexis Wright
Reposted by mdsumner
mdsumner.bsky.social
how's your day going

#rstats

paste0(sample(c(letters, LETTERS, 0:9,
strsplit("!@#$%^&*()_+=-~", "")[[1]]), 56),
collapse = "")
Reposted by mdsumner
mikemahoney218.com
my recent focus has been on a lot of point time series data -- I _think_ someone told me zarr handles this use case well enough, but honestly I've only seriously played with parquet so far
Reposted by mdsumner
lvulis.bsky.social
Oh huh. Like trajectories? I'd keep in mind what you're doing up/down as well and how that fits into the workflow. So what packages are loading the parquets etc because that ends up being a huge part of the decisionmaking for me
mdsumner.bsky.social
Oh great point ! is actually exactly where tabular spatial breaks down, trajectory data is an excellent crossover that doesn't quite fit either world, it's mostly tabular but ordered within groups (and is it points, segments, line strings or all the above, yes).

Argo long sits in this uneasy place
mdsumner.bsky.social
This is not parquet, but pretty essential to see where xarray and Zarr are going (raster yes, but not just raster)

youtu.be/I-NHCuLhRjY?...
mdsumner.bsky.social
It won't be long until Icechunk is used to virtualize Parquet stores, referencing compressed chunks by byte ranges, and stored in their mysterious Rust Zarrtabase
mdsumner.bsky.social
Neither is quite the superset, but both can use the other and this is not a hierarchy rather a thicket of experiments
mdsumner.bsky.social
Also, you already know will the difference imo because it's manifest in R, arrays (missing "labels) vs data frames

But, it's those crossover areas where they bleed into one another (nested parquet like json arrays, compressed partition chunks like in arrow) is where the blur is.
mdsumner.bsky.social
And this one:

earthmover.io/blog/what-is...

(The Earthmover stance that "labelled arrays are inefficient as fully materialized tables" is a bit disingenuous ... obvious actually, I wonder if they will review better now xarray has coordinate indexes)
Fundamentals: What Is Zarr? A Cloud-Native Format for Tensor Data - Earthmover
What Zarr is, and how it enables fast, scalable access to multidimensional array data in the cloud.
earthmover.io