Lightnews — Scholar-powered news

Reposted by Mimoune Djouallah

David Neuzerling @md.neuzerl.ing · 1d

Any system that allows exchanging real money for stuff with an element of chance is morally equivalent to a casino.

Corollary: Pokémon cards, Roblox, Labubus, and even claw machines should all be 18+

Zach Weinersmith @zachweinersmith.bsky.social · 5d

Do you have any extremely niche, but serious, ethical stances?

2 3 20

Mimoune Djouallah @mimdj.bsky.social · 2d

running #duckdb at 10 TB scale using #tpch like benchmark

#delta #onelake #singlenode #MicrosoftFabric
datamonkeysite.com/2025/10/19/r...

1 5

Mimoune Djouallah @mimdj.bsky.social · 11d

you are looking at #duckdb running tpch 1 TB with only 16 cores
it used to crash even with 64

pip install duckdb --upgrade is an act of faith basically

13

Mimoune Djouallah @mimdj.bsky.social · 18d

2

Mimoune Djouallah @mimdj.bsky.social · 18d

Put together a small python package duckrun :) point it at a folder of SQL/Python files, define a pipeline, and it will create Delta tables in #OneLake with #DuckDB and #delta_rs

github.com/djouallah/du...

2 1 9

Mimoune Djouallah @mimdj.bsky.social · 26d

actually #Microsoftfabric Datawarehouse automatically expose an Iceberg rest Catalog
thanks to #duckdb UI extension, you can see proper catalog

2

Mimoune Djouallah @mimdj.bsky.social · Sep 20

First Look at #onelake #apacheiceberg REST Catalog, please notice it is coming soon and not in production yet #MicrosoftFabric
www.youtube.com/watch?v=_QRE...

First Look at Onelake Iceberg REST Catalog

YouTube video by DataMonkey

www.youtube.com

1 2

Reposted by Mimoune Djouallah

nextdayvideo.bsky.social @nextdayvideo.bsky.social · Sep 19

#pyconau @mimdj.bsky.social Life Beyond Pandas: Workflows with DuckDB, Daft, Polars, and Datafusion http://youtu.be/SnogunyMnE8

1 1

Mimoune Djouallah @mimdj.bsky.social · Sep 16

2 months ago, I got access to a beta release of #onelake #Apacheiceberg REST Catalog, first thing I run it with #duckdb 😀

6

Mimoune Djouallah @mimdj.bsky.social · Sep 15

storage format should not be tied to #SQL logic, #duckdb got it so right !!! but a bit sad that #deltalake is left behind :(

7

Mimoune Djouallah @mimdj.bsky.social · Sep 9

you know me too well :)

Mimoune Djouallah @mimdj.bsky.social · Sep 9

First look at incremental framing in #PowerBI directlake mode

#Parquet #OLAP #optimization #Performance #onelake
datamonkeysite.com/2025/09/09/f...

First Look at Incremental Framing in Power BI

TL;DR: Incremental framing is like CDC to RAM :) It significantly improves cold-run performance of Direct Lake mode in some scenarios, there is an excellent documentation that explain everything in…

datamonkeysite.com

Mimoune Djouallah @mimdj.bsky.social · Sep 7

first look at geometry data type in #parquet
datamonkeysite.com/2025/09/07/f...

First Look at Geometry Types in Parquet

Getting different parties in the software industry to agree on a common standard is rare. Most of the time, a dominant player sets the rules. Occasionally, however, collaboration happens organicall…

datamonkeysite.com

2

Mimoune Djouallah @mimdj.bsky.social · Sep 6

first #apacheiceberg table written by #duckdb

6

Mimoune Djouallah @mimdj.bsky.social · Sep 5

good news #duckdb added support for reading and writing geometry data type

Bad news : other Fabric engines don't support it yet, so it is not very useful for now :(

3

Mimoune Djouallah @mimdj.bsky.social · Sep 1

I nearly get the logic behind delta parquet, but clearly people did not like it 😅

1

Mimoune Djouallah @mimdj.bsky.social · Sep 1

I was listening to a guy from S3 and he describe it beautifully, those table format are a way to manage Parquet !!!

1 2

Mimoune Djouallah @mimdj.bsky.social · Sep 1

#duckdb is the goat

github.com/djouallah/Fa...

1 1 11

Mimoune Djouallah @mimdj.bsky.social · Aug 25

Third time is the charm ✨
With the much needed improvements to the #MicrosoftFabric scheduler, I revisited my review of Fabric F2.
#duckdb #sql
www.youtube.com/watch?v=tchY...

Third Look at Fabric F2

YouTube video by DataMonkey

www.youtube.com

3

Mimoune Djouallah @mimdj.bsky.social · Aug 21

maybe decoupling storage from compute was not a very good idea 😅 I am joking, it was worth it

1

Mimoune Djouallah @mimdj.bsky.social · Aug 20

Btw, because engines are getting so fast the bottleneck become remote storage bandwidth 😁 not compute

1 1

Mimoune Djouallah @mimdj.bsky.social · Aug 15

new world record 😝 using #duckdb and #ducklake
22 cents for the 3 B rows, coffee benchmark :)

not bad at all for a single node :)

www.linkedin.com/posts/mimoun...

☕ Coffee benchmark at 3B scale factor : $0.22 total cost 💰 | Mimoune Djouallah

☕ Coffee benchmark at 3B scale factor : $0.22 total cost 💰 7 Months later, using the latest dev release of #DuckDB with #DuckLake, we cut the cost of running the unfamous :) coffee benchmark to just ...

www.linkedin.com

1 9

Mimoune Djouallah @mimdj.bsky.social · Aug 13

Writing #ApacheIceberg in Azure is not particularly hard, but you do need a catalog (essentially a database). For simple tests, you can use an in-memory DB
#ADLS #opentableformat #PyIceberg.

4

Reposted by Mimoune Djouallah

buckwoodymsft.bsky.social @buckwoodymsft.bsky.social · Aug 12

mssql-python vs pyodbc: Benchmarking SQL Server Performance - devblogs.microsoft.com/python/mssql...

A pretty big rewrite, actually

mssql-python vs pyodbc: Benchmarking SQL Server Performance - Microsoft for Python Developers Blog

Learn how the python driver for SQL Server, mssql-python, outperforms pyodbc in terms of latency and throughput for developers.

devblogs.microsoft.com

2 4 13

Mimoune Djouallah @mimdj.bsky.social · Aug 13

Apologies, I thought you were joking, for any new project, I don't think using pandas is a good idea

1 1