Lightnews — Scholar-powered news

Joe Barrow @jbarrow.bsky.social · 21d

Now, some acknowledgments: this work was made possible thanks to a generous compute grant from Lambda!

And I've got a hosted version of the model that I'll be sharing in a couple days hosted on @modal-labs.bsky.social, which makes it basically free for me to host and scale

Joe Barrow @jbarrow.bsky.social · 21d

As part of the paper, I'm working on releasing the dataset and FFDNet models on HuggingFace.

Those will be out in the coming days, you can follow along here: github.com/jbarrow/comm...

🤗Paper: huggingface.co/papers/2509....
arXiv: arxiv.org/abs/2509.16506

GitHub - jbarrow/commonforms: CommonForms dataset and models

CommonForms dataset and models. Contribute to jbarrow/commonforms development by creating an account on GitHub.

github.com

1 1

Joe Barrow @jbarrow.bsky.social · 21d

Now, just because we filtered for the cleanest forms doesn't mean we got _perfect_ forms. There are still a lot of inconsistencies in how people prepare forms! In future work I'll be looking at mitigating data quality issues like these.

1

Joe Barrow @jbarrow.bsky.social · 21d

(Note, this doesn't _just_ apply to Acrobat, it's also better than Apple Preview -- neither Acrobat nor Preview even make an attempt at checkboxes, and they're often fooled by any straight, horizontal line. Left: Acrobat, Right: FFDNet)

1

Joe Barrow @jbarrow.bsky.social · 21d

If we train object detectors to find the form fields on these pages, we get a much cleaner set of forms than if you used Acrobat to automatically prepare your form. (Left: Acrobat, Right: FFDNet).

1

Joe Barrow @jbarrow.bsky.social · 21d

Step 1 is to filter out for the cleanest forms possible. We start with 8MM PDFs from Common Crawl, and work our way down to ~60k of the cleanest forms we can find. The results is a ~500k page dataset, called CommonForms.

1

Joe Barrow @jbarrow.bsky.social · 21d

Paper thread of some work I’m *incredibly* proud of, my first single author paper!

Converting a PDF to a fillable form is a hard problem, and a lot of solutions don’t work very well! In CommonForms, I show that you can train models that outperform Adobe Acrobat for <$500! 🧵

1 1 2

Joe Barrow @jbarrow.bsky.social · May 31

Yeah I wonder if that statistic is flipped between the cities (though operated by the same provider — Lyft — I assume?)

No way that 99 out of every 100 riders in Boston have visited more than 27 stations?

3

Joe Barrow @jbarrow.bsky.social · May 31

Pretty sure you want that number to be lower. :p (my stats for DC ridership)

1 2

Joe Barrow @jbarrow.bsky.social · Apr 23

In which I argue that LLM-generated bounding boxes are impressive, but not that useful (yet): notes.penpusher.app/Misc/Horsesh...

Horseshoes (and Hand Grenades) - LLM Localization is not Close, but not Close Enough - Joe Barrow

TL;DRLarge Multimodal Models (LMMs) can now output bounding boxes when given images as inputs. The results are impressive, but for documents they aren't good enough for real world use, yet. The Probl…

notes.penpusher.app

1

Joe Barrow @jbarrow.bsky.social · Mar 11

Would absolutely love that!

2

Joe Barrow @jbarrow.bsky.social · Jan 7

"AI TOPS our stock price"
- Nvidia, today

1

Joe Barrow @jbarrow.bsky.social · Jan 7

Ah, yes, that ol' familiar unit of measure "AI TOPS"

1 4

Joe Barrow @jbarrow.bsky.social · Dec 22

Agree, my ideal would be if you could type into an old, cheap, refurb kindle personally.

Here’s a video of a person typing into the Palma: www.reddit.com/r/Onyx_Boox/...

My experience (tablet) is that it’s maybe 10s from pickup to writing — wake up (3s), navigate to apps (2s), open app (3-5s)?

Palma as a FreeWrite

www.reddit.com

1

Joe Barrow @jbarrow.bsky.social · Dec 21

I’ve got one of the older, larger eInk tablets and use it for reading books/papers and taking notes. Battery after several years lasts about a week of average use, longer if I keep WiFi off.

1 1

Joe Barrow @jbarrow.bsky.social · Dec 21

Not necessarily hitting the price point but there are eInk mini tablets (e.g. Boox Palma at around $200) that have Android, no sim (so no phone distractions), and long battery life (thanks to the eInk and being generally underpowered). They accept keyboards, too.

1 2

Joe Barrow @jbarrow.bsky.social · Dec 20

I put together a little guide on getting started with Google Gemini -- how to make multimodal calls, get structured outputs, and image bounding boxes to build an object detector.

notes.penpusher.app/Misc/Google+...

Google Gemini 101 - Object Detection with Vision and Structured Outputs - Joe Barrow - Obsidian Publish

This is a missing manual for how to get a simple working prototype up and running with Gemini's vision mode and structured outputs. I'm confident that manual exists elsewhere, but I haven't been able…

publish.obsidian.md

2 3

Reposted by Joe Barrow

Joe Barrow @jbarrow.bsky.social · Dec 18

Drake meme template.
No to: clear, concise prose
Yes to: negative vspace

1 3

Joe Barrow @jbarrow.bsky.social · Dec 18

1 3

Joe Barrow @jbarrow.bsky.social · Dec 18

Holy moly that created an extra half page of space!

1 2

Joe Barrow @jbarrow.bsky.social · Dec 18

Gemini 2.0 Flash is pretty good at localization in images. for an LMM (much better than GPT-4o in my experiments).

A picture of a teapot to the right of a teacup, both in a flat-bottomed basket. The teacup has eta and flowers in it. There are 2 blue bounding boxes on the image, one labeled "Teapot" and one labeled "Teacup" that are over the teapot and teacup respectively.

1

Joe Barrow @jbarrow.bsky.social · Dec 9

ML history question: is there an earlier reference to pixel-only in-context (i.e. no fine-tuning) DocVQA performance than the GPT-4 announcement from OpenAI?

Joe Barrow @jbarrow.bsky.social · Nov 25

Aged white tea, the kind that comes in a compressed disk or ball. My favorite kind of tea, imo they taste naturally quite sweet. Yunnan Sourcing has a bunch!

1 1