Lightnews — Scholar-powered news

Xenova

@xenova.bsky.social

1.9K followers 92 following 32 posts

Bringing the power of machine learning to the web. Currently working on Transformers.js (@huggingface 🤗)

Posts Media Videos Starter Packs

Pinned

Xenova @xenova.bsky.social · Nov 14

Introducing Transformers.js v3: State-of-the-art Machine Learning for the Web. New features include:
⚡ WebGPU support (up to 100x faster than WASM)
🔢 New quantization formats
🏛 121 supported architectures in total
🤖 Over 1200 pre-converted models

Get started with `npm i @huggingface/transformers`

4 7 33

Xenova @xenova.bsky.social · Aug 6

As always, the demo is open source (which you can find under the "Files" tab), so I'm excited to see how the community builds upon this! 🚀

🔗 Link to demo: huggingface.co/spaces/Liqui...

LFM2 WebGPU – In-browser tool calling - a Hugging Face Space by LiquidAI

In-browser tool calling, powered by Transformers.js

huggingface.co

1 4

Xenova @xenova.bsky.social · Aug 6

The next generation of AI-powered websites is going to be WILD! 🤯

In-browser tool calling & MCP is finally here, allowing LLMs to interact with websites programmatically.

To show what's possible, I built a demo using Liquid AI's new LFM2 model, powered by 🤗 Transformers.js.

2 2 7

Xenova @xenova.bsky.social · Jul 24

That's right, we're running Mistral's new Voxtral-Mini-3B model 100% locally in-browser on WebGPU, powered by Transformers.js and ONNX Runtime Web! 🔥

Try it out yourself! 👇
huggingface.co/spaces/webml...

Voxtral WebGPU - a Hugging Face Space by webml-community

State-of-the-art audio transcription in your browser

huggingface.co

Xenova @xenova.bsky.social · Jul 24

Introducing Voxtral WebGPU: State-of-the-art audio transcription directly in your browser! 🤯

🗣️ Transcribe videos, meeting notes, songs and more
🔐 Runs on-device, meaning no data is sent to a server
🌎 Multilingual (8 languages)
🤗 Completely free (forever) & open source

1 3 4

Xenova @xenova.bsky.social · Jul 22

Model: huggingface.co/lazy-guy12/c...

Online demo: lazy-guy.github.io/chess-llama/

lazy-guy12/chess-llama · Hugging Face

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

huggingface.co

Xenova @xenova.bsky.social · Jul 22

A community member trained a tiny Llama model (23M parameters) on 3 million high-quality @lichess.org games, then deployed it to run entirely in-browser with 🤗 Transformers.js! Super cool! 🔥

It has an estimated ELO of ~1400... can you beat it? 👀
(runs on both mobile and desktop)

1 3 11

Xenova @xenova.bsky.social · Feb 7

The most difficult part was getting the model running in the first place, but the next steps are simple:
✂️ Implement sentence splitting, enabling streamed responses
🌍 Multilingual support (only phonemization left)

Who wants to help? 🤗
huggingface.co/spaces/webml...

Kokoro Text-to-Speech (WebGPU) - a Hugging Face Space by webml-community

High-quality speech synthesis powered by Kokoro TTS

huggingface.co

Xenova @xenova.bsky.social · Feb 7

We did it! Kokoro TTS (v1.0) can now run 100% locally in your browser w/ WebGPU acceleration. Real-time text-to-speech without a server. ⚡️

Generate 10 seconds of speech in ~1 second for $0.

What will you build? 🔥

1 6 22

Xenova @xenova.bsky.social · Jan 16

The model is also extremely resilient to quantization. The smallest variant is only 86 MB in size (down from the original 326 MB), with no noticeable difference in audio quality! 🤯

Link to models/samples: huggingface.co/onnx-communi...

Xenova @xenova.bsky.social · Jan 16

You can get started in just a few lines of code! 🧑‍💻

Huge kudos to the Kokoro TTS community, especially taylorchu for the ONNX exports and Hexgrad for the amazing project! None of this would be possible without you all! 🤗

Try it out yourself: huggingface.co/spaces/webml...

$import { KokoroTTS } from "kokoro-js"; const tts = await KokoroTTS.from_pretrained( "onnx-community/Kokoro-82M-ONNX", { dtype: "q8" }, // fp32, fp16, q8, q4, q4f16 ); const text = "Life is like a box of chocolates. You never know what you're gonna get."; const audio = await tts.generate(text, { voice: "af_sky" }, // See `tts.list_voices()` ); audio.save("audio.wav");$

1 2

Xenova @xenova.bsky.social · Jan 16

Introducing Kokoro.js, a new JavaScript library for running Kokoro TTS, an 82 million parameter text-to-speech model, 100% locally in the browser w/ WASM. Powered by 🤗 Transformers.js. WebGPU support coming soon!

👉 npm i kokoro-js 👈

Link to demo (+ sample code) in 🧵

1 3 19

Xenova @xenova.bsky.social · Jan 10

For the AI builders out there: imagine what could be achieved with a browser extension that (1) uses a powerful reasoning LLM, (2) runs 100% locally & privately, and (3) can directly access/manipulate the DOM! 👀

💻 Source code: github.com/huggingface/...
🔗 Online demo: huggingface.co/spaces/webml...

Llama 3.2 Reasoning WebGPU - a Hugging Face Space by webml-community

Small and powerful reasoning LLM that runs in your browser

huggingface.co

1 1 3

Xenova @xenova.bsky.social · Jan 10

Is this the future of AI browser agents? 👀 WebGPU-accelerated reasoning LLMs are now supported in Transformers.js! 🤯

Here's MiniThinky-v2 (1B) running 100% locally in the browser at ~60 tps (no API calls)! I can't wait to see what you build with it!

Demo + source code in 🧵👇

1 7 32

Xenova @xenova.bsky.social · Jan 1

This project was greatly inspired by Brendan Bycroft's amazing LLM Visualization tool – check it out if you haven't already! Also, thanks to Niels Rogge for adding DINOv2 w/ Registers to transformers! 🤗

Source code: github.com/huggingface/...

Online demo: huggingface.co/spaces/webml...

Attention Visualization - a Hugging Face Space by webml-community

Vision Transformer Attention Visualization

huggingface.co

2 4

Xenova @xenova.bsky.social · Jan 1

Another interesting thing to see is how the attention maps become far more refined in later layers of the transformer. For example,

First layer (1) – noisy and diffuse, capturing broad general patterns.
Last layer (12) – focused and precise, highlighting specific features.

1 3

Xenova @xenova.bsky.social · Jan 1

Vision Transformers work by dividing images into fixed-size patches (e.g., 14 × 14), flattening each patch into a vector and treating each as a token.

It's fascinating to see what each attention head learns to "focus on". For example, layer 11, head 1 seems to identify eyes. Spooky! 👀

1 2

Xenova @xenova.bsky.social · Jan 1

The app loads a small DINOv2 model into the user's browser and runs it locally using Transformers.js! 🤗

This means you can analyze your own images for free: simply click the image to open the file dialog.

E.g., the model recognizes that long necks and fluffy ears are defining features of llamas! 🦙

1 1 2

Xenova @xenova.bsky.social · Jan 1

First project of 2025: Vision Transformer Explorer

I built a web app to interactively explore the self-attention maps produced by ViTs. This explains what the model is focusing on when making predictions, and provides insights into its inner workings! 🤯

Try it out yourself! 👇

1 3 9

Xenova @xenova.bsky.social · Dec 18

Yeah, I ran into this during development, and is unfortunately a bug in Firefox:
- bugzilla.mozilla.org/show_bug.cgi...
- bugzilla.mozilla.org/show_bug.cgi...

1725336 - Error in AudioContext.createMediaStreamSource when AudioContext is constructed with a custom sample rate

UNCONFIRMED (nobody) in Core - Web Audio. Last updated 2023-12-29.

bugzilla.mozilla.org

Xenova @xenova.bsky.social · Dec 18

Huge shout-out to the Useful Sensors team for such an amazing model and to Wael Yasmina for his 3D audio visualizer tutorial! 🤗

‍💻 Source code: github.com/huggingface/...
🔗 Online demo: huggingface.co/spaces/webml...

Moonshine Web - a Hugging Face Space by webml-community

Real-time in-browser speech recognition

huggingface.co

1 4

Xenova @xenova.bsky.social · Dec 18

Introducing Moonshine Web: real-time speech recognition running 100% locally in your browser!
🚀 Faster and more accurate than Whisper
🔒 Privacy-focused (no data leaves your device)
⚡️ WebGPU accelerated (w/ WASM fallback)
🔥 Powered by ONNX Runtime Web and Transformers.js

Demo + source code below! 👇

1 4 28

Reposted by Xenova

Reuters Institute @reutersinstitute.bsky.social · Dec 10

🤗NEW PIECE:

‘Open-source’ is becoming a buzzword for many aspects of modern journalism, including open-source AI. But what is it, and how can journalists benefit from it?

@marinaadami.bsky.social spoke to @fdaudens.bsky.social to find out.

reutersinstitute.politics.ox.ac.uk/news/journal...

This journalist wants you to try open-source AI: “AI is shiny, but value comes from the ideas people have to use it”

Hugging Face’s Florent Daudens on what open-source AI is, how journalists can use it and why he thinks they should.

reutersinstitute.politics.ox.ac.uk

2 5 12

Xenova @xenova.bsky.social · Dec 8

Huge shout-out to OuteAI for their amazing model (OuteTTS-0.2-500M) and for helping us bring it to the web! 🤗 Together, we released the outetts NPM package, which you can install with `npm i outetts`.

💻 Source code: github.com/huggingface/...

🔗 Demo: huggingface.co/spaces/webml...

Text-to-Speech WebGPU - a Hugging Face Space by webml-community

WebGPU text-to-Speech powered by OuteTTS and Transformers.js

huggingface.co

Xenova @xenova.bsky.social · Dec 8

The model is multilingual (English, Chinese, Korean & Japanese) and even supports zero-shot voice cloning! 🤯 Stay tuned for an update that will add these features to the UI!

More samples:
bsky.app/profile/reac...

vb @reach-vb.hf.co · Nov 25

Smol TTS keeps getting better! Introducing OuteTTS v0.2 - 500M parameters, multilingual with voice cloning! 🔥

> Multilingual - English, Chinese, Korean & Japanese
> Cross platform inference w/ llama.cpp
> Trained on 5 Billion audio tokens
> Qwen 2.5 0.5B LLM backbone
> Trained via HF GPU grants

2 6

Xenova @xenova.bsky.social · Dec 8

Introducing TTS WebGPU: The first ever text-to-speech web app built with WebGPU acceleration! 🔥

High-quality and natural speech generation that runs 100% locally in your browser, powered by OuteTTS and Transformers.js. 🤗 Try it out yourself!

Demo + source code below 👇

2 12 45