Keith Duke
@kmduke.bsky.social
keithduke.net
Full stack engineer, experience at scale (decade at CBSInteractive), web focused, all delivery methodologies. Currently loving Vite and Web Components.
Agent design, loving Qwen 3 4b, FastAPI, MLX
Seeking opportunities in related fields.
Full stack engineer, experience at scale (decade at CBSInteractive), web focused, all delivery methodologies. Currently loving Vite and Web Components.
Agent design, loving Qwen 3 4b, FastAPI, MLX
Seeking opportunities in related fields.
Won't convert to MLX format, it's also MoE and MLX doesn't support those layers yet.
There is a tiny-preview that's older on HF that has been converted. Pulled it and kicking the tires a little. Prompt given was "Why is the sky blue"
There is a tiny-preview that's older on HF that has been converted. Pulled it and kicking the tires a little. Prompt given was "Why is the sky blue"
![A screenshot showing the following text.
% python g4test.py 7:20:27
Fetching 12 files: 100%|██████████████████████████████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 133505.70it/s]
==========
The sky appears blue because of a process called Rayleigh scattering. As sunlight reaches Earth's atmosphere, it is scattered, or redirected, in all directions by the gases and particles in the air. Blue light is scattered more than other colors because it travels in shorter, smaller waves. This scattered blue light is what we see when we look up at the sky.
==========
Prompt: 65 tokens, 64.747 tokens-per-sec
Generation: 82 tokens, 79.240 tokens-per-sec
Peak memory: 7.248 GB](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:tlmx4akhvj2hw6snht4nqedb/bafkreiad57d5s23pgwj3t7huf4zbwj5hz326ulea2xf3xhfrkcywxk73dq@jpeg)
October 2, 2025 at 2:28 PM
Won't convert to MLX format, it's also MoE and MLX doesn't support those layers yet.
There is a tiny-preview that's older on HF that has been converted. Pulled it and kicking the tires a little. Prompt given was "Why is the sky blue"
There is a tiny-preview that's older on HF that has been converted. Pulled it and kicking the tires a little. Prompt given was "Why is the sky blue"
But changing it to just "read" and it has no problem
![A screenshot of the CLI app showing the LLM executing the tool "read"
You: use tool read on ./main.py
⠙ 🤔 Thinking...[10:26:25] INFO 🔧 Found 1 tool call(s) main.py:379
INFO 🔧 Executed read: {'content': '#!/usr/bin/env python3\n"""\nGranite 3.3 2B CLI with MLX - main.py:265
Interactive AI Assistant\n"""\n\nimport json\nimport logging\nimport time\nfrom datetime import...](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:tlmx4akhvj2hw6snht4nqedb/bafkreigoj6ovccf2ce6ndpf5krjlasuadcem2dw6mplh3hrp6ajxy4lxqy@jpeg)
September 19, 2025 at 5:37 PM
But changing it to just "read" and it has no problem
Finally getting it to somewhat follow tools. Interesting when I gave it a read_file tool it balked at using it
September 19, 2025 at 5:35 PM
Finally getting it to somewhat follow tools. Interesting when I gave it a read_file tool it balked at using it
Got session memory going, this is it running on my iPad.
Checking out tool use next. What's really cool is you get all the OS things like text to speech for free, on top of the free LLM. This is really cool so far.
Checking out tool use next. What's really cool is you get all the OS things like text to speech for free, on top of the free LLM. This is really cool so far.
September 16, 2025 at 7:19 PM
Got session memory going, this is it running on my iPad.
Checking out tool use next. What's really cool is you get all the OS things like text to speech for free, on top of the free LLM. This is really cool so far.
Checking out tool use next. What's really cool is you get all the OS things like text to speech for free, on top of the free LLM. This is really cool so far.
Tweaked UI a bit and some RAM usage if curious
September 16, 2025 at 5:05 PM
Tweaked UI a bit and some RAM usage if curious
Logic questions tripped it up, trick things like
September 16, 2025 at 2:21 PM
Logic questions tripped it up, trick things like
Apple's Foundation Model is available, I put together a quick demo to try it out. It can code.
September 16, 2025 at 2:19 PM
Apple's Foundation Model is available, I put together a quick demo to try it out. It can code.
That made my morning, great job, loved it!
November 28, 2024 at 1:03 PM
That made my morning, great job, loved it!