@halvarflake.bsky.social
Ok, convergence with SGD was too painfully slow, so I did a few thousand iterations with Adam, then switched to SGD. The result looks like this:
October 5, 2025 at 11:37 AM
Ok, convergence with SGD was too painfully slow, so I did a few thousand iterations with Adam, then switched to SGD. The result looks like this:
Current state is this. It's early in the training process, though - I chose SGD instead of ADAM (which becomes unstable when trained to very low loss), and the weight initialization which led to many creases in the upper left corner is still visible.
October 5, 2025 at 7:51 AM
Current state is this. It's early in the training process, though - I chose SGD instead of ADAM (which becomes unstable when trained to very low loss), and the weight initialization which led to many creases in the upper left corner is still visible.
Today's experiment: What shape will a deep neural network learn if trained on the sparse set of training points on the left? Will it learn a circle?
October 5, 2025 at 7:20 AM
Today's experiment: What shape will a deep neural network learn if trained on the sparse set of training points on the left? Will it learn a circle?
This is kinda wild, but kinda good?
September 23, 2025 at 7:58 PM
This is kinda wild, but kinda good?
Life's good. Got a surf in at 7 am, harvested two hands full of wild blackberries on the walk back, daughter taking swimming lesson by the beach now & I am watching the crowded peak while drinking a decaf (giving up caffeine has been great for me, at least for now).
August 17, 2025 at 8:15 AM
Life's good. Got a surf in at 7 am, harvested two hands full of wild blackberries on the walk back, daughter taking swimming lesson by the beach now & I am watching the crowded peak while drinking a decaf (giving up caffeine has been great for me, at least for now).
Incompetent even at incompetence?
April 5, 2025 at 3:10 AM
Incompetent even at incompetence?
Going to sleep while my computer wastes cycles on PyTorch. But I'm visualizing the polytope boundaries nicely after each training step. Right hand side is the target picture, red pixels are training set, blue pixels validation set (to be used later).
March 1, 2025 at 11:02 PM
Going to sleep while my computer wastes cycles on PyTorch. But I'm visualizing the polytope boundaries nicely after each training step. Right hand side is the target picture, red pixels are training set, blue pixels validation set (to be used later).
Check the date and price.
January 29, 2025 at 2:38 PM
Check the date and price.
Hilarious. German Satire magazine titled "controversial gesture: Is Hitler doing a Musk salute?"
January 21, 2025 at 4:25 PM
Hilarious. German Satire magazine titled "controversial gesture: Is Hitler doing a Musk salute?"
Just like it's creator, ChatGPT has a rather funny ideas about what is a copyright violation and what is not.
January 17, 2025 at 4:23 PM
Just like it's creator, ChatGPT has a rather funny ideas about what is a copyright violation and what is not.
I think my math hunch from early 2023 will age pretty well.
December 21, 2024 at 3:00 PM
I think my math hunch from early 2023 will age pretty well.
I frequently wonder why I insisted on Google paying me in cash instead of stock in 2011, and then I remembered: The total real return of the SP500 between March 2000 (when I turned 18) and February 2011 (when we sold zynamics) was negative, and it would take until June 2013 before it was positive.
December 13, 2024 at 8:25 PM
I frequently wonder why I insisted on Google paying me in cash instead of stock in 2011, and then I remembered: The total real return of the SP500 between March 2000 (when I turned 18) and February 2011 (when we sold zynamics) was negative, and it would take until June 2013 before it was positive.
Sketching while the kids draw.
December 7, 2024 at 3:50 PM
Sketching while the kids draw.
This table is wild. 200 32 bit multiplications for the price of a 32 bit DRAM read? And this is 45nm, what about modern process nodes?
December 4, 2024 at 4:36 PM
This table is wild. 200 32 bit multiplications for the price of a 32 bit DRAM read? And this is 45nm, what about modern process nodes?
Post a pic YOU took (no description) to bring some Zen to the timeline.
November 16, 2024 at 8:18 PM
Post a pic YOU took (no description) to bring some Zen to the timeline.