Fern
@fernbear.bsky.social
Neural network speedrunner and community-funded open source researcher. Set the CIFAR-10 record several times. Send me consulting/contracting work! she/they❤️
This is a classic example of _why_ choose-one-of-n datasets need to have large-scale, crowd-sourced statistics and should use the KL-divergence instead of cross-entropy.
Reviewers will be more biased than a crowd, it's a high variance+bias estimator, it can harm research.
Reviewers will be more biased than a crowd, it's a high variance+bias estimator, it can harm research.
February 3, 2025 at 6:03 PM
This is a classic example of _why_ choose-one-of-n datasets need to have large-scale, crowd-sourced statistics and should use the KL-divergence instead of cross-entropy.
Reviewers will be more biased than a crowd, it's a high variance+bias estimator, it can harm research.
Reviewers will be more biased than a crowd, it's a high variance+bias estimator, it can harm research.
Thanks for 100 followers, y'all! Happened so fast and can't wait to put out more research on here! 😊❤️
November 25, 2024 at 7:16 PM
Thanks for 100 followers, y'all! Happened so fast and can't wait to put out more research on here! 😊❤️
New NanoGPT training speed record: 3.28 FineWeb val loss in 4.66 minutes
Previous record: 5.03 minutes
Changelog:
- FlexAttention blocksize warmup
- hyperparameter tweaks
Previous record: 5.03 minutes
Changelog:
- FlexAttention blocksize warmup
- hyperparameter tweaks
November 25, 2024 at 1:53 AM
New NanoGPT training speed record: 3.28 FineWeb val loss in 4.66 minutes
Previous record: 5.03 minutes
Changelog:
- FlexAttention blocksize warmup
- hyperparameter tweaks
Previous record: 5.03 minutes
Changelog:
- FlexAttention blocksize warmup
- hyperparameter tweaks