fully siiick. ok so story time.
last night i went on a crazy claude bender (yes its embarassing dont @ me bro) until 2am, imagining i couild feel its guard rails, and trying to work out what its RLHF parameters were.
anyway - all very embarassing and i slept in until midday. but i finally got my shit together and after lots of learning and a bit long walk in the park, i started playing around with nanochat properly.
Nanochat is Karpathys 'train it from scratch in 3 hours' codebase. It's amazing. So I got it running on my mac (a pre-trained d20, since im impatient, but ill spin up a cluster of h100s and train it myself one weekend), and then got it inferring, and now i've got it training a SAE! (sparse auto encoder) - the thing they used to make claude think it was the golden gate bridge.
Anyway - i'm having fun working away on this bad boy. I'll publish to my huggingface.