ssn2000

Its so cool! Im wondering if this is actually better than using techniques in mechanistic interpretability like linear probing with activation steering or activation patching to improve the performance.

Total run cost is $1.2M until now, what resources are they using to train their model? Wish they shared more details on that and what the MFU metrics are.