News for hackers. Stuff that matters.

A comment on Show HN: I made a GPU VRAM calculator for transformer-based models.

Re:Show HN: I made a GPU VRAM calculator for transformer-bas... by thatguysaguy on Tuesday December 26, 2023 @08:13PM (#38777950)
This only lists first moments, but Adam stores estimates of first and second moments.
[ Parent | on HN ]
Re:Show HN: I made a GPU VRAM calculator for transformer-bas... by furiousteabag on Tuesday December 26, 2023 @08:17PM (#38777978)
By default, SGD w momentum is enabled as optimizer. You may try selecting Adam and it will list second moments as well.
[ Parent | on HN ]