Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on We fine-tuned Llama 405B on AMD GPUs. Parent comment
Were you using activation checkpointing? https://jax.readthedocs.io/en/latest/_autosummary/jax.checkp... is very important for keeping memory usage reasonable when training large models.
There are no comments on this comment yet.
Were you using activation checkpointing? https://jax.readthedocs.io/en/latest/_autosummary/jax.checkp... is very important for keeping memory usage reasonable when training large models.