GPU vs TPU, and good software managing large clusters of them across all sorts of failure.
the funny bit from the above article is the incident when someone forgot about a training job at google, and month later had the model fully trained without an alert of any kind. "outrageously good infra"
https://www.yitay.net/blog/training-great-llms-entirely-from...
GPU vs TPU, and good software managing large clusters of them across all sorts of failure.
the funny bit from the above article is the incident when someone forgot about a training job at google, and month later had the model fully trained without an alert of any kind. "outrageously good infra"