_ntka

Radiation would perhaps be better pictured with a light blue glow (Cherenkov radiation).

The green glow used in pop culture has its origin in the widespread use of radium paint to achieve a glow-in-the-dark effect (e.g. on watch faces) in the early 20th century. I still own a radium watch. The paint was always fluorescent green. And it did glow.

Isn't JAX the most widely used framework in the GenAI space? Most companies there use it -- Cohere, Anthropic, CharacterAI, xAI, Midjourney etc.

No, I am saying, with JAX you train on G.P.U., with a G, and your training runs are >2x faster, so your training costs are 2x lower, which matters whether your training spend is $1k or $100M. You're not interested in that? That's ok, but most people are.

Or alternatively, do you want faster training runs (and thus lower training costs)? Then JAX is a good choice for you.

It's all about cost and performance. If you can train a foundation model 2x faster with JAX on the same hardware, you are effectively slashing your training costs by 2x, which is significant for a multi-million dollar training run.

JAX is used by almost every large genAI player (Anthropic, Cohere, DeepMind, Midjourney, Character.ai, XAi, Apple, etc.). Its actual market share in foundation models development is something like 80%.

Also JAX is not just for TPU. It's mainly for GPU. It's usually 2-3x faster than torch on GPU: https://keras.io/getting_started/benchmarks/

Far more industry users of JAX use it on GPU compared to TPU.

This is not true, the initial Keras port of the model was done by Divam Gupta who is not affiliated with Keras or Google. He works at Meta.

The benchmark in the article uses mixed precision (and equivalent generation settings) for both implementations, it's a fair benchmark.

In the latest StackOverflow global developer survey, TensorFlow had 50% more users than PyTorch.

7 cubic km of olivine is a staggering amount. Makes you wonder whether the effect of deploying the olivine would even offset the energy cost and CO2 emissions of extracting and moving around all that rock.

Agreed, isolating the contribution of the "model" itself can be borderline impossible, and the details of model engineering have a considerable impact on performance.

That's why I like Kaggle competitions: most approaches tried on a given challenge will likely be optimized to their very limits, so that it's the nature of the different approaches that ends up making the difference.