GPT-J-6B: 6B JAX-Based Transformer

Re:GPT-J-6B: 6B JAX-Based Transformer by londons_explore on Wednesday June 09, 2021 @05:12AM (#27445858)

It's design is pretty specific to Google's TPU.

If you ran it elsewhere, you'd likely need to tweak the design or suffer quite a big performance penalty.

[ Parent | on HN ]

Re:GPT-J-6B: 6B JAX-Based Transformer by devetec on Wednesday June 09, 2021 @12:13PM (#27449612)

Nope, probably not. It's using JAX, which works on both GPUs and TPUs.

[ Parent | on HN ]

Re:GPT-J-6B: 6B JAX-Based Transformer by londons_explore on Wednesday June 09, 2021 @04:10PM (#27452508)

All the constants in the design will be tuned to fit perfectly in TPU hardware dimensions.

[ Parent | on HN ]