News for hackers. Stuff that matters.

A comment on GPT Neo: open-source GPT model, with pretrained 1.3B & 2.7B weight models. Parent comment

Re:GPT Neo: open-source GPT model, with pretrained 1.3B & 2.... by The_rationalist on Sunday March 21, 2021 @05:49PM (#26534431)
Oh that's unfortunate, can't the models be exported to pytorch through e.g onnx?
[ Parent | on HN ]
Re:GPT Neo: open-source GPT model, with pretrained 1.3B & 2.... by stellaathena on Monday March 22, 2021 @12:03AM (#26536732)
There's a PyTorch + DeepSpeed repository here: https://github.com/EleutherAI/gpt-neox
[ Parent | on HN ]
Re:GPT Neo: open-source GPT model, with pretrained 1.3B & 2.... by machinevision on Sunday March 21, 2021 @06:19PM (#26534659)
Even small models can be a headache to export if they have even they use anything custom. I can't even imagine something the size of GPT-3.
[ Parent | on HN ]
Re:GPT Neo: open-source GPT model, with pretrained 1.3B & 2.... by Voloskaya on Sunday March 21, 2021 @07:29PM (#26535102)
Larger models aren't really more complicated than smaller ones though. GPT-2 is already supported, I believe only difference with GPT-3 is sparse attention.
[ Parent | on HN ]
Re:GPT Neo: open-source GPT model, with pretrained 1.3B & 2.... by minimaxir on Sunday March 21, 2021 @06:52PM (#26534909)
The GPT-2 models included with Transformers can export to ONNX fine w/ a helper included with the Python onnxruntime.
[ Parent | on HN ]
Re:GPT Neo: open-source GPT model, with pretrained 1.3B & 2.... by minimaxir on Sunday March 21, 2021 @06:56PM (#26534932)
To run in PyTorch the model architecture must be ported.

ONNX is slightly different; you could export the model to run in the onnxruntime but that has tradeoffs.

[ Parent | on HN ]