All of this (and more) was possible before LLaMA hit the scene. Not one feature about LLaMA is novel, which is why we were able to build wrappers like llama.cpp so quickly.
Edit: The only optimization that BERT/GPT-Neo needed was Apple to enable Pytorch acceleration. LLaMA is exclusively an inferencing tool that runs on the CPU, it's "optimization" on both Apple Silicon and x86 are hacked wrappers around SIMD instructions.
I'm only pushing back here because I was running GPT-Neo for a Discord bot on cheap-as-dirt ARM hardware a year ago. It's been possible long before LLaMA brought it to the attention of bored entrepreneurs.
> And that, in turn, makes it harder for anybody to use it.
Only one of these models can be downloaded legally. Between the DMCA takedowns and broken IPFS gateways LLaMA faced, I can confidently say BERT, T5 and GPT-Neo Just Work faster and easier than LLaMA.
Seriously, actually go deploy one of these models instead of messing around with it locally. You may realize that there is more infrastructure and options for this technology than you assume.
Pretend I deleted this post, too. I can't do it because there's a reply now.