it was difficult to run all those models. now gamers follow youtube tutorials

Name 1 way GPT-Neo was harder to run than LLaMA.

Edit: Never mind, I deleted my posts. I keep forgetting that posting to HN is an exhausting mistake that I don't have the energy to deal with. One day I might learn better.

Pretend I deleted this post, too. I can't do it because there's a reply now.

GPT-Neo did run on Macbooks, though. Here's some random guy off YouTube doing it right now: https://youtu.be/sjLDuQrCCro

All of this (and more) was possible before LLaMA hit the scene. Not one feature about LLaMA is novel, which is why we were able to build wrappers like llama.cpp so quickly.

Edit: The only optimization that BERT/GPT-Neo needed was Apple to enable Pytorch acceleration. LLaMA is exclusively an inferencing tool that runs on the CPU, it's "optimization" on both Apple Silicon and x86 are hacked wrappers around SIMD instructions.

I'm only pushing back here because I was running GPT-Neo for a Discord bot on cheap-as-dirt ARM hardware a year ago. It's been possible long before LLaMA brought it to the attention of bored entrepreneurs.

> And that, in turn, makes it harder for anybody to use it.

Only one of these models can be downloaded legally. Between the DMCA takedowns and broken IPFS gateways LLaMA faced, I can confidently say BERT, T5 and GPT-Neo Just Work faster and easier than LLaMA.

Seriously, actually go deploy one of these models instead of messing around with it locally. You may realize that there is more infrastructure and options for this technology than you assume.

In my experience GPT-Neo never produced any useful output.