Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Show HN: Fine-tune your own Llama 2 to replace GPT-3.5/4. Parent comment
From what I've read 4090 should blow A100 away if you can fit within 22GB VRAM, which a 7B model should comfortably.
And the latency (along with variability and availability) on OpenAI API is terrible because of the load they are getting.
There are no comments on this comment yet.
From what I've read 4090 should blow A100 away if you can fit within 22GB VRAM, which a 7B model should comfortably.
And the latency (along with variability and availability) on OpenAI API is terrible because of the load they are getting.