News for hackers. Stuff that matters.
Re:How to run Llama 13B with a 6GB graphics card by jstarfish on Sunday May 14, 2023 @03:28PM (#35940667)
Maybe others' experiences are different, but I find smaller models to work just as well for "reductive" tasks.

Dolly sucks for generating long-form content (not very creative) but if I need a summary or classification, it's quicker and easier to spin up dolly-3b than vicuna-13b.

I suspect OpenAI is routing prompts to select models based on similar logic.

There are no comments on this comment yet.