Re:Orca 2: Teaching Small Language Models How to Reason
by yujian on Tuesday November 21, 2023 @09:24AM
(#38363996)
I'm not sure if I'm missing something from the paper, but are multi-billion parameter models getting called "small" language models now? And when did this paradigm shift happen?
[ Parent
| on HN ]
Re:Orca 2: Teaching Small Language Models How to Reason
by hmottestad on Tuesday November 21, 2023 @09:33AM
(#38364131)
All the llama models, including the 70B one can run on consumer hardware. You might be able to fit GPT-3 (175B) at Q4 or Q3 on a Mac Studio, but that's probably the limit for consumer hardware. At 4-bit a 7B model requires some 4GB of ram, so that should probably be possible to run on a phone, just not very fast.
Re:Orca 2: Teaching Small Language Models How to Reason
by sa-code on Tuesday November 21, 2023 @01:43PM
(#38368111)
Gpt 3.5 turbo is 20B
Re:Orca 2: Teaching Small Language Models How to Reason
by kristianp on Tuesday November 21, 2023 @03:32PM
(#38369645)
I doubt that. What's your source?
Re:Orca 2: Teaching Small Language Models How to Reason
by sa-code on Tuesday November 21, 2023 @06:34PM
(#38372190)
There was a paper published by Microsoft that seemed to leak this detail. I'm on mobile right now and don't have a link but it should be searchable
Re:Orca 2: Teaching Small Language Models How to Reason
by nl on Tuesday November 21, 2023 @07:16PM
(#38372638)
The paper was https://arxiv.org/abs/2310.17680
It has been withdrawn with this note:
> Contains inappropriately sourced conjecture of OpenAI's ChatGPT parameter count from this http URL, a citation which was omitted. The authors do not have direct knowledge or verification of this information, and relied solely on this article, which may lead to public confusion
(the noted URL is a just a Forbes blogger with no special qualifications that would make what he claimed particularly credible).
Re:Orca 2: Teaching Small Language Models How to Reason
by Chabsff on Tuesday November 21, 2023 @09:26AM
(#38364022)
Nowadays, small essentially means realistically useable on prosumer hardware.
Re:Orca 2: Teaching Small Language Models How to Reason
by nathanfig on Tuesday November 21, 2023 @09:27AM
(#38364028)
Relative term. In the world of LLMs, 7b is small.
Re:Orca 2: Teaching Small Language Models How to Reason
by moffkalast on Tuesday November 21, 2023 @09:41AM
(#38364237)
When 175B, 300B, 1.8T models are considered large, 7B is considered small.