Re:1-Bit LLMs Could Solve AI's Energy Demands
by d13 on Friday May 31, 2024 @02:57AM
(#40532196)
How does it compare against unequalised Llama 3 8B at 16fp? I’ve been using that locally and it’s almost replaced GPT4 for me. Runs in about 14GB of VRAM.
[ Parent
| on HN ]
Re:1-Bit LLMs Could Solve AI's Energy Demands
by iwontberude on Friday May 31, 2024 @10:31AM
(#40535697)
llama3 is nowhere near gpt4, though it is cool
Re:1-Bit LLMs Could Solve AI's Energy Demands
by LordDragonfang on Friday May 31, 2024 @05:09PM
(#40540278)
What is your use case where you find it comparable to gpt4?
Re:1-Bit LLMs Could Solve AI's Energy Demands
by stavros on Friday May 31, 2024 @06:52PM
(#40541162)
For creative tasks, for example, Llama 3 is much better. GPT-4 is very sterile, Llama is much more whimsical, and has a lot more character.
Re:1-Bit LLMs Could Solve AI's Energy Demands
by CaptainOfCoit on Friday May 31, 2024 @07:55PM
(#40541491)
> For creative tasks
What, specifically, are you asking of these LLMs? "creative tasks" can be anything from programming to cooking recipes, so a tiny bit more specificality would be appreciated :)
Re:1-Bit LLMs Could Solve AI's Energy Demands
by stavros on Friday May 31, 2024 @08:35PM
(#40541747)
Sorry, I meant making up stuff, e.g. creative writing.
Re:1-Bit LLMs Could Solve AI's Energy Demands
by CaptainOfCoit on Saturday June 01, 2024 @08:02AM
(#40545013)
So like making presentations? Composing poems? Writing novellas/novels? Technical articles?
Re:1-Bit LLMs Could Solve AI's Energy Demands
by LordDragonfang on Sunday June 02, 2024 @04:19AM
(#40552417)
I've used pretty much every major LLM out there for a specific type of creative writing, and none of them are as good at it as GPT4 with the exception of maybe Claude (Opus is actually probably even better regarding the sterility). Llama 3, even 70b, is definitely not better by any measure of actual quality - it's more random, at best.