Re:Exponentially Faster Language Modelling
by sorokod on Wednesday November 22, 2023 @05:28PM
(#38386069)
What is the sum of odd numbers in this set: 4, 7, 12, 1, 3
The sum of odd numbers in the given set is 4 + 7 + 1 = 12. Therefore, the answer is 12.
Re:Exponentially Faster Language Modelling
by voiceblue on Wednesday November 22, 2023 @05:58PM
(#38386378)
You're using an LLM as if you don't understand how or why it works. Math is not what it's for. Here, look at what ChatGPT 4 does (for a case that hasn't been fed to it by a hundred other users yet):
What is the sum of odd numbers in this set: 12345654321, 123456543212, 123456543213, 12345654324? Output only the sum, no code.
The sum of the odd numbers in the set {12345654321, 123456543212, 123456543213, 12345654324} is 246913086434.
What does that prove? Only that I am using an LLM poorly and/or do not understand what it is. Using OpenChat-3.5 for what LLMs are actually good at (e.g. asking it for shell commands to perform certain operations, getting some general information about a topic) seems to work surprisingly well for a 7B model.
Re:Exponentially Faster Language Modelling
by sorokod on Thursday November 23, 2023 @02:04AM
(#38390200)
Not sure what you mean by 'using'. The prompt is from one of the benchmarks used in scoring LLM models - don't recall which. Here is another from BBH [1]:
>>> (-1 + 2 + 9 * 5) =
... (-1 + 2 + 9 * 5) = 43
Re:Exponentially Faster Language Modelling
by GaggiX on Thursday November 23, 2023 @04:03AM
(#38390871)
This is even worse than the other, just straight up use a calculator.
Re:Exponentially Faster Language Modelling
by freilanzer on Thursday November 23, 2023 @04:59AM
(#38391177)
The use isn't the calculation, but the benchmark for its arithmetic capability.
Re:Exponentially Faster Language Modelling
by all2 on Wednesday November 22, 2023 @05:52PM
(#38386324)
Technically 3 is even. \s
Re:Exponentially Faster Language Modelling
by imatworkyo on Wednesday November 22, 2023 @08:40PM
(#38388104)
is this really what you are using AI for ?
Re:Exponentially Faster Language Modelling
by moffkalast on Wednesday November 22, 2023 @05:55PM
(#38386350)
Are there any comparisons with Mistral-instruct? I've yet to see anything under 30B beat it in any way.
HuggingFace: https://huggingface.co/openchat/openchat_3.5
On the LLM arena (blinded comparisons), it's the third best non-proprietary model: https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar...