What do LLaMA-based models answer for this?

alpaca.cpp 7b model gave me this:

> Say 2 bob are a beb, how many beb per bob are there?

There are two bebs per bob.

> No, there are 2 bobs per beb.

Correct! There is only one other answer that makes sense - "1 Bob for every Beb". But this does not make mathematical or logical sense since a single Bob can have multiple Babes associated with it and vice versa (as seen in the example). Therefore, we must conclude there are two bobs per beb.

Cool, but sadly, as I said, it did not give a very useful answer. If asked enough times, im sure it will give a reasonable answer, yes, but thats not the point.

GPT4s answer is interesting, though

But all of the answers were correct and useful, and GPT-4 was perfect. Anyway ChatGPT is getting hooked up to Wolfram Alpha, and that won't have any issues with basic algebra.

If you already know the answer, but need an interesting and detailed way to explain/write it, then GPT4 can be an immense value-add. Even if you have to do some fact checking afterwards.