News for hackers. Stuff that matters.

A comment on Forget ChatGPT: why researchers now run small AIs on their laptops. Parent comment

Re:Forget ChatGPT: why researchers now run small AIs on thei... by Jedd on Saturday September 21, 2024 @08:51PM (#41613763)
Often you'll find there's '-chat-' and '-instruct-' variants of an LLM available.

Trying to chat to an INSTRUCT model will be disappointing, much as you describe.

[ Parent | on HN ]
Re:Forget ChatGPT: why researchers now run small AIs on thei... by kaoD on Saturday September 21, 2024 @09:28PM (#41613951)
This was on their example LLaVA 1.5 7b q4 with all default parameters which does not specify chat or instruct... but after the first message it actually worked as expected so I guess it's RLHF'd for chat or chat+instruct.

I don't know if it was some sort of error on the UI or what.

Trying to interrogate it about the first message yielded no results. It just repeated back my question, verbatim, unlike the rest of the chat which was more or less chat-like :shrugh:

[ Parent | on HN ]