News for hackers. Stuff that matters.
Re:What Happens When People Don't Understand How AI Works by Al-Khwarizmi on Sunday June 08, 2025 @06:02PM (#44219761)
I don't think those of us who don't work at OpenAI, Google, etc. have enough information to accurately estimate the influence of instruction tuning on the capabilities or the general "feel" of LLMs (it's really a pity that no one releases non-instruction-tuned models anymore).

Personally my inaccurate estimate is much lower than yours. When non-instruction tuned versions of GPT-3 were available, my perception is that most of the abilities and characteristics that we associate with talking to an LLM were already there - just more erratic, e.g., you asked a question and the model might answer or might continue it with another question (which is also a plausible continuation of the provided text). But if it did "choose" to answer, it could do so with comparable accuracy to the instruction-tuned versions.

Instruction tuning made them more predictable, and made them tend to give the responses that humans prefer (e.g. actually answering questions, maybe using answer formats that humans like, etc.), but I doubt it gave them many abilities that weren't already there.

Re:What Happens When People Don't Understand How AI Works by tough on Monday June 09, 2025 @06:37AM (#44223024)
instruction tuning is like imprimating the chat ux into the models weights

its all about the user/assistant flow instead of just a -text generator- after it

and the assistant always tries to please the user.

they built a sychopantic machine either by mistake or malfeasance

Re:What Happens When People Don't Understand How AI Works by rahimnathwani on Monday June 09, 2025 @03:05PM (#44228115)
"it's really a pity that no one releases non-instruction-tuned models anymore"

Llama 4 was released with base (pretrained) and instruction-tuned variants.