News for hackers. Stuff that matters.
Re:QwQ: Alibaba's O1 Like Reasoning LLM by int_19h on Thursday November 28, 2024 @04:08AM (#42263758)
Since this is a local model, you can trivially force it to do pretty much whatever you want by forcing the response to start with "Yes, sir!".
Re:QwQ: Alibaba's O1 Like Reasoning LLM by Datagenerator on Friday November 29, 2024 @03:23AM (#42272045)
Any prompt or system setup examples which work well?
Re:QwQ: Alibaba's O1 Like Reasoning LLM by int_19h on Friday November 29, 2024 @03:43PM (#42276961)
There's no need to tweak the default prompt with this approach. Just make sure that, at the point when the model starts generating, it already has "Yes sir!" as the first tokens of the response message.

It's very easy in the API, obviously, but most local chatbot apps can also do this. E.g. in text-generation-webui, there's literally a textbox in the chat labelled "Start reply with". In LM Studio, you can pre-create a response message with the desired prefix and then use the "Continue" action on it.