News for hackers. Stuff that matters.

little_name

Re:LLMs are not suitable for brainstorming on Thursday May 16, 2024 @02:15AM (#40375705)
There is a thread below discussing tuning temperature and why it's not tied to creativity. I think the point is temperature and other sampling parameters don't have any inherent mechanism to control the adherence to training pattern as a whole (all top-X candidates are all from training data). Deviating from the max probability in next token generation has no relation with thinking out of box. It does give more generation results that may be lucky to hit something, but I doubt the cost effectiveness
Re:LLMs are not suitable for brainstorming on Thursday May 16, 2024 @01:51AM (#40375589)
> LLM’s degree of adherence to training patterns can be directly tuned, per request (up to whatever the maximum it is capable of.)

I think people often forget the behavior of instruction following is still based on SFT training data. This is exactly "adherence to training patterns". Prompt engineering is not elixir, it's just a way to utilize the patterns seen in training data.

Re:LLMs are not suitable for brainstorming on Thursday May 16, 2024 @12:22AM (#40375169)
Just googled, TIL. It's very much based on the idea of replicating patterns though, so I think it doesn't contradict with the original post.

Was thinking if you can apply TRIZ method to invent Transformer before 2017 - hard to imagine it working on me

Re:LLMs are not suitable for brainstorming on Wednesday May 15, 2024 @09:28PM (#40374321)
I don't know a case study but I was personally told by 2 different founders who had 100MM+ exits that their startup ideas were formed (at least partially) from brainstorm sessions. One guy even pointed to me the library where they had the whiteboard session which led to the idea that they exited in the end.

The SPC also have a blog that proposes brainstorming from -1 to 1: https://blog.southparkcommons.com/how-to-go-from-minus-1-to-...