News for hackers. Stuff that matters.
Re:LLMs are not suitable for brainstorming by hobobaggins on Wednesday May 15, 2024 @09:51PM (#40374460)
> people do the same thing

You're right, most people do that, but that's because they haven't trained themselves to be inventors. This is a skill that definitely needs to be developed more, but finding truly creative (and sometimes backwards-seeming) solutions can sometimes mean thinking about problems in a completely different way.

However, all LLM's do this. That's one of the points of the article.

Re:LLMs are not suitable for brainstorming by refulgentis on Wednesday May 15, 2024 @10:42PM (#40374700)
There was an actual peer-reviewed article from an institution yesterday showing, numerically, robustly, human evaluated, that LLMs are better at creative thinking than humans.

I don't think handwaving about exclusively outputting training data and random punditry via article really holds up here.

It's wrong for so many reasons, from the training data having more perspectives, to the "Fat Tony" test, empirically, we can go make it output novel combinations right now, to the CS test, we can have it emulate arbitrary Turing machines.

Re:LLMs are not suitable for brainstorming by wrycoder on Thursday May 16, 2024 @12:07AM (#40375102)
Might you possibly have a link?
Re:LLMs are not suitable for brainstorming by refulgentis on Thursday May 16, 2024 @12:29AM (#40375202)
My bad, looked it up and meant to edit it in, dunno what happened: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10858891/
Re:LLMs are not suitable for brainstorming by barfbagginus on Thursday May 16, 2024 @12:53AM (#40375299)
The methodology of this paper is seriously suspect.

First they have a very small sample size of around n = 150. Second, they don't represent average husband, they use people working on mechanical Turk for $9 an hour.

Finally, they don't have any standard measures of divergent thought - they just use 4 different types of question answering problems, and those are supposed to measure creativity. Whether they do or don't would require us to evaluate the questions.

It's important to be clear that this is saying that LLMs might be more creative than low paid workers on mechanical Turk, given no training time. That's not a very good representation of the average human, or the fact that humans need training.

So for example, an average doctor or scientist might still be more creative than LLMs.

Tldr; this is not a good paper to throw around if you want to convince people. It's truly awful in its design, and the way it's presented is not always honest.

Re:LLMs are not suitable for brainstorming by refulgentis on Thursday May 16, 2024 @01:03AM (#40375353)
No results for mechanical turk, n=151 is perfectly significant for what amounts to a taste test (I was quite taken aback to read that, 151 is low in very few contexts), I'm not "throwing it around", whatever that means in this context, and the extreme reaction and 100% focus on it belies that it was 1 of 3 short points I made, as well as that the research backs up the claim that was shut down, that is intuitively correct, I'd rather have a 1000 specialists at undergrad level ideating than me and Einstein and Picasso and Steve Jobs.

So, yeah, between that, and that both your average person and someone arguing from Turing tapes can show AI doing things not in the training data, I'm going with actual data thats being "thrown around" rather than the rando blog post that argues from "by definition it can't be creative because it isn't original because it only repeats training data" Call me when they're at N=152 on a blind test.

Re:LLMs are not suitable for brainstorming by rafaeljimenez on Monday May 27, 2024 @03:18PM (#40493717)
They also don’t seem to be taking into consideration the fact that answers to those tests may already be present in the training data.
Re:LLMs are not suitable for brainstorming by imtringued on Thursday May 16, 2024 @02:21AM (#40375729)
There is a difference between a verbatim answer and an answer in the shape of a verbatim answer. The latter is not much better if you actually want to do new things.
Re:LLMs are not suitable for brainstorming by dragonwriter on Thursday May 16, 2024 @01:39AM (#40375529)
> However, all LLM’s do this.

LLM’s degree of adherence to training patterns can be directly tuned, per request (up to whatever the maximum it is capable of.)

Can this be leveraged to build a framework around an LLM for brainstorming usefully? I’m not sure. But “Nope, because LLMs just directly adhere to existing patterns” is definitely not a useful answer to that question. It is the kind of answer someone who just parrots common patterns with surface-level knowledge of the subject area would come up with. (Huh, did a default-settings LLM write this article?)

Re:LLMs are not suitable for brainstorming by little_name on Thursday May 16, 2024 @01:51AM (#40375589)
> LLM’s degree of adherence to training patterns can be directly tuned, per request (up to whatever the maximum it is capable of.)

I think people often forget the behavior of instruction following is still based on SFT training data. This is exactly "adherence to training patterns". Prompt engineering is not elixir, it's just a way to utilize the patterns seen in training data.

Re:LLMs are not suitable for brainstorming by dragonwriter on Thursday May 16, 2024 @02:03AM (#40375643)
> Prompt engineering is not elixir

I’m not talking about prompt engineering when I talk about per request tuning of how closely it follows established patterns, I’m talking about inference parameters (temperature, top_p, top_k, are common for most models and ways of calling them, some others may be available, too.)

Re:LLMs are not suitable for brainstorming by little_name on Thursday May 16, 2024 @02:15AM (#40375705)
There is a thread below discussing tuning temperature and why it's not tied to creativity. I think the point is temperature and other sampling parameters don't have any inherent mechanism to control the adherence to training pattern as a whole (all top-X candidates are all from training data). Deviating from the max probability in next token generation has no relation with thinking out of box. It does give more generation results that may be lucky to hit something, but I doubt the cost effectiveness
Re:LLMs are not suitable for brainstorming by imtringued on Thursday May 16, 2024 @02:16AM (#40375709)
How will increasing temperature improve the arithmetic performance of the model with numbers it has never seen? When prompted the right way, the model has trouble sticking to the answer that is implied by its chain of thought process. For some strange reason it knows how to perform the task, but when it is time to output the answer, it decides to recalculate it from scratch using attention. Before you complain about me benchmarking arithmetic performance. Consider that I could have defined any algebraic structure and I could have made the exact same complaint. It's just that arithmetic is very easy to verify using a calculator. Dismissing the easy arithmetic also dismisses all operations on any algebra.

The obvious problem is that these logic and arithmetic operations don't really need exhaustive training examples. Training the LLM is supposed to teach it a method or process by which it arrives at the answer, not just the question answer pair itself, which tends to only lead to very good approximate results. That is ok when using language, since your exact words don't really matter if you can express yourself in another way, but when it comes to algebras, accuracy is key.

It really is strange that when one wishes for a more capable LLM that one is told how stupid one is for wanting such a thing. What exactly is so wrong about giving the model a price list and an example of an order at a restaurant and then asking how much the whole order costs and expecting the correct total sum? For some strange reason I'm getting numbers that would make sense if I generated the text in a vacuum without the price list, as if they are following some existing pattern...

Re:LLMs are not suitable for brainstorming by dragonwriter on Thursday May 16, 2024 @02:30AM (#40375763)
> How will increasing temperature improve the arithmetic performance of the model with numbers it has never seen?

“Brainstorming” and “doing precise arithmetic” are wildly different tasks. It is a complete non-sequitur to ask how a suggestion about the former will help the latter.