Unsure if this is a useful answer. But Searle/LLM could make something that looks like it has a creative spark, and that's it.
Why I think that's different is in the case of a human artist, they create something because they have something they want to say. Whatever they produce is a way of saying 'this is what the world feels like to me, is it the same for you?'. And if it is, it resonates.
But I cannot see how an LLM would 'want' to say anything. If we're talking psychoanlytically of where wanting comes from, and call it a desire to fill a void of how incoherent you actually are, then an LLM doesn't go through that process.
Maybe Searle does, and still wants the characters to make you feel a certain way, in which case the comparison doesn't fit.
The whole free will debate seems a bit out of scope (and out of my reach, hah), but nonetheless it feels interesting in the LLM context.
edit: Note that i don't necessarily think LLMs are there or even can be. We seem to technologically small to produce the complexity in ourselves. Nonetheless i'm always interested in how far reduced complexity can take us.
Ironically, many people complain LLMs are too incoherent, with all their confabulations and hallucinations.
But I agree. Desire is a good verb. I think that's what differentiates us from the 'machines'. In art, we try to create meaning. From our lives. From our discontents. Even a million LLMs cannot be in deficit of meaning; they are precisely tuned to their own capacity. Whereas something strange about humans is our endless desire for 'more'.
> _if_ you split the process up appropriately I believe this pre-requisite is very important. LLMs so are terrible at planning and splitting a complex task into simpler steps. This might be natural limitation of `next token prediction`. For complex planning, each step should be the result of both the previous and speculative future steps. We try to tackle this by dividing a plan into two: a macro and a micro plan but still a lot to improve there.
p.s. thanks! aider is awesome too!
It seems we've reached the point that understanding of LLMs would be a great candidate for the beginner/intermediate/expert meme. "It's just autocomplete" -> "It's got a world model, it's thinking for itself" -> "It's just autocomplete".
Why not?
> Of course, at some level of complexity, it will be stuck in a local maximum of work quality simply because the book has no guide on how to solve the problem at hand.
I find this a pretty un-optimistic view, especially from someone building a coding autopilot. Having myself used LLMs for a bunch of software development in the last year, it seems its 'local maximum' is no different from a developer's _if_ you split the process up appropriately. The author alludes to this when they mention 'workflow'.
Everyone is trying to use LLMs in a 'single inference pass', assuming that's as good as it gets, but that's like trying to get find human creativity in a single cascading activation of neurons. A brain doesn't fit on an axon. So, I kinda think the author should be less shy about their optimism. Inference is soon ~free, as they say, so to me, naive as I might be, the future of AI coding agents is not limited to grunt tasks, it is as creative and exploratory as any human coder.
Ps. Fume looks cool. I'd suggest people take a look at aider.chat and claude-engineer too (on github).