I agree but I think it may be privileging the human intelligence mechanism a bit too much. These LLMs are polymaths that can spit out content at a super human rate. It can generate poetry and literature similarly to code and answers about physics and car repair. It’s very rare for a human to be able to do that especially these days.

So I agree they’re transitional but only in the sense that our brains are transitional from the basal ganglia to the neocortex. In that sense I think LLMs will probably be a part of a future GAI brain with other things tracked on, but it’s not clear it will necessarily evolve to work like a human’s brain does.

Do you mean in theory or currently? Because currently, LLMs make simple errors (eg [1]) and are more capable of spitting out, well, nonsense. I think it's safe to say we're a long way from LLMs producing anything creatively good.

I'll put it this way: you won't be getting The Godfather from LLMs anytime soon but you can probably get an industrial film with generic music that tells you how to safely handle solvents, maybe.

Computers are generally good at doing math but LLMs generally aren't [2] and that really demonstrates the weaknesses in this statistical approach. ChatGPT (as one example) doesn't understand what numbers are or how to multiply them. It relies seeing similar answers to derive a likely answer so it often gets the first and large digits of the answer correct but not the middle. You can't keep scaling the input data to have it see every possible math question. That's just not practical.

Now multiplying two large numbers is a solvable problem. Counting Rs in strawberry is a solvable problem. But statistical LLMs are going to have a massive long tail of these problems. It's really going to take the next generational change to make progress.

[1]: https://www.inc.com/kit-eaton/how-many-rs-in-strawberry-this...

[2]: https://www.reachcapital.com/2024/07/16/why-llms-are-bad-at-...

I think that adding different forms of assistance remains the most interesting pattern right now.

https://chatgpt.com/share/670bfdbd-8624-8011-bc31-2ba66eab3e...

I didn't realize that it had come that far with the delegating of those problems to the code writing and executing part of itself.

I still think transformers and LLMs will likely remain as some component within that next gen architecture vs something completely alien.

Sometimes online forums are the only place where you can find solutions for niche situations and edge cases. Tricks which would have been very difficult to figure out on your own. LLMs can train on the official documentation of tools l/libraries but they can't experiment and figure out solutions to weird problems which are unfortunately very common in tech industry. If people stop sharing such solutions with others, it might become a big problem.

LLMs train on way more than just the official documentation: they train on the code itself, the unit tests for that code (which, for well written projects, cover all sorts of undocumented edge-based) and - for popular projects - thousands of examples of that library being used (and unit tested) "in the wild".

This is why LLMs are so effective at helping figure out edge-cases for widely used libraries.

The best coding LLMs are also trained on additional custom examples written by humans who were paid to build proprietary training data for those LLMs.

I suspect they are increasingly trained on artificially created examples which have been validated (to a certain extent) through executing that code before adding it to the training data. That's a unique advantage for code - it's a lot harder to "verify" non-code generated prose since you can't execute that and see if you get an error.

If understanding the code was enough, we wouldn't have any bugs or counterintuitive behaviors.

> and - for popular projects - thousands of examples of that library being used (and unit tested) "in the wild".

If people stopped contributing to forums, we won't have any such data for new things that are being made.

I would argue that code in GitHub is much more useful, because it's presented in the context of a larger application and is also more likely to work.

That's the most valuable aspect of it. When you find yourself in these niches situations, it's nice when you see someone has encountered it and has done the legwork to solve it, saving you hours and days. And that's why Wikis like the Arch Wiki are important. You need people to document the system, not just individual components.

I've been thinking about this a lot lately. Could we train an AI, e.g. using RL and GAN, where it gets an IT task to perform based on a body of documentation, such that its fitness would then be measured based on both direct success on the task, and on the creation of new (distilled and better written) documentation that would allow an otherwise context-less copy of itself to do well on the task?

'Listening to different spiritual gurus saying the same thing using different words' is like 'watching the same coloured glass pieces getting rearranged to form new patterns in kaleidoscope'

So I think future AI has to read also everything if it is used like ChatGPT these days by average people to ask for advice about almost anything.

> Another issue is how much IP consumption constitutes theft? If an LLM watches every movie ever made, that's probably theft.

It's hard to reconcile those two views, and I don't think theft is defined by "how much" is being stolen.