News for hackers. Stuff that matters.
Re:100x defect tolerance: How we solved the yield problem by immibis on Thursday January 16, 2025 @05:08AM (#42723475)
Why do you think that an AGI can't be a token predictor?
Re:100x defect tolerance: How we solved the yield problem by Shorel on Thursday January 16, 2025 @07:53AM (#42724525)
By analogy with human brains: Because our own brains are far more than the Broca's areas in them.

Evolution selects for efficiency.

If token prediction could work for everything, our brains would also do nothing else but token prediction. Even the brains of fishes and insects would work like that.

The human brain has dedicated clusters of neurons for several different cognitive abilities, including face recognition, line detection, body parts self perception, 3D spatial orientation, and so on.

Re:100x defect tolerance: How we solved the yield problem by myrmidon on Thursday January 16, 2025 @09:33AM (#42725680)
> Evolution selects for efficiency.

I think this is a poor argument here. From an evolutionary point of view, our brains are optimized to:

- Provide fine-motor control to craft weapons and tools (enhancing adaptibility and enabling us to hunt way outside our weight class)

- Communicate/coordinate effectively in small groups

- Do sensor processing and the above with a low energy budget

Our brains are *not* selected to be minimum-complexity intelligences, and a lot of what our brain does is completely useless for AGI building (motor control, sensor processing, ...).

Furthermore, the cost/complexity (from a evolutionary PoV) is a totally different beast from what complexity means to us.

Just consider flight as an example: A fruitfly is an insanely simple and straightforward beast, but to us, a biochemically fuelled, beating-wing design is still infeasibly complicated. If our approach to flight had been to ape after how nature does it in detail, we likely still would not have planes.

I do agree that todays LLMs still have clear architectural flaws that we need to overcome (online learning being a very glaring one), but, to pick up the flight analogy, we might well have the main wing structure already down, and we won't necessarily have to make those wings beat to get into the air...

Re:100x defect tolerance: How we solved the yield problem by Shorel on Thursday January 16, 2025 @11:32AM (#42727502)
Just because there are some parts of our brains that are not needed for an AGI...

Doesn't mean that there aren't some part of our brains that are needed for an AGI, and are not present in the current crop of LLM.

Re:100x defect tolerance: How we solved the yield problem by bcrl on Sunday January 19, 2025 @08:55PM (#42764177)
I think it means that AGIs trained in different ways will have different strengths, just like people.
Re:100x defect tolerance: How we solved the yield problem by immibis on Thursday January 16, 2025 @09:43AM (#42725833)
What do our brains do that isn't token prediction?

They receive information about photons and air vibrations and control muscles, okay. If a human brain was hooked up the way ChatGPT was, only to text input and output, would that make it not intelligent?

Re:100x defect tolerance: How we solved the yield problem by Shorel on Thursday January 16, 2025 @11:29AM (#42727464)
> What do our brains do that isn't token prediction?

I am planning a masters and phd on that question, so give me a few years to answer.

Re:100x defect tolerance: How we solved the yield problem by mschuster91 on Thursday January 16, 2025 @05:18AM (#42723558)
Because an LLM _by definition_ cannot even do basic maths (well, except if you're OpenAI and cheat your way around it by detecting if the user asks a simple math question).

I'd expect an actually "general" intelligence Thing to be able to be as versatile in intellectual tasks as a human is - and LLMs are reasonably decent at repetition, but cannot infer something completely new from the data it has.

Re:100x defect tolerance: How we solved the yield problem by versteegen on Thursday January 16, 2025 @06:02AM (#42723816)
Define "by definition".

Because this statement really makes no sense. Transformers are perfectly capable (and capable of perfectly) learning mathematical functions, given the necessary working-out space, e.g. for long division or for algebraic manipulation. And they can learn to generalise from their training data very well (although very data-inefficiently). That's their entire strength!

Re:100x defect tolerance: How we solved the yield problem by dogcomplex on Thursday January 16, 2025 @06:56AM (#42724146)
Yet they can get silver medal PhD level competition math scores.

Perhaps your "definition" should be simply that LLMs have temporarily seen limitations in their ability to natively do math unassisted by an external memory, but are exceptionally good at very advanced math when they can compensate for their lossy short-term attention memory...