News for hackers. Stuff that matters.
Re:GPT-3 by the8472 on Thursday June 25, 2020 @05:07AM (#23638492)
> That makes sense but it kind of falls into my "brain has a better architecture" possible explanation. What's interesting is that the thrust of the GPT approaches is "architecture doesn't matter while we can just keep building bigger models".

Those are not mutually exclusive. Architecture doesn't matter asymptotically but it does matter for task-specific performance for any concrete complexity budget. Compare to Big-O behavior of algorithms (asymptotes) vs. doing hardware specific optimizations (cache-friendliness, hand-rolled assembly, outsourcing specific parts to ASICs...).

OA is doing the former, looking at asymptotes. Nature has done that too (mammalian brains scale from rodents to humans) but also has applies all kinds of task-specific optimizations. Compare the cerebral cortex and cerebellum for example, the latter acts similar to an application accelerator but if it doesn't work software fallbacks are possible albeit slower.

> It would be surprising to me if AGI is achievable via two such different pathways.

Have you considered cephalopod or bird brains?

There are no comments on this comment yet.