News for hackers. Stuff that matters.
Re:Why Computers Won't Make Themselves Smarter by simonh on Wednesday March 31, 2021 @04:59AM (#26644984)
>I take it as a given that some kind of computation in the brain is responsible for human intelligence...

Roger Penrose argues that human intelligence can't be a computation because we can arrive at true conclusions even though they are not formally provable. Consistent logical systems can't do that as proved by Goedel. My answer to that is, there's no reason to suppose human mental processes are formally consistent, and there's no reason why AI computer systems have to be formally consistent either. I just wanted to address that objection because I agree completely with your arguments.

Aside from that, Chiang's arguments if true would make Deep Blue and Alpha Zero impossible. Clearly we can make computer systems that are dramatically better at solving many problems than ourselves. For Chiang's arguments to carry any force, he'd have to show why general intelligence is an exception to this very extensively demonstrated capability.

Re:Why Computers Won't Make Themselves Smarter by benlivengood on Wednesday March 31, 2021 @03:08PM (#26651628)
> Roger Penrose argues that human intelligence can't be a computation because we can arrive at true conclusions even though they are not formally provable.

Tarski proved that it's impossible to prove truth within the same system anyway, so this isn't a strong objection and is in fact subtly incorrect. Humans don't arrive at "true" conclusions either, we just fail to find contradictions and call it good enough.

Probabilistic reasoning is just as powerful in practice as tractable computations.

Re:Why Computers Won't Make Themselves Smarter by jqpabc123 on Wednesday March 31, 2021 @10:45AM (#26647974)
... and there's no reason why AI computer systems have to be formally consistent either.

There is no reason why an inconsistent AI system should be considered reliable either.

Formal consistency is baked into the design of modern computer hardware. A computer that is not formally consistent at the hardware level is random and thus not very practical.

Likewise at the software level, the only alternative to formal consistency is randomness. Which may provide an answer but with no real assurance of accuracy or correctness.

It may be possible to obtain a correct answer to very specific problems from random logic through iteration and testing --- but a lot of time, effort and energy will almost certainly be wasted on wrong ones. Once again, not terribly promising on a practical AGI level.