News for hackers. Stuff that matters.

A comment on A Google Brain engineer’s guide to entering AI. Parent comment

Re:A Google Brain engineer’s guide to entering AI by ben_w on Saturday November 10, 2018 @01:53PM (#18422830)
Once an AI gets that good, good enough to reproduce even a junior developer, nobody can ever get paid to work again. Before that point, it’s still useful to know the basics of AI in the same way knowing about arrays or higher order functions is useful.
[ Parent | on HN ]
Re:A Google Brain engineer’s guide to entering AI by yters on Saturday November 10, 2018 @02:24PM (#18422997)
Do you know of a Turing complete AI/ML model?
[ Parent | on HN ]
Re:A Google Brain engineer’s guide to entering AI by currymj on Saturday November 10, 2018 @05:57PM (#18423971)
many of them are Turing-complete almost by accident, if you are willing to add enough parameters. E.g. a shallow neural network with one layer is Turing-complete.

It's actually Turing-complete for rational-valued weights (countable). If you are willing to grant real valued weights (uncountable) then it is actually more powerful than a Turing machine, able to compute things in P/poly, a very weird complexity class that includes some encodings of the halting problem. But this is just because you're "cheating" by allowing infinite precision which is not possible in the real world.

[ Parent | on HN ]
Re:A Google Brain engineer’s guide to entering AI by yters on Saturday November 10, 2018 @06:27PM (#18424076)
That doesn't sound like TC for real world usage...unlike a regular programmer.
[ Parent | on HN ]
Re:A Google Brain engineer’s guide to entering AI by ben_w on Sunday November 11, 2018 @01:53PM (#18427874)
One layer is a surprise to me, I thought xor needed two?
[ Parent | on HN ]
Re:A Google Brain engineer’s guide to entering AI by chillee on Sunday November 11, 2018 @08:51PM (#18430022)
One hidden layer. Commonly known as the "Universal approximation theorem"
[ Parent | on HN ]
Re:A Google Brain engineer’s guide to entering AI by ben_w on Tuesday November 13, 2018 @08:33AM (#18440778)
Ah, I misunderstood GP saying one layer.
[ Parent | on HN ]
Re:A Google Brain engineer’s guide to entering AI by solomatov on Saturday November 10, 2018 @02:37PM (#18423066)
- Neural turing machines: https://arxiv.org/abs/1410.5401

- Universal transformers: https://arxiv.org/abs/1807.03819

These are just 2 of most well known.

[ Parent | on HN ]
Re:A Google Brain engineer’s guide to entering AI by yters on Saturday November 10, 2018 @03:06PM (#18423207)
Definitely interesting stuff. Do you know of real world tasks where these sorts of models have proven more useful than non TC models?
[ Parent | on HN ]
Re:A Google Brain engineer’s guide to entering AI by solomatov on Saturday November 10, 2018 @03:30PM (#18423321)
I am not aware of any real world usage of these models. They are hard to train, and most of the papers consider toy examples.
[ Parent | on HN ]