News for hackers. Stuff that matters.
Re:The Myth of AI – Jaron Lanier by VLM on Wednesday January 06, 2016 @09:52AM (#10850749)
"There has been a domineering subculture—that's been the most wealthy, prolific, and influential subculture in the technical world—that for a long time has not only promoted the idea that there's an equivalence between algorithms and life, and certain algorithms and people, but a historical determinism that we're inevitably making computers that will be smarter and better than us and will take over from us."

On a different discussion site I recently pointed out there is a similar religious belief or religious dogma in the idea of the self driving car, that went over like a lead balloon. People acting in a religious manner and expressing religious belief don't like to have that pointed out to them. Probably vestigial monotheism, if they act religiously at their traditional church on Sundays they don't like it pointed out that they worship a different altar online, or the atheists get really wound up when they are called out for joining the atheist movement but acting as deacons of the church of the self driving car of the future.

Anyway just saying its a line of reasoning, that while true, can only lead to unpleasantness. Like discussion of scientific racial / gender differences or discussions about IQ, all you're gonna get is social signalling screaming "see no evil".

Re:The Myth of AI – Jaron Lanier by golemotron on Wednesday January 06, 2016 @11:09AM (#10851237)
> Anyway just saying its a line of reasoning, that while true, can only lead to unpleasantness. Like discussion of scientific racial / gender differences or discussions about IQ, all you're gonna get is social signalling screaming "see no evil".

I'm waiting for it to fall apart. The amount of information that you have to willingly ignore these days to participate in the ''progress solves all ills'' narrative is huge.

On vestigial monotheism read Straw Dogs by John Gray if you haven't.

Re:The Myth of AI – Jaron Lanier by pron on Wednesday January 06, 2016 @12:03PM (#10851674)
I fully agree with the religious aspect of it (religion doesn't have to be theistic at all, but the particular beliefs in an after-life/resurrection/omniscient-omnipotent being do resemble the theistic, and indeed, monotheistic religions), but I think there's another aspect to the "AI threat" belief that is more than just "vestigial monotheism".

I think this is (in part) a power fantasy. The believers in the AI threat normally perceive themselves to be different from other people due to their (self-perceived or real) higher intelligence. They therefore describe that property (that they believe makes them special) as being particularly dangerous, and therefore powerful, because they want to believe that intelligence imbues its possessor with power and a lot of it. Sadly, superbugs, meteorites, H bombs, climate change, human-rights violations, inequality and other threats don't require or possess much intelligence, but an unstoppable AI is the purest manifestation of intelligence-as-power.

In the human world, intelligence is not very correlated with power, except very roughly. The most powerful people are generally more intelligent than average, but the correlation doesn't extend far beyond that. If anything, charm, confidence and courage seem to imbue their bearers with much more power than raw intelligence. Which is why I fear (only partly jokingly) a super-charming machine than a super-intelligent one. A super-charming machine is far more likely to bend humanity to its will -- and therefore pose a greater danger -- even if what it wants may be truly idiotic; a super-intelligent machine would be far more limited in its effects...

Re:The Myth of AI – Jaron Lanier by VLM on Wednesday January 06, 2016 @12:47PM (#10852077)
"A super-charming machine"

A television, or a tablet with facebook and youtube apps?

Its interesting to think of mass propaganda a la Bernays as an example of "super charming machine". In the old days the cogs and wheels in the marketing business are/were humans, just like in the old days "math calculator" was a job description but it got automated and accelerated into computers.

Populist political machine might be another example.

Re:The Myth of AI – Jaron Lanier by tome on Thursday January 07, 2016 @05:30AM (#10857151)
> unstoppable AI is the purest manifestation of intelligence-as-power

Brilliant observation.

Re:The Myth of AI – Jaron Lanier by AndrewKemendo on Wednesday January 06, 2016 @10:51AM (#10851109)
vestigial monotheism

That's a fantastic metaphor.

I'm curious though how someone could act in a religious manner about a self-driving car.

To the original point though, I am very much in the camp of "historical determinism that we're inevitably making computers that will be smarter and better than us" and here is why: I believe that intelligence (and "consciousness" if you want to go down that rabbit hole) is completely material and as such it is possible that we will eventually understand the mechanics of them.

If we can understand those, then history would indicate (Historical fallacy) that we can replace or replicate those mechanics with more durable systems or materials than the fairly fragile.

Re:The Myth of AI – Jaron Lanier by TheOtherHobbes on Wednesday January 06, 2016 @11:20AM (#10851331)
That's a religious position. We have absolutely no idea what consciousness is or how it works.

More than that, we have don't even have ad hoc good models for motivation and personality, never mind useful formal and explicit models.

I think Lanier is absolutely right, and I think there's a strong quasi-theological element running through all of AI and programming.

Programming is a lot like making spells and incantations. If you get them just so you get the outcome you want. But you have to speak the language of the system to make that possible. And you have to be very careful about unintended consequences.

There's something very medieval about this - both in the sense of the pure scholasticism of academic CS, but also in the practical sense of knowing how to formulate the correct "prayers" to make useful things happen.

Lanier seems to be pointing out that CS is still haunted and influenced by these religious metaphors, and that AI is the most visible example of that.

I think he's right - and more, I think that real AI, in the sense of autonomous personalities, won't be possible until that's no longer true.

Re:The Myth of AI – Jaron Lanier by AndrewKemendo on Wednesday January 06, 2016 @11:41AM (#10851482)
How is it a religious position? A religious position would be thinking that some unknowable process would deliver some kind of techno-rapture.

So because we don't know how something works now, we will never know? C'mon.

Programming is a lot like making spells and incantations.

No, it's not. Maybe for laymen it looks like that, but for us developers it is definitely not the case. Granted, sometimes something works and you don't realize why, but that just means you have a big debug problem on your hands - not that it's unknowable. Usually only happens when you port something or copy code over from one system to the next.

Not sure how you come to those conclusions.

Re:The Myth of AI – Jaron Lanier by moron4hire on Wednesday January 06, 2016 @11:45AM (#10851520)
Because it's based in faith that something will happen, not a rational determination based on facts on hand.
Re:The Myth of AI – Jaron Lanier by AndrewKemendo on Wednesday January 06, 2016 @12:10PM (#10851726)
What? We are actually building this stuff...

It's about as much faith as Doug Englebart had about the PC.

Re:The Myth of AI – Jaron Lanier by moron4hire on Wednesday January 06, 2016 @12:35PM (#10851961)
No, we're building ELIZA with larger databases. You can't take any research we have on hand today and say "a natural extension of this leads to consciousness". Part of that is because we don't even understand what consciousness is. Part of that is because we don't have any realistic work beyond pattern matching.

That's why it's not rational. There is no fact you can exploit to come to this conclusion. That's called "faith", regardless of the particular subject matter.

A little faith is important. We didn't know if the first astronauts would survive orbit, and we didn't really have any data to prove it unequivocally. We just had faith that our limited understanding wasn't missing anything and the only way we could find out would be to put people in orbit. We found out soon enough.

But it's really, really a bad idea to call something rational when it's really faith, just because you don't want to be associated with religion.

Re:The Myth of AI – Jaron Lanier by AndrewKemendo on Wednesday January 06, 2016 @12:51PM (#10852113)
"a natural extension of this leads to consciousness"

Did I say that? In fact I didn't.

I don't care what you call anything, the fact remains that AGI doesn't exist today, but we can work on the best guesses at what we think will get us there. That is the exact OPPOSITE of faith if there ever was one.

We are working on computer vision with segmentation. That's a tiny subsection of supervised learning. By the way you say pattern matching as though it's some trivial thing. In fact that's what a hefty portion of our brain does to understand anything - so it's actually very important and our visual system (eyeball to cortex to representation) is arguably the most important part of that.

I couldn't care less about whether a computer has consciousness anyway. I think the whole consciousness argument is a waste of time (which is why I called it a rabbit hole previously).

I think you're hung up on this religious thing and not able to discriminate between actually building things that may be able to lead to AGI versus writing novels and daydreaming like so many "futurists."

Re:The Myth of AI – Jaron Lanier by vinceguidry on Wednesday January 06, 2016 @12:11PM (#10851738)
> We have absolutely no idea what consciousness is or how it works.

We know what it is, we just don't know how it works.

Self-awareness comes in stages. We generally move through those stages throughout our daily lives. Animals display wide variations in qualities of self-awareness as well. So we can define consciousness as the sum of all of these qualities.

What I think is going to happen is that we'll start separating out lots of aspects of consciousness and explore them in software, adding more and more of them in as the state of the art of hardware gets better. The consciousness algorithms will slowly, over time, shed complexity.

Eventually, well before we get to even human-level self-awareness, we'll run into hard physical limits and realize that biology is way more effective at making the sort of compact, yet incredibly complex evolved system than any design process could be.

Biology has an advantage we don't have, it does not have to understand what it is doing, and it works ceaselessly. It can simply try over and over again over millions of years.

I predict that getting machines to become truly self-aware will be more trouble than it's worth, and that then you'll be choosing levels of comparatively-lower self-awareness for each individual component of a software system as part of architecting it. In fact, we have that tradeoff now. Do I really need to pull out ML to write a shell script?

Re:The Myth of AI – Jaron Lanier by dennisgorelik on Wednesday January 06, 2016 @09:44PM (#10855621)
It is surprising how even after 50+ years of Moore law technology is still does not cope with biology. Single cell carries about 700MB of genetic information in its nucleus.

https://medium.com/precision-medicine/how-big-is-the-human-g...

It is orders of magnitude more dense than current technology.

Re:The Myth of AI – Jaron Lanier by simbilou on Wednesday January 06, 2016 @12:18PM (#10851802)
> That's a religious position. We have absolutely no idea what consciousness is or how it works.

Well we know that it exists, or at the very least seem to exists. And there is nothing in the physical word that we have understood so far that cannot be ultimatly expressed computationally—even if it’s usually not the most useful formalism. It is of course somewhat a leap of faith to then say _everything_ is computation (or can be expressed as computation), but I would argue that science has already made this leap in it’s infancy: The Book of Nature is written in the language of mathematics… Admittedly, modern philosophy of science is a bit more nuanced but the old the statement from Galileo basically stands. Now I agree that it’s not entirely rational, but it certainly isn’t “religious”.

So yeah, I think there are very good arguments to be made that conciousness can at least in principle emerge from some kind of computation. That computation could be a complete simulation of every atom in a brain or maybe some shortcut is possible it doesn’t matter for my point. That does not mean however that I think it can be made or will be made, since has you pointed out, we have no idea what we are talking about. I completely agree with Lanier in this, the tech industry should stop focusing on this nonsense.

Re:The Myth of AI – Jaron Lanier by AndrewKemendo on Wednesday January 06, 2016 @12:22PM (#10851829)
It is of course somewhat a leap of faith to then say _everything_ is computation (or can be expressed as computation)

You don't need any kind of leap of faith to actually start working on it. That's the wonderful thing about AI and AGI more broadly. You can actually go work on it, today. Are you going to solve it immediately? Of course not, but at least you can chip away at the problems.

Re:The Myth of AI – Jaron Lanier by TheOtherHobbes on Wednesday January 06, 2016 @04:09PM (#10853798)
> And there is nothing in the physical word that we have understood so far that cannot be ultimatly expressed computationally

Except for those physical phenomena that can only be modelled with a recursive form that doesn't start with coefficients defined with infinite precision - which unfortunately includes a lot of useful things like weather prediction, and even the classical mechanics of n-body systems.

How about telling me what tomorrow's Dow close is going to be? Or the oil price four years from now? Or which parts of the universe that electron over there passed through on its way to your monitor?

I think a lot of the responses here just make the point for Lanier:

"There's a whole other problem area that has to do with neuroscience, where if we pretend we understand things before we do, we do damage to science, not just because we raise expectations and then fail to meet them repeatedly, but because we confuse generations of young scientists."

Anyone who believes that consciousness is computable by definition, because it just has to be, has a dilettante-level view of the problem. Philosophers have been arguing about this for centuries, and on the whole they're not so certain about it.

So I'll repeat - we don't have adequate models for human or even animal personality, or for emotional responsiveness, or even for the self-aware perception of qualia, which is apparently one of the key things that defines consciousness.

Big data isn't a solution to this, any more than throwing the works of Shakespeare into a database and looking for clustering statistics about word proximity and sentence structure will get you a new Julius Caesar.

The reality is there are levels of intelligent behaviour - especially self-aware creative behaviour - that are completely opaque to any modelling technique available in current CS. I believe that anyone who thinks the problem is simple and just needs moar code and a faster processor thrown at it is expressing a faith-based position of hopeful belief, not a reality-based fact.

Shakespeare is Shakespeare because the writing isn't word salad. Shakespeare is interesting because of the unusual the density and richness of the experiences that are referred to - not just in the plot, but in the details of the metaphors in each sentence.

You need to model experience before you can recreate that, and you can't do that unless you have a deep understanding of what experience is. Big data gives you slightly more focussed monkeys and more typewriters, but it's in no way a solution to the basic modelling problem.

Re:The Myth of AI – Jaron Lanier by VLM on Wednesday January 06, 2016 @12:54PM (#10852128)
I'll take your analogy and run with it, to note the extreme focus on symbolism in programming.

Its all about the copy by reference vs copy by value, type systems, etc.

Now when the medieval mystics wanted to talk enlightenment values, politics, psychology, political science, but the local powers that be were not down with the cool new stuff, they got obscurely alchemical, astrological, or divinational to avoid having the local leaders figure out what they were saying and therefore separate head from body via guillotine. Long after both the topics, and schemes they used to obscure the topics, were no longer contemporary and relevant, we laugh at the ancient alchemists.

Come to think of it, don't the cool kids programmers laugh at BASIC, assembly, C++, perl? Yet another similarity. Ha ha those ancient mystics trying to write quicksort in BASIC, LOL.

Re:The Myth of AI – Jaron Lanier by qbrass on Wednesday January 06, 2016 @11:24AM (#10851358)
Self-driving cars will come and save humanity from driving related deaths and ensure a new era of prosperity as humans get to relax while self-driving cars do everything for them.
Re:The Myth of AI – Jaron Lanier by VLM on Wednesday January 06, 2016 @12:36PM (#10851969)
I did a poor job of providing examples of religious behavior, but as history shows people can believe and behave in a religious manner about most anything. Including self-driving cars.

I took a comparative world religion a long time ago; I can't easily find the handy behavior list we had to memorize. We were taught something vaguely Mandaville-ian, I don't know if that outlook is still current and cool, but at least you can find Mandaville's opinion in the wikipedia. The "aspects" section of the wikipedia religion article would be a good start.

Shared faith based belief in the inevitability of some rather objectively unrealistic future outcome, in this case "We're all gonna use self driving cars", critically based solely on faith. A mythology, using the "a story that is important for the group whether or not it is objectively or provably true" wiki definition, where progress has always happened in the past and therefore will always happen in the future, when reality is progress only comes from obtaining and using energy, and our petroleum is running out while our population expands, so you do the math on the realistic future trajectory of progress, furthermore there's no question in the religion about the narrative that progress necessarily automatically equals self driving cars, instead of "busses and trains and subways" or not commuting at all or whatever. There's a fair amount of superstition involved aka everything the general public or journalists say about anything technical, legal, or economic WRT the topic of self driving cars, they're just making unreasoned noises at each other. Its always accompanied by extensive and exhaustive ethical baggage, in this case about preferred lifestyles, the value of work, urban planning, as if the self driving car is actually important relative to VPN deployment or subsidized public transit or star trek style transporters, yet despite its irrelevance there's a great pile of associated ethical baggage and related extreme judgment. There's a strong self policing society, where certain aspects of the religion are strongly amplified via endless ceremonial repetition, and the not-devout-enough are tossed out or punished via ridicule or otherwise.

Note that you can discuss self driving cars in non-religious ways, much as you can scientifically study aspects of a religion. That doesn't mean the adherents are not thinking/believing, communicating, and organizing in a religious style or manner, that just means you looked at them from a non-religious perspective.

Some historical religions being a little hard to believe in the modern scientific world, doesn't mean all religions are false; quite possibly we will have self driving cars in the future. That future success won't be a disproof of past observational behavior, under the "walks like a duck, quacks like a duck, it IS a duck" doctrine of what is a religion. Praying to each other for deliverance in the form of a self driving car won't bring it here any faster, but if it does somehow arrive anyway, the physical manifestation of that idol/miracle doesn't prove the praying was the cause of the effect/idol/miracle.

Re:The Myth of AI – Jaron Lanier by AndrewKemendo on Wednesday January 06, 2016 @02:03PM (#10852745)
the physical manifestation of that idol/miracle doesn't prove the praying was the cause of the effect/idol/miracle

Interestingly enough, if enough people "pray" in the form of demanding these products through the marketplace, then yes actually it can cause an actual product to be developed. See: Oculus. It might be expensive but damnit there it is. So I think this actually weakens your argument.

I can't in my mind compare techno-optimism with religious behavior it's way too different.

The self driving car as a technology actually physically works. Sebastian Thrun proved that back in 2001 and google has been proving it over the last 5 years.

There is no possible way to prove - or disprove for that matter - that your Prayer had anything to do with your uncle/grandmother/wife's cancer going into remission.

Those are infinitely different things.

I think you are turning some people's hyperbolic exuberance into something more than it is. Just like people say "everyone loves X" or "everyone is going to want to X."

Re:The Myth of AI – Jaron Lanier by language on Wednesday January 06, 2016 @12:34PM (#10851955)
With regard to "vestigial monotheism": I think what you're describing is just a property of discourses in general, in that they're all built around bodies of knowledge which have fixed points [assumptions/truths/whatever], and that participants act in such a way as to keep those things stable.

Definitely seen things like this before, very poignant. Not to knock on Kurzweil and friends too hard [lots of interesting stuff in both spaces obviously], but I've had this vibe from talking with self-described Singularitarians as well (there's some overlap between those circles).

Re:The Myth of AI – Jaron Lanier by ArkyBeagle on Wednesday January 06, 2016 @11:59PM (#10856083)
It is kind of religion - sort of classical American Yankee machine-worship. Even the Puritans/Pilgrims thought that organization - essentially a technology - could save them from the swamp gases of British life.

And I can't say it didn't work, but I'm probably a fanatic at this religion anyway.

I think Mark Twain more or less says this outright in "Connecticut Yankee." Speaking of Twain, he grievously injured himself financially chasing the dream of a typesetting machine far ahead of its time.

If it's religion, it's religion that (mostly) works. <insert Panglossian diatribe here>

Re:The Myth of AI – Jaron Lanier by api on Wednesday January 06, 2016 @12:11PM (#10851735)
Religion and ideological herd behaviors are a core part of human nature. You can't expect these to magically vanish just because people have abandoned belief in supernatural deities and magic. Those are just one form of it. Religion is a social phenomenon and cognitive style that is independent from the subject matter.

That being said: the techno-critics also display these behaviors as do the new agers, "naturalists," and others with which they tend to travel. Head over to one of those circles or boards and start arguing the futurist line and see what it gets you.

Both sides have valid points.

Re:The Myth of AI – Jaron Lanier by randallsquared on Wednesday January 06, 2016 @10:38AM (#10851032)
While I don't agree with Lanier, being an AI skeptic isn't quite the same as saying that self driving cars can't happen: we don't yet have a good understanding of either consciousness or tractable general optimization algorithms, so it's still possible that there's something that will make it impossible. Unlikely, perhaps, but possible.

Also, your sarcasm about the self driving car might have been too subtle. ;)

Re:The Myth of AI – Jaron Lanier by ghaff on Wednesday January 06, 2016 @12:01PM (#10851663)
What you see with respect to self-driving cars IMO is a lot of people who SO want them to exist that they assign timelines that are so optimistic they'd be rolling on the floor laughing if applied to projects they had direct familiarity with. Of course, it's also fueled by a lot of hype and hand-wavy statements about exponential growth and the like.

Like many other aspects of AI, I'm not sure just about anyone thinks we need to have strong AI to make self-driving cars work under the vast majority of conditions. We "just" need to create automation that can handle a reasonable number of corner cases without flipping out or grinding to a halt. But that's a debate about engineering timelines not something more philosophical.

Re:The Myth of AI – Jaron Lanier by randallsquared on Wednesday January 06, 2016 @12:40PM (#10852002)
I'm a little confused. We have self-driving cars. Of course there are still corner cases and bugs and regulatory issues, but cars have driven for fair distances on existing roads without a human driving. I had assumed, therefore, that the point @VLM was making was that, like self-driving cars, we already have AI by many of the standards of earlier AI researchers (the "when it works, it's not called AI any longer" argument).
Re:The Myth of AI – Jaron Lanier by ghaff on Wednesday January 06, 2016 @12:47PM (#10852069)
I was reading "self-driving cars" as "Level 4 autonomy" under general conditions. Which IMO is many decades away.

But we're in violent agreement about the "when it works, it's not called AI any longer" argument.

Re:The Myth of AI – Jaron Lanier by Jach on Wednesday January 06, 2016 @11:13AM (#10851277)