News for hackers. Stuff that matters.
Re:The Last Invention of Man by andrewla on Friday October 06, 2017 @05:43PM (#15420699)
I think it’s a little absurd that we would make the leap to machine super-intelligence without going through the step of machine low or medium intelligence.

If I could write, I’d write a companion story to this called “Omega-1, the artificial super intelligence that wasn’t really that good at things”. They try to have it write mini-AIs to solve MT problems, but it’s slow and inaccurate. They have it write and produce TV shows, but they’re bland and poorly received. They have it make video games but the controls make no sense and they aren’t fun. They have it make trading strategies and it loses all the money.

They ask it to build a smarter version of itself, but it sees no way forward that’s fundamentally better because it lacks comprehension of what it would mean to be “smarter” - it can add more memory or more computing power, but without changing the way that it stores and indexes the information, it can’t make something that can solve problems that it can’t.

Eventually Omega team gives up, publishes the results, but doesn’t have the heart to shut down Omega-1, a machine that passes the Turing Test but isn’t really that good at things.

Re:The Last Invention of Man by thaumaturgy on Friday October 06, 2017 @06:10PM (#15420837)
Yeah. Maciej wrote a pretty good piece rebutting AI alarmism and kind of alludes to that as one of several points.

http://idlewords.com/talks/superintelligence.htm

> With no way to define intelligence (except just pointing to ourselves), we don't even know if it's a quantity that can be maximized. For all we know, human-level intelligence could be a tradeoff. Maybe any entity significantly smarter than a human being would be crippled by existential despair, or spend all its time in Buddha-like contemplation.

and

> But the hard takeoff scenario requires that there be a feature of the AI algorithm that can be repeatedly optimized to make the AI better at self-improvement.

And so on. It's a good read.

Re:The Last Invention of Man by apsec112 on Friday October 06, 2017 @07:37PM (#15421265)
Rebuttal to that essay by the Machine Intelligence Research Institute (AI safety group):

https://intelligence.org/2017/01/13/response-to-ceglowski-on...

Re:The Last Invention of Man by crooked-v on Friday October 06, 2017 @08:00PM (#15421360)
That write-up seems to mainly try to shrug off the "what is intelligence" issue by virtue of "maximization", while ignoring that when dealing with human intelligence, selecting for maximization of a single field gets you autistic savants, not Lex Luthor-grade manipulative geniuses.
Re:The Last Invention of Man by SiVal on Friday October 06, 2017 @09:15PM (#15421663)
My own rebuttal to that unimpressive Maciej argument is that he merely listed many ways in which something that seemed as though it ought to work didn't work out at all as expected, so we can safely assume that something as challenging as "superintelligence" won't work out either, and is therefore no danger.

But the argument that there are countless ways for plans to go awry and instead cause something unexpected to happen is hardly reassuring evidence that there is, therefore, "nothing to worry about".

Re:The Last Invention of Man by spynxic on Friday October 06, 2017 @07:27PM (#15421213)
> With no way to define intelligence (except just pointing to ourselves..

..and anything else considered to be alive)

Not to make an attempt at arguing whether or not machines can be intelligent but machines are not submerged in an environment that effects them, giving them no merit to be intelligent about anything; and even if they were they still have no ability to influence the environment. (Sensors for example)

It seems many that speak on AI have completely disregarded coming up with a reasonable implementation of intelligence in favor of pondering its potential uses. (Typically in ideals central to humans)

Re:The Last Invention of Man by xarball on Sunday October 08, 2017 @02:21AM (#15427187)
> machines are not submerged in an environment that effects them, giving them no merit to be intelligent about anything

You're alluding to the problem of having to 'raise' a super-intelligence with a vested interest in the continuity of its physical existence. I think I agree with where you're going with that.

Our understanding of 'self' or 'self preservation' likely comes from a long history of suffering, pain, and failure brought on by trials at understanding the self from evolution's attempts at imbuing the information for longevity into cognition. Mother nature has had years of exponential growth with an inconceivable number of branches attempting to understand the 'self', that lead to failure, all necessary because the idea of 'self preservation' or 'grounding one's sense of ego into a concept of physical sensory data' is very obscure to precisely define, let along to define such that it can be TESTED between generations.

Based on how much of a headstart evolution has on that refining process, and how bad humans are at implementing that or even exposing sensory information about that that plug directly into concepts that necessarily affect an artificial's sense of self, I'm willing to bet that we are completely ill-equipped to grow any real intelligence that's truly grounded in the real world, until it can make all the same failed attempts at living longer or evolving + testing those attempts, with that exact same directional goal of 'self-preservation' which humans have.

Self-preservation is really the only thing you can evolve to, to guide a physical evolution. I'm not aware of any other physical goals that could lead to the same human-like intelligence or human-like neural models.

Most creatures seem to hit the exact same point. An agent-based survival of the fittest with generational physical trials that punish poor understandings of the self's physical manifestation, trying to optimize for its longevity.

Anything that doesn't hit that kind of model is strictly not aware of the manifestation of its ego, as we know it (e.g. plants, bacteria, etc), and I don't think the physical sense of self can evolve into grounded concepts without allowing for generational failure on the understanding of self, due to incorrect concepts of 'physical self' between competing generations.

TL;DR - The concept of physical self is an evolved trait only if there is a goal tied to it, it cannot arrive without the concept of 'self existence' being threatened or when not allowed to fail.

Re:The Last Invention of Man by sampo on Friday October 06, 2017 @08:03PM (#15421381)
> Maybe any entity significantly smarter than a human being would be crippled by existential despair

This was addressed in a Spider-Man story where a villain enhances his intelligence, then first becomes a very successful crime boss, then writes novels and music, and then falls into existential despair.

https://en.wikipedia.org/wiki/Flowers_for_Rhino

Re:The Last Invention of Man by kobeya on Friday October 06, 2017 @08:19PM (#15421471)
I think it was done first, and probably better, in Flowers for Algernon.
Re:The Last Invention of Man by kwhitefoot on Saturday October 07, 2017 @03:16AM (#15422694)
Just in case anyone her is unfamiliar with the story here is a link to the Wikipedia synopsis: https://en.wikipedia.org/wiki/Flowers_for_Algernon.

Daniel Keyes should be more famous and the story more widely known.

And now for something weird: http://www.unboundworlds.com/2008/10/challenged-and-banned-f...

How anyone could think of banning one of the most affecting and humane books ever written is beyond me.

Re:The Last Invention of Man by perl4ever on Friday October 06, 2017 @09:35PM (#15421740)
"it is a pastiche of the science fiction story Flowers for Algernon."
Re:The Last Invention of Man by doktrin on Saturday October 07, 2017 @07:22AM (#15423346)
> pretty good piece rebutting AI alarmism

I only read the text, but I was under the impression it was mainly meant to be a funny Wat-style talk and not a meaningful rebuttal.

If, in fact, this piece is intended as a serious rebuttal then I honestly think it fails. The overwhelming majority of the content is either a funny whataboutism (e.g. Gilligan's Island, pot smoking roommate, the Great Emu War, etc.), or jokes about what could be loosely termed the "AI community".

AI 'alarmism', as I understand it, is not predicated on the idea that super-AI is a certainty - merely that it's a possibility. No point raised in this piece gives me a reason to believe that AGI (or whatever) is either impossible or so unlikely that it can be disregarded.

Re:The Last Invention of Man by saulrh on Friday October 06, 2017 @07:59PM (#15421356)
You're betting everything on vacuous what-ifs. I mean, let's turn this around:

With no way to define intelligence (except by pointing to ourselves), we don't even know if we're good at it. For all we know human intelligence is a dead end. Maybe any entity that is not a product of evolution, free of genetic detritus that becomes maladaptive around advanced technology, will outstrip human civilization like human civilization outstripped evolution.

I don't doubt the book is full of entertaining soundbytes, but given these two quotes I am unconvinced that it is any more than entertainment.

Re:The Last Invention of Man by westoncb on Friday October 06, 2017 @07:12PM (#15421132)
Similarly, I wonder if it's something where adding to it doesn't actually do much, like all the variations you see done on Turing Machines (adding more tapes or 2d tapes etc.) which seem like they would increase its capacity somehow, but it turns out they have the same power as the original Turing Machine. Or like the arithmetic of infinite numbers, e.g. aleph-0 + aleph-0 = aleph-0.
Re:The Last Invention of Man by felipemnoa on Friday October 06, 2017 @09:02PM (#15421622)
>>> With no way to define intelligence (except just pointing to ourselves), we don't even know if it's a quantity that can be maximized. For all we know, human-level intelligence could be a tradeoff. Maybe any entity significantly smarter than a human being would be crippled by existential despair, or spend all its time in Buddha-like contemplation.<<

The only intelligence that I believe is important is problem solving. If an AI is as capable as humans at problem solving then that is all you need.

>>crippled by existential despair

Sounds to me that you are talking about feelings. I really do not see the purpose of purposely giving the AI feelings. A strong AI should not have any feelings just like your google maps application does not have any feelings when giving you directions.

A general AI will simply be a general problem solver at the same level of a human but with a virtual infinite capacity to scale its computational abilities.

You could definitely give it free will so that it decides its own goals but why would anybody do something like that unless they want it to turn on them?

It would be like somebody launching a nuclear ballistic missile and letting it decide which city to land in. It is possible to program that type of free will but why? It could decide to destroy your city.

Re:The Last Invention of Man by perl4ever on Friday October 06, 2017 @09:30PM (#15421718)
Feelings regulate actions. People don't blindly follow rules that would lead to disaster because of more fundamental regulatory mechanisms such as feelings. In order to make an AI that is as flexible as a smart human, it can't be brittle, and therefore will need ways to break out of logical traps. That is what a general problem solver must be like, inherently.
Re:The Last Invention of Man by felipemnoa on Saturday October 07, 2017 @11:23AM (#15424099)
Feelings can be irrational so I really do not think you want to employ a mechanism that uses anything resembling feelings. If it can be irrational that means it can also go crazy and decide to exterminate all human kind.

>>In order to make an AI that is as flexible as a smart human, it can't be brittle, and therefore will need ways to break out of logical traps.

I think this can be achieved without having to give the machines flaws we have just like we did not need to create machines with flapping wings to fly.

Remember that the only goal of a living organism is to replicate and have its DNA live another generation. That's it. Feelings helps us achieve that goal I suppose. An AI does not need to have that goal. If anybody were to give it that goal it would only be a matter of time before it decided that humans were standing in the way of its goal.

Re:The Last Invention of Man by therein on Saturday October 07, 2017 @01:12AM (#15422374)
Very interesting point about using feelings and intuition to break out of logical traps. Does that also mean this AI will be fallible or perhaps even stubborn? Will it ever refuse to accept that it is wrong about a result it produces?
Re:The Last Invention of Man by CrystalGamma on Saturday October 07, 2017 @03:21PM (#15425247)
Where did you get that URL? It's not linked on http://idlewords.com/talks/ …
Re:The Last Invention of Man by 5_minutes on Friday October 06, 2017 @06:32PM (#15420953)
Thanks for this
Re:The Last Invention of Man by rictic on Friday October 06, 2017 @09:06PM (#15421635)
That's pretty likely, but not certain. Go-playing-AI went from "talentless amateur" to "better than every living human" in about a year.

Sometimes things progress rapidly, and some kinds of intelligence could recursively self-improve.

Re:The Last Invention of Man by edanm on Saturday October 07, 2017 @04:03AM (#15422845)
"I think it’s a little absurd that we would make the leap to machine super-intelligence without going through the step of machine low or medium intelligence."

Why? The leap from calculators that were worse than humans, to calculators that were much, much better than humans, didn't take too long. No one knows how fast the leap will be from what we currently have, to general intelligence, to super intelligence.

I don't think anyone in the AI safety movement will tell you that we know for sure the leap will be fast. I think they'd say we simply don't know how fast it will be, but the stakes are pretty high so it's worth preparing for.

Re:The Last Invention of Man by dTal on Saturday October 07, 2017 @07:47AM (#15423401)
I feel this lacks imagination. In actual fact, a computerised person of average intelligence would be an incredible piece of technology and could easily lead to an AI singularity, given enough funding - which it would probably receive, because holy shit. What happens when you set thousands of average people on a problem, and give them thousands of simulated years to work on it? When you can spin up a research team as easily as an AWS instance?

>but without changing the way that it stores and indexes the information, it can’t make something that can solve problems that it can’t.

And yet people routinely grow up to be smarter than their parents, with no special effort on their parents' part. All you need to make a smarter version of yourself is the ability to make inaccurate copies and dumb luck. What, besides ethics and money, stops you from running an evolutionary process that selects for intelligence at a vastly higher rate than real time?

Re:The Last Invention of Man by KidComputer on Saturday October 07, 2017 @09:08AM (#15423627)
> What, besides ethics and money, stops you from running an evolutionary process that selects for intelligence at a vastly higher rate than real time

You can already do that today. So why don't we have a superintelligence? I can train a NTM to learn counting and give me the next natural number in a single step. I can't train it to give me the next prime in a single step, why?. Computability and complexity, both conveniently forgotten in the discussion of superintelligence.

Re:The Last Invention of Man by eggpy on Friday October 06, 2017 @07:28PM (#15421215)
But this part isn't science fiction, we already have AIs that do this. See AIVA for pretty good musical composition[0], new sites are using AI to write and compress news stories[1], and screeplays are being written via machine learning[2].

Obviously these are still in their infancy, but doesn't this perfectly encapsulate your point? There's not really an interesting story or thought piece there, because people are working on these in all sorts of fields all over the world, today.

If we can build AIs that create "good" art, is it much of a stretch to teach a super AI what "better" means? By default a super intelligence doesn't have to start out as super intelligence, just have the capability to improve.

[0] https://www.youtube.com/watch?v=Ebnd03x137A

[1] https://www.wired.com/2017/02/robots-wrote-this-story/

[2] https://www.youtube.com/watch?v=LY7x2Ihqjmc

Re:The Last Invention of Man by daveguy on Friday October 06, 2017 @05:59PM (#15420786)
I think the storyline should include that no one wanted to shut it down and despite the fact that it broke their hearts they couldn't justify spending hundreds of thousands of dollars on the electricity that it cost to keep Omega-1 running. It was simply unable to sustain itself like 90% of startups.
Re:The Last Invention of Man by ndkskshfr on Friday October 06, 2017 @06:39PM (#15420975)
> I think it’s a little absurd that we would make the leap to machine super-intelligence without going through the step of machine low or medium intelligence.

I’m not so sure about this, which is not so say i’m arguing FOR the leap. But, intuitively, there is an absolute massive leap in terms of perceived intelligence among humans, let alone with chimps. It seems some of the magic sauce for passing a turing test is, evolutionarily speaking, rather small.

Re:The Last Invention of Man by dmix on Friday October 06, 2017 @05:58PM (#15420783)
> I think it’s a little absurd that we would make the leap to machine super-intelligence without going through the step of machine low or medium intelligence.

I have the same feelings about socialism...

If we can draw any lesson from the history of our biological existence from which we expect machine intelligence to spring from it will likely be a series of smaller pieces that will join together to form like Voltron. Combined will be the best of our ability.

I highly doubt there will ever be some centralized system/organization that has 'figured out' intelligence. Although that doesn't have to necessarily mean we will forever be limited to niche/vertical intelligent systems in the long run either.

Re:The Last Invention of Man by PoachedSausage on Friday October 06, 2017 @07:07PM (#15421099)
Perhaps we will find that when we give it control of weapons, despite its inadequacies in other areas, it will excel at killing.
Re:The Last Invention of Man by kevin_thibedeau on Friday October 06, 2017 @06:50PM (#15421027)
The first intelligent AIs will be weak and have limited influence on real world human affairs. The worst we have now is poorly supervised automatic trading systems that can shift massive amounts of money through electronic markets.

However, once AIs can design and implement their own successors all bets are off.

I would also be concerned about the development of non-theraputic neural implants and the ability of an AI to surreptitiously influence human behavior through them. Thankfully that is likely to to be further off than when we start bumping into the AI singularity.

Re:The Last Invention of Man by pdimitar on Friday October 06, 2017 @07:26PM (#15421201)
In the absolutely epic space opera "Hyperion" (and its sequel "Endymion") the author Dan Simmons theorizes -- through the plot, not through a monologue! -- that AI creatures would offer us tech that we could never invent -- impossibly minituarized personal computers (the size of a discrete and thin silver bracelet, an earring or even smaller implants under the skin, and which are basic AI by themselves), and real-time worm-holes through which humanity finally achieves inter-stellar civilization status.

However, the author is very realistic (pessimistic?) about such an AI community -- actually there's billions of them living in a place humanity can't ever reach or even guess where it is, with internal politics as foreign for the people as our societal dynamics are a mystery to the bacteria around us. Anyhow -- the tech they give to humanity is heavily backdoored and this becomes very apparent and dangerous as the plot progresses.

I'd recommend those books to anybody with a heart and some curiosity about a seemingly optimistic future (which is actually pretty dark).

Be warned though: if you decide to read them, the first ~120 pages of the first book are DARK AND HEAVY. However, enduring it was the best sacrifice of several hours of patience I've ever done in my life. I just couldn't stop reading before I finished all 4 books.

Re:The Last Invention of Man by ericfrenkiel on Friday October 06, 2017 @08:28PM (#15421509)
Echo the sentiments about "Hyperion" - one of my favorite sci-fi sagas of all time. The author is incredible because his prose are mellifluous regardless and across genre - sci-fi, drama, horror, and historical fiction are all in Dan Simmons' corpus of work.
Re:The Last Invention of Man by crooked-v on Friday October 06, 2017 @08:02PM (#15421374)
That kind of falls apart in this context because a set of mysterious unreachable AIs bearing gifts are a de facto foreign nation and would be treated appropriately.
Re:The Last Invention of Man by pdimitar on Friday October 06, 2017 @08:11PM (#15421422)
Well, I agree that I got a bit off-topic. The general analogy is still there but you're right about the details.

That being said, the author handled the problem you outlined with the simple premise of -- that humanity wouldn't be able to even survive as a whole if it delayed the decision how to treat the AI nation.

Without giving spoilers, the fledgling AI nation planned and executed things so well that humanity either had to embrace them and let its core be governed by black-box (and backdoored) tech like teleporters and miniscule personal assistants, or face literal extinction. Needless to say, the humans made a Faustian deal.

Co-evolution of species with wildly different values is the core of the conflict in these books, and even as a fanboy I dare saying it has great value outside of a sci-fi / space opera book.

Re:The Last Invention of Man by gertef on Friday October 06, 2017 @08:19PM (#15421465)
We also have virus botnets