In case you are other readers are interested, what you describe is an old philosophical debate spanning mostly morals but as you point out, also the pursuit of happiness and other areas. It is the opposition between consequentialism (=what is important is the result of your actions) versus deontologism (=what is important is what you are doing, not the result).

I think that as machines become smarter and smarter, it is good that we dive a bit more in our philosophical world and reach for what makes us human.

It took me a long time to understand that if you remove the pure rational intellect, something precious still remains: the driving force that shapes our morals. There is not rational reason to like survival. To love our children, to do what we call good. The resons for that are moral and are, I believe, what defines us as human at the core.

Ian Banks was spot on in calling a future civilization a "Culture". Because when machines take over the productive work, defining our values is going to be our full time activity. Machines may participate, but why would they add anything we consider valuable to the core?

That's going to be an interesting transition and I am happy that I'll probably live to see it!

> There is not rational reason to like survival. To love our children, to do what we call good.

Of course there is and very well described in evolutionary science

This is an is/ought confusion. You can explain why something is that way, but you can't explain why it must be that way.

You can explain why our survival instinct evolved so it is tempting to jump to the conclusion that we ought to survive but it is a fallacy (called the naturalistic fallacy): saying that something being natural means it is good.

>I think that as machines become smarter and smarter, it is good that we dive a bit more in our philosophical world and reach for what makes us human.

That's a recurring theme in Ghost in the Shell, at least the anime series. That as the line between human and machine is blurred, the general trend is in the direction of homogeneity, but here and there you still see people doing "deontological" things, ostensibly as a grasp at uniqueness to preserve self.

An interesting thought occurs to me though, with the proliferation of various NN architectures, and the idea that such nets, if scaled and installed to power humanoid robots, would effectively learn different heuristics after training on chaotically different data, it's quite possible that machines will also gradually evolve individuality and something rather close to personality.

Perhaps individuality is an intrinsic, emergent property in any system of generally intelligent and learning agents. Like a 3+ body on steroids, millions, if not billions of dimensions.

Also interesting to consider that the emergence of uniqueness among a system of said agents seems to increase entropy, a presently unique to life property given that everything inanimate in the universe does exactly the opposite. The more I think of general AI, the more I have trouble distinguishing it from the only other sentient intelligence we know of.

On the other hand, insects are also kind of like NN powered robots, and they don't (necessarily?) have individuality. Especially not ones that form colonies, like ants or bees.

I don't know if we have an answer to the question of whether insects have personalities. But perhaps their intelligence is not general enough, they're not, as far as we know, sentient. Maybe that's the difference between an automaton and a soul

I don't think it is very hard to make a personality or an individuality appear. Feed GPT-2 a lot of opinnonated texts about itself and you will already see a bit of it emerge.

The difference is that this AI agent was built to beat humans. Horses were not built to run faster than humans.

You have to remember the specific comparison and the art that humans tried to partake in to master go.

Humans do not run to beat horses. We never compete. That would be silly.

Cars and humans also do not compete at the 100 meter dash.

If humans and robots compete at 100 meter dash than the olympics will similarly lose its luster.

> Humans do not run to beat horses. We never compete. That would be silly

I hate to be that guy, but:

https://en.m.wikipedia.org/wiki/Man_versus_Horse_Marathon

It is very silly indeed.

2004 2007 a human won. Wow.

The race's setup is horse-friendly as well: Shorter than a regular marathon by a few miles and on flat terrain. A longer distance on hillier terrain would be won by humans much more often.

There was an interesting article a couple of years back, positing that the years that humans won coincide with unusually warm weather. The takeaway was the humans have a superior cooling system for efficient long distance running, compared to most animal's superior sprinting capabilities.

I believe only humans and dogs hunt via stamina. You can walk a horse to death or simply chase an antelope until it overheats and dies.

'AI' is not an adversary. It is simply a tool created by hundreds of humans coming together.

It's like watching 1-v-100 boxing match. Of course the 100 are going to win.

Enjoyable sport has always been about ~similarly matched opponents. When we have DeepMind AI Go vs. MindDeep AI Go, that's when things get interesting.

I interpreted breathoften’s comment differently. As someone who’s played chess at a very high level, I feel like I understand people who play these games for the art behind it. It’s difficult to describe, but chess for me is beautiful because to win you must be patient, careful, understand what is and isn’t important, etc. playing chess is like learning virtues. An algorithm playing chess doesn’t care about any of that. I wonder if it’s similar to the difference between meditation and acid. One gives you time to understand and the other just gets you there without explanation.

About jazz improvisation: I knew people at the creative labs research center back in 2003 that told me a researcher once showed them a program that did just that. It was able to improvise « the way player x would » by just listening to it, and it would continue the impro in the same style.

Imagine a world where no human contribution matters because a different species already thought of it. No improvisation on any instrument can be novel because a machine already played it exactly like you would have. A world where your children look to a philosopher machine for guidance, because it is wiser, kinder, and deeper than you could ever be.

Human existence will still occur, however, our species's defining characteristic will be no more consequential than a meadowlark's greeting of the sun every morning. Beautiful things will occur, but they will be ghostly imitations of the creations of some other being. Humans will create nothing, except, possibly, more humans.

It's not necessarily about your success vs. the machines. It's about every human who will ever live in the future's impossibility of success vs. the machines that is depressing and inevitable.

> Imagine a world where no human contribution matters because a different species already thought of it

You are begging the question in in the original sense of the phrase "begging the question."

You start by stating your conclusion as an axiom! You state that "no human contribution matters because a different species already thought of it" as if that is necessarily true.

It isn't.

We choose what matters. I play Bach, badly. Bach already thought of that music, and many thousands, possibly millions of people have played the particular pieces I have worked on (The first suite for unaccompanied cello in Gmaj, the Prelude to the first Fuge in Cmaj from Book One of WTC).

Does my playing not matter?

I say it does. Furthermore, Bach's music can be encoded as a number. All numbers already exist. Bach did not create that number any more than I created the number that encodes this comment.

Does my comment not matter?

You find this general idea depressing, and so do many other people.

But it isn't "depressing" in an absolute sense. That's just a word we made up to describe a feeling many of us happen to have.

Your playing doesn't "matter" in the way Bach's original composition "matters." One of these things has been remembered for hundreds of years, and one of these things would never have been remembered unless you reinterpreted the work in some novel way that reached millions of people.

The fact that Bach's compositions can be encoded as a number doesn't mean that it wouldn't take a genius of Bach's level to deserialize that particular number, which could literally be deserialized into a representation of any arbitrary thing in existence with the correct algorithm, any less novel when it was created. The same holds true of your comment. Just because I declare 1001 represents a beautifully unique masterpiece, once it is decoded, does not make that masterpiece actually exist.

Bach was important. In a world with general AI no human will have the ability to be important in that way ever again. At first AI will create crude imitations of human art. Then it will create hundreds of billions of creative works that are more human than human. Then it will create artworks that surpass our ability to comprehend. I don't know about you, but the inability to do anything novel as a species, to learn anything new, is a terribly bleak possibility.

Does the fact that we were the ones who created them in the first place count? Why should we feel depressed at one of our own creations? Why be depressed by looking at a car just because it can move faster than us, while in actuality it was created for that purpose?

'AI' is not an adversary. It is simply a tool created by hundreds of humans coming together.

It's like watching 1-v-100 boxing match. Of course the 100 are going to win.

Enjoyable sport has always been about ~similarly matched opponents. When we have DeepMind AI Go vs. MindDeep AI Go, that's when things get interesting.

That's kind of what we already have though, given that these things are often trained by playing against themselves these days

Well said !!

> I will never play any instrument as well as a sequencer. I believe algorithms will make beautiful Jazz improvisations in real time, in my lifetime. But playing is still beautiful.

Having dug quite deep into algorithmic music generation of all sorts, as well as having studied Machine Learning as my uni masters, I still believe that you still need an actual artist to make a music generator program.

A machine simply isn't going to figure out "swing" if you don't tell it to. And swing is one of the easiest things. Yes if you look just at note generation, I think algorithms can go a long way. But the subtleties of timing and timbre, it can only imitate in context. Which definitely is good enough for many purposes, and I agree with your prediction that algorithms will be able to generate beautiful music, but I also think there will always remain an "edge" for the artist. If only to discover novel things that are also cool, and then working those out in order to fully express the coolness of that new thing.

I'm also thinking about all the evolutions the many genres of electronic music have gone through the past decades. New and novel "sounds" (or moods, or styles, etc) are still being create/discovered. It is that process, that I don't think we're there yet. Yes the algorithm can probably generate beautiful psytrance, lo-fi hiphop beats and if I'm generous probably eventually also really complex (and jazzy!) stuff like squarepusher.

But what I'm not seeing happening any time soon (barring any breaking general AI type of advances) is if you give the algorithm a TB-303 for the first time and see if it figures out acid house. Yes you can probably teach it the origins of neurofunk DnB (think 1999 Optical & Ed Rush's Wormhole album) and produce super awesome dance music. But I don't see how it could ever develop what happened to DnB beyond that. Wavetable synthesis didn't really exist that way back then, and the bass came from more classic synthesis like the Reese bass. Nowadays, what you can do with a wavetable synth VST like Serum, almost defines what modern DnB sounds like. That particular sound was evolved and shaped through the genre of drum'nbass and became part of it, a new style of synthesis, heavily facilitated through the particular UX controls of these synth plugins, which again caused the author of Serum to amplify by creating his vision of what this UX should be like, is almost like the birth of a new instrument, together with artists having to learn the correct style to "play" this instrument. Which has settled enough now that it is appearing in other new genres. Yet is also still developing. And that is just one genre of music I happen to be somewhat familiar with, I'm sure similar examples can be named in many other genres (for instance I don't know much about the history of dubstep).

Those evolutionary steps, invention of truly novel things, for the foreseeable future, I don't think AI is there yet and artists do still have an edge, even if it's a very thin one.

People need to experience a sense of competency and power as a part of feeling satisfaction with their lives. It is not the only route to satisfaction, but in my opinion it's one of the few roads to great satisfaction.

I agree with the sentiment that one can't be good at everything, and that there is a radically high-performance exemplar for anything, but for machines to dominate in almost every category of luxury endeavour -- yes I can see that as demoralizing. Even the useless things we spend our time on, we can be no good at that.