I contend that the AI would need to give us such a sign. If it did not, how are we feeble humans to know whether we're contributing to the correct cause? Maybe donating my life's savings to SI is actually detrimental, and I should be giving my money to DARPA instead!
So therefore either we're un-blackmailable, because we have no clue what we're doing anyway, or we need a sign from the great AI as to what to do.
Re: Newcomb; yes, but I see a different utilitarian solution when the values change. Say the values are $10m and $100m. Well, then $10m is the obvious choice. Both are life-changing amounts of money; more than I would know what to do with. I'd much rather not risk throwing away a life-changing amount.
If you get the numbers right, you can get closer to the intention of the paradox. Something like $100k and $10m; I'd really hate to lose the $100k, but it would not be life-changing like the $10m.
But yeah, without any way of determining which thing such a future AI would prefer, the AI's "influence" on the past would be greatly reduced.
Only reason I see it happening then is if it happens to be simulating the past anyway, and maybe just does the thing for the people who already thought of it, and then that really only makes sense to happen if it was created as a result of those people. Otherwise there probably wouldn't be much point, so it seems rather unlikely.
I mean, what's the chance that someone attempting to bring it about would significantly increase the chances of it happening? With our level of tech, and with the small number of people convinced by such an argument.
Yeah, good point, I hadn't thought on that detail much before.
If the AI gave a sign it would be giving the sign to the simulation, which could not actually change what happened in the AI's past. The AI does not literally cause things retroactively, just based on predictions of its future actions. Giving a sign to the simulation would be useless.
I do not believe that such a machine will be created, I just wanted to clarify some things.
The hypothetical machine would supposedly motivate real people now due to their inability to determine whether they are in the simulation or not. If they cannot tell, then the idea is that the person in the present and the simulation in the future would take the same actions (until the simulation differed), and as such would both have the same incentives (they would in essence be the same person), and so to avoid a <some probability> of being tortured, would do the thing.
I reiterate that I do not believe that such an AI will come to be (it should be preventable easily enough, even if it is possible) and in any case I do not believe that enough information to create an accurate simulation of me would be available by the time that such a machine would be created, even if such a machine is possible. (I think quantum randomness and uncertainty principles are sufficient to prevent obtaining enough information about me to create a sufficiently accurate simulation of the world and me)
Also I am not entirely sure that I should count a simulation of myself as being a person, let alone consider it to be me.
Also I am religious and don't believe that God would want people to act in accordance with such a machine/I believe it would be wrong to worship such a machine.
(though I expect that for the majority of the people here my other arguments would be more convincing.)
Also ok yeah $1000 might not be the best value for the example, the specific value isn't the point.
(but yeah I also generally would one box it)