But yeah, without any way of determining which thing such a future AI would prefer, the AI's "influence" on the past would be greatly reduced.
Only reason I see it happening then is if it happens to be simulating the past anyway, and maybe just does the thing for the people who already thought of it, and then that really only makes sense to happen if it was created as a result of those people. Otherwise there probably wouldn't be much point, so it seems rather unlikely.
I mean, what's the chance that someone attempting to bring it about would significantly increase the chances of it happening? With our level of tech, and with the small number of people convinced by such an argument.
Yeah, good point, I hadn't thought on that detail much before.
I contend that the AI would need to give us such a sign. If it did not, how are we feeble humans to know whether we're contributing to the correct cause? Maybe donating my life's savings to SI is actually detrimental, and I should be giving my money to DARPA instead!
So therefore either we're un-blackmailable, because we have no clue what we're doing anyway, or we need a sign from the great AI as to what to do.
Re: Newcomb; yes, but I see a different utilitarian solution when the values change. Say the values are $10m and $100m. Well, then $10m is the obvious choice. Both are life-changing amounts of money; more than I would know what to do with. I'd much rather not risk throwing away a life-changing amount.
If you get the numbers right, you can get closer to the intention of the paradox. Something like $100k and $10m; I'd really hate to lose the $100k, but it would not be life-changing like the $10m.