Perhaps you should have finished reading. What you suggest the author missed is essentially the core thesis of the piece.
The remainder of the comment indicates at least 1/3.
The comment is trying to mirror my criticism - "I read the first part, found an error, and stopped". My response is trying to highlight that, in fact, their response does not mirror mine because I had actual errors that I pointed out that motivated me to stop reading, whereas that comment did not (apparently).
In other words, if I had actually made substantive errors in my first sentence or so, it might make sense to stop reading. I'd have already demonstrated that my thinking wasn't very clear. If that was the case though, then it would be an invalid criticism of my reasoning (read a bit, saw an error, stopped) because that comment author would be following the same paradigm. On the other hand, if I didn't actually make any substantive errors in the first sentence or so my post, then the criticism is still invalid, because, while I actually pointed out substantive errors in the OP, this comment doesn't point out substantive errors in my comment.
(like I said, it was mostly in jest, so don't take it too seriously, please)
I believe he just used Tesla as an example of corporation not as something new or special.
> the point of the paperclip thought experiment is, broadly, that an AI's interests might not be aligned with our own and that misalignment may cause serious problems.
But (some) corporations clearly do share this aspect to some extent. They maximize profit while environmental concerns are not prioritized.
And he pointed out later that corporations are slightly more complex than just maximizing 'one' thing. But a range of things and most importantly profit.
This is okay though because corporations are the same class and kind of intelligence as the rest of humanity and so they can be (somewhat) predicted and constrained by our laws and norms.
Because corporations are the same class of intelligence, they can't outsmart everyone. We can catch them when they do wrong and punish them and use laws and courts to control them.
Because corporations are the same kind of intelligence they (usually) aren't going to do things we might consider insane or sociopathic. For example, if corporation X realized they might make a profit by taking out life insurance on workers and then getting them to do very risky things, X Corp probably wouldn't do it and if they did people would leak it and laws would be made to stop it.
The paperclip thought experiment is an illustration of a different kind of intelligence that wouldn't be constrained by human morality or norms and would surpass human intelligence and not be constrained by laws or force either.
While the corporation might not possess superintelligence, trying to thwart it it when its actions are harmful to society is like having a crazy neighbor. You just want to relax when you're home from work; you can't dedicate all your free time and attention to constantly monitoring and pushing back against someone who is on a crusade to achieve a petty, limited goal and seemingly has no other interests to occupy their time.
Tesla is just past 1/3 down, the page has almost 800 comments with no pagination.
Your consciousness uploaded and then downloaded into a synthetic human like body isn't ever going to be you. As the man said “transhumanism is a warmed-over Christian heresy”.
The Ship of Theseus is the warmed-over Christian heresy. A brain is an organism with a symbiotic tight coupling to a meat bag, and the brain itself a similar cooperation by accident.
An organism can develop that builds new tight "synthetic" couplings; it isn't sci-fi. It's partially released today. It's just that we are so normalized to the interfaces of smart phones, keyboards, monitors, steering wheels, and so on that we didn't notice it's arrival.
More exotic, more tightly coupled interfaces also exist today (there exist direct-to-brain interfaces that provide a new sensory feedback loop that you brain can adapt to), they're just not as competitive currently with thousands of years of neural optimization of hands/eyes/nose/touch.
Why would you want to do this? Who knows. Why do we want to do anything? Maybe a person wants to be part machine as a lizard-brain driven reaction to a low-oxygen office environment that sees no viable way out.
If I replace a single one of your neurons with a mechanical replacement, are you still you? What if I replace half? All?
If you walk across a room, are you still you? What if I break you down, and reassemble you at the other side of the room, using the same atoms? What if I only use half the original atoms? What if I freeze you, move you across the room, and then dethaw you?
As far as I (crudely) understand it, we don't know what exactly the cause is, other than it's tied extremely tightly to the brain and seems to be an accident of evolution/an illusion.
My guess is that as long as the neurons keep firing continuously and the illusion is unbroken, the "you" that is you right now will remain you. So you may well be able to go full ship-of-theseus and piece-by-piece completely replace the physical layer without breaking the consciousness, but if you stop it completely with an upload/download* , continuity is broken and the new "you" will be an exact duplicate while the "you" right now will cease to be. Because it's an exact duplicate there would be no way to confirm this, though - the new "you" would think it had continuity due to how complete their memories are.
* I'm thinking of switching to a robotic brain all at once here, instead of piece-by-piece. I'm not going to touch a transporter as suggested in your comment, as that probably depends on technical details of the transporter for what exactly is going on.
I think that conventional notions of identity are probably incoherent, and mean you die from second to second anyway.
I'm not likely to have the same reactions after lunch that I had before lunch, or consistency from yesterday to today.
Can it be done in a way that doesn't imply the death of the subject?
The key word is replacement. Can this replacement vote or inherit property? What happens if there is a clerical error and they make two replacements. Which one is the real you?
Incidentally, this is why I can't take Roko's Basilisk at all seriously - a future simulation of me is not me. Well, it;s one reason anyway.
One was the idea that the author could rule out the singularity because he wasn't aware of progress toward self-motivated AI that would be something like our own intelligence. This seems like a limited view to me because we wouldn't need self-motivation at all in AI to hit the singularity. Humans can supply the self motivation.
Suppose I'm using GPT-4 or GPT-44 trained on the corpus from sci-hub and it recommends experiments to me, or explains physics to me, etc. I could be the self-motivating part and the AI could be the intelligence part, and it seems we'd still hit the technological singularity.
Another problem I had was when the author characterized Elon Musk's "obsession" with the paperclip maximizer and described Tesla as a battery maximizer. It seems like the author kind of misses the point of the paperclip thought experiment, which is, broadly, that an AI's interests might not be aligned with our own and that misalignment may cause serious problems.
Tesla is clearly not a battery maximizer and it is clearly not a different class of intelligence from the humans and corporations existing today (though it may be towards the top of that class). Neither of those things would necessarily be true of an AI.
Given the position of my scrollbar it seems I was only starting to read this piece, but already finding what I think are significant problems as the author sets up the argument, I'm hesitant to spend more reading.