p.s. great story and the comments too! "the Rapture of the Nerds". priceless.
There's a lot of reason to recommend Cory Doctorow's "Walk Away". It's handling of exactly this - brain scan + sim - is very much one of them.
But yeah, it's too gratuitously bleak for me. I feel like that's a crutch, a failure of creativity.
I love it too, one of my favorite episodes of TV ever made. That being said, the ending wasn't all rosy. The bank of stored minds was pretty unsettling. The closest to a "good" ending I can recall was "Hang the DJ", the dating episode.
Humans and animals have: a stream of consciousness, deeply tied to the body and integration of numerous senses, a survival imperative, episodic memories, emotions for regulation, full autonomy, rapid learning, high adaptability. Large language models have none of those things.
There is no reason to create these types of virtual hells for virtual people. Instead, build Star Trek-like computers (the ship's computer, not Data!) to order around.
If you make virtual artificial people, give them the same respect and rights as everyone.
Nonetheless, it brings forward a couple of other issues. We might never know if we have achieved sentience or just the resemblance of sentience. Furthermore, many of the concerns of AGI might still become an issue even if the machine does not technically "think".
It's highly unlikely that we'll be scanning, uploading, and booting up brains in the cloud any time soon. This isn't the direction technology is going. If we could, though, the author's spot on that there would be millions of people who would do some horrific things to those brains, and there would be trillions of dollars involved.
The whole attention economy is built around manipulating people's brains for profit and not really paying attention to how it harms them. The story is an allegory for that.
This ignores neurons since there are only like 86 billion of them, but they could be sufficiently more complex than synapses that they'd actually be the dominant factor. Who knows.
This also ignores glia, since most people don't know anything about glia and most people assume that they don't do much with computation. Of course, when we have all the neurons represented perfectly, I'm sure we'll discover the glia need to be in there, too. There are about as many glia as neurons (3x more in the cortex, the part that makes you you, coloquially), and I've never seen any estimate of how many connections they have [1].
Bottom line: we almost certainly need exaflops to simulate a replicated brain, maybe zettaflops to be safe. Even with current exponential growth rates [2] (and assuming brain simulation can be simply parallelized (it can't)), that's like 45 years away. That sounds sorta soon, but I'm way more likely to be underestimating the scale of the problem than overestimating it, and that's how long until we can even begin trying. How long until we can meaningfully use those zettaflops is much, much longer.
[1] I finished my PhD two months ago and my knowledge of glia is already outdated. We were taught glia outnumbered neurons 10-to-1: apparently this is no longer thought to be the case. https://en.wikipedia.org/wiki/Glia#Total_number
[2] https://en.wikipedia.org/wiki/FLOPS#/media/File:Supercompute...
,_OWID.svg){kind=link}
To me, it seems more like we've frozen the language processing portion of a brain, put it on a lab table, and now everyone gets to take turns poking it with a cattle prod.
We can't even decide how much, and of what, would constitute a person. If like you said, the best AI right now is just a portion of the language processing part of our brain, it still can be sentient.
Not that I think LLMs are anything close to people or AGI. But the fact is that we can't concretely and absolutely refute AI sentience based on our current knowledge. The technology deserves respect and deep thoughts instead of dismissing it as "glorified autocomplete". Nature needed billions of years to go from inert chemicals to sentience. We went from vacuum tubes to something resembling it in less than a century. Where can it go in the next century?
This is practically the same conversation many places are having about abortion. The difference is that we know a human egg eventually becomes a human, we just can’t agree when.
"What is the distribution of human consciousness?"
"How do the most conscious virtual models compare to the least conscious humans?"
"If the most conscious virtual models are more conscious than the least conscious humans... should the virtual models have more rights? Should the humans have fewer? A mix of both?"
At no objective point. Rights and ethics are a social constract, and as such can be given (and taken away from) some elite, a few people, most people, or even rocks and lizzards.
Yes, unless we stretch the definition of conscious most people use beyond recognition.
At that point, though, it will be so remote from what we use the term for, that it could just be any random term, like doskoulard!
"Can we even refutre that a rock is doskoulard?"
The rock is neither "aware" nor "responsive". It just stands there. It's a non-reactive set of minerals, lacking not just any capacity to react, but also life.
Though that's overthinking it. Sometimes you don't need decicated testing equipment to know something, just common sense.
Common sense is valuable, but it has a mixed scientific track record.
You can always find small exceptions to everything. But you know what I mean.
Except if your point is that, like the vegetative victims, the rock's brain is still alive.
The concept is similar where the protagonist loads his consciousness to the digital world. There are a lot of interesting directions explored there with time asynchronicity, the conflict between real world and the digital identities, and the basis of fundamental reality. Highly recommend!
The short story obviously takes the very bad angle. But there's a lot of reason to believe it could be very good instead as long as we protected basic human rights for digital people from the very onset -- but doing that is critical.
1. You didn't have the rights to the model of your brain - "A series of landmark U.S. court decisions found that Acevedo did not have the right to control how his brain image was used".
2. The virtual people didn't like being a simulation - "most ... boot into a state of disorientation which is quickly replaced by terror and extreme panic"
3. People lie to the simulations to get them to cooperate more - "the ideal way to secure ... cooperation in workload tasks is to provide it with a "current date" in the second quarter of 2033."
4. The “virtual people” had to be constantly reset once they realized they were just there to perform a menial task. - "Although it initially performs to a very high standard, work quality drops within 200-300 subjective hours... This is much earlier than other industry-grade images created specifically for these tasks" ... "develops early-onset dementia at the age of 59 with ideal care, but is prone to a slew of more serious mental illnesses within a matter of 1–2 subjective years under heavier workloads"
it’s wild how some of these conversations with AI seem sentient or self aware - even just for moments at a time.
edit: Thanks to everyone who found the article!