bondarchuk

Sort of a recap for normies of Scott Alexander's Whither Tartaria from 2021: https://www.astralcodexten.com/p/whither-tartaria

Well yes, they are supported by a religious charity https://www.templeton.org/grant/nautilus-magazine

I brought up Unit 731 because an argument like "there's so many potential insights to be gleaned from research like this" is also applicable to it, yet it is obviously deemed invalid by moral standards nonetheless. It is not a 1 on 1 comparison but a more subtle argument, I thought hackernews would be able to handle it...

Sure (it was a mistake to say "sufficiently complex" without qualification, obviously only certain very specific complex assemblages will actually be conscious). But if you think the "honed by billions of years of evolution" thing is relevant (on which I agree with you), you also have to acknowledge that the same goes for the neurons themselves. And not just the neurons in isolation, but also their capacity for self-organizing into computationally useful structures, possibly in response to certain stimuli, which you can be sure this type of research will try to take advantage of.

I'm sure they know more about brains than I do but I'm also sure they cannot derive from an abstract description of a computational process whether that process implements suffering or self-awareness or whatever, which is what I think we'd need to do this human-neurons-as-lego-bricks type research safely.

Or if it ran a couple million soft-body simulations with the goal of creating a reusable temporary link between certain kinds of materials, I think it would probably get there. Just another argument from lack of imagination on the part of the author.

Is it really so wacky to suggest we might want to be careful with bootstrapping complex computation from human neurons? It's quite different from an LLM and the fact that it's not made out of neurons seems to be one of the core arguments for opponents of the possibility of LLM consciousness...

>that just pops out if you have a large enough system.

certainly not a claim I would make.

>No proof is required

Isn't it legitimate to reason on a more abstract level and try to make inferences that way?

>psuedoscientific tests

no clue what you're referring to here or in the rest of your comments about saints and authorities and sins. Another parallel to religion could be drawn w.r.t. the idea that only human souls could ever feel suffering but I don't think it's such a productive level to have this conversation on.

Certainly it wouldn't be the end of science, we could still do all the other science that doesn't involve trying to massage human neurons into doing complex computations.

You could do that if you knew enough about how it works in detail. In practice they're just gonna make as big a blob as they can muster and apply RL on it or something, and nobody will be any the wiser as to what the thing's feeling.

I honestly don't know if better arguments are needed. I think most people simply have no idea that this research is happening right now, and if they did the vast majority would be against it. Feels fundamentally different from stem cell or IVF.

We know that human brain cells in sufficiently large and complex assemblages are sentient (I don't know if the word "inherently" has any purpose here). We know this research aims at making the assemblages larger and more complex. It's a fundamentally different kind of argument IMO compared to those against e.g. stem cells or IVF.

(Slight aside but did you know that they're also doing tests on whole human brains taken from corpses, and they give them tranquilizers "just to be sure"? https://news.ycombinator.com/item?id=48212992)

Sure, and you could say the same thing about unit 731. The argument is not that it is inconceivable that there are benefits, but that they should be weighed against the drawbacks, which you don't mention at all.

One of those research areas where nothing they currently do is unethical, but also nothing they currently do is useful, and right about the point where it might ever become useful is also the point where it becomes hugely unethical. So we'd do well to outlaw it right now, but seems like we don't really have a moral framework for cases like this.

There are ways of approaching this attack you're not currently considering. It simply won't work to go "here's how I would do this, that's impossible though, therefore this whole range of attacks is impossible".

So you both thought about it for about 10 seconds and came up with some surface-level objections. Doesn't really prove anything when dedicated attackers with 10s of millions in potential profits are in play. Probably you would've written very similar dismissals about the idea of implanting a mini esim modem in a hardware wallet if someone would've brought it up back then.

Would be good if Apple (or anyone) could have some kind of user-verifiable supply chain integrity thing, if they don't already.

Or just a single apple store sales employee...

More tech-literate representation would just be more effective at coming up with filesharing blocking solutions that actually work. What we should do is vote for people who will restrict copyright laws to reasonable proportions.

>but the desire to attribute major scientific discoveries to a single author or team is based on an incorrect interpretation of the scientific process.

I mean yeah, in a sense, sure.. But would people be writing articles arguing that point now if some dude with wrong opinions on race and genetics had taken the picture and a woman had first realized the double helix structure?

Anyway, yeah I also read that about Pauling and helices, seems someone would've figured it out eventually and it's somewhat a matter of luck who got there first.

So did Franklin provably realize the double-helix structure of DNA first or not? A lot of beating about the bush here but the article stops just short of actually explicitly making the claim it's working so hard to imply, AFAICT...

(edit: should note the HN submission title (at present "Despite what Watson said, Rosalind Franklin understood structure of DNA first") does not actually occur in TFA)

On the one hand given the current level of technology I agree with you. On the other hand if, say, an atom-level simulation of a human brain were created I would think it's reasonable to assume that it is conscious even if we don't have a theory of consciousness. And we can imagine different levels of detail in-between current LLMs and atom-level simulations, and different people would draw the line for worrying about consciousness in different places...

A crucial thing being missed here is most of the present discourse (as far as I've seen) happens between people who are uncertain about it but do think it is possible that an AI might be conscious now or in the future and this should be taken seriously, and people who totally rule out the possibility that an AI might be conscious now (or, in many cases, ever). Then the philosophical picture starts to look very different.

PS the whole "a group of things [can't] share an internal world" spiel is weird and just plain wrong IMO. Humans could perfectly well share an internal world if the technical means were there. The author has not yet realized that the idea of "the self" as a single well-defined and enduring object-type thing is only a convenient illusion created by our minds. Alas, the style of the piece does not really lend itself to a coherent counter-argument, it would be a useful companion piece if the author simply laid out their personal beliefs on this matter plainly.

Conditional logic works just fine!

Do you think an atom-for-atom software simulation of a human would be able to feel pain?

Sure. And possibly it is indeed in pain then, if a computational process equivalent to that which makes up pain in animals is being performed. Or possibly not. But the criterium the GP poster laid out is not a valid way to decide that.

>nothing in their mathematical equations are different, when the result of these vector calculations produce text, that to humans look like someone in pain.

So at least something is different, namely the text output changes (and therefore some internal state too, of course). I think your analogy is too simple, it does not have to be the case that the GPU has to act like the equivalent of the body for an LLM in this respect.

In order to have a reasonable view of the possibility of AI consciousness, one first has to have a reasonable view of the relation between human consciousness and the physical human brain. Unfortunately most people do not have that. Instead they hold on to received quasi-christian dualistic concepts about souls - even if they would never admit it in those terms, the way most people conceptualize the relation between mind and body it comes to basically the same thing.

And in fact in our culture most people are VERY attached to these ideas, because it is tied up with "what makes us human", morality, and also death if you want to go there. So I believe for many people, for whom seriously entertaining the idea of a conscious computer program is so far out of the realm of possibility it hardly registers, talking about AI consciousness is simply a way to work through and re-assert these beliefs about human consciousness that they feel so strongly about.

Hence you get articles like this with barely any substance, but some weird need to signpost every sentence with social ridicule. "Dumbest", "absurd", "obsession", "viral", "wildly tiresome", "third-rail", "psychosis", "sect", not to mention the scare quotes everywhere. Come on...

When you lose a limb you'd die of blood loss though, right, not the mere lack of a limb.

Copacetically?

It is really quite staggering how many commenters here seem to believe that the fact that there is no state carried from one chat session to the next proves that there can't be consciousness. It is obviously wrong if you think about it for a minute but I guess people just aren't used to thinking about the relation between the physical and the mental with any level of seriousness.