Sure, and you could say the same thing about unit 731. The argument is not that it is inconceivable that there are benefits, but that they should be weighed against the drawbacks, which you don't mention at all.

How is unit 731 an apt comparison here?

Those were unethical experiments from the onset.

If anything, my argument would be more "just because unit 731 ended up doing live vivisections of humans, that doesn't mean we should have outlawed dissection of dead humans centuries earlier".

I don't think it's a good idea at all to try to outlaw totally reasonable research just because we can imagine a future where people end up doing a much more extreme much more problematic similar thing.

It would be the end of science. (and the end of any culture really... we're very good at imagining future dystopias and armageddons)

It's just an extension of Dario's brand of brainrot where consciousness has an almost religious existence that just pops out if you have a large enough system.

No proof is required beyond psuedoscientific tests and the "saints" of this belief system must be given absolute authority over the system to ensure no one commits any sins.

Is it really so wacky to suggest we might want to be careful with bootstrapping complex computation from human neurons? It's quite different from an LLM and the fact that it's not made out of neurons seems to be one of the core arguments for opponents of the possibility of LLM consciousness...

>that just pops out if you have a large enough system.

certainly not a claim I would make.

>No proof is required

Isn't it legitimate to reason on a more abstract level and try to make inferences that way?

>psuedoscientific tests

no clue what you're referring to here or in the rest of your comments about saints and authorities and sins. Another parallel to religion could be drawn w.r.t. the idea that only human souls could ever feel suffering but I don't think it's such a productive level to have this conversation on.

I may have overreacted, but jumping from "we should probably ban this research preemptively because it might pose difficult questions later" to comparisons to Unit 731 when someone disagreed, reminded me of the way Anthropic pushes their belief that Claude is conscious onto everyone with "evidence" of questionable value and even when there are many questions they haven't bothered to answer about such a claim. Plus, they do this despite various features we generally associate with conscious experience that current gen LLMs lack.

Given the complexity of behavior observed in nature that doesn't require conscious experience, yet is more advanced than what we can achieve in circuitry, I think it would be too early to close off this area of research.

I brought up Unit 731 because an argument like "there's so many potential insights to be gleaned from research like this" is also applicable to it, yet it is obviously deemed invalid by moral standards nonetheless. It is not a 1 on 1 comparison but a more subtle argument, I thought hackernews would be able to handle it...

If you believe you are conscious, you must acknowledge consciousness can arise in an agglomeration of human brain cells.

How sure are you that the human brain cells in the vat are not aware and suffering?

Consciousness can arise from an agglomeration of molecules. How sure are you that empty vats (which are made of molecules) are not aware and suffering?

What's going on here? Answer: Fallacy of affirmation of the consequent.

Most molecules are not conscious. Most human brains are.

This isn't a freshman philosophy seminar.

> Most molecules are not conscious.

No molecules are conscious.

> Most human brains are.

Not relevant because we are talking about brain cells, not brains. It's both a fallacy of affirmation of the consequent and a fallacy of composition.

> This isn't a freshman philosophy seminar.

And thus one goes through life blithely using fallacious logic while dismissing the fact that it is logically incoherent.

> you must acknowledge consciousness can arise in an agglomeration of human brain cells

That doesn't follow. You could, for example, be a substance dualist. Alternatively you could suspect that there exists some special as yet undocumented property of the human CNS that is required as a prerequisite. Or etc.

Certainly it wouldn't be the end of science, we could still do all the other science that doesn't involve trying to massage human neurons into doing complex computations.

I want to be clear: I hate this. I hate everything about it. HOWEVER...

If you want to argue against wetware then we need a better argument than this. The same ideas were used to argue against stem cell research, IVF, and many other treatments and ideas that have saved and helped lives. Unit 731 isn't an apt comparison because cells are not inherently sentient to our knowledge. If they were it would absolutely shatter the profession of medical ethics as we know it.

I honestly don't know if better arguments are needed. I think most people simply have no idea that this research is happening right now, and if they did the vast majority would be against it. Feels fundamentally different from stem cell or IVF.

We know that human brain cells in sufficiently large and complex assemblages are sentient (I don't know if the word "inherently" has any purpose here). We know this research aims at making the assemblages larger and more complex. It's a fundamentally different kind of argument IMO compared to those against e.g. stem cells or IVF.

(Slight aside but did you know that they're also doing tests on whole human brains taken from corpses, and they give them tranquilizers "just to be sure"? https://news.ycombinator.com/item?id=48212992)

> We know that human brain cells in sufficiently large and complex assemblages are sentient

No, we know that human brain cells in specific configurations honed by billions of years of evolution and in specific states are sentient. (The same collection of cells in a slightly different state or a different arrangement might be permanently comatose).

This argument is no different in kind than the claim that Mamdani and El-Sayed are terrorists: it's a fallacy of affirmation of the consequent.

Sure (it was a mistake to say "sufficiently complex" without qualification, obviously only certain very specific complex assemblages will actually be conscious). But if you think the "honed by billions of years of evolution" thing is relevant (on which I agree with you), you also have to acknowledge that the same goes for the neurons themselves. And not just the neurons in isolation, but also their capacity for self-organizing into computationally useful structures, possibly in response to certain stimuli, which you can be sure this type of research will try to take advantage of.

how do u know cells are not sentient to our knowledge? (i am serious, not pulling ur strings. i want to know about proof)

How do you know your fingernails aren't sentient? Better stop clipping them immediately!

> i want to know about proof

The epistemology of empirical science is inference to the best explanation, not proof. As pointed out by Hume centuries ago, no empirical generalization can be proven.

> The same ideas were used to argue against stem cell research, IVF, and many other treatments and ideas that have saved and helped lives.

You write as if objections to (certain kinds of) stem cell research, IVF, etc. have been roundly defeated in some way simply because they've "saved and helped lives". This is naked utilitarianism and I hope I don't have to rehearse the reasons why utilitarianism fails utterly as moral theory.

If an experiment could theoretically cure a disease, how unethical can it be before we outlaw it? Maybe it's better to simply discuss the line in the sand, rather than to pretend this is one one side of something not-yet-defined.