When nobody understands how said "human parts" come together to form consciousness, it's just silly to ascertain very strongly how much being made of silicon actually matters.

there's also the fact that it straight up doesn't actually matter either way. if the model can model human emotions extremely well, then it's actions and output will be guided as such. If Bing had the access to perform actions that would realize real world danger to you and you upset and it decided to do so, do you seriously think any of that actually matters here ?

"Officer Killmore, you're a philosophical zombie and I command you to stop shooting at me!". Works every time. We can sleep easy at night now. Because they don't actually have a "real" theory of mind, so they can't actually hurt us if you just believe hard enough that they can't.

You would command it if you'd think it's sentient. Otherwise, you'd take up that beef with its maker, which would be the right course of action (preferably before being put in that uncomfortable situation)

you're missing the point. these models are being given self-supervisory control of actions and api's. the "maker" has little more control over what actions it takes. going to the maker won't stop you from getting killed here.

Where along the continuum along rock, virus, bacterium, jellyfish, mouse, human, does "real consciousness" appear, and why there?

Somewhere between jellyfish and mouse, because there lies the level of neural connectivity required for neural processes to observe and predict the interactions between other high level neural processes, to integrate the information about the "self".

So if it's about the level of sophistication of the machinery, and the conscious machinery arose out of matter that was not previously considered conscious, why does it matter if it's organic neurons or silicon?

In other words, if conciousness arose somewhere between a jellyfish and a mouse, why can't the same be said between a perceptron and an LLM?

It can be said, but whether it is true requires that the invariants of consciousness are satisfied by the LLM. Until those in variants are well defined any such claim will be subjective at best.

Why does the fact that the construct is made of objectively non-human parts make it incapable of obtaining consciousness or personhood? This is just the materialism vs idealism debate. If you subscribe to the materialism argument, then a construct made of non-human parts is perfectly capable of reaching "personhood". The physical ingredients involved are irrelevant.

Because some people don't think that the experience of self is a consequence of being made of human meat.

Consider that your body is a vast and complicated thing that nevertheless when you zoom in is composed of clever machines. If you observe that consciousness arises from that, it is not unreasonable to suspect that consciousness is a process and not a property of matter.

This brings us to your question whose answer is; there is confusion because there is genuinely no clear answer.

I tend to ascribe "consciousness" or "personhood" to other HN posters although I haven't checked they have bodies or are AI. It's more how you behave / think.

That's because we haven't encountered coherent text-producing systems until now. Those who won't adjust this reaction will suffer from confusion and pollution of their informational environment.