It does not matter if your conversation partner is made up of a giant turing machine, a huge set of prolog rules, a giant neural network with a single hidden layer, a really deep transformer, a convolutional kernel, or a bustling pile of ants drawing words into the sand.
When conversing with one another we have the implicit assumption that we are not the only conscious being in existence despite it being a completely subjective phenomena. Everything else would be quite impolite.
Turing extends that (im)politeness to non-human phenomena. So long as there is the appearance of intelligence and consciousness it is the polite thing to assume that there is actually consciousness.
End of the day, you are arguing that since you could explain away the process, it is not conscious.. but you could explain away anything. How is human brain conscious? It is just activation of neurons across a large network. We can even poke electrodes and simulate various feelings.
It would be helpful if you can explain why you believe this to be true rather than what you believe to be true.
I can say with absolute certainty, that you don't either.
These billion parameter models are way to big for you or me to understand.
You make the same fallacy that Searl does. You assume that unconcious substrate can't give rise to conciousness through their configuration.
Just like we have a good working theory on how the brain operates, the devil seems to be in the detail. It's neither in the big connectome/architecture, nor in the small mechanisms neurotransmitters/activation functions (or the attention mechanism that you somewhat wrongly describe), but somehow in the system's dynamics. Heck for all we know it might actually be the appearance of consciousness that actually gives rise to consciousness.
I can't say with 100% certainty that a calculator isn't having some form of ghost, after all some ants are little more than neurological counters and shift registers, and people would probably place them somewhere on the spectrum for having a ghost that is closer to a human than to a sewing machine.
Turing adressed what little points your argument has 70 years ago in subsection 6. (4) to (8): https://academic.oup.com/mind/article/LIX/236/433/986238?log...
On a side note. What is it with neuroscientists and arrogance? As if that stuff was some secret knowledge, that's unpublished or too arcane for anybody but them to understand.