Yes, in the same sense a modern digital camera is a glorified photodiode. In both cases, light comes in, voltage comes out, and we can use it to count how much light came in.

Why stop there, it's just ones and zeroes.

It's "glorified markov chain" in the same sense that sqlite is just "glorified bubble sort".

Don't you know that "attention is all you need"? Attention is non-markovian. It's all-to-all with some masking, not a chain.