So obviously there is a lot of data in the parameters. But by stateless, I mean that a forward pass is a pure function over the context window. The only information shared between each forward pass is the context itself as it is built.

I certainly can't define consciousness, but it feels like some sort of existence or continuity over time would have to be a prerequisite.