Yes it would have to be transient, though thats not to say a continuous experience couldn’t be created by connecting together the transient conscious experiences in sequence because of the persistent state of the context window. My only point though was that nobody who says an LLM is conscious thinks the conscious part is the weights sitting on a hard drive.