Surely you are not suggesting LLMs pick a/an according to statistical distribution and then choose vowel starting/non-starting words, not vice versa? This is testable: let it complete "I climbed up the apple/pear tree and picked". It always picks "an" for apple and "a" for pear, and never reaches a dead end by picking the wrong article so that it either fails syntax (wrong article) or semantics (pear growing on apple tree).
We actually have fairly good idea how LLMs pick a/an. Have a look at https://www.lesswrong.com/posts/cgqh99SHsCv3jJYDS/we-found-a....
For sure there has to be some internal high level representation of the entire input, and I would expect also some representation of possible continuations at a higher level than just alternate word choices, but I wouldn't exactly characterize this as a plan since it's so statistical in nature, and the model has to continually adapt to what is actually generated (sampled).
At a low temperature setting "an" might always be picked, making "apple" more likely to follow, but at a higher temperature maybe another word such as "the" might be chosen, perhaps with a continuation of "biggest apple i could find", etc. In a 350B token training set I'm sure there are a number of other alternatives too.
So, does the model plan ahead ? Well, sort of, but this "plan" has to represent multiple branching alternate futures, and continually adapt as the output is sampled, so it's a very fluid kind of plan. It's also not clear how far ahead the plan needs to represent .. maybe one word only (like a state machine), even if the reason those alternate one-word continuations are there is because they were seen as part of specific longer continuations.
The lack of any internal state preservation from one word generated to the next puts a huge limit on what can be planned, as does the extent of how complex a plan can be generated from the flow of a single input sequence forwards thru the layers of the model. You can see the limitations of this with prompts such as "pick two primes > 200, and tell me their product, but not the numbers themselves"... if you play with it and analyze the mistakes, it makes it clear how much the output really is generated word by word, adapting to what came before, much of the time.
What's going on here is that at the simplest level of explanation these models are statistical word generators: you feed some input (words) in and ask the model to generate a word that it thinks is a statistically likely continuation/follow-on to that input. If you want more than one word of output then take the previous output word, add that to the input, and ask the model to generate the next word, etc, etc...
What's happening with "hallucinations" is a bit like when we accuse a human of "starting to speak before engaging their brain"... the model generates words one at a time as asked (i.e. "starts speaking") but isn't explicitly planning ahead (hasn't engaged it's brain) to any great extent.
If it starts down a path of "the source for that is ..." (not because it actually has a source, but rather because it thought it statistically likely that it would be quoting one), then to complete that sentence it's going to have to quote a source - as usual what ever it thinks is most statistically likely in the context (maybe something highly relevant from it's training set, maybe something of poor relevance, or maybe something whose only relevance is that it looks source-y).