In the context of LLMs it's only ever going to work out that wath math/stats, because garbage in garbage out will taint LLM output roughly to the same degree as human generation. Because humans are the garbage producers in both scenarios.

To bypass the garbage in problem, you'll need a different AI structure from LLMs almost certainly, a proper logic model.

Is it the fundamental architecture of an LLM that makes this problematic, or the input?

Based on how it's reported that they work, I assume that even without garbage in their statistical reasoning of the next word would occasionally come up with bad output. E.g. they might come up with 2 + 2 = 5 because of mathematical inputs such as "1 + 2 + 2 = 5".

Except it's unlikely because they have a bit larger context and are modeling a bit more than one-dimensional probabilities of "what comes after this?".

I'd say that LLMs are more resistant to GIGO, as long as the fraction of garbage in training data is small - it'll look like outliers to the larger model.