That's an illogical counterargument. The absence of published research output does not imply the absence of intelligent brain patterns. What if someone was intelligent but just wasn't interested in astronomy?

Yes but this was just to make a blatant example. The questions still stands. If you feed a LLM certain kind of data is it possible it strays from it completely - like we sometimes do in cases big and small when we figure out how to do something a bit better by not following the convention.