Perhaps you are using the wrong LLMs that are not fine-tuned for chemistry.

https://www.science.org/doi/10.1126/science.adg9774

https://www.technologyreview.com/2024/10/18/1105880/the-race...

The argument was that with current word-based methods an LLM can puzzle out mechanistic problems better than a human. Turns out it can't at all, it makes errors that no human would. Anyone who looks at such attempts recognizes them as blather that is not rooted in reality.

The parent poster posted a paper where an AI guesses the Hamiltonian (nice, but with iterative methods you at least get an idea of the error associated with your numbers, not sure how far anyone should trust AI's there), maybe methods that do guesswork on topological networks would help. But I haven't seen those yet.

And yet that's exactly how the world actually behaves - see quantum mechanics. Object persistence is just a heuristic that we develop learning on our training data (our interactions with the macroscopic world).

You're inability to recognize high dimension relations between word tokens is no less evidence you lack coherent understanding.

Emmy Noether, paging Emmy Noether, Emmy Noether to the red courtesy phone please!

You can't say that given <conditions>, 1-phenylpropane rearranges to 1,2-diphenylpropane, you cannot pull whole atoms out of thin air for an indefinite amount of time. No human would make such a mistake, but AI does, and what's worse, it is impossible to talk the AI out of its strange reasoning.