A more interesting question is...does differential equations based models like mamba/state space models perform better on this sort of physics problem than pure transformer LLMs?