LLMs at their core do produce reproducible results with a given seed. it's all the workflow stuff people do on top that tends to break reproducibility.

This is not the case for LLMs running on GPUs (which is most of them); GPUs are non-deterministic for this use-case due to the floating point math involved. there is no way to get perfectly deterministic output from OpenAI despite the presence of seed and temperature parameters.