oxidi

Good point. Though sampling generally happens on the CPU in a linear way. What you describe might influence the raw output logits from a single LLM step, but since the differences are only tiny, a well designed sampler could still make the output deterministic (so same seed = same text output). With a very high temperature these small differences might influence the output though, since the ranking of two tokens might be swapped.

I think the usual misconception is to think that LLM outputs are random "by default". IMHO this apparent randomness is more of a feature rather than a bug, but that may be a different conversation.

I think lots of people misunderstand that the "non-deterministic" nature of LLMs come from sampling the token distribution, not from the model itself.

I was referring to the "creative syntax" and it wasn't meant to be an attack on Python.

We cannot deny that Python has some interesting solutions, such as the std lib namedtuple implementation. It's basically a code template & exec().

I don't think these are necessarily bad, but they're definitely funny.

I think the point was that Python syntax is simpler than e.g. borrow checking.

Although Python has some seriously PERLesque YOLO moments, like "#"*3 == "###". This is admittedly useful, but funny nonetheless.