Sorry but that is exactly what I meant: the world is a) either deterministic or b) partially deterministic and partially random (for example quantum), and in either of these cases there's no way you can construct free will agents.

Arguably the b) version is not simple physics, but that's what I meant.