I've run into places where it would be handy to conditionally use hooks, but never the useContext hook, so I can't imagine feeling strongly about that case.

Define "legal". Calling glibc memcpy with overlapping arrays worked right up until it didn't. The general contract of react hooks is that they may break in unexpected ways if you call them conditionally. The current implementation of useContext happens to not break.