Free speech, as a value, is a broader concept than mere lack of government restraint on speech. It can also pertain to the exchange of ideas in certain private forums.

And by the way, the First Amendment disallows not only prior restraint, but many other government restraints on speech as well. It's just that prior restraint is considered the most egregious and hence subject to the greatest judicial scrutiny.