They are more common at public companies, but a company doesn't need to be publicly traded to give people RSUs. Conversely, a public company can pay in options rather than RSUs (or both) if they want.

Source: Work for a non publicly traded company and have RSUs not options, previously worked for a publicly-traded company that issued RSUs and options as part of comp