I've had a similar thought process. We had an implicit right to privacy for abortion, but not for practically anything else (a couple other things, but some of them were also bizarre. Like how can you have privacy over who you marry if you're required to get a license before hand and they can still ban some classes/conditions but not others). If it is a right, it should apply universally. If not, then we need to start creating those rights in the constitution (where it requires larger agreement and isn't as easy to overturn). But maybe I'm crazy.