In an academic manner, you are right. Its silly that we have to do anything to keep any government from interfering with things that we consider to be fundamental rights.
I didn't suggest that the constitution mentions marriage. However, its failure to explicitly state it as a fundamental right is a large part of the reason gay people were oppressed for so long in our country. Historically, rights not outlined in the constitution fall under the authority of state governments.
Also note that the constitution and the bill of rights are more of a statement of rights we already have. The document isn't intended to actually grant the rights. No one can grant fundamental rights. However, having a constitution as a statement of rights can help protect them when others want to take them away.
Because the constitution says what it says and it doesn't say what it doesn't say. It is a fundamental right, but the drafters of the constitution failed to mention it in explicit enough terms to adequately protect people. In fact, the people that voted in favor of gay marriage must have agreed that the constitution didn't adequately protect these rights, because they heavily referenced amendments (AKA modifications) to the constitution in their decision.
The constitution isn't perfect. If it was, we wouldn't have needed amendments to give women the right to vote, or to make minorities count as whole people. We need amendments to make it better.
For all I know, Scalia may have had bigoted alterior motives for his vote. However, I'm simply evaluating his argument (which if you'll reread my statement I wasn't actually agreeing with).