You're comparing apples and oranges. People know who Babe Ruth and Michael Jordan are, the same way they know the Empire State Building and Beethoven.
Besides, I'd be shocked if fewer people know who played the Monday Night Football game this week than who was nominated for best album at the 2013 Grammy awards, 5 days earlier.
An individual game will typically have no permanence to it, but the nature of the game can last a lot longer. In many cases, the games we are playing today originate from older versions that pre-date the 1700s, so if 'permanence' is your qualification for cultural relevance, then sports seem to have as much (if not more) relevance than music.
It's certainly an important aspect of our culture, but it has a much smaller role than music and architecture. Everybody knows the Empire State Building and Beethoven, but most people don't know who won last week's football game and by what margin.