In my Ph.D. studies in Educational Psychology we were made aware of cultural bias. More than a hundred years ago, when people made tests to assess "intelligence" they may have not been so careful as we try to be now.
Are you sincerely willing to claim that it's impossible to create a test that doesn't make it easier for some subset of the population to outperform another?
If a student couldn't be bothered to learn about the culture, he isn't a good fit.
That said, the questions really are about reading. If you read lots of books, not counting the ones for little kids, you'll have no problem with the questions.
It is desirable to maintain a college culture that is biased towards people who read about lots of different things, including yachts. Reading is good.
The choice of reading material matters. I don't count "The Cat In The Hat" and "Teen Magazine" the same as "The Grapes of Wrath", "Hamlet", "The Scarlet Letter", and "As I Lay Dying".
Supposedly it was racist. Supposedly the white kids are all out doing yacht races, so they alone would know the vocabulary.
That is ridiculous. The portion of the population of white kids that go yachting is minuscule. Practically all white kids don't even know a person who does that. Practically all white kids have never even been on a yacht.
The question was fair. It, and others like it, are a test of reading. If you read well and have read a large variety of books, you'll know the words. The whole point of the test is to see if a student has that sort of college-capable ability (Can you read very well?) so of course the question belongs on the test.
The same goes for polo. Nobody does that. The same is nearly true for golf, and I think the desire to watch Tiger Woods might flip any advantage toward the black kids.
Really, it's about reading.