There are a lot of cultural factors involved in believing a seemingly pointless exercise like matching squares is worth doing. You have to believe that something good can come out of it in general; and you have to believe that something good can come out of it specifically for yourself.
More seriously, random pattern matching questions are the best you can do to control for the not-caring factor, because they eliminate the advantage of test prep. The SAT has a significant test prep factor, which I agree is highly confounded by "culture" (a byword for how much your parents make you study) and the circumstances and opportunities related to test prep, including the motivation of the student, and the availability of a place to study and study materials. IQ tests are significantly more meritocratic than algorithms interviews or the SAT.
OT, but I really love GlaDOS and hope there's another Portal soon.
I'm not sure if we want to do that because the capacity for working hard is often more valuable than "g". One's capacity and willingness to work hard for a long time is impossible to untangle from one's circumstances including one's upbringing. Fairly testing for that is much harder than testing for g but it may be too crucial to ignore.
Now, cultural fluency questions have appeared on IQ tests, especially in the past. However that's not so much an attack on the idea of "g" as it is a problem with antiquated methods of measuring it.
That's not to say that IQ tests are that accurate. The biggest confounder I know of is ADHD, which causes the tests to come out five whole points lower on average. I'm sure there are others.