Neither do white people look pure white, nor are blacks actually black...

True, however, Asians aren't even close to yellow.

These terms usually have long and weird history and often do not base on any actual observation but on layers of theory that for modern eye would justifiably appear completely bonkers. See e.g. https://en.wikipedia.org/wiki/Historical_race_concepts Of course, people using these terms now often have no idea where they come from.