In fact, I would argue that disagreements and ambiguities as to what makes a person "white" or "black" are astronomically more frequent than difficulties in understanding what is meant by the words "whitelist" or "blacklist".
Neither should be used to refer to people. “White” and “Black”, capitalized as proper nouns and proper adjectives are in English, to refer to particular identity groups and association with those identity groups are, OTOH, a different story.
"Whitelist" and "blacklist" are abstract terms which are useful because they have general and non-specific meanings which can be adapted to many contexts. Artificially prescribed alternatives, introduced only to satisfy the current American cultural Zeitgeist, are not.
It dismays me to see programmers arguing loudly and stubbornly for bad terminology, when a key skill for designing a good API is to choose good names.