It was the weirdly sexualized race and gender based cannibalism that I think puts it over the line, not the actions of Joe, except in the context of the former.
I mean I see what he was trying to do, but rather than basing his African elites on anthropological evidence, or on real white cultures that were horrible to other races, (which would have gotten his point across better, I think,) he based them on the most disturbing (and to me, last believable or sympathetic) racial fears of the time, and because of that, it is kinda uncomfortable.
1964 was the same year the Civil Rights Act was passed. I imagine ideas of baser racial instincts were different when there were still lynchings and fear of sexual racial mixing. :(
As someone born in the very early 90's, it's weird seeing evidence of people taking seriously these thoughts I've only seen caricatured.