You should perhaps look a little bit into the history of slavery. It was not white against black or Europeans against Africans. It was everybody against everybody else.

The Muslim history of slavery is quite interesting, for example.

http://en.wikipedia.org/wiki/Anthony_Johnson_(colonist)

http://en.wikipedia.org/wiki/Arab_slave_trade

http://en.wikipedia.org/wiki/Slavery_in_China