Non-intellectual people can and do question white privilege stuff in the United States, saying that you have to be "intellectual enough" to have that type of conversation sounds very classist to me (am not native English speaker, maybe that's a better word for it).
That Lebanese population will most probably not be labeled as "white" in most of the Anglo world, first of all in the United States, while South Africa is only one country among many dozens of African countries (I choose to ignore white presence in Zimbabwe because Mugabe). There is indeed a sizeable French expat community in many African countries but they are afraid of physically getting out of their bubbles, usually located in a posh district in the respective country's capital city. My friend (also French) was not afraid of getting out of that bubble hence his direct interactions with people who had almost never interacted directly with a white person before.
Lebanese people are historically considered "white" in the USA, at least the Maronite Christian community. Most of the Lebanese who moved to the USA in the late 19th and early 20th century are long since assimilated to the white mainstream.
French retirees in Senegal and Cote d'Ivoire are certainly not limited to a posh neighbourhood in the capital and can be found spread out across the country. Dakar is a dismal place, and certainly retired foreigners find Senegal's small towns and villages more relaxed and peaceful.
This is not a subject where mention of "my friend said" is helpful. One needs first-hand experience with Africa (and experience outside the expat bubble that is all that some foreigner workers see).
I mantain my opinion about the Lebanese and how they’re viewed in the US, heck, the Italians were not seen as white-white until very recently while the Mexicans or the Argentinians still aren’t.
Any African who is intellectual enough to possibly participate in the international debate mentioned above, will almost certainly be part of a cosmopolitan class that has interacted with white people. Also, West African countries have a prominent Lebanese population that runs import–export based businesses like supermarkets, Senegal has a huge community of French retirees, and Zimbabwe and South Africa still have their white populations. So, no, Africans are not as isolated as you suggest.