It's projected to peak at a bit over 10B before the end of the century and then steadily decline: https://en.wikipedia.org/wiki/Projections_of_population_grow... (see solid red median line).

I think it'll plateau well before that though because they'll see so much pressure on food and water resources. Africa doesn't feed itself now and if the rest of the world needs to direct more food production locally due to lower yield caused by climate change then Africa is positioned to suffer the most.

But yields continue increasing at a relatively fast pace overal?

Yup, it's by no means equally distributed, but fertility is dropping fast in Africa too.