Many parts of Africa are not politically stable enough to support this manufacturing. You should watch this Planet Money article to understand why Manufacturing (of T-Shirts in this case) move from one country to the next.

http://apps.npr.org/tshirt/#/title