The problem is more about latency, not so much bandwidth. Also, not everyone has a good connection.

Cache only works the second time the user comes to your page. By definition, every user will experience slow loading the first time.

That said, it probably depends on your use case if it is worth bothering with a CDN or not.