One downside to CUDA is its massively powerful for single FP precision, but double FP precision performance is less than 1/10 of single FP performance.

When precision is needed CUDA is much less useful, say you're running 10^10 simulations then with single FP precision you will only have a result accurate to 5 significant figures.