Most of the time spent in typical "big data" analysis is actually I/O. GPU doesn't help in making the data reads/writes any faster.

Keeping that aside for a moment, even though GPUs are the rage these days, it's not very easy to integrate with them. For e.g. language binding. You also need to model your program in a way that optimizes for GPU-based processing.

Actually, people are already doing this; Amazon e.g. provides GPU-heavy instances with (IIRC) Tesla GPUs.