I've "read" most of the first three volumes if it counts to read the text and skip most of the math. I always figured it was enough to "get" the algorithm, and go back to the math if I ever needed it. So far, years later, I've never needed to go back to the math, so I'm glad I didn't spend much time on it.

No doubt the math is useful in a theoretical way, and I'm pretty sure I could follow through most of it if I really needed to, but as far as proving algorithms and asymptotic run times, I've yet to need more than a general understanding of big O notation in my real work.

How long it takes is entirely up to you. If you try to understand every detail of every proof and theorem it will probably take a very long time.

I'm not sure it's worth reading all the way through, to be honest. I guess I'd suggest the first volume for a solid, math heavy intro to algorithm analysis and design. But after that, there's not much point unless you're going to read it for fun, and in that case, why ask about it on HN? The reality is that few people will ever need to know that much detail about any of the algorithms covered because almost all of them are library functions now.