Which GPU(s) to Get for Deep Learning

Re:Which GPU(s) to Get for Deep Learning by tensor on Monday May 22, 2017 @09:58PM (#14398270)

The problem is that a month of GPU time on a cluster can buy you the hardware itself. If you are doing serious deep learning work it's just not cost effective at this point. If the costs come down by half or more, it may start looking viable for people who need a lot of resources.

[ Parent | on HN ]

Re:Which GPU(s) to Get for Deep Learning by boulos on Monday May 22, 2017 @10:36PM (#14398426)

Can you explain your math? A K80 on GCE is $.7/hr x 730 => $511/month if you were really 24x7. A K80 (and really we sell them by the die not the board) is more than $1000.

I don't disagree that a consumer board is about that price, but they're not apples to apples. (Either in memory size, reliability or both). I'm fine with that being the real complaint: (major) cloud providers only sell the Tesla class boards, and they're really expensive ;).

[ Parent | on HN ]

Re:Which GPU(s) to Get for Deep Learning by marmaduke on Tuesday May 23, 2017 @03:51AM (#14399606)

> but they're not apples to apples

It would be nice to see Nvidia or someone expand on this, so that users who have to make this choice can do so without guessing. If Google or AWS or M$ could publish reliability information, that'd be cool too.

Illustrative case: I run Monte Carlo work on GPUs and administer a local compute cluster. I tested a workload on a 16 GB P100 and a GTX 1080. A 12 GB P100 costs (academic, EU) 5000 euros while the latter costs 700 euros, but the performance difference is about 2x. Still, when we ask Nvidia reps, they say not to bother installing GTX cards in our cluster, because they aren't designed for 24/7 work, not commercializable etc. Even so, the GTX would have to burn out 3 times before the choice of P100 breaks even. Burning out three times means GTX 1080, then 1180, 1280 etc.

[ Parent | on HN ]

Re:Which GPU(s) to Get for Deep Learning by technics256 on Tuesday May 23, 2017 @04:04AM (#14399659)

Thank you for the answer. When you say performance difference is 2x, I presume the P100 is 2x faster than the 1080 in training epoch time?

[ Parent | on HN ]

Re:Which GPU(s) to Get for Deep Learning by marmaduke on Tuesday May 23, 2017 @09:44AM (#14401088)

yep

[ Parent | on HN ]

Re:Which GPU(s) to Get for Deep Learning by tanilama on Monday May 22, 2017 @10:56PM (#14398478)

A GTX 1080 is like $550 at this point, K80 cannot really compete with the cost effectiveness here. And it is probably not going out of fashion in one year, so the price is totally worth it.

The real attractiveness for cloud at this point is if you are going to train your model with 8-GPU or more, that is likely not feasible for individual enthusiasts, but demand for such machine is rare for hobbyists anyway.

[ Parent | on HN ]

Re:Which GPU(s) to Get for Deep Learning by shaklee3 on Tuesday May 23, 2017 @01:27AM (#14398992)

This is kind of the key. If P100s were common on gce, it would be a better comparison. But the k80 is very old by today's standards, and a consumer card will run circles around it.

[ Parent | on HN ]

Re:Which GPU(s) to Get for Deep Learning by tensor on Tuesday May 23, 2017 @01:05AM (#14398921)

Sure, in looking at what it would cost to replicate this:

https://arxiv.org/pdf/1704.01444.pdf

They say they used four Titan X cards for a month. I was actually looking at google cloud machine learning since I thought being targeted to tensorflow would be the most cost effective. But it's 0.49/hour per ML training unit, or 1.47/hour per GPU. A basic gpu gives 3 training units, so I thought that something approximately similar would require 12 training units, which comes to something like $4000+ a month. Maybe I completely misunderstood the resources being offered though, because you are right that cloud engine costs seem much lower.

I'll have to revisit the math here, though it worries me that it's not at all clear that a K80 will be much faster than a Titan X for a given problem. E.g. https://www.amax.com/blog/?p=907. It would also be really nice to get some pricing and benchmarks for the new TPUs, assuming they are priced better.

Maybe part of the problem is that vendors are not making it remotely easy to even understand what performance you'll actually get for a given price.

[ Parent | on HN ]

Re:Which GPU(s) to Get for Deep Learning by jacquesm on Tuesday May 23, 2017 @03:56AM (#14399625)

The comparison should be between a machine off-premises with as many GPUs in it of the same type that you intend to use and power + cooling.

Payback time according to my own calculations depending on GPU model and machine used compared to the various cloud offerings is between 4 and 6 months when used continuously.

[ Parent | on HN ]

Re:Which GPU(s) to Get for Deep Learning by zo7 on Monday May 22, 2017 @10:56PM (#14398480)

Isn't there an abstraction cost though, since that price is only for half a board? In my experience (with AWS) I've had a machine with a Titan X outrun a cloud instance by 1.5-2x, which is significant.

There are also other hidden costs that aren't factored into that number. I'm having a hard time getting anything reasonable for under $750/month for 24/7 usage: https://cloud.google.com/products/calculator

[ Parent | on HN ]