That would have been hard to write out and genuinely support a couple years ago, when NVidia ruled the top-of the line as well as the best value-for-money most of the way down the chain. It was a hefty hit to your framerate or your wallet to support AMD.

But these days AMD is a serious contender at all levels! If the performance or value isn't actually in AMD's favor, it's near enough to be undetectable outside of a benchmark, and very easy to support the good guys.

> very easy to support the good guys

But AMD doesn’t run CUDA? Is it very easy to switch a stack relying on CUDA to OpenCL? I don’t think so.

AMD's ROCm suite has a tool for converting CUDA to C++ code: https://github.com/ROCm-Developer-Tools/HIP

How well it works is likely codebase and application dependent.

I’ve been using OpenCL for some neutral net image optimisation recently, I’ve actually found it very efficient (citation obviously needed), I personally welcome the competition of AMD and it does seem like Nvidia is digging its own grave (not that I think it’ll die).

I don't think it's that hard, but it depends maybe a bit on how long your codebase has evolved through CUDA.

Here was my experience. Everything in ocl ended up slower, with 30%+ longer (walltimes) on nvidia hardware. Also some problems with floating point precision. Real problem was there was no business case to switch because we could always make customers eat the cost of taking nvidia gpus.

The added code complexity was not worth it - we originally did it as NRE for a big client who wanted OEM capabilities to our IP.

OpenCL is slower than CUDA, but it's not the end of the world.

got a reference?

That is simply not true.

Please be more specific. What is not true? That it is slower or that this is not the end of the world?

Wow, people are butthurt. It's a wildly known fact in the industry. A simple google search for benchmarks or a PDF like this would have satisfied them. But they didn't want the truth they wanted to be butthurt.

https://arxiv.org/ftp/arxiv/papers/1005/1005.2581.pdf

In basically every chart and graph, OpenCL is slower.

OpenCL has a higher level of abstraction, so it's got a higher penalty for it. But, you get cross-platform support. nVidia doesn't optimize for OpenCL either because they don't want you to use it over their own competing framework.

But 'dats cool, keep downvoting, guys. Sources aren't real if you can downvote 'em enough. God, I love Hacker News.

Would you kindly raise the level of discourse here? You have useful information, so thanks for sharing it. But please leave the swipes and complaints about downvotes and HN members out of it. It worsens discussion and is against the guidelines. You’ll likely receive downvoted for that, more than anything else.

This reddit-style, highly emotional commenting is not terribly welcome on HN.

There are zero DL frameworks that support AMD cards as a primary target.

Most have some kind of branch or patchset with OpenCL support. The problem is that they aren't great. If you need any new layers there is no support. There is nothing like CuDNN so you don't get the high speed convolutional kernels.

It's great to blame developers for supporting NVidia, but the thing is NVidia are great to work with. They dedicate large teams to deep learning support (not like the 2 or 3 part time Devs AMD does), and they publish good research and tutorials. AMD does nothing like this.

Amd had always had first class support for openCL, cuda is Nvidia proprietary and although Nvidia "supports" OpenCL, it's quite bad. Issue #22 on tensorflow is regarding openCL support.

AMD has done some interesting work on HCC (a more proper gpgpu compiler approach with llvm base) and that is showing promise. See here: https://instinct.radeon.com/wp-content/uploads/sites/4/2017/...

Additionally, they support cuda decompiling to HIP which is an intermediary that can be built to target Nvidia (via nvcc) or amd (via HCC).

Nvidia has built a lot of tooling for DL, such as cudnn, and the new Tesla cards have dedicated silicon for tensor calculation. Amd does have a cudnn equivalent called MlOpen. They have also ported caffe via HIP and it works well. Work is being done by amd right now to torch, mxnet and tensorflow to add support for amd hardware with minimal burden to the maintainers of these projects.

You can read about some of the DL toolkits available here: https://instinct.radeon.com/en/6-deep-learning-projects-amd-...

I think it's particularly bad form on behalf of everyone in the DL framework and library world to cator only to Nvidia and cuda, and that they very much walked into this shake down with open arms.

The original comment is correct in that contributing support for OpenCL (which works on mobile too) will alleviate this to a fair degree. It's one of those things where the more momentum is behind it, the more device manufactures will focus on ensuring their opencl compiler is building properly optimized kernels for their hardware.

Start contributing to OpenCL or adding hip support to existing projects and we'll see some viable alternatives pop up from not only AMD, but players like Qualcomm and Samsung.

Until quite recently OpenCL was a C only game, so blame Khronos for not embracing all HPC relevant languages, like CUDA does.

OpenCL doesn't have an equivalent to CuDNN.

I'm not saying it's a great situation, I'm saying that NVidia has always had better libraries, tools and performance, and it isn't surprising that developers use them.

Deep learning is hard and slow enough without using second class tools.

Much respect to NVIDIA and their software team but the situation is changing. PlaidML is like cuDNN for every GPU. Fully open source, faster in many cases than TF+cuDNN on NVIDIA, beats vendor tools on other architectures, Linux/Mac/Win. Supports Keras currently but more frameworks are not difficult (patches welcome).

The PlaidML benchmarks are suspect. They compare to Keras + Tensorflow, which is a really unfair comparison since 1) Tensorflow is probably the slowest of the big deep learning frameworks out there (compared to PyTorch, MXNet, etc.), and 2) Keras itself is quite slow. Keras is optimized more for ease of use, introduces lots of abstractions, and often doesn't take advantage of many TF optimizations, (for just one example until very recently Keras did not use TF's fused batch norm, which the TF docs claim provides a 10-30% speedup in overall network performance, which alone could be enough to account for many of the benchmarks showing PlaidML ahead).

In my opinion it's extremely fair. The benchmarks are Keras+PlaidML compared to Keras+TensorFlow, it allows running exactly the same nets (just imported from the Keras included applications) and whatever penalty Keras might impose is equal in the two cases. Having one very direct comparison is actually why we constructed the tests that way (none of the other frameworks run on our high priority platforms).

That said we'd be pretty excited if someone wanted to add support for TF, PyTorch, MXNet, etc. We like Keras but are happy to have integrations for all frameworks. With work you could pair it with Docker and containerize GPU-accelerated workloads without the guests even needing to know what hardware it's running on. Lots of possibilities.

No, no, no.

> whatever penalty Keras might impose is equal in the two cases.

The penalty Keras imposes when using Tensorflow depends on its Tensorflow implementation. The penalty Keras imposes when using MXNet depends on its MXNet implementation. The penalty Keras imposes when using PlaidML depends on whatever the PlaidML devs implemented. When you build a Keras layer, it's calling different Keras code for each backend.

The comparison would be fair if Plaid claimed to be the fastest Keras backed, not if it were actually claiming to be faster than Tensorflow.

There was someone on reddit/ml who posted some pretty interesting numbers for training.

I think they have a lot of challenges ahead of them, but I’m still more optimistic about Plaid than AMD’s own efforts.

AMD says that they don’t care about ML[1], and their actions back that up.

Edit: and to be clear, I think comparing Keras+Plaid vs Keras+TF is an entirely valid thing to do. Lots of people work in Keras, and if you download a random NN code off github it likely to be Keras (or Pytorch now of course).

[1] https://www.reddit.com/r/MachineLearning/comments/66bgmf/com...

PlaidML seems entirely geared towards inference (I only see batch size 1 anywhere). Training is important.

Batch 1 inference on convnets is key for us internally but training does work pretty well. The underlying machinery can do much more. Here's a blog post that talks about how it works with some links to more detailed docs & the actual implementations:

http://vertex.ai/blog/tile-a-new-language-for-machine-learni...

Two of the big motivators for opening the code were 1) giving students taking the popular courses a way to get started with GPU in whatever machine they've got (recent Intel GPUs in say a MacBook Air are enough) and 2) giving researchers a platform where it's simple to add efficient GPU-accelerated ops.

For scale on #2 check out the entire implementation of convolution:

https://github.com/plaidml/plaidml/blob/master/plaidml/keras...

PlaidML is very promising I agree.

> Amd does have a cudnn equivalent called MlOpen.

MIOpen could be great one day. ATM you still get random problems like this: https://github.com/ROCmSoftwarePlatform/MIOpen/issues/19

That's an order of magnitude worse performance than NVidia on ResNet 52 for 2 months with no real reason.

Great idea, but no one can use it reliably yet.

apparently Caffe has support for AMD cards through ROCm

https://rocm.github.io/dl.html

That's exactly what I mean.

It's a patchset (note that it isn't upstreamed in their chart), and it's caffe. That was great in 2015.

For most Deep Learning developers, using AMD is out of question until DL frameworks like TensorFlow start natively supporting them. I am not really sure if/when this is going to happen. AMD needs to really step up their game.

One would think that this announcement from Nvidia would speed up this development. Doesn't this new policy render tensorflow + Nvidia pretty much useless ?

No. Tesla and Quadro are definitely exempt from this potential update to the EULA, and you can have a lot of GeForce cards outside of a datacenter.

Then this is again the same kind of money grab as they have done with active stereo rendering - it used to work even on consumer-level hardware.

But then it got popular outside of research labs too, so Nvidia has decided to milk the cow and restrict this feature only to their Quadro line-up, with ATI following suit. The result has been that they have pretty much killed the market - apart from those research labs nobody is going to buy a $3-4k Quadro card that has otherwise the same performance as a $300 GeForce only for the stereo support.

I think this sort of artificial crippling/restricting of usage to force industrial customers to use the more expensive hw they don't need otherwise will only lead to a proliferation of task-specific ASICs and Nvidia will hurt only itself with it. The reason why people use Nvidia GPUs for parallel computing is cost and ubiquitous availability, not because there aren't other options. This move will only accelerate their development - for which there were no reasons until now.

A HIP version is available. https://github.com/ROCmSoftwarePlatform/hiptensorflow

Hip caffe is also quite well underway.

Why is it on AMD to support TensorFlow? Pretty sure it is on the machine learning people to use the standard APIs like OpenCL or Vulkan etc instead of CUDA...

The’re not running a non-profit. It’s on them to respond to the market.

Goes the other way around too. This whole debacle demonstrates how dumb it is to base your product on a proprietary API with just one vendor.

Here you go: https://github.com/tensorflow/tensorflow/issues/22

Companies doing ML do not and should not base their decisions on some hypothetical considerations about long-term effects on market health, or just present/future hardware cost. They work with what's available. It is market's job to correct. And it surely will now, as current situation becomes more of a problem.

Nobody wanted to do this, but the lousy state of OpenCL compared to CUDA left them no choice. It really is so far ahead that OpenCL was never an option. It’s on AMD to fix that, since NVIDIA certainly won’t.

No, because CUDA has first class C++ support, while Khronos only added C++ support when it became visible no one wants to keep using C, when better options are available in CUDA via PTX.

As with many things: it depends on who wants it more. In this case, I think AMD probably cares more.

That’s all fine and great but CUDA. It’s a huge moat and enterprises will likely just fit the bill and pass on the costs.

Apparently tensorflow is quite close to supporting opencl, and now that nvidia put a 10x multiple on their cards for ML use you can be sure that there will be an army of people making sure that opencl becomes the standard.

Just going to say, just because tensorflow supports it won't mean much. The ecosystem as a whole is still largely cuda. This includes the database tech(kinetica,mapd), the resource managers (mesos/yarn), not to mention HPC.

There's also just not much incentive for people to move to AMD here. There's a reason NVIDIA gives out these graphics cards like candy to academia. It's because they want to make their margins in data center while keeping the broader developer community locked in to cuda.

People on HN make broad and sweeping comments about "if you open it they will come and everything will be magically better".

It's a lot more complicated than that. The market incentives just aren't quite there yet. Could it happen one day? Yes. Will it happen today? No. It's going to take a lot more than this for other vendors to start supporting AMD.

What I will say: There will be competition and the space is heating up. AMD is one player.

Now let me put my commercial hat on here: What would it take for me as a deep learning vendor to support/care about AMD?

1. Show me the money. I need a clear revenue stream. AMD has some hope here. Customers don't care about which gpu they use as long as it fulfills a use case they care about.

2. Code: Show me something that's actually a robust. A 1 off fork of any deep learning framework (see the random 1 off caffe forks that aren't actually caffe by nvidia, intel,amd)

3. Share the burden. Put the code out there and support the broader community. Actually maintain and follow up with the latest innovations (hint: this is hard. throwing code over the wall once doesn't mean anything)

4. Amazon will be key here. Google cloud (look google is great but they aren't the leading cloud player by a long shot and likely won't be anytime soon) - get them on board with some AMD cards.

5. (Disclaimer I know the founders) - make opencl not suck. https://vertex.ai/ is an interesting player in this space.

So look, I won't say it's impossible. Let's just not ignore the actual commercial market forces that are also at play here.

There used to be this gigantic company that ruled networking world. Its name was Cisco. No one could touch it. Costs for a basic series XX with 4 slots was N, they charged X, for 6 slots, it was N+20, they charged 4X, for 16 slots it was N+200, they charged 30X.

What is Cisco is the current question.

> What is Cisco is the current question.

A $190 billion company earning 4x what nVidia does, a mere $9-$10 billion in net income, with 12x the cash of nVidia. A gigantic money printing machine, is what it sounds like they are.

Have you heard about the company called Juniper? Brocade? Arista? The only reason they got anywhere is because of cisco's "We are cisco.Eff you!" attitude.

Nowhere did I say it's never going to happen. These things take time to evolve. I even talked about the market forces and what's currently plausible.

There's always big companies that get disrupted because they kill good will. That's the start to something. It takes more than 1 move. Will NVIDIA make this mistake over time? Likely.

Will it be AMD and Tensorflow support that does it? Not by a long shot. It will be by a series of players providing the right incentives. My response (nor any response to this) should not be treated as binary. Ultimately in order for a company to be "disrupted" you need to actually analyze and exploit the market forces involved here. just writing this off as "open ecosystem is all we need" is dangerous.

I have to say I disagree with you on the AMD/TensorFlow point here, Adam. Your previous points are completely valid. Nothing of note will happen in 2018, I expect. But by early 2019, maybe - if AMD (or Intel) gets their act together on the software side of things. I don't think the "community" will do it for them. Buy maybe a big player like Amazon will have had enough of Nvidia and support an open alternative. Although we support Cuda in our version of YARN (Hops Hadoop), I expect we'd add ROCm or OpenCL or whatever - if it was a serious alternative. That would happen quickly. The problem, of course, is that we would want great support in TensorFlow first before we do that. Data scientists need a seamless transition - including from a performance perspective. For us, that also means support for distributed deep learning (Ring AllReduce over infiniband). I don't expect that will happen in 2018, and could take until 2020, if i'm being realistic. That means when AMD finally get some good DL libraries, Nvidia will still have one-up on them with distributed DL (reduce training time linearly with more GPUs).

The other wildcard i haven't seen people mention here is the Neural Network Processor (NNP) from Intel Nervana. The hardware has potential. As long as the software doesn't force us to use BigDL, it has potential.

I'd expect competition to heat up in 2018.

Re: AMD/Intel. We've been waiting for them to get their act together for years. Nervana could be great but I'm going to wait on that one. So far their "launches" have been nothing more than marketing fluff.

As for your projections about tensorflow. It won't be tensorflow. Tensorflow will be 1 of many frameworks. Look I like HOPS but you guys push tensorflow explicitly. A startup running its own hadoop distro that happens to push tensorflow isn't going to move the needle. You guys are great middleware I'm sure but I haven't seen the customers where it might be viable. I hope you guys continue to grow though! While you're doing that, most hadoop vendors are focused on moving up the stack. It will take people with actual resources to move the needle in terms of enterprise adoption.

Amazon is doing this with mxnet and EMR, MapR is pushing tensorflow in their serving. CNTK is being pushed in SQL server and HDInsights. There's some competition there.

What I'm getting at here is: it will take multiple vendors and competition. I'm going to place my bet on the bigger players already involved with the foundations first though. Open standards (addressed below) where it commoditizes the chip will be key. The storage infra will follow from that. It should be something that doesn't displace current infra but allows interop.

Things like nvvm from the mxnet folks, onnx (where the framework doesn't matter anymore!) being pushed by the various hardware vendors etc will move the needle. You need buy in from the actual big players who can upfront the development time in to making these things viable alternatives.

For your "seamless transition" I'm not sure that would be that hard done right. Supporting "great tensorflow" can come in multiple flavors. As a separate issue, tensorflow's production story is horrible. That's another topic I could rant about all day though. It ultimately comes from abstracting it away though. That by itself is a hard problem (Disclosure: I have my own solution for this that I won't talk about here just know I'm biased :D)

Lastly, I question whether opencl can even be a viable alternative. It's a fragmented inconsistent standard with a worse API than cuda. One reason it "won" is because it's in general cleaner and a clear leader in the space.

Yeah, ROCm is the most viable candidate as of today. In general, Nvidia are not good for middleware vendors. They want to be one, but don't offer a platform that integrates with anything. Licensing costs for the DGX-1 are insanely high. My problem is mostly from a data scientist perspective - teams don't need a few high performance GPUs, like a couple of DGX-1 boxes. They need a hundred 1080Tis, maybe complemented by a DGX-1 (which would cost the same a 2 DGX-1s). That way they can do lots of parallel experiments (hyperparam optimization), and distributed training. Making GPUs a scarce resource just re-inforces the lead of the hyperscale AI companies, who have thousands of GPUs available for their data scientists.

Oh I agree that GPUs should be more commodity. We might see alternative ASICs rather than GPUs come out though. I'm personally more interested in seeing that succeed than just confining the solution space to gpus and discrete gpu competition. I'm just not keen on trying to predict what will win (I really don't know) I just have criteria I would be looking for before trying to implement support for it in either my deep learning framework or trying to support anything for customers.

A dick move like this by Nvidia will only spur the development of alternatives to CUDA. CUDA isn't the only game in town but until now nobody had much reason to invest in improving the alternatives - CUDA was free, worked and the hw was cheap and ubiquitous.

Right so like my other replies, I don't claim that alternatives won't come up eventually. I also don't claim that this is exactly how nvidia can unseat themselves.

What's wrong here with many of the guesses people are making though: They think simple open standards are enough.

It's a lot more than that. Don't alienate the market forces that also need to move to make this work. Too many coders write off the real business incentives side of this.

Viable competition will come from multiple vendors and possibly backwards compatible/interchangeable standards that don't require many changes in code. Give me a path forward as a vendor. Show me the money. Put code out there and maintain it/keep it up. Show me it's going to stick around.

It's like any SAAS you'd pay for or an IDE you invest in, vendors want to see an ecosystem that fulfills a set of requirements and allows them to get work done for their customers.

Right now cuda is still that best tool. I covered what will likely need to be considered in my parent post so I won't reiterate that here.

I only ask that folks don't assume that "open source linux drivers and open standards and 1 open source framework with opencl support" are enough to unseat nvidia. The market won't move for that.

btw since you know the founders of vertex.ai you should tell them their cert is messed up they probably want to do something like https://hackernoon.com/set-up-ssl-on-github-pages-with-custo...

Thanks will pass it on!

Just curious, what cards are you comparing when referring to the 10x multiple?

is this a serious reply? do you know how bad is AMD's OpenCL support? how bad was the attitude?

have a look, people had to make a petition to ask AMD to fix their craps.

https://www.phoronix.com/forums/forum/linux-graphics-x-org-d...

How would a open www.change.org petition be any indicator of the current state of things? Or even indication of the 2013 state of things? For christ's sake, anyone can start a petition about anything.

Obviously the petitioner is also clueless, or else how could they write "We feel that Radeon hardware is vastly superior to competition", when their hardware in 2013 was nowhere near NVIDIA. (See the review the petition links to for an example...).

You sir are a typical phoronix reader. Uninformed, biased and thick.

Shameless plug for the tvm project: http://tvmlang.org/2017/10/30/Bringing-AMDGPUs-to-TVM-Stack-... which is an effort towards broad (many vendors) hardware support and optimization for deep learning.

tvm was the first thing I thought of when I heard about this.

If you're involved in the tvm project - are you aware of any major deep learning libraries working on utilizing it? I'd imagine MXNet would be the first one to try it, but I haven't seen anything.

I can't speak for AMD's support towards the Radeon/OpenCL community, but their GPU and CPU support towards the Linux community has been less than stellar.

And while slightly off topic, their CPUs contain just as much Evil as Intel's CPUs (PSP/ME). I wouldn't be so quick to call them a "principled" underdog.

It doesn't seem to be enforcable anyway, what is the point of this change?

We already have software licences which grant you a copyright usage only when certain conditions are met like:

1. restrict its software to be deployed at one physical or virtual computer.

2. Similar to one but restrict it to be deployed the specific vendor hardware.

3. Non-commercial usage only

4. Commercial usage allowed but only for small company less than 50 employees, buy more expensive license for that.

What's the difference?

If those ridiculously existing consumer-unfriendly licenses are valid, I think NVIDIA's stupid no-data-center-deployment-except-for-blockchain-processing license is also valid.

Who says that those are valid too?

> AMD does not have this restriction

But AMD/ATI cards are heavily bought by crypto miners, so it's almost impossible to buy one, even at an insanely high prize. And NVidia cards are compatible with all games made between 2000 and 2017. AMD driver were very bad several years ago. So if you are playing GoG games or older games occasionally there is only NVidia. Yes, both NVidia with their spyware-driver and now this shit, and AMD driver sucks. For CPUs everyone suggests now AMD which are now better than the crippled Intel CPUs with their plastic pads inside the CPU so that a new Intel CPU will last only about 3 years.

As a holder of AMD stock, I completely agree

Sadly I did find that with the latest gen $130 card (cant remember the model) there is no driver support in Debian stable, or the driver was broken (I can’t recall which). I have to run Debian unstable to use my card, which unfortunately has a broken ZFS driver...

I read that AMD cards were friendly to Linux, so after removing my NVIDIA card for stability reasons, I switched to AMD only to have other issues.

I’m still looking for an easy Debian compatible graphics card so I can do CAD (Onshape - OpenGL). The AMD card works now, but I don’t know what to do next time around.

Yes AMD have open-source version of their drivers for linux, so they are generally more stable/friendly than Nvidia.

I was using the open source drivers, but the version in Debian stretch does not support my card. I had to go to buster to get it working after tracing a long list of forum posts of people having trouble with this card.