If you want maximum flexibility and least time wasted, go with NVIDIA. Most models, Github repos, papers, online questions - are on NVIDIA and the authors probably never bothered with any other architecture.

UPD link to GitHub issues:

https://github.com/RadeonOpenCompute/ROCm/issues/887

"One big benefit of using the packaged driver: working ROCm OpenCL compute! OpenCL compute benchmarks on the packaged driver are coming in a separate article today. The OpenCL/compute support for Radeon RX 6800 series is working and better off than the current RDNA 1 support. It's great to see things working in the limited testing thus far. Presumably for the imminent ROCm 4.0 release they will also have the Big Navi support independent of the packaged driver, but in any case great to see it coming."

> One big benefit of using the packaged driver: working ROCm OpenCL compute! OpenCL compute benchmarks on the packaged driver are coming in a separate article today. The OpenCL/compute support for Radeon RX 6800 series is working and better off than the current RDNA 1 support. It's great to see things working in the limited testing thus far.

Colab is free. Colab Pro is $10/mo. GPU instances give you access to better hardware and don't lock up your machine for hours/days at a time.

Colab notebooks are proving to be one of the most incredible resources out there right now. I liken it to the "free tier" cloud vm / app engine / heroku of a decade ago ;)

Wouldn’t it make more sense to let the CPU do it, or have specialized cards for the job. So you could have an AMD graphics card and an Nvidia ML card.

Graphics cards are the specialized cards for ML. They are several orders of magnitude faster than GP CPUs at ML because they are specifically designed for the mathematical calculations done in most ML.

We call them graphics cards because that was the first real general use-case for doing lots of linear algebra on large data sets. But long before CUDA came along, people were abusing graphics API to perform more general purpose high-performance calculations for things like video decompression or physics.

nVidia has good leadership and recognized that there would soon be a sizeable market for video cards as a cheap replacement for specialized hardware used in super computers.

Interesting tidbit, when I was in college, one of our CS professors invested in a cluster of PS3s as a cheap "super computer" as they were substantially faster at ML than anything else you could get for $600.

CPUs are not faster, in fact they are several orders of magnitude slower for these operations.

Typically you have to wait some time after launch (e.g. 3-6 months) before AMD gets their Linux drivers into a stable state.

If it becomes serious, rent cloud systems so you don't have to do all the maintenance.

Google doesn’t sell any hardware for ML training, but they offer Cloud TPU as a service for that.