It's certainly around 10-15% and it's not CPU bound, as the benchmark uses synthetic images without the need for data loading and preprocessing.
I cannot rerun them now for precise numbers because my ROCm packages are broken and I don't have the spare hours to fix things. Surely it can be fixed but needs fiddling.
It is just that "more normal" result that you would expect to get, would be a drastic differences in performance. Note, Cvikli comment on "May 12" on that GitHub page:
* The result with RNN networks on 1 Radeon VII and 1080ti was close to the same
* Comparing convolutional performance the 4AMD and 4Nvidia, difference got really huge because of cuDNN for Nvidia cards. We can get more than 10x performance from the 1080Ti than the Radeon VII card. We find this difference in speed a little too big at image recognition cuDNN, I can't believe that this should happen and the hardware shouldn't be able to achieve the same.
Note also in that comment, it looks like 4xGPU system just didn't work.
It is hard to fully utilize GPU resources. Getting same result normally means that the same bottleneck is being hit. Algorithmic differences could result in drastic performance differences (10x). And as a final point of small things making a difference, note it is 1080Ti in that comment.
The benchmark is not CPU bound, as evidenced by looking at rocm-smi, nvidia-smi (GPU usage at 100%) and htop (low CPU usage).
[1] https://developer.nvidia.com/cudnn . [2] https://scholar.google.com/scholar?as_ylo=2019&q=nvidia+gemm
Not sure what that means. I did this: https://news.ycombinator.com/item?id=21666411
I run the fan at 100% and even after several thousand iterations it keeps its speed at 274-275 im/sec and 68 C temperature with the stock fan and a standard desktop chassis kept open.
Is this 90% of fp32 performance, of 90% of fp16 performance?
Anyways, my complaints may be related partially to me using the xenial (Ubuntu 16.04) repo on bionic (18.04), because the there is no support for 18.04 LTS, even though it came out more than one and a half years ago. I also had to downgrade the kernel from 5.0 to 4.15 as 5.0 is not supported. I also had to force the installation of several 32-bit libraries on this 64-bit system, which took quite some googling and trial and error, otherwise installation didn't succeed. Little things like that.
But certainly bfloat16 sounds attractive with its larger dynamic range, as float16 requires workarounds like loss scaling to avoid rounding down to zero when the exponent gets too small.
Minor nitpick, but it looks like the Radeon vii is in the same tier as the 1080ti, and actually performs worse [1] in benchmarks.
1. https://gpu.userbenchmark.com/Compare/Nvidia-GTX-1080-Ti-vs-...
TensorFlow 1.14.4
git clone https://github.com/lambdal/lambda-tensorflow-benchmark.git --recursive
python lambda-tensorflow-benchmark/benchmarks/scripts/tf_cnn_benchmarks/tf_cnn_benchmarks.py --optimizer=sgd --model=resnet50 --num_gpus=1 --batch_size=64 --variable_update=replicated --distortions=false --num_batches=10000 --data_name=imagenet
FP32
- NVIDIA GTX 1080 Ti: ~215 images/sec
- NVIDIA RTX 2080 Ti: ~300 images/sec
- NVIDIA TITAN RTX: ~320 images/sec
- NVIDIA Tesla V100: ~383 images/sec
- AMD Radeon VII: ~275 images/sec
FP16
- NVIDIA GTX 1080 Ti: ~277 images/sec
- NVIDIA RTX 2080 Ti: ~495 images/sec
- NVIDIA TITAN RTX: ~518 images/sec
- NVIDIA Tesla V100: ~725 images/sec
- AMD Radeon VII: ~373 images/sec
So in FP32 for this ResNet50 benchmark the Radeon VII is about 9-10% slower than the 2080 Ti and about 25-30% faster than the 1080 Ti.In FP16 it is 25% slower than the 2080 Ti and 35% faster than the 1080 Ti.
Sure, it's just one benchmark, but ResNet is quite representative of deep learning architectures in vision.
Here it performs VERY well for machine learning. Look at PlaidML.
However, the tooling and installation procedure is still rough around the edges and the documentation is lacking.
Last time I tried to upgrade the rocm things and the drivers, and bumped up against an issue that one of the packages tried to write to /data/jenkins_workspace during installation. We have /data as an NFS mount and is not writable even for the root user, so the installation broke down. Worked around it, but there are other issues as well. The order of packages you install can matter as well, I had to purge packages and reinstall them a few times. It worked with TF 1.14.1 but not with 1.14.0 if I recall correctly. The first few iterations are very slow, it seems it needs to find the optimal conv implementation for all kernel sizes separately.
Also, the card does not have anything like NVIDIA's tensor cores for half-precision (float16/FP16) computation. It can do FP16 but won't speed up and delivers worse results (in terms of prediction accuracy) than NVIDIA cards running FP16. There is no prediction accuracy difference in FP32.
And some other small issues here and there. So be warned and test it yourself on one card before investing in big multi-GPU machines.
But I do wish AMD entering the field will benefit us through increased competition in the long run. But at the moment ROCm seems like just a side project for a small team in AMD and they can't yet deliver the streamlined experience we're used to from CUDA.