Oh I know for sure, it's still just amusing to me to look up these just slightly older cards and see them going for so little compared to what they were new;

I do also look at various benchmarks and comparisons over and over because I would like to build a home gpu server for some machine learning fun and I find it interesting that the TFLOPS/$ numbers always come out very much linear... A card that gives 2x performance costs 2x more.... if cards depreciate 50%, it's because new cards have been launched that have double performance for the same price... etc... Even simpler, the CUDA cores-per-dollar is pretty much a constant.

What isn't taken into account there is various optimizations new cards have that speed up training or inference, like better use of the tensor cores. At some point incompatibility with newer CUDA compute levels makes the older cards much less useable.

Not directly applicable for large language models, I was doing some searching for cards with lots of double-precision calc performance. Nvidia has been leaving that out of all of the consumer versions of hardware; cheap P100's seem to be by far the highest DP FLOPS per dollar solution available right now.

I have an interest in doing lattice Boltzmann fluid dynamics simulation on GPUs, and this is something I've noticed with the P100's as well. That and memory bandwidth is super important for such calcs, along with total memory. Commercial codes I've seen aren't using any ML ops compute optimizations or tensor cores and anything like that, it's mostly just OpenCL based and thus the P100s throughput per dollar is pretty high.

MI25

Faster, cheaper, crippled by AMDs poor software support.