These generations tend to be three years apart though, so if you're buying as the new generation comes out then your total TCO period has you running almost peak hardware (there were several versions of the V100, each with minor improvements). Many vendors also offer buy back and upgrade programs.

At the same time it's hard to understate how different the prices are here. Our break even point for using on prem compared to AWS was about nine months. After that we saved money for the rest of that hardwares lifetime.

I definitely agree that people shouldn't just rush out and buy these without benchmarking and examining their usecase. The cloud is really good for this! At the same time though I have yet to see any cloud provider with anything even approaching the interlink I can get on prem and that means, so it's basically impossible to get the same performance out of the cloud as it is on prem right now.