CFD and other high-fidelity multiphysics simulations can be efficient when GPU-bound. It all comes down to being able to write a matrix and stream a bunch of vectors to multiply it by. When considering Krylov subspace methods, this extends to covering a large part of the problem space of linear algebra.

I think the Tesla cards in particular are double precision, which is not necessarily the best for deep learning applications.

High-fidelity multiphysics seems incompatible with FP16, unless you're talking some kind of mixed precision method.

multiplying and adding matrices are fairly fundamental operations.

Here is a starting point: https://en.wikipedia.org/wiki/Multiply%E2%80%93accumulate_op...

Yes but they are limited in size and in precision to FP16, which is not sufficient for most applications, especially scientific computing... What this person is asking I think is: Can you actually think of another concrete application?