I don't doubt that but on today's web stacks you use either MySQL or Postgres typically and both those are written in C.
Yes, you can create your own DB written in a low level language, but you'll be hitting the same issues that the DB solves for you (indexing upon insertions, query optimizations, transactional operations, etc).
That's not how thing work. GPUs/TPUs aren't magic, they don't speed everything up: they are really good is some scenario, but they are really slow in many others: the cost of loading memory to the GPU and back from it has a huge cost and it's only worth it if the GPU has enough work to do with the workload.
In theory, you could write a web server that runs on the GPU (at least a majority of it, it will still need some CPU work obviously), but it would be incredibly slow, probably slower than if it was a regular web server running on a machine from the 90s.
FPGA are totally different beasts, you can theoretically speed up anything with them, but
1. that requires a LOT of work, and
2. they aren't new: anything you can do with FPGA today, you could do it in 1995 already.
But it's not true today, because today hand-tuned assembly is slow compared to what you can do on the GPU or an FPGA or a TPU.
And, of course, it's not true that you should do this if you can't spend unlimited time writing your code. It's entirely reasonable to use slower languages like Python and JS if you can hack faster as a result. Since many of us are taking that entirely reasonable option, the number of CPU cycles needed to run basic applications is going up, which increases power consumption.