FPGA is great if you need to talk to some hardware very fast/on many pins. E.g. something like a network router where you are shuffling packets between many high speed interfaces. Or doing a lot of measurements/interfacing a bunch of high speed sensors.
But not for general purpose software - GPUs are both faster, easier to develop for (and with good tooling) and much cheaper for doing that today.
Also, did they try to use enhanced locality introduced by processing several streams on GPU? E.g., if you keep states sorted as for tuple (state id, stream id) for all your streams, you may get more memory-controller friendly access pattern. I haven't seen mentions of that technique (which MUST be considered after Big Hero 6 [1] - they used that technique to never miss caches in whole movie rendering process). Big Hero 6 is 2014, the paper is 2017.
[1] https://en.wikipedia.org/wiki/Big_Hero_6_(film)
I really do not like papers like one linked by you. One system gets all of the treatment while other ones get... whatever is left.
I guess have they tried to use these techniques for GPUs, they would get performance gap that is much less than reported.
If Intel can release a CPU with a built-in FPGA and everyone has one, software developers will take advantage of them. I can see stuff like video editing programs, compression algorithms, etc taking advantage of that.