I think it's useful to have, even if just to have a calibration of what priority level is "never get done" (which will probably change over time) and a list of things that have been considered and fall below that level. Obviously rearranging the priority of stuff below that threshold is pointless but that's a seperate problem to having the list.
But unbounded backlog is dumb. Obviously some things will just never be done and it's better to be honest and drop them rather than just let the backlog grow forever.
The usual bottleneck is that you don't know which tickets in the backlog are higher or lower priority. You have to look and prioritize the backlog and... prioritizing tickets that aren't important enough to work on now is the definition of fluffy bs things to spend time on.
So in practice, the backlog gets "prioritized" by being ignored. If something is important, it'll come up again and reality will force you to do it, not the issue sitting unlooked at in linear