I'm currently using XFCE on 3x4k monitors on a desktop. You can control almost everything with the DPI setting in Settings > Appearance > Fonts and use the Display menu to adjust your screen resolution. (The font sizes do reflect correctly into Firefox, though - no tweaks needed there at all. One thing that I've noticed is that you have to completely quit or close some apps before the new DPI settings are used.)

I also frequently connect my XFCE laptop (only 1920x1080 native) running a totally different distribution to a 4K TV.

This requires a visit to the Appearance menu (as above) every time I plug/unplug in order to adjust font sizes. (Or, I could just run the 4k at 1080p, but either way it needs an adjustment.) This doesn't actually bother me, since it's just one step and I could script it if I did it all the time.

As far as using your laptop screen as a second screen, I've never really cared for that as a way to work anyway because of the size disparity (my 4k monitors/tv's are all much, much bigger than my little 13" laptop), but you could play with the resolution on each -- but you're right; the DPI settings in XFCE are for the whole desktop (and all apps, including Firefox), not per screen.