Not all routers support this. In fact at work we've got a router that seems to black-hole anything whose source or destination isn't in the DHCP table (TP-Link 480t+).

In the end I solved this by accessing this only while I'm tethered, that fixes the phone's IP address.

Too bad there doesn't seem to be an option to select keys according to host key instead of hostname or alias.