I'm not sure an IP whitelist gives you anything -- it provides a handy mechanism for escalating privileges, both for intrusions on a machine at the "known-good" IP and for unintended network access in general.