It's not strictly law, but at this point it might as well be. The last PM to be appointed without being an MP was Sir Alec Douglas-Home in the 1960s, and he immediately resigned his peerage and won a by-election to remedy that...