You do need a decent magnetic field next to the pacemaker - having it near a fridge magnet while you open the fridge looking to sneak a snack isn't going to do it.
I used to work as a neuro tech, and we would occasionally reprogram vagus nerve stimulators (sort of a defibrillator for the vagus nerve). Every few minutes it would send a charge up the vagus nerve, which could scramble rhythms that might be developing into a seizure. You could dell when it was firing if the person was talking - every few minutes, their voice would go funny for a couple of seconds as the vocal chords are innervated by the vagus. It wasn't the most pleasant experience, but you did get used to it, and it was better than the general level of seizures you'd have to have to get one implanted in the first place.
Edit: Implanted defibrillators, not pacemakers