Do you often do these things in the middle of running a playbook? I would hope not. :)

It's not meant to have the connections persist forever, just long enough for repeated short ssh connections to have almost no overhead, and therefore be much faster.