"Hijack your keys" is somewhat overstating it, all they can do with it is log into other hosts that you can log into, assuming key auth is the only factor. And only while you are connected to the original host.

Terminal security bugs are not more or less serious than your average Chrome 0day.