Your dexterity is lost without practice, but I find that the new licks, chord progressions and other ideas you add to your repertoire to be used in improvisation stick around. This to me is far more valuable than the dexterity which I can reacquire in no time. My jazz voicings today are far better than they were at a time when I practiced a lot more piano.