I had the experience of changing to colemak and then spending maybe two months not practicing deliberately. Progress was slow. But then, I started practicing every day and I was back to my qwerty speeds within a few weeks. The incidental practice you get from typing in your everyday life isn't enough to quickly get learn a new layout, unfortunately.