You can simply delay your "yak shaving with unknown problems" until too late. Your velocity with simple things will be fine just until few weeks (or days) before deadline. And then you're toast. ;)
So you should take into account uncertainty and importance.
First you should do all uncrtain and important things, then certain and important and, finally, certain and less important. And your velocity will vary greatly then, very slow at the beginning and fast at the end. It is almost meaningless, I say.
Your velocity maps points (assumed difficulty) to real time. Once you've done a few tasks - some of which will have involved massive yak shaving - your velocity figure starts to take in to account estimating for unknown problems.
Additionally you're meant to estimate in 'points' against other tasks - was this as difficult as x? And never never ever use hours for the estimation.