Exactly, that's the worry. If two similar updates happen close in time, orthogonality forces them apart for no reason. I like your weight preference idea. Maybe just apply orthogonality when updates are actually different, not when they are nearly the same. Probably you should try that if not tried it already. :)