Maybe retention is worse, but I'm still better off. Higher speeds are more engaging and enjoyable to me, as bad speakers and long pauses matter less. Therefore I benefit from watching more talks and lectures. For better retention it is preferable to occasionally re-watch good talks for spaced repetition.

The Video Speed Controller extension is indispensable part of my browsing experience, mostly for keyboard shortcuts to speed up/down and << >> videos.