The approach I usually go with is to initially create a full set of notes/slides that contain everything I'll want to cover, and then use a copy of that as the basis for my slides, which I then aggressively strip stuff out of. Then I've got the original notes still to refer to if needed while giving the talk, and to distribute.

Slides for during a presentation and slides/notes for reference without hearing the talk at the same time have two completely different needs to fill, and I don't think there's a one-piece solution to both. Especially since I feel the same way that, outside of a conference setting, referring to an audio/video recording of a presentation is much less efficient than a proper set of notes.