As you say, the very fact that we know the real direction counters that. They mean that within the abstract context of electronics presented in introductory physics, the real direction of charge doesn't matter and cannot be determined. As long as you pick one consistent convention and stick to it, the math will always work out the same, since depending on convention, all the directions and signs are equally flipped. The real direction of charge only matters when you get deep into the details (eg semiconductors).
At the level of detail of introductory physics, it's effectively a symmetry, similar to how given the simultaneous flipping of charge, parity and time, you cannot tell the difference.
I would suspect that the same goes for electrons.
An electron gun would (as used in old CRT monitors), would be a very striking example of this - I doubt we could make an electron-hole gun (though shooting positive ions could work, but that's not quite the same thing) - but it may be possible to observe in more normal conditions too?
An electron gun produces a beam that contains only electrons; there is no conductor, and I think holes can exist only in the presence of a conductor. So you can't shoot a beam of holes through a vacuum. But if the material between the gun and the screen were a semiconductor, maybe you could draw pictures on the screen using a beam of holes? I mean, I don't see why a beam of holes can't be focused just like a beam of electrons.
Wouldn't electrons rush in to fill the holes from every direction, rather than just the intended one? That's what my intuition says anyway. So yeah, my guess would be that it is in fact not possible.
Nobody seems to have mentioned Holes. Holes are positive charge-carriers. Yeah - they're virtual, they're not like positrons or protons. But they behave just like electrons going "the other way".
My understanding is that a hole represents the absence of an electron. If an electron is removed (e.g. by rubbing), there's remains a physical object bearing a positive charge: the proton that was originally associated with that electron.
I haven't heard anyone talking about holes for years. Are they now deemed an outmoded concept?
[Edit: should have read further down the comments :-)]
It's also usually brought up early on when teaching new students about circuit physics, but it's really not something that comes up in an EE's day to day.
Maybe it's more relevant if you're deep into analog or RF black magic, I wouldn't know.
But there is--otherwise we wouldn't know that Franklin got it backwards. He thought the charge carriers were going one way, and chose the convention he did because he thought it matched the way the charge carriers were going, but it turns out they were going the other way. The signs of the charges are a convention--and the fact that we still use Franklin's convention and it works just fine attests to that--but the direction the charge carriers move is not.