At that point we might as well skip the glasses entirely and have it output directly to the visual/audio cortex (with shut offs, of course)

I's possible that communicating brain > computer is much simpler than the other direction. I would expect that to be the case.