Re:How blind players succeed at sports video games they’ve n...
by gojomo on Friday October 05, 2018 @02:40PM
(#18150629)
And of course, you'd want to ensure that the helper-agent'ss audio-cues remain pleasant to human ears – and not just computer-to-computer screeches.
To achieve this, you'd add yet another adversarial agent, that's trying to discriminate original audio from the helper-enhanced audio – without access to the video.
Only audio-help that successfully fools this 3rd agent, and is thus relatively-indistinguishable from the unassisted-audio, gets added to the game.
For example, can a 'blind' agent, that's fed just the (stereo) soundtrack of a game, become good at it? (While not, add more audio cues.)
Can another cooperative agent, which can see the screen but only emit audio, learn to help the blind-agent do better? (Train up this helper-agent during playtesting, then add its audio cues to the main game – either always, or in a special 'audio assist' mode.)
(Parts of this could wind up like a rotated version of this "sound of pixels" work:
http://sound-of-pixels.csail.mit.edu/
But, instead of learning which pixel-regions are responsible for sounds, it'd be adding sounds to second-medium-communicate relevant visual information.)