Re:Live-caption glasses let deaf people read conversations u...
by morcheeba on Tuesday March 14, 2023 @06:24PM
(#35159926)
Maybe a combination? Even simple beamforming/stereo would be helpful to help display the speaker's location. For example, the "speaker 1" tag could appear on the left, center, or right of the display to give a spatial clue where they are located.
[ Parent
| on HN ]
Re:Live-caption glasses let deaf people read conversations u...
by kajecounterhack on Thursday March 16, 2023 @01:26PM
(#35185135)
You could only display the speaker's location if you had some way to associate streams with the individuals. So you'd have to train an audio-visual association model.
The thing you could do is train a localizer on the separated audio. Phase is estimated by the source separation process, so you can actually train an ML model provided you have some ground truth (e.g. estimated human locations from camera detections)