The point is that in a video player, you can predict the future: the video will continue playing.
> Otherwise, you still have to wait for 100ms of samples to output
No, you don’t, not if you can write into the buffer at an arbitrary point. This is what the whole page I linked is about.
You can do that if you can predict the future and fill the buffer with data you don't know (samples from the future). Otherwise, you still have to wait for 100ms of samples to output. So if you have to wait 100ms for the samples, then the output happens after 100ms, hence 100ms of latency.
100ms samples can be fine for a video player. For real-time you (usually) do this: 2 buffers of 10ms each. While one buffer is playing, you fill another buffer with real-time data. After 10ms has passed, you start playing the buffer with real-time data, while the other buffer gets filled.