News for hackers. Stuff that matters.
Re:StreamPot: Run FFmpeg as an API with fluent-FFmpeg compat... by jjcm on Sunday July 28, 2024 @04:53PM (#41095887)
Highly prefer and recommend websocket connections for polling. This is what I ended up doing for my side project's video encoding needs[1]. It lets me get silky smooth progress indicators on multiple encoding resolutions at once.

[1] https://github.com/jjcm/nonio-video-cdn/blob/master/route/en...

Re:StreamPot: Run FFmpeg as an API with fluent-FFmpeg compat... by LoganDark on Sunday July 28, 2024 @05:47PM (#41096221)
Do you mean over* polling (instead of polling)? Polling (repeatedly asking the server) and pushing (the server telling you directly) are two different things; polling over WebSocket seems sort of weird.
Re:StreamPot: Run FFmpeg as an API with fluent-FFmpeg compat... by inopinatus on Monday July 29, 2024 @04:47AM (#41098911)
A push over a websocket is still fundamentally a polling behaviour; it’s just happening further down the stack, below the awareness of application code on the client and implemented internally on the server. This does save you an explicit round-trip in your own code though.

Ultimately the only element that isn’t polling in a classical machine architecture is the interrupt handler in the top half of the kernel (or system equivalent).

Re:StreamPot: Run FFmpeg as an API with fluent-FFmpeg compat... by josephg on Monday July 29, 2024 @05:38AM (#41099160)
> A push over a websocket is still fundamentally a polling behaviour; it’s just happening further down the stack, below the awareness of application code on the client and implemented internally on the server.

Not necessarily. The server may internally use interprocess signalling or sockets or something to find out when the process is ready instead of busy-waiting. For example, if ffmpeg is run as a child process, the parent process can sleep a thread waiting for the child to terminate.

Just about any sort of busy-wait polling behaviour is a code smell in my book. Stop wasting cycles. Let the thing you're waiting on tell you when its ready. If it can't, fix it until it can. This is all opensource code.

Re:StreamPot: Run FFmpeg as an API with fluent-FFmpeg compat... by inopinatus on Monday July 29, 2024 @08:46AM (#41100023)
That will also eventually be polling, within a kernel subsystem or library function that one has merely chosen to perceive as a black box for convenience of reasoning.

Seriously, unless you know we’re delivering an interrupt or a signal, assume it’s polling all the way down. And I’m not so sure about the signals.

Re:StreamPot: Run FFmpeg as an API with fluent-FFmpeg compat... by josephg on Monday July 29, 2024 @09:07AM (#41100168)
When my program is waiting to read from a network stream, or from a USB device, it uses 0% CPU and responds instantly when the signal comes in. That is how software should be.

Maybe there's cases in the kernel where polling is unavoidable. But polling should happen as little as possible - for the sake of both latency and efficiency. I don't spend much time thinking about the kernel. In userland, polling is almost never necessary. I'd love it if just about all software that manages data that changes over time was written such that downstream consumers can be notified when that data changes. This should be the case for filesystems, databases, message queues, web servers, device lists over USB and so on.

Re:StreamPot: Run FFmpeg as an API with fluent-FFmpeg compat... by inopinatus on Monday July 29, 2024 @12:41PM (#41101736)
It isn’t.
Re:StreamPot: Run FFmpeg as an API with fluent-FFmpeg compat... by LoganDark on Monday July 29, 2024 @04:53AM (#41098932)
Servers doing polling internally and sending you periodic status updates over WebSocket is relatively common. I don't think polling over WebSocket is a good idea in this situation though (as in sending periodic requests to the server asking for a reply).
Re:StreamPot: Run FFmpeg as an API with fluent-FFmpeg compat... by mort96 on Monday July 29, 2024 @05:53AM (#41099229)
Eh adding websockets is a significant complexity jump, in my experience it's hard to write websocket client code which doesn't ever end up in a bad state despite connection breaks, and I still haven't figured out how to avoid random long delays in creating a websocket in my own projects. If you only need a completion callback and not continuous progress updates, long polling is an excellent solution which adds very little complexity compared to websockets.