News for hackers. Stuff that matters.
Re:Show HN: Attaching to a virtual GPU over TCP by steelbrain on Friday August 09, 2024 @01:47PM (#41203918)
Ah this is quite interesting! I had a usecase where I needed a GPU-over-IP but only for transcoding videos. I had a not-so-powerful AMD GPU in my homelab server that somehow kept crashing the kernel any time I tried to encode videos with it and also an NVIDIA RTX 3080 in a gaming machine.

So I wrote https://github.com/steelbrain/ffmpeg-over-ip and had the server running in the windows machine and the client in the media server (could be plex, emby, jellyfin etc) and it worked flawlessly.

Re:Show HN: Attaching to a virtual GPU over TCP by bhaney on Friday August 09, 2024 @03:17PM (#41204586)
This is more or less what I was hoping for when I saw the submission title. Was disappointed to see that the submission wasn't actually a useful generic tool but instead a paid cloud service. Of course the real content is in the comments.

As an aside, are there any uses for GPU-over-network other than video encoding? The increased latency seems like it would prohibit anything machine learning related or graphics intensive.

Re:Show HN: Attaching to a virtual GPU over TCP by trws on Friday August 09, 2024 @05:56PM (#41205686)
Some computation tasks can tolerate the latency if they’re written with enough overlap and can keep enough of the data resident, but they usually need more performant networking than this. See older efforts like rcuda for remote cuda over infiniband as an example. It’s not ideal, but sometimes worth it. Usually the win is in taking a multi-GPU app and giving it 16 or 32 of them rather than a single remote GPU though.
Re:Show HN: Attaching to a virtual GPU over TCP by tommsy64 on Friday August 09, 2024 @08:02PM (#41206401)
There is a GPU-over-network software called Juice [1]. I've used it on AWS for running CPU-intensive workloads that also happen to need some GPU without needing to use a huge GPU instance. I was able to use a small GPU instance, which had just 4 CPU cores, and stream its GPU to one with 128 CPU cores.

I found Juice to work decently for graphical applications too (e.g., games, CAD software). Latency was about what you'd expect for video encode + decode + network: 5-20ms on a LAN if I recall correctly.

[1] - https://github.com/Juice-Labs/Juice-Labs

Re:Show HN: Attaching to a virtual GPU over TCP by Fnoord on Saturday August 10, 2024 @12:42AM (#41207361)
I mean, anything you use a GPU/TPU for could benefit.

IPMI and such could use it. Like, for example, Proxmox could use it. Machine learning tasks (like Frigate) and hashcat could also use such. All in theory, of course. Many tasks use VNC right now, or SPICE. The ability to extract your GPU in the Unix way over TCP/IP is powerful. Though Node.js would not be the way I'd want such to go.

Re:Show HN: Attaching to a virtual GPU over TCP by lostmsu on Saturday August 10, 2024 @01:29AM (#41207533)
How do you use it for video encoding/decoding? Won't the uncompressed video (input for encoding or output of decoding) be too large to transmit over network practically?
Re:Show HN: Attaching to a virtual GPU over TCP by bhaney on Saturday August 10, 2024 @04:47AM (#41208181)
Well, the ffmpeg-over-ip tool in the GP does it by just not sending uncompressed video. It's more of an ffmpeg server where the server is implicitly expected to have access to a GPU that the client doesn't have, and only compressed video is being sent back and forth in the form of video streams that would normally be the input and output of ffmpeg. It's not a generic GPU server that tries to push a whole PCI bus over the network, which I personally think is a bit of a fool's errand and doomed to never be particularly useful to existing generic workloads. It would work if you very carefully redesign the workload to not take advantage of a GPU's typical high bandwidth and low latency, but if you have to do that then what's the point of trying to abstract over the device layer? Better to work at a higher level of abstraction where you can optimize for your particular application, rather than a lower level that you can't possibly implement well and then have to completely redo the higher levels anyway to work with it.
Re:Show HN: Attaching to a virtual GPU over TCP by lostmsu on Saturday August 10, 2024 @02:24PM (#41211299)
Ah, you mean transcoding scenarios. Like it can't encode my screen capture.
Re:Show HN: Attaching to a virtual GPU over TCP by johnisgood on Saturday August 10, 2024 @05:56AM (#41208424)
I am increasingly growing tired of these "cloud" services, paid or not. :/
Re:Show HN: Attaching to a virtual GPU over TCP by adwn on Saturday August 10, 2024 @06:47AM (#41208620)
Well, feel free to spend your own time on writing such a tool and releasing it as Open Source. That would be a really cool project! Until then, don't complain that others aren't willing to donate a significant amount of their work to the public.
Re:Show HN: Attaching to a virtual GPU over TCP by johnisgood on Saturday August 10, 2024 @07:37AM (#41208779)
Sounds angry. I have spent a lot of time contributing to open source projects, many of which I created by myself, what is your point? I still prefer being able to self-host.
Re:Show HN: Attaching to a virtual GPU over TCP by rowanG077 on Saturday August 10, 2024 @08:40AM (#41209045)
There is a vast gap between walled garden cloud service rent seeking and giving away software as open source. In the olden days you could buy software licenses to run it wherever you wanted.
Re:Show HN: Attaching to a virtual GPU over TCP by mhuffman on Saturday August 10, 2024 @05:32PM (#41212373)
I agree. Paying by the month for the rest of your life or they cut you off is not something I am a fan of. I feel sorry for people too young to remember that you could actually buy an app, get free bug updates and get a discount if they made some big changes on a new version that you might (or might not) want. But it was up to you when and where you ran it and it was yours forever. I have heard the arguments for why people enjoy this monthly subscription model, but my counter argument is that people did just fine before without them, so what is so different now? And I mean, in general, not that you need to use a GPU for 1 hour but don't want to buy one. I mean, for example, how Adobe products run on your computer but you rent them forever.
Re:Show HN: Attaching to a virtual GPU over TCP by crishoj on Friday August 09, 2024 @02:44PM (#41204329)
Interesting. Do you know if your tool supports conversions resulting in multiple files, such as HLS and its myriad of timeslice files?
Re:Show HN: Attaching to a virtual GPU over TCP by steelbrain on Saturday August 10, 2024 @06:07AM (#41208463)
Since it’s sharing the underlying file system and just running ffmpeg remotely, it should support any variation of outputs
Re:Show HN: Attaching to a virtual GPU over TCP by toomuchtodo on Friday August 09, 2024 @04:42PM (#41205253)