News for hackers. Stuff that matters.
Re:ChatGPT Hardware a Look at 8x Nvidia A100 Powering the Tool by synergy20 on Tuesday February 14, 2023 @04:14PM (#34796092)
it's way more than that, building an ecosystem takes time, like a decade or more, lots of tooling, libraries, optimizations etc have to be done, and they have to be done in a way users think it's either similar to CUDA, or is simpler, the moat is very wide and deep as far as I can tell.
Re:ChatGPT Hardware a Look at 8x Nvidia A100 Powering the Tool by fancyfredbot on Tuesday February 14, 2023 @04:26PM (#34796271)
I feel like very few ML researchers actually use CUDA. You'd likely get a long way with an optimised backend for XLA to cover tf/jax, probably something similar for pytorch. That's a lot of work but not sure it's a whole decade.
Re:ChatGPT Hardware a Look at 8x Nvidia A100 Powering the Tool by synergy20 on Tuesday February 14, 2023 @05:09PM (#34796879)
Not sure if that's entirely tree. CUDA has 85% or more of the market share last time I checked, intel's OneAPP and AMD's ROMA? are so behind that made their Machine learning hardware totally uninteresting for ML related purchases, sadly.
Re:ChatGPT Hardware a Look at 8x Nvidia A100 Powering the Tool by Const-me on Tuesday February 14, 2023 @05:15PM (#34796965)
I don’t think OpenAI needs a new ecosystem. I think they just need a cost-efficient way to run inference of their ML models, for large userbase.

That smaller task is way more manageable. It shouldn’t take a decade, maybe a month or two. Vulkan ain’t some new tech, the API arrived years ago and is widely used by game developers.

Thanks to Valve work on DXVK 2.0, there’s even an implementation of MS DirectCompute GPU API over Vulkan on Linux. I’m not saying doing that is necessarily the best approach for the job, just one of the many options.