News for hackers. Stuff that matters.
Re:Why ML Needs a New Programming Language by benreesman on Friday September 05, 2025 @06:56PM (#45144664)
I'm in kind of a different place with it on the inference side.

I've got these crazy tuned up CUDA kernels that are relatively straightforward to build in isolation and really where all the magic happens, and there's this new CUTLASS 3 stuff and modern C++ can call it all trivially.

And then there's this increasingly thin film of torch crap that's just this side of unbuildable and drags in this reference counting and broken setup.py and it's a bunch of up and down projections to the real hacker shit.

I'm thinking I'm about one squeeze of the toothpaste tube from just squuezing that junk out and having a nice, clean, well-groomed C++ program that can link anything and link into anything.

Re:Why ML Needs a New Programming Language by pjmlp on Saturday September 06, 2025 @01:43AM (#45146909)
CUTLASS 4 has first class support for Python.
Re:Why ML Needs a New Programming Language by saagarjha on Saturday September 06, 2025 @05:12AM (#45147770)
In fact I doubt the C++ API will be getting much love moving forward
Re:Why ML Needs a New Programming Language by pjmlp on Saturday September 06, 2025 @05:30AM (#45147855)
At GTC 2025, NVidia introduced two major changes in CUDA ecosystem.

First class support for Python JIT/DSLs across the whole ecosystem.

Change the way C++ is used and taught, more focused on standard C++ support and libraries, than low level CUDA extensions.

So in a way, I think you're kind of right.

Re:Why ML Needs a New Programming Language by benreesman on Saturday September 06, 2025 @10:18AM (#45149470)
Nah, their people are way involved in mdarray and ROCm is looking to have the "oh no its broken again" bit flipped off in the RDNA 4/5 cycle.

NVIDIA wants Python and C++ people, they want a new thing to moat up on, and they know it has to be legitimately good to defy economic gravity on chips a lot of companies can design and fan now.

Re:Why ML Needs a New Programming Language by pjmlp on Saturday September 06, 2025 @10:35AM (#45149634)
Intel and AMD don't have anyone but themselves to blame.

And Khronos always expecting the community to do the work for their standards, filling in the missing pieces for actually excellence in developer experience.

Re:Why ML Needs a New Programming Language by benreesman on Saturday September 06, 2025 @11:43AM (#45150267)
Well I blame our legislators, our regulators, and a public who tolerates low integrity, low competence leadership. A society gets what it pays for with the standards it sets for its leaders.

Out of a great many outcomes a very topical one today is semiconductors, and the outcomes rival any on anything for corrupt, incompetent, and entirely consistent with our speed run of the road to irrelevance.

Re:Why ML Needs a New Programming Language by bigyabai on Saturday September 06, 2025 @10:19PM (#45154804)
You don't seem to know what Khronos even is, at this point. It is a nonprofit consortium, joining the group for the community effort is the only reason it exists.

By blaming Khronos you're really just reiterating blame on Intel and AMD with the tacit inclusion of Apple. I suppose you could blame Nvidia for not giving away their IP, but they were a staunch OpenCL supporter from the start.