News for hackers. Stuff that matters.
Re:ILGPU: Write GPU programs with C# and F# by jvanderbot on Friday May 17, 2024 @06:45PM (#40394860)
I'm going through the CUDA courses. I've done GPU and CPU optimization as an enthusiastic amateur in my day job once or twice a year, but it's not my core focus.

It quickly seems that the low level C/Cpp is becoming obsolete, and it's hard to squeeze performance unless you're doing something truly green field / new. Otherwise someone has already optimized the hell out of it.

So what's the use case for porting GPU to higher level languages like C#? What would you use this for?

Re:ILGPU: Write GPU programs with C# and F# by almostgotcaught on Friday May 17, 2024 @07:32PM (#40395152)
> It quickly seems that the low level C/Cpp is becoming obsolete, and it's hard to squeeze performance unless you're doing something truly green field / new. Otherwise someone has already optimized the hell out of it.

I hear and see both of these sentiments frequently from the internet crowd (on-lookers). It's both wrong and humorously arrogant. I'll repeat what I said to someone on reddit yesterday: there are thousands (maybe 10s?) scattered around the FAANGs, NVIDIA, AMD, Intel, accelerator startups, boutiques, etc. whose day to day is both C/C++ and squeezing perf out of kernels and getting many points (sometimes 10s) improvement. Certainly the wins aren't daily but I'm saying they do steadily find room for improvement. How is that possible? I'll give you a hint: platforms, demands, hardware, use-cases all change essentially on a quarterly basis.

So before you proclaim victory on behalf of whatever high-level framework, ask yourself if you're really familiar with the production and business environment for this kind of code.

Re:ILGPU: Write GPU programs with C# and F# by jvanderbot on Friday May 17, 2024 @07:40PM (#40395212)
As a learner and self proclaimed amateur, I assume the vitriolic tone is directed elsewhere. Without that, you're saying that performant pipelines are definitely in demand and optimization is still a full time job. That's good! I'd like to get better at that.
Re:ILGPU: Write GPU programs with C# and F# by almostgotcaught on Friday May 17, 2024 @07:52PM (#40395286)
>I assume the vitriolic tone is directed elsewhere

I'm always really shocked on hn when people getting called out for arrogance respond with accusations (of vitriol). What day job do you have where you can make proclamations like "It quickly seems that the low level C/Cpp is becoming obsolete" while simultaneously admitting being an amateur and not get checked. Must be very different from my day job where being precise and accurate and conservative is paramount. Moreover what kind of habit of thought are you in that being called out for that is read as vitriol?

Re:ILGPU: Write GPU programs with C# and F# by ClassyJacket on Friday May 17, 2024 @08:17PM (#40395434)
Oh my god, calm down. Jesus christ.
Re:ILGPU: Write GPU programs with C# and F# by almostgotcaught on Friday May 17, 2024 @08:22PM (#40395457)
Same thing applies here: I'm not the one the one that's not calm. I very calmly pointed out that op is wrong. That's it. You painting that as "not calm" is the only abnormal thing going on here.
Re:ILGPU: Write GPU programs with C# and F# by jvanderbot on Friday May 17, 2024 @10:36PM (#40396116)
You can interpret text as being uncalm, but there's always room for misunderstanding, esp in a forum/thread. Nobody is actually upset I think, just using direct language.
Re:ILGPU: Write GPU programs with C# and F# by other-guy on Sunday May 19, 2024 @10:46AM (#40407313)
Calm or not, when you called their behavior "humorously arrogant" it set the tone of the rest of your comment to condescension. It distracts from the on-topic discussion, despite any informative value contained in your comment.

When people post like this it's hard to tell if they are trying to have a conversation in good faith, or they are seeking pleasure in the form of belittling someone.

Re:ILGPU: Write GPU programs with C# and F# by almostgotcaught on Sunday May 19, 2024 @01:56PM (#40408546)
> "humorously arrogant" it set the tone of the rest of your comment to condescension

it was as condescending as the original comment was arrogant. what exactly is the confusion here? if you want to a community where people are allowed to make bold claims without doing sufficient research (hn) then you should also allow for those people to treated as such.

Re:ILGPU: Write GPU programs with C# and F# by denom on Friday May 17, 2024 @09:33PM (#40395831)
> Moreover what kind of habit of thought are you in that being called out for that is read as vitriol?

That’s as much a matter of personal identity as a ‘habit of thought’, friend. Otherwise, we could we comment on the basis of _your_ perspective as some mutable affectation, no?

You see, we are shadows of ourselves. An inquisitive play, or let’s say a “judgment” about Cuda economics: aspirational, errant? Then, the linguistic phalanx: honed no doubt by long running needs to be heard and seen and listened to.

Re:ILGPU: Write GPU programs with C# and F# by jvanderbot on Friday May 17, 2024 @10:31PM (#40396094)
"it seems" combined with "I'm learning" was meant to convey ignorance not arrogance. I think I'm just being misread and that's probably my fault. It wouldn't be the first time - so I'm sorry if you found it off-putting

My day job is optimization for logistics and multi-agent systems on edge deployments. We vary between heterogenous compute environment on-platform and occasional connections to cloud-like servers, but only occasionally. The system has a lot to do and we're pushing what it can do for itself.

I like these problems and am always trying to find new ways to solve them. That's really it, I'm wondering if it's best to focus on learning to work the hardware as is, or learning to twist existing libraries into new problems.

For example if you look at this: https://weightythoughts.com/p/cuda-is-still-a-giant-moat-for...

It actually denigrates anyone who would consider starting at low level CUDA kernels. I liked your message though, you had a great point: what do I imagine all those CUDA programmers do if not write CUDA?

Re:ILGPU: Write GPU programs with C# and F# by jay-barronville on Friday May 17, 2024 @08:20PM (#40395444)
1. C and C++ aren’t going anywhere.

2. I don’t think you have an appreciation for the sheer amount of software—including lots of very critical software—implemented in C and C++ that are being improved upon daily.

A buddy of mine works on critical low-level software (including C) in the energy sector. Millions of lines of code. The effects, positive or negative, of any single code change can impact tens of thousands of Americans (likely even more). His job is to maintain and continuously optimize this software. Given your comment, I think you’d be surprised to learn that he never runs out of work.

Re:ILGPU: Write GPU programs with C# and F# by jvanderbot on Friday May 17, 2024 @10:37PM (#40396120)
Well, I write in those languages every day! It's not a statement about the languages, just wondering about whether CUDA is best approached low level or from well established libraries.
Re:ILGPU: Write GPU programs with C# and F# by vmsp on Saturday May 18, 2024 @07:04AM (#40397975)
Would you mind expanding on what your friend does? Sounds like energy trading which is something I’ve been taking a look at. I’m just curious.
Re:ILGPU: Write GPU programs with C# and F# by AndrewDucker on Monday May 20, 2024 @08:27AM (#40414724)
Seems more likely it's to do with running the systems and networks.
Re:ILGPU: Write GPU programs with C# and F# by sheepscreek on Friday May 17, 2024 @06:53PM (#40394906)
Make it more accessible? Prototyping? Eg. easily determining if your use case is even suited for a GPU workload. Best case - it is, and now you can write a custom CUDA kernel to squeeze out more performance. Worst case, you lose a couple of hours vs weeks before you discover it’s not going to work.
Re:ILGPU: Write GPU programs with C# and F# by neonsunset on Friday May 17, 2024 @06:59PM (#40394942)
Given IL itself is an abstract stack-based bytecode, it can be compiled to the corresponding IR, which can then target corresponding back-end (CUDA, OpenCL, CPU, etc.) - this is what ILGPU does.

Because all code is in the single repository in the post and is fairly easy to read, you can skim through it to draw your own conclusions if this interests you.

Also, very easy to start using: just `dotnet add package ILGPU` on most configurations (as ADHD puts higher mental strain on activities involving complex configuration, I try to keep to the tools that have minimal ceremony)

C# (and F# by extension) generally allow to write system-ish code, with references to locals and same C primitives, which means that you're likely not sacrificing in performance in this particular scenario by having the language be higher-level. After all, you're using ILGPU's APIs first and foremost.

As to why use it at all - you are likely to move faster with it than C++, especially if it's not your full-time job, with all the escape hatches to extract 99.9% efficiency still on the table (that is, if performance of the kernel emitted by ILGPU has issues in the first place - see below for alternative, cheap FFI and easy C/C++ integration are still there as well).

It also lets you do things like PTX assembly: https://github.com/m4rs-mt/ILGPU/blob/master/Samples/InlineP...

Re:ILGPU: Write GPU programs with C# and F# by fulafel on Saturday May 18, 2024 @02:32AM (#40396892)
Programmers generally use higher level languages to implement original code because the code is easier for humans to read, write and debug. Porting from low level languages (eg CUDA or C++ to F#) is less common.
Re:ILGPU: Write GPU programs with C# and F# by nullandkale on Saturday May 18, 2024 @12:11PM (#40400088)
I use ILGPU all the time for prototyping different image processing and rendering things for work and for my personal projects. I use cpp for a few things where the extra efficiency is worth it, but for most tasks c# is plenty fast enough.
Re:ILGPU: Write GPU programs with C# and F# by gtani on Monday May 27, 2024 @07:32AM (#40489808)
what do you mean obsolete, read about PTX target tests:

https://old.reddit.com/r/rust/comments/18209in/gpu_programmi...