News for hackers. Stuff that matters.
Re:Nvidia to join Dow Jones Industrial Average, replacing Intel by kiratp on Friday November 01, 2024 @09:57PM (#42023476)
Nvidia’s success is due to a decade+ of investment into their software stack (CUDA and friends), built on a world view that more software can and should be massively parallel.

They were a key cause for LLMs being a thing in the first place.

Re:Nvidia to join Dow Jones Industrial Average, replacing Intel by BobbyJo on Friday November 01, 2024 @10:39PM (#42023660)
This. I worked in HPC between 2010 and 2013, and people were trying to compete with NVidia in the GPGPU space even back then, seeing the way the wind was blowing. Compute bandwidth, and FLOPS/watt has steadily been more and more important the last 15 years, even before AI.

NVidia has continued to stay ahead because every alternative to CUDA is half baked trash, even when the silicone make sense. As a tech company, trading $$ for time not spent dealing with compatibility bugs and broken drivers pretty much always makes business sense.

Re:Nvidia to join Dow Jones Industrial Average, replacing Intel by otabdeveloper4 on Saturday November 02, 2024 @03:42AM (#42024717)
> half baked trash

> dealing with compatibility bugs

> broken drivers

Describes my experience trying to use CUDA perfectly.

We have a long way to go and we haven't even started yet.

Re:Nvidia to join Dow Jones Industrial Average, replacing Intel by initplus on Saturday November 02, 2024 @03:55AM (#42024771)
Given that experience, think about what state the alternatives must be in!
Re:Nvidia to join Dow Jones Industrial Average, replacing Intel by disgruntledphd2 on Saturday November 02, 2024 @04:23AM (#42024893)
To be fair, Cuda has improved a lot since 2014 or so. I messed up my Linux box multiple times trying to install Cuda but the last time it was just apt install and maybe setting ld library and it all just worked.
Re:Nvidia to join Dow Jones Industrial Average, replacing Intel by yieldcrv on Saturday November 02, 2024 @05:46AM (#42025214)
It’s pretty bad. just when you think you can order AMD chips since there is no shortage, and use a translation layer and have a cheap AI datacenter, it turns out AMD is fumbling the ball at every step of the way
Re:Nvidia to join Dow Jones Industrial Average, replacing Intel by bottled_poe on Saturday November 02, 2024 @10:07AM (#42026480)
It’s interesting. They have had plenty of time and resources available to mount solid competition. Why haven’t they? Is it a talent hiring problems or some more fundamental problem with their engineering processes? The writing has been on the wall for gpgpu for more than 10 years. Definitely enough time to catch up.
Re:Nvidia to join Dow Jones Industrial Average, replacing Intel by BobbyJo on Saturday November 02, 2024 @07:45PM (#42030072)
Its a commitment problem IMO. NVidia stuck with CUDA for a long time before it started paying them what it cost. AMD and Intel have both launched and killed initiatives a couple times each, but abandon them within a few years because adoption didn't happen overnight.

If you need people to abandon an ecosystem thats been developed steadily over nearly 20 years for your shiny new thing in order to keep it around, you'll never compete.

Re:Nvidia to join Dow Jones Industrial Average, replacing Intel by the__alchemist on Saturday November 02, 2024 @12:06PM (#42027207)
Have you tried Vulkan Compute or OpenCL? I recommend evaluating this relatively, vice compared to an ideal.
Re:Nvidia to join Dow Jones Industrial Average, replacing Intel by neurostimulant on Saturday November 02, 2024 @06:40AM (#42025444)
Wasn't Intel's commercial compiler also dominated HPC space around that time period? I remember back then people would use cuda and icc for hpc stuff.
Re:Nvidia to join Dow Jones Industrial Average, replacing Intel by rbanffy on Saturday November 02, 2024 @07:06AM (#42025604)
I believe their compilers still dominate in CPU-bound workloads.
Re:Nvidia to join Dow Jones Industrial Average, replacing Intel by knappe on Saturday November 02, 2024 @03:32PM (#42028646)
I can't speak to modern hpc, since I've been out of the game for a few years but icc was absolutely the preferred compiler for almost any workload and Intel procs were the desired hardware to run on. AMD was joked about as the only reason Intel wasn't considered a monopoly at the time and that Intel would happily take wheelbarrows of cash over to AMD to prop them up, just to ensure they didn't even appear like a monopoly. But no one in the field was buying AMD procs for their workloads.
Re:Nvidia to join Dow Jones Industrial Average, replacing Intel by BobbyJo on Saturday November 02, 2024 @10:07PM (#42030716)
Most super computers were still Intel/IBM, but around that time is when the shift to GPU clusters started. #1 super computer spot was taken by an Nvidia cluster in 2012, but I remember other big projects were done before as well.
Re:Nvidia to join Dow Jones Industrial Average, replacing Intel by ghaff on Saturday November 02, 2024 @08:52AM (#42026109)
Nvidia did a lot of things right. But it sure didn't hurt that the crypto bubble gave way to LLMs with almost perfect timing. Luck favors the prepared and all that but Nvidia also would have had trouble timing market changes better.
Re:Nvidia to join Dow Jones Industrial Average, replacing Intel by toasterlovin on Saturday November 02, 2024 @10:33AM (#42026618)
Yeah, exactly, it has never been clear that we would find a use for parallel computation outside of niches like graphics and HPC. And in fact most of the history of computing has been evidence of the opposite: useful computation has been predominantly serial in nature. But here we are now.
Re:Nvidia to join Dow Jones Industrial Average, replacing Intel by FredPret on Saturday November 02, 2024 @02:11PM (#42028073)
Nvidia did ride a perfect storm of gaming + bitcoin + AI.

But AMD and others could have done the same, had they been better at riding that wave. There’s a reason it was Nvidia who won out.

Re:Nvidia to join Dow Jones Industrial Average, replacing Intel by llm_trw on Friday November 01, 2024 @11:14PM (#42023795)
More than a decade.

I was using it as soon as it came out in 2007 and my dinky desktop workstation was out performing the main frame in the basement.

Re:Nvidia to join Dow Jones Industrial Average, replacing Intel by mullingitover on Saturday November 02, 2024 @12:35AM (#42024096)
> a world view that more software can and should be massively parallel.

Twenty years ago I was thinking we'd be speccing machines in kilocores by now.

Re:Nvidia to join Dow Jones Industrial Average, replacing Intel by otabdeveloper4 on Saturday November 02, 2024 @03:43AM (#42024720)
128 and 512 cores are normal now, so not that far off.
Re:Nvidia to join Dow Jones Industrial Average, replacing Intel by epcoa on Saturday November 02, 2024 @12:40AM (#42024116)
Re:Nvidia to join Dow Jones Industrial Average, replacing Intel by Keyframe on Saturday November 02, 2024 @04:38AM (#42024951)
4090 has 16384 cuda cores, so there's that!
Re:Nvidia to join Dow Jones Industrial Average, replacing Intel by atq2119 on Saturday November 02, 2024 @05:47AM (#42025220)
Nvidia's marketing is misleading. Those "cuda cores" are more SIMD lanes than cores. Number of SMs is a more appropriate equivalent to CPU core count.
Re:Nvidia to join Dow Jones Industrial Average, replacing Intel by Keyframe on Saturday November 02, 2024 @08:34AM (#42026013)
you're right
Re:Nvidia to join Dow Jones Industrial Average, replacing Intel by dahart on Saturday November 02, 2024 @11:26AM (#42026940)
Are you sure that isn’t what @mullingitover meant?

> Number of SMs is a more appropriate equivalent to CPU core count.

What do you mean by this? Why should an SM be considered equivalent to a CPU core? An SM can do 128 simultaneous adds and/or multiplies in a single cycle, where a CPU core can do, what, 2 or maybe 4? Obviously depends on the CPU / core / hyperthreading / # math pipelines / etc., but the SM to CPU-core ratio of the number of simultaneous calculation is in the double digits. It’s a tradeoff where the GPU has some restrictions in return for being able to do many multiples more at the same time.

If you consider an SM and a CPU equivalent, then the SM’s perf can exceed the CPU core by ~2 orders of magnitude — is that the comparison you want? If you consider a GPU thread lane and a CPU thread lane equivalent, the the GPU thread lane is slower and more restricted. Neither comparison is apples to apples, CPUs and GPUs are made for different workloads, but arguing that an SM is equivalent to a CPU core seems equally or more “misleading” when you’re leaving out the tradeoff.

I’d argue that comparing SMs to cores is misleading and that it makes more sense to compare chips is by their thread counts. Or, don’t compare cores at all and just look at the performance in, say, FLOPS.

Re:Nvidia to join Dow Jones Industrial Average, replacing Intel by atq2119 on Saturday November 02, 2024 @11:51AM (#42027102)
A single Zen5 core can do 32 single precision FMAs per clock.

That's using SIMD, but so is Nvidia for all intents and purposes. Those "cuda cores" aren't truly independent: when their execution diverges, masking is used pretty much like you'd do in CPU SIMD.

A lot of the control logic is per-SM or perhaps per-SIMD unit -- there are multiple of those per SM. You could perhaps make a case that it's the individual SIMDs which correspond to CPU cores (that makes the flops line up even more closely). It depends on what the goal of the comparison is.

Re:Nvidia to join Dow Jones Industrial Average, replacing Intel by Dylan16807 on Saturday November 02, 2024 @02:14PM (#42028096)
> An SM can do 128 simultaneous adds and/or multiplies in a single cycle, where a CPU core can do, what, 2 or maybe 4?

https://images.nvidia.com/aem-dam/Solutions/geforce/news/rtx...

An SM is split into four identical blocks, and I would say each block is roughly equivalent to a CPU core. It has a scheduler, registers, 32 ALUs or FPUs, and some other stuff.

A CPU core with two AVX-512 units can do several integer operations plus 32 single-precision operations (including FMA) per cycle. Not 2 or 4. An older CPU with 2-3 AVX2 units could fall slightly behind, but it's pretty close.

That doesn't factor in the tensor units, but they're less general purpose, and CPUs usually put such things outside the cores.

I would say an SM is roughly equivalent to four CPU cores.

Re:Nvidia to join Dow Jones Industrial Average, replacing Intel by dahart on Saturday November 02, 2024 @02:34PM (#42028235)
Yeah I totally forgot to consider CPU SIMD. Brain fart. The other comment corrected me too, you’re both right. When I said 2 or 4, I was thinking of the SISD math pipe and not AVX instructions.

Yes, considering CPU SIMD, maybe comparing a CPU core to a CUDA warp makes some sense in some situations. The peak FLOPS rate is still so much higher on Nvidia though, that the comparison hardly makes sense. So yeah like I and the other commenter mentioned, it depends entirely on what comparison is being made.

Re:Nvidia to join Dow Jones Industrial Average, replacing Intel by seanhunter on Saturday November 02, 2024 @03:32PM (#42028644)
100%. If you ever tried to build pytorch (or tensorflow) gpu accellerated using NVidia vs AMD it was absolutely chalk and cheese for the longest time.

Nvidia/CUDA process: Download package. Run the build. It works. Run your thing- it's GPU accelerated. Go get a beer/coffee/whatever while your net runs.

AMD process: Download package. Run the build. Debug failure. Read lots of articles about which patches to apply. Apply the patches. Run the build. It fails again. Shit. OK ok now I know what to do. I need a special fork of the package. go get that. Find it doesn't actually have the same API that the latest pytorch/tf relies on. OK downgrade those to an earlier version. OK now we're good. Run the build again. Aw shit. That failed again. More web searches. Oh ok now I know - there's some other patches you need to apply to this branch. OK cool. Now it compiles. Install the package. Run the thing. Huh. That's weird. GPU accelleration isn't on.... sigh....