News for hackers. Stuff that matters.
Re:Nvidia Unveils Blackwell, Its Next GPU by treme on Wednesday March 20, 2024 @09:32AM (#39766223)
Crazy that video game graphics business happened to be the secret level portal to AI land
Re:Nvidia Unveils Blackwell, Its Next GPU by dragontamer on Wednesday March 20, 2024 @10:35AM (#39767209)
Watch this 90s commercial for 3dfx: https://www.youtube.com/watch?v=ooLO2xeyJZA

GPUs always had more compute / Gigaflops than traditional computers. GPUs in fact have more to do with 80s-era supercomputer architecture than normal CPUs.

https://www.youtube.com/watch?v=ODIqbTGNee4

Re:Nvidia Unveils Blackwell, Its Next GPU by jdawg777 on Wednesday March 20, 2024 @12:11PM (#39768564)
That was an unexpected twist.
Re:Nvidia Unveils Blackwell, Its Next GPU by dehrmann on Wednesday March 20, 2024 @11:03AM (#39767627)
I'm not sure if it's just nostalgia, but I still like the final 3dfx logo, at least in a video game context.
Re:Nvidia Unveils Blackwell, Its Next GPU by pjmlp on Wednesday March 20, 2024 @10:25AM (#39767040)
Ever since GPUs became programable, which goes earlier than many think (see TMS34010 from 1986), there has been many attempts to use the cards for general purpose compute.

It turns out that anything related to neural networks, and similar AI approaches, is all about compute.

Re:Nvidia Unveils Blackwell, Its Next GPU by dragontamer on Wednesday March 20, 2024 @02:23PM (#39770243)
I'd say almost the opposite in practice.

GPUs, as they became programmable (and maybe even a little bit before that...), started to take cues from SIMD Supercomputers. So the compute methodologies were researched first, and then GPUs (ie: applications to graphics) were applied afterwards.

I've heard rumors that the first programmable GPUs were considered because GPUs already were in SIMD-style compute and running instructions in a programmable way at the hardware/firmware level. It just needed to be "revealed" to OpenGL or DirectX programmers.

Re:Nvidia Unveils Blackwell, Its Next GPU by pjmlp on Wednesday March 20, 2024 @02:52PM (#39770549)
TMS34010 used a C like SDK, in 1986, there were hardly any SIMD supercomputers, unless you consider Cray and Connection Machines as model.
Re:Nvidia Unveils Blackwell, Its Next GPU by dragontamer on Wednesday March 20, 2024 @03:06PM (#39770708)
I'm thinking closer to 90s period, where "graphics cards" developed by Voodoo / 3dfx and its Glide API were clearly along the DirectX / OpenGL inspiration path. (3d Graphics pipeline / transform & lightning / etc. etc.). I don't know the exact point when these 3d accelerators became programmable with vertex/pixel shaders, but Glide is clearly on the path and part of the history here.

--------

You're right in that 80s / Arcade chips look to be different. Its a programmable graphics chip alright, but I wouldn't call that chip SIMD, not from what I can see from Wikipedia at least.

Re:Nvidia Unveils Blackwell, Its Next GPU by pjmlp on Wednesday March 20, 2024 @03:47PM (#39771192)
I wasn't the one bringing SIMD into the picture. :)
Re:Nvidia Unveils Blackwell, Its Next GPU by falcor84 on Wednesday March 20, 2024 @09:41AM (#39766336)
In hindsight I think it makes good sense - game graphics were always aiming to represent worlds with as high fidelity as possible.
Re:Nvidia Unveils Blackwell, Its Next GPU by barumrho on Wednesday March 20, 2024 @09:52AM (#39766526)
Good point, but it is interesting that the computations to render visual worlds in high fidelity is the same computation to "ingest" the data to create a model.

Reminds me of how a mic is a speaker and speaker is a mic.

Re:Nvidia Unveils Blackwell, Its Next GPU by AYBABTME on Wednesday March 20, 2024 @10:17AM (#39766902)
I think it speaks to the generalizability of linear algebra.
Re:Nvidia Unveils Blackwell, Its Next GPU by p_l on Wednesday March 20, 2024 @09:44AM (#39766388)
nVidia claims that they went into video games because it provided a way to fund their goal of compute accelerators
Re:Nvidia Unveils Blackwell, Its Next GPU by packetlost on Wednesday March 20, 2024 @09:49AM (#39766480)
I don't believe that claim one bit. That reads like revisionist history if I've ever seen it. If there's sources that back it up, fine, but until I see pretty reasonable proof I'm going to take that as CEO grandstanding while the market is hot.
Re:Nvidia Unveils Blackwell, Its Next GPU by xyzzy_plugh on Wednesday March 20, 2024 @10:17AM (#39766912)
Nvidia has been peddling compute for decades. They've long been selling enterprise tier cards for industrial purposes that are ill-suited to games. In many ways CUDA has been their long running attempt to break out of gaming.

I'm not surprised here. They invested and invested and invested and now everyone is playing catch up while they nearly corner the market. In industry circles we've been lamenting their dominance in this area for well over a decade.

Google and Apple are the only companies I see that have a shot at disrupting their market but they tend to end up navel gazing instead of burning bridges. And Nvidia has bridges with everyone.

It's absolutely an industrial geopolitics mastercraft scenario. We'll be studying them for a century.

Everyone has long known that the secret to unlocking the next tier of performance is solving concurrency. Look at all the advances that have made it into production in the last twenty years. But threading models only get you so far on a CPU, and programming GPUs is comparatively very difficult. Nvidia has always aimed to displace the CPU's prominence, since day one.

It's horizontal scaling 101.

Re:Nvidia Unveils Blackwell, Its Next GPU by erupt7893 on Wednesday March 20, 2024 @10:35AM (#39767198)
They've been peddling compute since mid-2000's, however Nvidia has been around for much longer. To the revisionist history point, I'm unfamiliar seeing any of this prior to that. Unless I'm missing something, they were very much a graphics only company competing with 3dfx, Matrox, S3, ATi, etc
Re:Nvidia Unveils Blackwell, Its Next GPU by dotnet00 on Wednesday March 20, 2024 @10:51AM (#39767448)
They were founded in 1993, but until 1998 had only put out 2 chips, the first was a quadratic surface renderer that didn't sell well, and the second was a normal triangle rasterizer that saved them from bankruptcy. Then in 2001 they released the first GPU that could run code, each subsequent generation expanding on that capability until they were able to do general purpose compute with CUDA.
Re:Nvidia Unveils Blackwell, Its Next GPU by hibikir on Wednesday March 20, 2024 @10:46AM (#39767352)
Nvidia was selling cards with hardware acceleration since 1997. The first one I own is from 1998, and its advantage wasn't that it had superior compute, but that it also included the non-3d circuitry of the rest of the video card. So instead of having a 2d card and a Voodoo, which were daisy chained, you could use that a lone Riva TNT that will do both. So they started being less about compute than the competition.

Between the first nvidia cards and the first release of CUDA, you have a decade.

Re:Nvidia Unveils Blackwell, Its Next GPU by dehrmann on Wednesday March 20, 2024 @11:02AM (#39767604)
> So instead of having a 2d card and a Voodoo, which were daisy chained, you could use that a lone Riva TNT that will do both

Or get the slightly underpowered Banshee.

Re:Nvidia Unveils Blackwell, Its Next GPU by bcrosby95 on Wednesday March 20, 2024 @10:34AM (#39767187)
Decades is still long after the first GeForce. Before CUDA we had third parties using graphics primitives to run general purposes computing.

So I'm not buying it.

Re:Nvidia Unveils Blackwell, Its Next GPU by dotnet00 on Wednesday March 20, 2024 @10:15AM (#39766870)
You can find slides from IIRC one of their first GPUs talking about how they can be made into very capable compute accelerators. They've supported CUDA on most of their GPUs since 2007 while the competition is still struggling to put out a truly equivalent solution. Even before that they had Cg.

I think it's pretty obvious that they saw the writing on the wall about the value of GPUs for massively parallel computing and actively pushed to be in the right spot for when everything took off.

Re:Nvidia Unveils Blackwell, Its Next GPU by Symmetry on Wednesday March 20, 2024 @10:24AM (#39767023)
And probably they'd been at least thinking about that since the early 2000s to have it ready by then. But the claim is that they were thinking about it since their beginning in 1993, and that's the part that just seems hard to believe.
Re:Nvidia Unveils Blackwell, Its Next GPU by dotnet00 on Wednesday March 20, 2024 @10:35AM (#39767214)
That's the claim Jensen has made, and while there isn't any particular evidence besides his word to go off for the founding goal, it's worth considering that between 1993 and 1998 they only put out 2 chips. The first one was quadratic primitive based (which, sounds more focused towards a future of compute applications, due to the ease of representing equations in such a form), and the second one was a last ditch effort triangle primitive chip to avoid bankruptcy after DirectX settled on triangle primitives. So I don't really think those years count for much in terms of determining what their goal was.

Their release cadence only really picked up in 1998, and just 2-3 years later they had the first 'programmable' GPUs with early shaders.

As such, saying that NVIDIA wasn't always aiming for GPU compute feels kinda like saying that SpaceX wasn't always aiming for feasibly reusable rocketry because it took them ~15 years to build up to their current version of it.

Re:Nvidia Unveils Blackwell, Its Next GPU by agumonkey on Wednesday March 20, 2024 @10:49AM (#39767413)
Also a funny 'existential' point to remember. You have a clean idea (doing math / quad-splines) but the market sinks it.. doesn't mean it won't be good later. It's a game of long and patient grind.
Re:Nvidia Unveils Blackwell, Its Next GPU by nabla9 on Wednesday March 20, 2024 @10:33AM (#39767177)
It's not hard to believe.

The idea that future of computing is related to parallel numerical computing and linear algebra (HPC, graphics ,scientific computing, data science, machine learning) predates current deep learning boom.

Already in 2003 and GPGPU (General-purpose computing on graphics processing units) was a thing. OpenVidia came out in 2003.

Re:Nvidia Unveils Blackwell, Its Next GPU by Scene_Cast2 on Wednesday March 20, 2024 @10:20AM (#39766956)
They tried to do physics acceleration at one point. I think they were trying to find a niche for their DSP-style compute.
Re:Nvidia Unveils Blackwell, Its Next GPU by dragonwriter on Wednesday March 20, 2024 @01:50PM (#39769916)
> They tried to do physics acceleration at one point

Well, there is PhysX which they bought in 2008 and still have; when they bought PhysX they shifted the acceleration implementation from relying on dedicated physics accelerator hardware to GPGPU.

There’s APEX which was built on top of PhysX but later discontinued.

There’s FleX…

“At one point” might be a slight misstatement.

Re:Nvidia Unveils Blackwell, Its Next GPU by agumonkey on Wednesday March 20, 2024 @10:48AM (#39767386)
I'm 50/50 on this. I don't think they ever envisioned (npi) that gaming would grow that much and would allow them to pivot back into GPU based HPC. That said they might always had a dream hinting at this.
Re:Nvidia Unveils Blackwell, Its Next GPU by Murfalo on Wednesday March 20, 2024 @09:57AM (#39766603)
What is so hard to believe? Technology evolves rapidly. I can't imagine that anyone investing big money in compute technology wouldn't have expected that.
Re:Nvidia Unveils Blackwell, Its Next GPU by Zenul_Abidin on Wednesday March 20, 2024 @10:26AM (#39767054)
Because that happened in a time when there were many players in the 90s each making their own GPUs for gaming purposes specifically. Compute was not even on the picture until things like CUDA and OpenCL came out.
Re:Nvidia Unveils Blackwell, Its Next GPU by froonly on Sunday March 24, 2024 @12:14AM (#39805124)
Back in the late 90s, there was a project at SGI, called Bali, to make all their pipelines work in IEEE 32-bit floating point (they were using Intel i860 chips) so that they could do HW rendering of scenes written in Pixar's Renderman language.

Sony copied that idea for the 1st Playstation, and then folks like NVidia & 3DLabs quickly followed suit, the idea being they would enable that functionality for games like Final Fantasy.

In the early 2000s, the HPC folks realized that you could use a GPU for physics & engineering codes, and here were are 20 yrs later.

Re:Nvidia Unveils Blackwell, Its Next GPU by izacus on Wednesday March 20, 2024 @10:10AM (#39766812)
You do understand nVidia dates back to fixed pipeline accelerators for essentially some vertices and textures, right?
Re:Nvidia Unveils Blackwell, Its Next GPU by pxtail on Wednesday March 20, 2024 @10:03AM (#39766693)
Of course, obviously ever since inception of the company they had the noble goal of uplifting human race. It was just the set of unfortunate circumstances that forced them to make trivial utility devices just to survive long enough
Re:Nvidia Unveils Blackwell, Its Next GPU by cma on Wednesday March 20, 2024 @10:21AM (#39766977)
They did a lot on medical scan visualization pretty early on too.
Re:Nvidia Unveils Blackwell, Its Next GPU by ripe on Wednesday March 20, 2024 @10:17AM (#39766901)
"When we were selling shovels to coal miners, we always secretly knew there was gold in those hills."

Yeah, sure.

Re:Nvidia Unveils Blackwell, Its Next GPU by tgtweak on Wednesday March 20, 2024 @09:55AM (#39766568)
Highly doubt this was the initial idea for nvidia given they were graphics-only for a very long time. CUDA definitely felt like more of a value-add for the first 5-6 years than a concerted effort to build accelerators and to fund that with graphics demand. First "tesla" line of GPUs - which had very little compute-only focus - were in 2007.
Re:Nvidia Unveils Blackwell, Its Next GPU by ksec on Wednesday March 20, 2024 @10:09AM (#39766792)
>First "tesla" line of GPUs - which had very little compute-only focus - were in 2007.

CUDA was first released in ~2006 but started at least in 2004, largely building on top or the momentum of Cg which was released in 2002 and being worked on since 2000. There was at the time in the late 90s about general purpose parallel programming and how it could possible be done on GPUs or doing it on something like PS2 Cell.

I dont know if Nvidia was really started with CUDA in mind in 1993. But Nvidia was into CUDA like GPU usage WAY ahead of anyone else in the field.

Re:Nvidia Unveils Blackwell, Its Next GPU by cma on Wednesday March 20, 2024 @10:23AM (#39767000)
Cell was PS3 (2006)
Re:Nvidia Unveils Blackwell, Its Next GPU by gosub100 on Wednesday March 20, 2024 @10:56AM (#39767529)
Wasn't there a time when people were hacking shaders to do compute tasks and read the results back instead of displaying to the frame buffer?
Re:Nvidia Unveils Blackwell, Its Next GPU by p_l on Friday March 22, 2024 @05:02AM (#39788702)
Yes, multiple such attempts, including complete languages. Very thin hardware support as different GPUs varied very much on what could be done.

GeForce 8 introduced CUDA because graphics pipeline similarly evolved into position where instead of forcing the use of dedicated cores for specific tasks (vector and fragment shaders) due to simple economies of chip making, the Shader Model 4 introducing yet another stage justified making unified shader cores... which could be just as easily used for other tasks.

Re:Nvidia Unveils Blackwell, Its Next GPU by m3kw9 on Wednesday March 20, 2024 @10:37AM (#39767231)
And tell me what is Nvidia trying to fund with the AI business?
Re:Nvidia Unveils Blackwell, Its Next GPU by amelius on Wednesday March 20, 2024 @10:29AM (#39767108)
Then why does my graphics card have 5 video outputs?
Re:Nvidia Unveils Blackwell, Its Next GPU by MangoCoffee on Wednesday March 20, 2024 @09:51AM (#39766517)
Nah, Intel said Nvidia just got "lucky"
Re:Nvidia Unveils Blackwell, Its Next GPU by xyzzy_plugh on Wednesday March 20, 2024 @10:20AM (#39766964)
So says every loser when they are bested by their own ignorance.
Re:Nvidia Unveils Blackwell, Its Next GPU by baq on Wednesday March 20, 2024 @09:51AM (#39766506)
Does a series of matrix multiplications have a soul?
Re:Nvidia Unveils Blackwell, Its Next GPU by AYBABTME on Wednesday March 20, 2024 @10:20AM (#39766962)
Matrix multiplication is just an effective representation for the dense graph structures. The same graphs could be implemented in other ways (adjacency list, edge list), perform the same overall logic, without doing vector/matrix/tensor math. It seems like the magic is in the overall idea of neurons and networks of them following increasingly interesting architectures and training mechanisms.

But because these graphs are mostly dense and involve numerical operations, matrices/tensors are a great implementation.

Re:Nvidia Unveils Blackwell, Its Next GPU by sirsinsalot on Wednesday March 20, 2024 @09:53AM (#39766546)
Does a series of electrical impulses in a brain?
Re:Nvidia Unveils Blackwell, Its Next GPU by amelius on Wednesday March 20, 2024 @10:30AM (#39767136)
Obviously no, because signals take time to move from A to B, while humans live in the now.
Re:Nvidia Unveils Blackwell, Its Next GPU by VladimirGolovin on Wednesday March 20, 2024 @11:47AM (#39768218)
Does a bunch of atoms arranged into proteins have a soul?
Re:Nvidia Unveils Blackwell, Its Next GPU by bheadmaster on Wednesday March 20, 2024 @10:29AM (#39767109)
Define "soul".
Re:Nvidia Unveils Blackwell, Its Next GPU by paulmd on Wednesday March 20, 2024 @04:49PM (#39771844)
does a SQL table provide a stochastic representation of conceptual symbolics?

oh, sorry, I thought we were just asking questions