the GPUs these guys would be using are not the same ones you are using to play crysis, we're talking more about this kind of purpose-built thing: https://www.nvidia.com/en-us/data-center/a100/

It's become more of a term for highly parallel processor units in general, one which NVidia encourages because it ties their product offering together

True. I guess those are almost ASICs in a way just with gpu flavour interface

Way less specialized though. I think why most don't go for ASICs at this point, is because once you actually have units being produced, things have changed so much that you'd wish you had something more flexible. That's why general purpose GPUs are used today.

Didn't this happen in mining? When finally they got their machines new better tech came out already.

Well, Bitcoin ASICs are still the beast when it comes to Bitcoin mining. Some other cryptocurrencies use other methods for mining, so those ASICs won't work for that, but who's to say what's the better tech in the cryptocurrency space :shrug:

GPGPU

There are AI / datacenter focused GPUs (like the A100, H100 etc.). They do not have any graphics rendering circuitry.

Does it seems silly to anybody else to even call these GPUs? That's a GPU minus the G.

Agreed. I guess it's because of the architectural heritage but at this point GPU is something of a misnomer.

Very much absurd. A user above posted GPGPU, which I guess stands for General Purpose Graphics Processing Unit.

In the beginnings of computation these kinds of cards were called accelerators. Dedicated consumer sound cards were a thing, the venerable SoundBlaster. I really would like an AI-Blaster coming out.

There is actually ML hardware that is not based on GPU technology: it's called TPU (Tensor processing unit) but only Google uses it. I guess it is easier to repurpose existing technology even if a specialized approach is more efficient in theory.

> They do not have any graphics rendering circuitry.

What? Not having a display output is not the same as not having graphics rendering circuitry. Here's vulkaninfo from an A100 box: https://gist.github.com/eiz/c1c3e1bd99341e11e8a4acdee7ae4cb4

This may not contradict what I said. Do you know for a fact these things are implemented using dedicated hardware?

Fair enough. In the GH100 architecture doc https://resources.nvidia.com/en-us-tensor-core/gtc22-whitepa... (page 18) they do mention retaining 2 graphics-capable TPCs but it's clearly not the focus.

They do have graphics rendering circuitry, but e.g. fewer shading units and more graphics memory, or support for faster interconnects. You can look up the specs and compare. The differences are varied, but IMO not enough to claim they're not GPUs anymore. Even gaming focused GPUs are GPGPUs these days: the RTX 4090 has as many Tensor Core units as the A100. And you can still use e.g. DirectX, OpenGL with a datacenter grade GPU.

> fewer shading units

This is incorrect. NVIDIA uses a unified graphics and compute engine. A CUDA core is a shading unit. These datacenter GPUs have a shit ton of these (CUDA cores).

Edit: actually the point I want to make is the A100 only retains those hardware units which can be used for compute. Some of these units may have a (dual) use for graphics processing but that is besides the point (since this is true of all CUDA enabled NVIDIA GPUs).

Do GPUs give you more flexibility to take different approaches? Maybe they’re paying extra for optionality. Or maybe (most likely) TechCrunch is using the term “GPU” imprecisely.

To programmer_dude’s point, compute center GPUs don’t have hardware for the rasterisation stage, which is a particularly inflexible bit of the graphics pipeline. Omitting it and emphasising the geometry (matrix multiplication) capabilties is meant to give it more flexibility/less of a strongly-opinionated graphics focus.

As for the “GPU” term, it’s a bit of a historical relic, presently it serves as a useful indicator of compute hardware (in contrast to CPU and Google’s TPU.) Nvidia itself calls its A100 a “Tensor Core GPU.”

GPGPU is not new. If it's good for your use case, then it's what you need.

It's not like they are getting RTX cards with useless raytracing shit.

Unneeded stuff would be the cost of making and ASIC for a workload that GPUs already handle well. GPU manufacturing already exists.