News for hackers. Stuff that matters.

A comment on The History of CUDA.

Re:The History of CUDA by ly3xqhl8g9 on Wednesday June 07, 2023 @03:17AM (#36223864)
Ian Buck's doctoral thesis Stream Computing on Graphics Hardware (2006) [1] and a shorter article about it (2004).

From the 2004 article: "It is also possible that future streaming hardware will share the same memory as the CPU, eliminating the need for data transfer altogether." Unified memory foreseen 16 years before Apple Silicon.

[1] http://graphics.stanford.edu/~ianbuck/thesis.pdf

[2] https://www.cs.cmu.edu/afs/cs/academic/class/15869-f11/www/r...

[ Parent | on HN ]
Re:The History of CUDA by sharpneli on Wednesday June 07, 2023 @03:37AM (#36224004)
In 2020 Apple invented unified memory?

More seriously people need to stop with the Apple comparisons. Unified memory has been a thing for a way longer time. Heck around 2014 AMD had integrated GPUs with not just unified memory but fully unified address spaces with the host. Unified memory in itself happened way before that.

Not to mention that mobiles have always been unified archs. It’s just a design decision.

[ Parent | on HN ]
Re:The History of CUDA by ly3xqhl8g9 on Wednesday June 07, 2023 @03:52AM (#36224112)
Ian Buck's 2004 prediction is still 10 years before 2014. I did not say Apple invented unified memory, it just got popular with Apple Silicon, and looking at local LLM inference on M1/M2 and the 192 GB of memory M2 Ultra allows, it will surely get more important.
[ Parent | on HN ]
Re:The History of CUDA by captainbland on Wednesday June 07, 2023 @04:09AM (#36224225)
Even the Xbox 360 had unified memory. It's old as the hills at this point.
[ Parent | on HN ]
Re:The History of CUDA by muziq on Wednesday June 07, 2023 @03:54PM (#36232572)
The original Xbox was UMA..
[ Parent | on HN ]
Re:The History of CUDA by kkielhofner on Wednesday June 07, 2023 @09:08AM (#36226320)
More recent but the ARM based Nvidia Jetson line of hardware has had unified memory since 2017.
[ Parent | on HN ]
Re:The History of CUDA by kurtoid on Wednesday June 07, 2023 @03:51AM (#36224102)
doesn't intel integrates graphics share memory?
[ Parent | on HN ]
Re:The History of CUDA by ly3xqhl8g9 on Wednesday June 07, 2023 @04:24AM (#36224332)
Yes, but the point is "future streaming hardware" allowing for "stream computing" as Ian Buck puts it in 2004. The prediction is not: we will have unified memory and we will be able to boil eggs on a chip (such as Intel integrated graphics chip), but that we will have unified memory and be able to run Vicuna 13B locally (such as the M1/M2 chips).
[ Parent | on HN ]
Re:The History of CUDA by ChuckNorris89 on Wednesday June 07, 2023 @03:51AM (#36224105)
>Unified memory foreseen 16 years before Apple Silicon.

Honestly, it's good to get some more background information before claiming that Apple invented every innovation till sliced bread.

Microcontrollers, SoCs from various vendors, gaming consoles and Intel CPUs with integrated graphics also had unified memory since .. forever(?), or at least nearly 30 years, because it was as efficient back then as it is now for silicone and SW usage.

Apple didn't reinvent the wheel in this regard, it was already there as a low hanging fruit.

[ Parent | on HN ]
Re:The History of CUDA by ly3xqhl8g9 on Wednesday June 07, 2023 @03:57AM (#36224151)
I did not say Apple invented unified memory, just that Ian Buck's prediction was 16 years before Apple Silicon. Also, in practice, is not only about unified memory, but also performance. The Intel chip with integrated graphics is able to boil an egg with the heat it dissipates, the M1/M2 are cool as if not even running while handling way more workload.
[ Parent | on HN ]
Re:The History of CUDA by ChuckNorris89 on Wednesday June 07, 2023 @04:03AM (#36224191)
Having unified memory and having much more compute performance/watt are two orthogonal issues. Unfired memory had already existed before Apple silicone and was already present in most consoles, cheap tablets and smartphones due to how ubiquos SoCs and Intel chips with iGPUs were.
[ Parent | on HN ]
Re:The History of CUDA by ly3xqhl8g9 on Wednesday June 07, 2023 @04:21AM (#36224310)
Ian Buck's thesis is about how GPUs can be used as "stream computing": "In this paper (Buck, 2004), we present Brook for GPUs, a system for general-purpose computation on programmable graphics hardware. Brook extends C to include simple data-parallel constructs, enabling the use of the GPU as a streaming co- processor." That's all the point: Apple Silicon allows for running local large language models (and other ML models/algorithms) in a way, at a price point, with enough performance, and so on in which other chips with unified memory don't.
[ Parent | on HN ]
Re:The History of CUDA by dogma1138 on Wednesday June 07, 2023 @04:37AM (#36224393)
Ian Buck is now one of the top execs at NVIDIA right now btw.

https://blogs.nvidia.com/blog/author/ian-buck/

[ Parent | on HN ]
Re:The History of CUDA by hbbio on Wednesday June 07, 2023 @07:11AM (#36225403)
Quite deserved: CUDA is probably the reason Nvidia became a trillion dollar co.

In his 2004 PhD, he tells ATI had a much better performance than nvidia... even if it was still the case, it would not even matter as their tools and drivers are terrible.

[ Parent | on HN ]
Re:The History of CUDA by orbital-decay on Wednesday June 07, 2023 @05:36AM (#36224762)
"Unified memory" has been around forever in one form or another, as it's simply a single address space (and physical location) for various independent subsystems. In graphics, it's probably been used since Amiga. (?) This is common in console GPUs which always punched above their weight. The ubiquitous Intel shared memory has been around for ages, although it was not entirely unified (reserved area for GPU, which it cannot escape; zero-copy still possible by allocating inside it and addressing data on CPU).
[ Parent | on HN ]
Re:The History of CUDA by ly3xqhl8g9 on Wednesday June 07, 2023 @06:29AM (#36225079)
I obviously did not mean "unified memory" in that sense. In that sense even Apple I in 1976 had "unified memory" [1] [2]. The sense in which I meant it, following the spirit of the above paper/thesis, which no one seems to have read because they were too quick to jump on the bashing bandwagon, was unified memory performing "stream computing", e.g. an Apple Silicon chip running a local large language model. And if you get to run Vicuna 13B or something else on an Intel Tiger Lake or similar, more power to you, and to us if you make it open source.

[1] "The display section contains its own memory, leaving all of RAM for user programs", http://s3data.computerhistory.org/brochures/apple.applei.197...

[2] "Running Apple 1 software on a breadboard computer (Wozmon)", https://www.youtube.com/watch?v=HlLCtjJzHVI

[ Parent | on HN ]
Re:The History of CUDA by legosexmagic on Wednesday June 07, 2023 @08:07AM (#36225777)
integrated gpus do perform "stream computing" tho
[ Parent | on HN ]
Re:The History of CUDA by HarHarVeryFunny on Wednesday June 07, 2023 @10:55AM (#36227561)
There's a difference between CPU-GPU shared memory and unified memory, although not everyone seems to be using "unified" in the same sense.

What Apple appear to have with their M2 chips is shared memory meaning that the CPU and GPU are accessing the same memory chips. On the just-announced M2 Ultra chip they are claiming 800GB/sec memory bandwidth, which compares well to the 1TB/sec on a recent NVIDIA card.

Unified memory, at least as NVIDIA use the term, only refers to a unified address space such that the GPU and CPU (located on opposite sides of the PCI bus) can use the same address space to access memory. However, the memory being mapped to by this unified address space may be on either side of the PCI bus (i.e be CPU memory or GPU memory) and may migrate from one side to the other to optimize performance. Given how slow PCI bus transfers are compared to GPU memory bandwidth, the use cases for this is not at all the same as true shared memory... It's really just a developer convenience feature to not have to explicitly orchestrate CPU-GPU memory transfers (which you may be better off doing to maximize performance).

NVIDIA seem to be going in the same direction as Apple here, with their latest designs integrating GPU and CPU on a single module.

[ Parent | on HN ]
Re:The History of CUDA by gregw2 on Thursday June 08, 2023 @02:32AM (#36238071)
Unified memory was a key feature of Silicon Graphics's low end O2 workstation released in 1996. It enabled some unusual texture mapping capabilities at the time.
[ Parent | on HN ]