ggreg84

Chips and Cheese GPU analysis are pretty detailed, but they need to be taken with a huge grain of salt because the results only really apply to OpenCL and nobody buying NVIDIA or AMD GPUs for Compute runs OpenCL on them; its either CUDA or HIP, which differ widely in parts of their compilation stack.

After reading the entire analysis, I'm left wondering, what observations in this analysis - if any - actually apply to CUDA?

Asynchronous clean-up is a solved problem in C++ senders & receivers.

I wonder what's so different about rust that they can't solve it in the same way.

> My biggest concern with Rust is the sloppiness around the distinction between a "compiler bug" and a "hole in the type system."

That bug is marked as I-unsound, which means that it introduces a hole in the type system.

And so are all other similar bugs, i.e., your concern seems to be unfounded, since you can actually click on the I-unsound label, and view all current bugs of this kind (and past closed ones as well!).

Excellent example. A supercomputer built with somebody elses money (tax payers) for use by relatively cheap phd students funded by somebody elses money (tax payers), to... maybe some day do some AI... but if they don't, that's fine. [0]

The level of risk tolerance of such a user is essentially infinite, to the point that your AI cluster can arrive 8 years late (Intel and Aurora), and you are still fine.

What makes sense for such user, might not make sense for a private company whose survival depends on expensive engineers quickly generating value in a competitive market.

So the example is really perfect: those with infinite risk tolerance go with AMD GPUs for AI today.

Maybe some day this will change.

[0] https://lumi-supercomputer.github.io/LUMI-training-materials...

> OpenCL based machine learning libraries, no?

Have you used OpenCL and CUDA C++ ?

CUDA C++ is single-source easy to use, has good tools, has a lots of libraries you can build your new library on top (empowered by generic libraries), ...

OpenCL... is not single-source, not easy to use, does not have good tools, does not have good libraries that you can reuse (does not have generics), ...

AI is like a F1 race, you can either go race with a F1 car (CUDA C++), or don't go at all. Trying to go with the horsecar OpenCL is, is a waste of time and money. Every other AI startup is going to loop you a million times.

> Nvidia deliberately undermined that at every turn, and pushed a proprietary solution, hard.

Undermined who at what? Nobody was interested in OpenCL succeeding: not Intel, not AMD, not Apple. All these companies push for their own proprietary incompatible ecosystems to try to create a platform like NVIDIA has with CUDA. Intel pushes for OneAPI which is Intel only, AMD pushes for their CUDA clone, and Apple pushes for Metal.

Its easy to create a new standard. You and me can get together, write something on a napkin, call it a standard, and we are done. We could go around telling people that it's going to be "The Future", like it happened with OpenCL, but that doesn't mean anyone will actually ship something that's usable.

From all vendors, ironically, it actually seems that the only vendor working on a portable way to program GPUs is NVIDIA itself, since they have sent a bunch of people regularly to the ISO Fortran and ISO C++ standard committees to extend these languages to allow portable code to run on GPUs without changes. In contrast to OpenCL, ISO Fortran and ISO C++ are actual international standards.

I think Fortran supports this since 2018 standard, and C++ since 2017. Ironically, NVIDIA is the only vendor that actually ships this. Intel and AMD will talk about how "portable and cross-vendor" OneAPI and ROCm are, yet their Fortran and C++ implementations still, 5 years later, can't actually use their GPUs. The reason is simple, AMD and Intel don't care / believe / want a portable programming model for GPUs. They want their own closed platform. Unfortunately, since they are at a disadvantage, they need their closed platforms to be able to run on NVIDIA GPUs because otherwise nobody would use them, but that doesn't mean that their platforms are portable or open, and they don't care about code using their platform being able to run well on NVIDIA GPUs since that would be counter productive.

Indeed, wow.

Which is (EDIT: NOT) the most widely sold console ever.

I just use inotifywait for that. What does entr do differently?

This will run a binary when any file on the directory tree changes:

    #!/usr/bin/env sh
    set -e
    while true; do
        inotifywait -e modify,create,delete,move -r $1 $2
    done

and then just call it: run_on_modify path/to/dir path/to/bin

If someone thinks this is magic or wizardy, instead of teaching them that fire exists (entr), teach them how to make fire first (inotify, linux inodes, etc.). Knowing what kind of events inodes support and how to do something when they happen is pretty powerful.

If you then want to use "entr" or similar instead of just calling inotifywait yourself, that's fair game. But TBH I have a hard time justifying a C or Rust program that's not going to be available / installed everywhere when a 3 LOC POSIX shell script can solve the same problem (and many many more).

> Is this a serious comment? Alwrapper doesn't use emulation. An API wrapper simply translates calls to the appropriate API. B

QEMU simply translates calls from one hardware API to another hardware API, so it isn't an emulator either right?

Since you can't really buy apple cpus or gpus and put them in a PC, for benchmarking... today at least one can't compare apple's hardware against intel hardware.

What one can compare, is the performance of the "Apple platform" and the "PC platform" at similar price points, power budgets, features. This is more meaningful for most people, which mostly care about how fast can the computer do X (whether it uses the CPU, GPU, or some other chip, most people don't care).

Buy professional appliances, for restaurants.

For some reason, they are often better along many axes: cheaper, easier to clean, more customizable, more powerful, more flexible, etc.

I really like the professional induction stoves, with actual physical dials, no touch crap.

And the best part, there is a huge used market for professional appliances in really good shape, for even lower price.

> Isn't there a single GPU benchmark that actually does the same work so that comparisons can actually be made?

Apple does not support OpenGL or Vulkan, only Metal, and most app devs have better things to do than rewriting code for the Mac.

The recommended way of gaming on a mac is to use emulation by emulating whatever the game is used with a metal wrapper.

IMO the claim that these benchmarks aren't fair is naive. I don't care about how good the hardware is, but about what performance I get. If I get poor performance because the software, drivers, etc. are poor, I want to know that.

If you can re-compile all your source code, then ABI stability is not that important.

In the real world,

- people use operating systems to get work done,

- such work is often done with proprietary software that people buy, and that lot of people get paid to develop

- breaking such software is not acceptable, because it means that suddenly dozens of thousand of people can't work

- telling all those people to "stop working" until their software is fixed is not acceptable either.

People use Linux for work in the real world, and that's why Linux tries very hard not to break ABIs.

Microsoft has had a stable ABI for 30 years, and people still run software compiled 30 years ago on the latest Windows version today.

What this tells you about OpenBSD users, is another story.

+1. Asking the right questions.

What went wrong is that people learned how much NVIDIA pays, figured out they were significantly underpayed, Arm is not a very profitable business so couldn't compete, and the moment it started looking like the NVIDIA deal was not going to fly, top talent started jumping ship.

Their goal - and only option - is to sell the company for as much $$$ as possible, and this strategy helps them towards their goal.

This is the simplest and most effective way to make Arm books look good, which is what regulators all around the world - including the UK - have decided is what's best for the world.

So this is not idiotic, this is effective, and is finance 101.

For Arm's long term success this is a very stupid decision. But that's not what any of the parties involved need or care about. What they care about is $$$ right now.

I won't be buying and holding Arm stock, particularly not as a long term investment, cause it looks like its going to be a poor one.

To many on the internet which was financed by DARPA?

The irony is strong on this thread.

And Mac Studio probably.

Still, closing your online store for days during the weeks around releases seems dumb.

Someone might want to buy an Apple watch or something :laugh:

Imagine closing Amazon for a week every time Amazon rips some startup product onto their own branded one.

You can save the annotated slides after the lecture, and just share them?

When the M1 was released... a year ago...

You could buy a 900$ laptop with a better CPU than any laptop out there. 20 hours battery life (almost 3x any other laptop out there). Silent (no fans). With a great screen, a great keyboard, lightweight, well built, etc.

Basically a machine that was more than 2x cheaper of competing laptops at 2500 $ or more, yet had more than 2x of everything.

Do you have a reference?

> Optimizing for speed usually causes increased memory consumption,

Lol, no?

If your benchmark is memory bound, reducing memory usage is probably the simplest way to make it faster.

If by software throttling hardware you mean LHR.

Then as other have mentioned there are two options for GPU vendors:

* Hardware unblocked: GPU can be used for mining, scarcity becomes worse, good luck finding gaming GPUs for less than 2000$ for as long as it is profitable to use them for mining (~2 years maybe?)

* Hardware blocked: GPU can't be used for mining (profitably), gaming GPUs can be bought by gamers (still scarcity due to COVID, but not as bad)

The main reason only NVIDIA does this, and AMD and Intel don't isn't because of "freedom", but rather because AMD and Intel GPUs are so bad at compute that nobody can use them for anything but gaming anyways.

Independently of whether one agrees with nvidia's decision here, and whether it actually achieved its goal or not, the thought process isn't really hard to follow.

And I mean, before LHR people were complaining that miners were buying all nvidia gpus, and they couldn't get any, and asking nvidia to do something. nvidia did something, and now that people got to get the GPUs for gaming, they are complaining that they can't mine on them, which is probably the only reason they actually were able to get one in the first place.

So :shrug:. To be honest, it was foreseeable that people were going to complain either way.

Reply in this comment thread: https://news.ycombinator.com/item?id=30568961

> Because the hardware already does all of those things whether you paid for them or not.

AMD and Intel GPUs have the hardware capability of being great at CAD, but in practice they suck and nobody uses them because AMD and Intel do not have driver integration & certification with CAD software.

Why? Cause developing that costs money.

NVIDIA hardware and software only has this _because_ people that need it pay 10k$ for GFX cards that do these things.

If you'd wanted gamer cards to do these things, you'd need to add to the price tag of gamer cards the cost of developing and maintaining these things, which would make them more expensive.

The claim that this should be free because software and hardware costs nothing once you already have developed is illogical. CAD software, GPU driver software for CAD, and GPU hardware to meet CAD's demand, continues to evolve, which costs money.

> I'm not a big fan of limiting hardware to run only approved software. One day it's for limiting crypto mining, the next day it will be for limiting our ability to use strong encryption. It's a slippery slope.

I'd assume that developing extra features costs money. If a buyer doesn't need a feature, why should they have to pay for it?

For example, if a gamer does not need drivers for a CAD program, or crypto features, why should the gamer have to foot the bill for developing these features, and suffer the consequences of them (e.g. scarcity due to cryptominers buying all gaming cards) ?

From the Nouveau devs themselves: https://nitter.eu/_Lyude/status/1498811646697000961

They claim that leaking anything would actually make their work significantly harder.

> And for that one, exceptions as they are today are often faster than the equivalent error-code-based code: https://nibblestew.blogspot.com/2017/01/measuring-execution-...

The example on that blog post is not representative of how error codes are used.

On all programming languages, C included, but also Rust, functions return a completion value that's either success, or error (and when its error, it includes which error it is, and the error state, etc.).

    int foo();
    switch (foo())
       case 0: break; // success
       case 1: /* error case 1 */
       case 2: /* error case 2 */
       ...

Instead, the blog post has the function return some value in registers that is not an error, and passes the function an ErrorState* as an argument, that the function first sets via memory, and then the handler reads, also via memory.

That this is slow is not rocket science, which is why nobody does this.

I get why the author has to do it. If they were to do it right, error codes would perform significantly better than exceptions. Since returning the error or success is "free" (the function has to return anyways, and up to 2 registers are reserved by the ABI anyways).

Somebody mentions this in the comments, were as far as fixing it and sending a PR (https://github.com/jpakkane/excspeed/pull/1) but the author complains that this is not how GLib does error codes, so they don't care.

So yeah, if you do C error codes like GLib does, which is essentially a poor man's emulation of exceptions, then sure, exceptions are faster than doing error codes wrong.

Most people using C do it right, and so does Rust's Result (and all Rust programs).

Note that every member can decide how it aids.

Germany aid to Ukraine was to send them 5k helmets.

Those helmets didn't do anything against Russia's missiles.

---

That is, attacking a member of NATO just means that all other members must help it, but how they help, is up to them, and that help can be just a pat in the back.