News for hackers. Stuff that matters.
Re:Spectre mitigations murder userspace performance by nickpeterson on Saturday June 19, 2021 @08:35AM (#27559929)
I’ve made similar comments in the past but I think we’re just trying to predict too much about what programs are trying to do in hardware.

I’d rather have simple hardware that is light on energy requirements and easier to understand. I don’t think software as an industry really has a “this chip isn’t fast enough problem”. Most of the real slowdowns anyone has in day to day performance has more to do with inefficient code than hardware anyways.

Look at old game consoles (like a regular old PlayStation) and what they were able to render on what today would be laughably slow hardware. They could do it because developers learned the stack inside and out.

I’m hopeful that the emergence of arm as a desktop/server competitor will help in some ways by putting enough pressure on companies like intel to really rethink some of their core stuff, but who knows.

Re:Spectre mitigations murder userspace performance by giantrobot on Saturday June 19, 2021 @05:12PM (#27564509)
> Look at old game consoles (like a regular old PlayStation) and what they were able to render on what today would be laughably slow hardware. They could do it because developers learned the stack inside and out.

You're oversimplifying the situation and drawing conclusions from it. The PSX had dedicated hardware for geometry transforms, video decoding, sound processing, and rasterizing. The CPU was used for game logic and queuing up the render pipeline. So it's not like all PlayStation developers were doing black magic in every game, they were using a lot of dedicated hardware with well known and relatively modest and fixed constraints (NTSC/PAL, stereo audio, and relatively small frame buffers).

It's interesting you picked the PSX as your exemplar because it's CPU/GPU/MDEC and audio processor were meant to be black boxes so developers didn't have to code to bare metal. The IO stack was much shorter and the OS (such as it was) was single tasking, single threaded, and didn't have any networking.

Slowdowns in modern systems are much more complicated that a simplistic "inefficient code makes them slow". There's hundreds of small stupid latencies all over the place in modern systems. The USB and Bluetooth stacks are filled with little latencies, retransmissions, and just wait loops. Unless your inefficient code is full of accidentally quadratic loops it's usually compounded latencies making systems "feel slow".

Re:Spectre mitigations murder userspace performance by p_l on Sunday June 20, 2021 @08:44AM (#27569434)
The description sounds more like PS4 than PS2 (maybe PSX was simpler - PS3 is closer to PS4 and "normal" programming). I've seen numerous examples of how involved CPU was in rendering, especially on PS2, due to special units being exposed as MIPS coprocessors. Efficient rendering meant you had to keep the pipelines happy and that could require significant amount of assembly wrangling.

I remember reading about how GTA3 on PS2 was a watershed moment, because it was a proof you could use a somewhat more generic engine - whereas console programming had a heavy amount of assembly focus due to historical reasons.

Re:Spectre mitigations murder userspace performance by giantrobot on Tuesday June 22, 2021 @02:36PM (#27595397)
I'm sure there's plenty of people on HN with first hand experience developing for the various PlayStation models. I only know what I've read about the experience and technical tear downs. So maybe one will swoop in dropping knowledge bombs and tell me I'm wrong.

The original claim was 3D worked on the PSX because of "efficiency" of the code. While I'm not claiming games on the PSX were inefficient, the hardware was designed specifically to do real-time 3D rendering. It's not like the PSX could only do 3D because everyone somehow had super deep mastery of the system.

To my understanding the PSX was a big change in console development. While consoles always had some dedicated hardware (blitters, audio DSPs, etc), development was a lot of beam racing. There were no frame buffers, code ran as graphics were being drawn to the screen and during the VBI.

The PSX was more like a modern system where you had a CPU and a GPU (albeit fixed function) with an OpenGL-ish drawing model. It was a far more capable 3D machine than PCs of the time that had to do all 3D work on the CPU with no acceleration.

Tomb Raider on the PSX was impressive because the console only cost $300. While I'm sure Tomb Raider on the PSX wasn't inefficiently coded, the design of the PSX console contributed as much to its existence as the code. Quake on the PC on the other hand could only exist because of the developers' deep system mastery pushing generic hardware to its limits (before GPUs were prevalent).

Re:Spectre mitigations murder userspace performance by UncleMeat on Saturday June 19, 2021 @08:42AM (#27559972)
> I’d rather have simple hardware that is light on energy requirements and easier to understand. I don’t think software as an industry really has a “this chip isn’t fast enough problem”.

Turning off speculative execution reduces performance enormously. Yes, code is often less efficient than it could be but "surprise, you need 5x as large of a datacenter because your hardware isn't doing fancy stuff" is not going to be an acceptable thing for any large organization.

Re:Spectre mitigations murder userspace performance by infogulch on Saturday June 19, 2021 @09:02AM (#27560100)
I suspect our heavy reliance on speculative execution is just a local optimum, and we can grow out of it by providing the processor more information about the data flow, making memory access more explicitly asynchronous, and simplifying the hot paths.

I like the way the mill approaches these architectural problems.

Re:Spectre mitigations murder userspace performance by tux3 on Saturday June 19, 2021 @09:26AM (#27560267)
I'm very unconvinced by these arguments. Providing enough information to the processor to claw back all the performance that predictors give you, that requires some way of knowing all those things in advance. Statically.

And so you fall into the pit of tar and despair that is relying on Sufficiently Smart Compilers. Same one that couldn't save Intel's shiny new IA-64 architecture (the "Itanic").

Static analysis is just not a tractable way to replace hardware predictors. Even profile-guided information is a questionable answer. Could it be we're missing some key idea that'll take us far beyond all of this? Maybe, but it's probably not the same WLIV designs that require impossible feats of software comparable to asking for the weather 2 years in advance so you can save yourself the hardware cost of an umbrella.

Now, perhaps it would be unfair to criticize the Mill for not taping out an actual chip, after all having new ideas is valuable and fabs are very expensive. But we know this is a tricky subject. Performance optimization ideas only have value after you benchmark them, and they don't have an RTL implementation. Nothing that can run on an FPGA, not even "abstract" Verilog running in a testbench on a software simulator.

If what you want is a DSP that runs super simple hot loops very fast, then build a DSP and make it VLIW all you like. But that's not the workloads people run on a general purpose computer.

Re:Spectre mitigations murder userspace performance by wizzwizz4 on Saturday June 19, 2021 @01:55PM (#27562680)
> And so you fall into the pit of tar and despair that is relying on Sufficiently Smart Compilers.

How will we ever know that a Sufficiently Smart Compiler is impossible, if we never have processors where compiler intelligence is useful?

Re:Spectre mitigations murder userspace performance by saagarjha on Saturday June 19, 2021 @02:36PM (#27563098)
> And so you fall into the pit of tar and despair that is relying on Sufficiently Smart Compilers. Same one that couldn't save Intel's shiny new IA-64 architecture (the "Itanic").
Re:Spectre mitigations murder userspace performance by wizzwizz4 on Saturday June 19, 2021 @05:55PM (#27564840)
And how can one get their hands on an Itanium machine? It's a dead architecture in less than two weeks, and for the last decade the things have been almost exclusively enterprise machines; not something the hobbyist (or even the academic!) is likely to get access to.

Do you have an HP zx6000 lying around, perchance?

Itanium failed early; it had many more problems than compilers. And if we don't have a replacement, we'll never get those compilers from anyone but the most obsessed.

Re:Spectre mitigations murder userspace performance by temac on Saturday June 19, 2021 @09:36AM (#27560350)
This has been tried (not on vaporhardware Mill, I'm more thinking about e.g. Intel Itanium) and it failed.

The reason is extremely simple: a speculative OOO processor optimizes dynamically. If you switch that with static compile time optimizations, you are bound to only be as fast as before in some quite limited parameter ranges (like: number of entries in a hash table, size of an image, etc.)

Re:Spectre mitigations murder userspace performance by homerowilson on Saturday June 19, 2021 @11:32AM (#27561367)
Also don't forget the really interesting Transmeta processors, also ultimately a failure. VLIW is very difficult in practice.
Re:Spectre mitigations murder userspace performance by p_l on Sunday June 20, 2021 @08:59AM (#27569515)
Transmeta were VLIW, yes, but weren't EPIC - which is the main difference involved. VLIW just means there's a long instruction word, usually targeting multiple units, but it doesn't necessarily mean there's no dynamic reshuffle in the CPU architecture or anything like that.

EPIC stood for Explicitly Parallel Instruction Computing, and took the "let's push it all to compiler" to the extreme. Itanium was never supposed to have any form of branch prediction or OOB, because it was supposed to be handled by the compiler.

This led to someone quipping that Itanium was a very fast DSP, but ridiculously expensive.

Re:Spectre mitigations murder userspace performance by monocasa on Saturday June 19, 2021 @02:04PM (#27562786)
I agree with what you're saying around the Mill and dataflow in a sense, but feel like we disagree in the details. The issue with Spectre, etc. is that data of different trust levels are comingled in a single address space. We want the processor to speculate on some data, but not others and we don't tell it how to tell the difference. What's going to save us isn't dependency information or static scheduling, but more fine grained permissions around the data accessible at a single time.

IMO we need to bring back segmentation hardware. Not the x86 version of segmentation (there's not a feature that x86 wasn't able to make twice as complicated as it needed to be while only giving you half the use cases), but the cleaner object capability on top of paging hardware versions of segmentation. That solves the really rough Spectre cases like even NetSpectre where you can slurp out kernel state remotely from untrusted network packets. Just stick them in a "this memory is untrusted" segment. From there the CPU's dynamic dataflow optimizations can include speculation where memory is marked as trusted.

Re:Spectre mitigations murder userspace performance by pjc50 on Saturday June 19, 2021 @09:27AM (#27560273)
Is the Mill dead? I've not seen the regular flow of HN posts about it recently. Everyone got excited about RISC-V instead.
Re:Spectre mitigations murder userspace performance by mhh__ on Saturday June 19, 2021 @01:09PM (#27562260)
They're still going - Ivan said on their forum that they were planning on going full steam ahead last summer, but scuppered by lockdown. I don't totally buy that, but that's the story.
Re:Spectre mitigations murder userspace performance by mburns on Saturday June 19, 2021 @12:53PM (#27562112)
Re:Spectre mitigations murder userspace performance by zozbot234 on Saturday June 19, 2021 @09:23AM (#27560248)
The interesting part about the Mill is that it's not even all that complex. It's just taking the existing VLIW architectural approach and adding a number of custom "tweaks" to improve the programming model over what VLIW provides.
Re:Spectre mitigations murder userspace performance by verall on Saturday June 19, 2021 @12:58PM (#27562160)
This is sort of what GPUs do, since a GPU driver is essentially a big JIT compiler. Branching is very expensive compared to CPUs. Remember that 95%+ of branches of predicted successfully, and CPUs work on more stable ISAs.
Re:Spectre mitigations murder userspace performance by icegreentea2 on Saturday June 19, 2021 @08:59AM (#27560081)
More specifically, we should point out that speculative execution is important for papering over IO/memory latency. We can incant all we want about 'inefficient code', but the compute vs memory latency imbalance is a whole different beast.
Re:Spectre mitigations murder userspace performance by astrange on Saturday June 19, 2021 @08:46AM (#27560000)
> I’d rather have simple hardware that is light on energy requirements and easier to understand. I don’t think software as an industry really has a “this chip isn’t fast enough problem”.

Have you seen how happy people are with M1 machines? That's because it's faster. It's definitely not simpler.

> Most of the real slowdowns anyone has in day to day performance has more to do with inefficient code than hardware anyways.

Not true, especially not for heat. The energy use of on-CPU/on-GPU tasks is more up to the hardware. It might be true for things like network bandwidth though.

Re:Spectre mitigations murder userspace performance by saagarjha on Saturday June 19, 2021 @02:38PM (#27563111)
Wait until Slack outfits their developers with M1 machines and we’ll quickly be back to being unhappy again.
Re:Spectre mitigations murder userspace performance by josteink on Sunday June 20, 2021 @05:16AM (#27568408)
I know this sounds radical, but uh, if everyone hates slack how about not using slack?
Re:Spectre mitigations murder userspace performance by saagarjha on Sunday June 20, 2021 @05:52AM (#27568584)
This is often the same as saying "how about you quit your job".
Re:Spectre mitigations murder userspace performance by josteink on Sunday June 20, 2021 @06:15AM (#27568692)
But why does companies use slack if “everyone” hates it? For once, I’ve never been to a company which uses it.

There are literally countless options in the IM/groupware field which is not Slack.

Re:Spectre mitigations murder userspace performance by mhuffman on Sunday June 20, 2021 @10:30AM (#27570367)
The only two I have seen in most places are Slack and Teams.
Re:Spectre mitigations murder userspace performance by temac on Saturday June 19, 2021 @09:32AM (#27560315)
You would be surprised how "slow" a non speculative processor is.

Even Intel Atoms have been speculative since Silvermont.

If you take more mainstream historical processors, for example:

https://www.cpu-world.com/Compare/385/Intel_Pentium_Dual-Cor...

Let say the Pentium MMX 166 is roughly 60x slower, per core, than the Pentium G6950. So the "IPC" (I'm not sure of what the benchmark are doing, but I'm doing rough evaluations anyway) is approx 3.5 lower for the Pentium MMX.

And that is while the speed gap constantly increased between memory and CPU (esp latency), so a Pentium MMX cpu core somehow speed up to 2.8 GHz would actually probably perform even worse in a design with memory that actually exists. Maybe way worse.

The complexity of modern good ARMs is comparable or even higher in some area than the complexity of x86 processors.

Re:Spectre mitigations murder userspace performance by josh2600 on Saturday June 19, 2021 @08:44AM (#27559983)
We’re seeing a full frontal assault on Intel by competing computing architectures. Everyone is jockeying to see who is going to have the better next gen architecture. The problem is that in order to start attacking general computing as a problem, you need first customers who buy a lot of chips, and those customers tend to have specific workloads they want accelerated. Now we’re back to designing for customers instead of abstractly making the absolute best chip.

And that’s the reality right? Demand drives innovation it private enterprise is funding it. If you want a chip for the public good, you’d need to fund it with public money and that’s a different dance with the devil.

In short, I think the way that technology is produced is fundamentally customer focused right now.

Re:Spectre mitigations murder userspace performance by im3w1l on Saturday June 19, 2021 @08:49AM (#27560013)
I think this has cause and effect reversed. Advancing general purpose CPU's is very hard and expensive nowadays. The slower rate of progress means that custom chips make more sense, they take longer to become obsolete.
Re:Spectre mitigations murder userspace performance by quotemstr on Saturday June 19, 2021 @07:25PM (#27565491)
I'll make a deal with you: I'll agree to turn down speculative execution in hardware if

1) you can solve the VLIW scheduling problem, and

2) you transition the entire computing ecosystem to a JIT model.

These are the things we would need to claw back the performance we would lose through disabling speculation. You can look at speculation (I'm handwaving a lot here, bear with me) as the processor hardware dynamically recompiling your program code depending on observed behavior of the program. That's well and good and it gets us a huge performance boost.

You can, in principle, do the same thing in pure software. But every single attempt to do has ended in total algorithmic failure.

The last serious attempt I'm aware of to "drive" a uOP scheduler explicitly in software was Itanium, and that failed, in part, because compilers couldn't take advantage of the processor's instruction level parallelism. There's nothing in math or mathematics or computer science that forbids a magical compiler of the sort the Itanium people wanted to create. But nobody's made one. Your first task in your project of eliminating speculative execution is to solve this algorithmic problem.

But solving problem #1, while necessary, is insufficient. No static ahead-of-time compiler can adjust the compiled code depending on the actual execution history of the program. To really get back to speculative execution par, you have to give your already-magical compiler the ability to recompile code at runtime. That means turning everything into a JIT. Your /bin/ls would actually be LLVM bytecode, not machine code, and some runtime system would be responsible for dynamically generating the machine code and adjusting it depending on execution history. After all, that's what current superscalar CPUs do internally, transparently, all the time. This is problem #2.

Honestly, I think the world we'd create by solving both these problems would be a better world. I really don't like how we can't program the processor's speculation engine and uOP scheduler. I'd love to be able to do that.

But I don't think we can get there from where we are, so we're going to be stuck with speculation and hardware mitigation forever. Please, prove me wrong.

Re:Spectre mitigations murder userspace performance by rmdashrfstar on Sunday June 20, 2021 @02:16AM (#27567611)
> Honestly, I think the world we'd create by solving both these problems would be a better world. I really don't like how we can't program the processor's speculation engine and uOP scheduler. I'd love to be able to do that.

Have you started working on any solutions for 1 and 2?

Re:Spectre mitigations murder userspace performance by wffurr on Sunday June 20, 2021 @07:55AM (#27569197)
#2 sounds like WebAssembly with WASI.
Re:Spectre mitigations murder userspace performance by mhh__ on Saturday June 19, 2021 @09:34AM (#27560341)
But rendering isn't what makes a modern computer slow. For example, Assassin's creed isn't even that well optimized but it maxes all threads, uses all the GPU and looks beautiful.

And people have tried "simple" hardware (it ends up being not simple) that the programmer (compiler) understands before, it doesn't work.

Re:Spectre mitigations murder userspace performance by jcelerier on Saturday June 19, 2021 @11:58AM (#27561604)
> I don’t think software as an industry really has a “this chip isn’t fast enough problem”.

yet this is what I'm saying to myself every day, even with very fast computers available, and very efficient code running that hardware pretty much to its limits.

Re:Spectre mitigations murder userspace performance by paulryanrogers on Saturday June 19, 2021 @09:50AM (#27560442)
Consider the philosophy of MAME which plays old games and avoids optimizing for GPUs and (IIRC) most specialized graphics layers like Direct3D or Metal. As I understand it this makes it more portable and easier to maintain, at the cost of some performance.

Now MAME's scope is limited to aging games which aren't evolving. Only the underlyibg OS's and runtime hardware are changing.

Console games pushing aging hardware have a very small variety of runtime hardware to optimize for. And optimizations pay off in better graphics or features that differentiate. Optimizing for ever evolving desktop computers with near infinite hardware is a whole different story.

Re:Spectre mitigations murder userspace performance by mkr-hn on Saturday June 19, 2021 @08:41AM (#27559964)
See: racing the beam

https://www.youtube.com/watch?v=sJFnWZH5FXc

The things they did with 128 bytes of RAM were amazing.