Engineers have known for a long time that we aren't as efficient as we could be.
[0] Not attempting to beat up on either of those companies.
I never dreamed that software developers would manage to make things slower still (by orders of magnitude) by throwing everything onto the 'net and using multiple round-trip web-api calls to do the most trivial things. I just didn't anticipate that this source of inefficiency was there to be harnessed.
(This came to a head for me recently when I was trying out planner apps for my phone. It boggled my mind that, even on a dual 1.8 GHz processor, Asana and MS Planner do not respond instantly.)
The Dylan folks were all trained computer scientists, mostly MIT PhDs, almost all with years and even decades of experience in writing garbage collected languages on resource-constrained machines (e.g. MACLISP). By contrast many of the apps you write are written by much less trained folks using a lot of automation and boilerplate.
While I personally prefer to write tight, clean code directly to the underlying system, I am really glad for this democratization of programming which has lead not only to good careers for people who didn't get to go to MIT/CMU/UCB but also lead to a profusion of great apps. Not everybody has to design TCP but everybody gets to use it.
And note that Dylan had little impact on society while Tinder sure has.
Don't know what I was thinking.
But it also means there's some relatively low-hanging fruit these days if you find a niche where performance is much more important than accessibility :)
We think we have been optimizing for those things, but I don't know that we actually have. Does Ruby/Python really help you get to market faster than would similar languages with static typing and compilers or JITs? (Kotlin, C#, F#, Nim, etc)
There is research that suggests not, certainly I don't think the effect is large if it even exists. Many people use TypeScript because they feel it is a productivity boon over Javascript. We might one day leverage that and use the static types to compile TypeScript more efficient WASM or something, win win situation.
- Garbage collection make performance unpredictable because of collection spikes.
- If you use a big library, you now have to load the library, which is an extra time-sink.
- Likewise for frameworks.
- Optimizing compilers optimize the 80% of our programs that don't need optimizing, and under-optimize the 20% that do.
(This is DJB's argument.)
Also, there's a counter-argument to this, that writing non-garbage-collected code might result in security bugs (to do with memory management). And security is a non-functional requirement that some might say is more important than speed. It's hard to judge though; there are loads of requiremenets in software dev, and it's hard to balance them.
That's falacy. Don't mix "fully manual memory management" with "I have no GC". Those are 2 totally unrelated topics. GC is one of the way to manage memory automatically. There is hundreds with their own plus-sides and downsides. Here's the least unpopular:
* Stop the world and incremental GC: Makes it easy to write simple code. Doesn't really make the lifecycle explicit so when the codebase grow larger you start to have boilerplate code to manage/close connections/files/transactions. (all toys languages, also Java and Go).
* Reference counting: Can be made fast enough by using some of the pointer 64bits to store its state. A giant problem is that all your libraries have to support compatible "smart pointer" formats or the whole thing become unmanageable. (Modern C++)
* Contacts/Lending: Does it's job, is safe but makes the code more complex by making a lot of implicit things explicit (Rust, Modern C++)
* Explicit ownership + message passing: works well enough for GUI because just like HTML everything is less or more a tree. Also consuming messages allow "higher level" objects not to have to care about pointers. For backend code it's not very good. (Qt-C++)
* Pure functional: Everything is a copy, they get freed when out of scope. Works well for some domain, but gets in your way for more stateful constructs.
* Copy-on-Write: Pass everything by reference (and count them). Copy when they are modified, otherwise point to the real memory. Only solves half of the problem. Assuming most allocations are containers it works well enough. It doesn't attempt to solve objects management.
* Pipelining: Get input, process them, pass them on or free them. The processing should not have to dynamically allocate at all (Bash, Erlang).
* Pre-allocation: Allocate everything at compile time and use buffers big enough for your use case. (C)
* State machines: Encode the object lifecycle as a series of state changes. Release when it reaches end-of-life. Great for protocols, parsing, transactions. You usually can use code generators to generate proven safe C from an higher level format or proof. However it doesn't scale and only matches a subset of programming projects. It's also hard to learn for non EE programmers because it doesn't map perfectly on the iterative Von Neumann CPU. Easy to read but hard to write.
No-GC != security-bugs because of memory management. If a C program uses pre-allocated buffers, then the security issues is because the language is unsafe, not because it has no GC.
I think we can go even stronger than 'probably' - we know that there are people who do work to these constraints, and it's mostly experienced devs from a few starting tracks working on years-long projects.
As far as I know, NASA's coding standards haven't slipped from their famous "0 bugs per 500,000 lines", and that code (along with JPL, Orbital, SpaceX, etc) is often written to harsh physical constraints. Systems like the Mars entry controller have to operate unsupported in environments where slowness, power, and weight are all serious limiting factors. For all that the IoT is a disaster, embedded control software for non-consumer projects is often concise, fast, and reliable.
It would probably be possible to train a lot more high-performance developers than we have now, but the demand certainly hasn't been present and even if we could there'd be a real price exacted in lost development time and features.
It'll be interesting to see what happens as chip speedups become increasingly unavailable; it's possible we'll revisit the old stuff and make it faster, but so far the leading answer appears to be falling back on networking to handle tasks on bigger hardware.
Isn't the problem really memory bandwidth? Aren't most CPU cycles "wasted" on speculative execution while waiting on fetches from memory slower than CPU caches?
Wouldn't a hardware model where all of the RAM (SRAM?) was on the CPU die (fabrication and DRAM clock speed aside) solve this problem and make everything an order of magnitude faster? What am I missing? Is this really the programmers fault?
The CPU literally spends most of its time waiting on my instructions, not choking on them, so why is this my fault for being lazy?
You are missing money...
SRAM and wider data paths are expensive.
You are totally right about that taking more logic into synchronous domain domain is an almost universal solution to getting more performance. The faster the chips become, the more IO gap becomes.
Experienced MCU programmers know quite a number of algorithms that may run faster on "dumb" 100mhz MCUs than on a 20-cored Xeon, and are proud to show them off.
Even when some of MCU cores may take like 6 cycles to do an arithmetic op, their strong side is that it will be taking them these 6 cycles to do that all and every time consistently, and that their memory runs synchronously.
Say, in a 1000000 cycle loop, where every operation results in cache miss, Xeon will spend more time on a loop than a wimpy 100Mhz MCU that simply don't take any penalty on unpredictable, heavy on synchronous ops cope.
Apple A11 is so fast for an ARM just because they choose the simplest, the surest, most direct, but also the most expensive way to scale performance: tons of SRAM and prefetch logic everywhere to reduce IO gap as much as possible.
I was reading a thing a while back on here or Reddit where someone asked about an OS that ran entirely in cache, after all in principle a modern processor has more L3 than an original Unix minicomputer.
Since many of the memory subsystems operate on the assumption that changes should be propagated to memory that you would still need to have RAM there even if you weren't really "using" it.
There was a suggestion that since the bringup code begins entirely on-processor (before initializing RAM) that you could hack the Management Engine and keep running in that mode forever.
I have read estimates that go as high as 1/3 of the CPU lost to it (sorry, no reference here), but even those were not usual.
At least one aspect of windows10 design will add input latency, which is the forced vsync in the windowing system.
Disagree. Much of it keeps getting slower.
I'm sure it solves problems they just aren't my problems.
Which frankly applies to nearly everything AWS offers.
I've seen people design for massive scaleout before they even have a final idea of the product.
Frankly for the kind of systems I work on I have collosal vertical headroom before I even need to think about going lateral with scaling.
I mean the largest database I deal with would fit into the RAM of a single mid-range 2018 server (192GB would do it).
The most important database on our main work system will fit into the RAM on my Thinkpad with enough space to run an IDE and Vagrant (32GB of RAM in the laptop so high but insane).
My iPad Pro 9.7" (one version old, came before the 10.5") drops frames in iOS 11's full screen blur transition animations. Before iOS 11 it was fine, but either the iOS team didn't test animations on anything but their brand new hardware, or they tested and didn't care if it ran smoothly.
Having one of the OS's most frequent animations choke on anything but the brand newest GPU in the $650+ model? Not a great look, especially with Apple's reputation for making devices that last a while.
Fingers crossed that iOS 12 fixes it.
Power consumption scales roughly quadratically in relation to frequency.
Modern cores all have features like power and frequency gating, performance counters - the stuff needed to allocate just enough of computing power to complete computations in time. This is very, very evident in things like music playback - a lot of intermittent io and computational spikes.
It's true that switching power scales with the square of voltage (P=V^2Cf), but that's not really a problem for the race to sleep as switching power even with V as low as you can get it is going to be orders of magnitude higher than your static power.
The "race to sleep" is a misconception long held by software people.
The situation you describe is only true when the part of the chip doing computation is completely off, and is shunted off from power. That's so called deep sleep.
There is a limit to how frequent you can switch the big and complex chip to low power state, as doing so by itself takes some power and computational overhead to keep track of your power use profile.
For many cases, one have to endure leakage and keep working, like when you have to do IO have to keep the chip "on" anyways, and want to utilize your "on" time as much as possible.
A ton of work have been done on power saving strategies like partial device shutdown, shutting down part of registers, trying to operate different parts of the chip at different frequencies to match io wait and processing time, downshifting to more iterative computational paths, and on and on and on
It's very easy to stamp out more ALUs on a chip than we can reasonably fill with data (and, arguably, we've already hit that point with consumer hardware). Improving parallelization nearly always comes at a cost to serial performance, and if you have code that requires a latency-sensitive, unparallelizable critical loop... well, you're basically saying "screw you."
They're working on a new processor architecture, which is very interesting in many, many ways.
And they're working on ways to parallelize existing code, in ways that can't really be done with conventional superscalar architectures.
The Mill has some sets of innovations that seem to come together, for instance the Spiller, Belt, and memory speculation model seem hard to seperate. But I'm pretty sure you could detach that from having a virtually addressed cache and their protection model. And then there's stuff like the super weird organization structure they have. I feel like they're setting themselves up for failure by trying to make changes in so many ways all at once.
However, starting up an entirely new computer architecture isn't easy. They want to retain control of their company, so they aren't accepting tons of VC money. Progress is slow.
That's the biggest problem with Moore's law fizzling out - computer architecture isn't actually that hard, most of the work is just figuring out how to apply the extra transistors you're given. So no matter how specialized you get, you aren't going to get more than 1-2 generations of a chip out of a single semiconductor process, as there is only so much that clever architecture can accomplish.
That was sort of my point; however cool the conceptual architecture is, they've definitely reached the point where they ought to be able to get at least rough comparisons to competitive hardware. That such comparisons are lacking should be taken as a sign that the architecture is too immature to make those claims. I just wish people would stop bringing it up as the magic solution to parallelism, since there's insufficient evidence for those claims.
> That's the biggest problem with Moore's law fizzling out - computer architecture isn't actually that hard, most of the work is just figuring out how to apply the extra transistors you're given. So no matter how specialized you get, you aren't going to get more than 1-2 generations of a chip out of a single semiconductor process, as there is only so much that clever architecture can accomplish.
I've been swayed by the idea that the future of processor technology is essentially accelerator land. You're going to have a few big CPU cores because they're good for code that has no real parallelization potential, but you're going to also have clusters of accelerators that are optimized for different kinds of workloads (such as data-heavy, communication-light workloads that are good for GPGPU).
That is possible, their progress has been slow.
> For the most part, the ISA just needs to be 'good enough' to allow things like branch prediction, speculative execution, out-of-order execution, etc. You can do that with x86, ARM, POWER, whatever - the instruction set doesn't matter a whole lot, it's just how many/what type of transistors you can throw at it, and how hard you want to work on timing and power optimization.
If you watch all their videos, then this isn't really correct.
Architecture matters a lot. With modern processors, there is a vast gulf between the instruction set used (the programming model) and how the processor actually works. A lot of the die area on the chip (aside from caches) isn't devoted to actual computation (like adding numbers in an ALU) but moving data around and scheduling.
The Mill Computing approach is to move all this functionality back out of the hardware into the software compilation / loading step. Where it can be done once and more efficiently. They can then use that die area to add in more functional units (because the architecture is wide-issue) and more cache.
There is only so much that clever architecture can accomplish, but that is still an order of magnitude more than what is being done now with existing architectures, so this seems a worthy pursuit.
Why will they succeed where Itanium failed? There's a long, long history where people expect compiler smarts to maximize performance of hardware, and compilers just aren't up to it. It's also very necessary to demonstrate this on larger, more complex code, rather than focusing on things like GEMM kernels.
There's a lot that needs to be justified, and the Mill people just haven't presented the justification.
Ivan Godard mentions Itanium a few times during the talks he has given. So they are at least aware of that architecture and its faults.
I guess it all comes back to the question of where all this instruction scheduling ought to be. Yes, it is being done in hardware now, because that is the path that offers software compatibility at the binary encoding level.
But in a sensible world, is that really where the instruction scheduling should be? And if it can be done in hardware, I don't see any theoretical reason it can't be done in software.
The question, then, becomes how to do this in a practical manner. It remains to be seen if the Mill Computing scheme will be successful. I would argue that we (as an industry) haven't failed enough with the software scheduling approach.
Hardware instructions can take a variable amount of time to complete, depending on their inputs--particularly branches and memory instructions. That means that the set of operations that are ready to execute is a fundamentally dynamic property, and software can only make a static decision.
For memory accesses yes, that's a big problem which is why there's so much effort in the design devoted to making software memory speculation feasible. The Itanium put some effort into that too but generally at a much higher overhead and the Mill's solution for dealing with memory faults in speculated accesses was frankly brilliant.
Everything on the Mill CPU architecture is statically scheduled, because everything has a fixed latency.
Loads / stores from L1 cache are also a fixed latency. If you're going out to DRAM, then you'll have to wait a lot no matter what architecture. Branches are also fixed latency, assuming the target has been prefetched.
So then it is a question of how good is the prefetching, and they've got some techniques to improve that too. They've also got techniques to help with cache pressure and effectively double the I-cache size without the usual power and latency penalty.
Details are in the papers and videos, if you are interested.
So each 'problem' that's worth it might get a specialized ASIC or two, but all of that could very well run its course in the next 5-10 years, and things look kind of bleak for performance gains after that.
But you were still getting more transistors at the time. You just had to use them in different ways.
We've gotten complacent. Software keeps getting faster, despite bad coding - just because of the improvements in hardware.