If you think it will accelerate more while hardware gains drop to zero, why were we leaving these absolutely massive gains on the table? This means that, if a new processor gave us double the speed, we could get more than double just from software alone and get the hardware doubling on top of that.
If you think it will decelerate, why are you happy Moore's Law ends? We're worse off than we were before, even though we aren't "lazy" now.
This approach seems to me a bit like saying "Oh I'm happy I lost my job, now I can finally make money by looking for pennies on the ground all day, which the job made me too lazy to do".
There have been plenty of small experiments in other architectures but never enough of a mindshare investment to really evaluate viability and performance of alternatives. There are costs to a more diversified architecture eco-system, sure, but I fully believe we will find more value in that space than we have so far taken advantage of.
ASIC implementation of a PLC, anyone?
It probably means we're worse off, in that spending on software engineering to get gains rather than expecting it out of new hardware processes will almost certainly be more expensive. On the other hand, software gains give us more computing with the same energy which is a good thing for the planet.
By FLOPS I mean the classical FP64, while OPS can be any bit width, even 1 bit.
1. it wastes energy, increasingly a non trivial share of total energy use by humans
2. it often manifests itself as the user (me) having to wait for things that should be instant.
If the latter never happened, I would be mostly on board with what you are saying. I strongly wish we could improve the state of software such that you could achieve the same productivity we have now, with only a factor of 2 to 10 slowdown vs optimum instead of the 10x to 1000x we accept now. I think that should be possible. As a start, we can make first class jits for languages that are usually interpreted( python / ruby ) and possible, come up with statically typed languages that people find as nice to use as they find dynamically typed ones. I think this should be possible with techniques like type inference. Doing those two things can improve the efficiency of a lot of code without any downsides imho.
I'm very mean to my slaves. I have absolutely no issue leaving a rack of machines to slave away all weekend while I enjoy myself as opposed to spending that same weekend myself making it more efficient. (e.g. training models)
I don't care if a CPU is running at 80% consuming more electricity when I could work for a week and make it do the same job at idle. It's an inefficient use of my time. That same week's worth of effort could be spent delivering more value than the $10k server + $500/year operating cost. (I'm paid less than that, but I deliver more value to my company than my paycheck.) Also, risk. If it works now, it may not after I spend a week mucking with it.
That attitude is perfectly fine and reasonable until people start applying it to user-facing software. That's how you get software that wastes millions of user hours and immense amounts of energy just so some programmer could "deliver more value" and, sadly, that applies to an alarming proportion of modern software.
Ultimately, my work is dictated by what the customers whine about. If they make more noise over missing feature X than slow feature Y, X gets my time.
This kind of oversimplified faith in the idealized notion of the free market completely ignores the myriad of externalities that systematically distort software competition. General arguments aside, it suffices to go through almost any set of software niches to note that the dominant products in most of them are dominant for reasons that have nothing whatsoever (and are frequently inversely correlated with) any reasonable sense of technical merit or even economic value produced (for the users) in the present (though many of them were once upon a time arguably the best in their respective niche).
> Ultimately, my work is dictated by what the customers whine about. If they make more noise over missing feature X than slow feature Y, X gets my time.
Non-technical users rarely articulate performance-related concerns, but that doesn't mean they don't have them. In my experience, if you have prolonged free form conversations with them, they will often voice impressions of the software in question getting slower and less reliable overall, whereas targeted questionnaires and focus groups will tend to laser in on specific features simply because those are much easier to articulate. Also, bad software is so common by now that some people can't even imagine that there isn't actually any sane technical reason for, say, Photoshop to take almost 10 seconds to open a simple image file. That doesn't mean you're not wasting their time, money and energy.
Things like that already happened - for example slow HDDs are now replaced by very fast SSDs. There is Intel Optane that is only 1 order of magnitude slower than RAM and Intel Opate DIMM is closing gap even further.
I think you're romanticizing a bad thing.
It simply isn’t possible to make transistors infinitesimally small (just as a horse’s metabolism could never keep up with the relativistic muscle movements required for faster than light travel.)
Sure, that sucks. But realistically, the only way to progress is to design better architectures, or think of new computing paradigms, or deepen our understanding of the laws of nature.
All of those, if realized, could be very good things.
I sometimes get bummed out about the end of Moore’s law, but I try to remind myself to be grateful and appreciate what we have. Computers are wonderful, and it’s amazing that we have them at all: the universe didn’t owe us computing; we figured it out in order to solve problems. Now we have another big problem to solve.
Ultimately, we will see huge investments in pushing the boundaries of computing. Whether it’s biological, quantum, or just massively parallel, I’m excited to see what we can come up with in the years ahead.
I think if you fully understood the implications of this, you wouldn't be so sanguine.
We're already seeing the effects of a stalling out of process improvements.
Starting a couple years ago, there were articles by enthusiast PC builders who were buying used Xeon processors on eBay, and building new gaming PCs around them, with comparable performance to the i7's available at the time. Just to put some round numbers on that, you had people spending $100 or so on a used CPU (none of that money goes directly to Intel) vs. spending nearly $1000 on a brand new i7.
You can find other instances where a company is competing with their own older, used products... it isn't fun for the company in question.
What's going to happen to Intel's stock price when that sort of thing becomes more common? The price of their stock (and all other tech companies) is based on future value, rather than other companies that pay dividends but otherwise aren't expected to expand significantly.
If investors come to accept the fact that Intel can't really make a chip next year (that will drive hardware and software sales for the entire PC industry), better than last year's chip, their stock will crash. As will all the others in the tech industry.
New hardware (with more capabilities at a reasonable price) drives new software / new applications. If next year's phone isn't better than last year's phone (assuming you can replace the battery that is losing capacity), that will kill valuations in the mobile segment.
Some tech like stacking higher in 3-D on the chip will help applications like NAND Flash storage that aren't heat-dissipation limited. But even there the cost per bit stored will won't be going down at nearly the rate like we're used to seeing.
A tech crash across the board will tank the entire world economy as well, because much of the productivity improvements are based on computer technology.
This is a massive, massive upheaval we're looking at. Unless we switch to new technology that can keep the productivity party going for a while...
The question is, what drives stock values? What drives the entire industry?
I was mentioning Intel because they have been a leader in semiconductor process technology, often the first to ship product at a new process node.
This race to increase various aspects of performance has long let the fields of architecture and the software on top of it be lazy in their optimizations. When there's no more juice in that fruit to squeeze, I know there is much more to be found elsewhere.
/I can't wait for my non-von neumann SoC FPGA hybrid to use for...I'll figure it out then I'm sure.