Yeah of course most of that slowness is not even because of ignorance of the low level stuff but complete disregard of performance aspect of software. If it's even slightly more convenient for a programmer to do something that is very stupid performance wise, they will do it.
Sure developer time costs, but this cannot actually be the reason. Because tooling is also ridiculously slow in exactly same way and wastes countless of hours of that costly developer time.
This is absolutely true, but the problem is usually a lack of due care about performance at higher levels, rather than low level optimization. Things are usually slow because they do entirely too much of the wrong things, than because nobody optimized the things, or the wrong registers were used or what not.
I get that like the major parts of picking algorithms that are N over N^2 is always going to be more important, but it doesn't seem like the article is disagreeing with you at all, just another area where people need to consider their decisions.
And OOP is simply not contradictory to DOD.
Maybe it insists on a deep heirarchy of widget inheritence as many do. Maybe the framework wants to load and configure all the possible widgets, even though the application only uses a few. Maybe theres a bunch of images loaded from disk even though they're not used. Maybe it's just the fantastic default compositing system that puts everything a frame behind adding up with everything else.
There certainly are applications where data structures matter, and time spent processing data is significant to user experience, but that's not why the whole computing experience feels slower (although maybe prettier) than 25 years ago, despite capability being so much more for most things (as pointed out in the article, ram is better than years ago, and we certainly have more of it, but ram access takes a lot more cpu cycles now)
Also, actual native GUI apps are quite fast imo. Only electron apps are slow, but I really do think that it is mostly due to the web having a wrong abstraction for GUIs, not due to other reasons (of course there are shitty web apps, as well as shitty programs everywhere else)
It totally is because we've abstracted the cost of software engineering unto the hardware.
Back in the 90s, people had to basically beg for memory when writing software while modern development is built on the theory of getting the fastest delivery speed at the cost of performance. We outsource the cost of software unto millions of customer's hardware and it's never coming back because going to market quick makes you money.
In your example of the watch window, updating it at more than 30 fps is probably overkill.
One very common mistake is optimising the wrong thing. My advice is always the same: first run a profiler with a real workload (absent that, eu it over your automated tests), then look at what you need to optimise for speed. All the rest should be optimised for readability.
In the article’s case, if counting warriors is so important, keeping a counter at the AntColony class that’s updated every time you write to an Ant would be the obvious solution.
Quantify important into how much you spend on your tooling.
Aside from some key bits of development software at the IC / board layout, almost everything a software developer needs is free, open source or cheap. Noone pays $1000 for an IDE anymore.
How many minutes per developer per day before that level of expenditure pays for itself inside 2-3 months?
Who cares? The premise is false, you can't buy it. There is no "fast tools" you pay for. Every decent shop got their devs an SSD at the first available opportunity because it paid for itself really quickly, cost you money _not_ to buy them. The very definition of a false economy was "no the devs aren't getting new shiny sportsmode SSDs for their boxes."
We all use gcc and clang (for relevant languages), for example because everything else you might pay for sucks a lot harder. Whatever the ideology you claim or deny. Intel literally damaged their customers' products to try to make themselves look better than AMD with theirs! They're a non-starter after showing themselves willing to do that but if you must, yes, their compilers still suck exclusively on intel more than gcc. How is microsoft's compilers standards chasing going? In the 21st century yet or /still/ not? (Oh come one, we're mostly there but for ... yeah). Compiling template heavy C++? Does developer time matter?
(But just quietly, terminals & cmdline tools feel a lot more productive than IDEs to me and I use both in differing circumstances).
If developers would love to spend thousands of bucks here and there, paid developer tools industry would flourish. But I think that it's even degraded compared to the past. I remember paid Delphi IDE and I remember paid components and that was a thing back then. I can't remember any popular paid React component right now, despite the fact that much more people are using React compared to Delphi.
Spotify is slower than Winamp, Slack is slower than IRC, Webmail is slower than my e-mail client, VS Code is slower than Visual C++ 6.0.
And nothing else related to the functional differences between Spotify and WinAMP can explain why Spotify is slow. Done properly, it should feel snappier than WinAMP felt in 2000.
On that level, Winamp did the job just fine.
If we're trading music player performance for the massive increase in availability I'll make that trade.
But really, 99% of the value-add of Spotify over Winamp is stuff that happens server-side. Your Spotify client doesn’t have a database of all the songs, nor the ML models for computing recommendations. As far as playing streaming music goes, Winamp would happily play an m3u (mmmm soma.fm).
Don’t get me wrong, the discovery and search features in Spotify have brought me a ton of value. But wow is the client resource usage dramatically disproportionate for the added functionality
This is actually a point in favor of directly comparing WinAMP and Spotify - the important "value-add" bits of Spotify all happen server side, so they should have exactly zero impact on client performance.
And it was almost instant.
All of which are Electron apps, apart from "Webmail" which is, well, what one might call a "distributed Electron app" (a website.)
Maybe, just maybe, Electron is the problem.
One could write those programs in a truly inefficient manner in a not too performant language and it would run without problem. Today’s computers are really fast.
Whether or not the new features make up for the general perceptual slowness of modern computers is a somewhat open question.
Then you look at the Windows 10 list of crapware and it becomes a lot clearer.
It’s incredible how much grief Windows Vista got in comparison.
Is there an actual causal relationship? And which crapware specifically? Asking because I only have access to a bunch of Windows 10 Pro machines which already don't seem to have most of crapware on there (i.e. often I see threads here where people complain about all kinds of ads and other things I never even knew existed) and the rest disabled (as far as I can tell) and still it's less snapy than this one Windows Server instance on comparable machine. (and sadly no single modern OS feels as snappy as even Windows XP SP2 for a similar level of functionality on not-so-recent hardware)
Of course IRC is much faster, it's also few orders of magnitude simpler than Slack. For work related communications I prefer Slack to IRC, but for chatting with friends IRC does just fine. One simple protocol vs hundreds of APIs that provide extremely rich content. Once again, Slack takes few seconds to launch on a modern machine, it's not that bad considering it does so much.
Yeah shit is slower, shit is also way more connected and complex than in the 90s.
Unfortunately not all kinds of slowness only happen in hotspots. This is true for CPU cycles, but if an occasional task uses all memory, it's going to mess up everything downstream as well.
Like, unrolling a `for` loop that only has 5 iterations makes sense. But if you have 100 iterations, then the larger memory footprint of all the code might actually make it slower than just keeping the `for` loop.
In some cases, you can use Duff's Device. https://en.wikipedia.org/wiki/Duff%27s_device
I disagree. Programmers may care about performance, but project managers certainly don't and they always have the final say.
LOL no. Offer a dev choice between a framework that is trending on Twitter, vs one that is 100x faster but "legacy" (meaning more than a year old) and she'll choose the first one every time, and the project manager won't know the difference, or care if he did.
When I take an hour to profile and analyze some request and say “I think we can cut this thing’s response time by 80% by optimizing $X, which will result in an overall page load time reduction of 3s”, that’s way easier to sell than “I want to figure out why this page load is so slow” and then reporting back a week later with “shrug I optimized 5 things and it didn’t get faster. Dunno.”
The business model isn't right for performant experiences
Simply, performance is expensive and for most businesses, the additional revenue is not worth the additional expense.
Productivity, energy usage, e-waste in landfill as people are forced to upgrade in order to do simple tasks with the current software... The world is paying a high price for programmer laziness.
There's insane demand for games, and those programmers are able to optimise their code, insane demand for trading software and those programmers are able to optimise their code. But webdevs just don't care. I look at pages now that do nothing more in functionality than the same app would have done 20 years ago, yet they are slower now, and I have literally 1000x more CPU power. It's insane.
And fast is one aspect of performance. What's the memory footprint?
JS is plenty fast and can be used for "large, complicated programs"[1] just fine. The problem with most JS written today is in programmer practices—the way that the community associated with NodeJS pushes one other to write code (which is, ironically, not even a good fit for the JavaScript language). It turns out that how you write code actually matters (which is the central point of the article here to begin with...)
1. https://web.archive.org/web/20070127104743/http://www.spread...
Are you joking?
I have seen progressively worse performance from even the best browsers, and page load times that should be instant often literally take minutes or never load unless I completely kill the browser process and return. The slowdown is nearly inexorable, with occasional improvements in some versions before resuming the dismal trend.
Simple word processing and spreadsheets are so laggy on keystrokes as to be almost unusable, and by unusable, unresponsive on a level hundreds of times worse than DECADES ago, on computers orders of magnitude less powerful. Simple cursor movements are so laggy that I must set aside my train of thought to attend to the tool.
And this is on a very solid CAD-level computer, FIOS connection, etc.
It is disgusting and unforgivable. I quit software career 15 years ago for a new industry in no small part because I saw this trend of ever more complex "tools" & "frameworks", etc. creating a situation of building castles on shifting sands, with serious declines in the ability to reason about debugging, performance, or security. It is only worse now, probably exponentially. Worse yet, it seems to have yielded no perceptable "programmer productivity".
It is one thing to architect and program to take advantage of upcoming advances in hardware. It is quite another thing to ignore it and assume that the hardware builders will save you from the bloatware that you foist on the world without a serious thought.
Where is the evidence that you're seeing poor performance from the browser itself and that the source of the problem does not lie in the difference between what the server is sending down the tubes today compared to what it was sending 10 years ago?
But this is the root cause we're discussing - programmers selecting tools for their convenience (and worse yet, cool factor), instead of FIRST considering the responsiveness of the system as they design and code.
Optimization as an afterthought is about as good as security as an afterthought - anything from a complete waste of time to a disaster.
There are indeed pages that load like lightning, so it can be done (e.g., HN takes about 1.5sec to create a new window and load, so not exactly lightning, but usable), but many are horrible, and clearly due to bad programming.
For starters, when I see a page that loads code from 25 different sites that need NoScript privs to even display, that alone is pretty questionable - license and manage your own damn code (for the sake of minimizing dependency alone!). Twitter is particularly egregious in the last year or so, a new page taking 10sec-?? to load, and the LAST thing that loads is the list of posts -- the same load times it would feel so much more responsive if that was the first to load, and the other navigation, news, etc. panels loaded later while I was reading. That is a many bad programming choices.
It's not. Someone explained that browser makers spend billions on top talent to make browsers fast, and you posted a flippant comment — "are you joking?" — about your observations that performance of browsers is getting worse.
Now you're talking about stuff that web developers do on the pages that you visit.
Browsers are an example of software that is fast because companies have put effort into making them that way, instead of not caring. That's the claim made by the person you responded to. Dispute it, if you want, but don't make claims and then change the subject or shut down inquiry into the things that you're saying.
The problem is some websites
I believe the GP post was thinking an application which has some heavy computational kernel to it, where most of the processor time (and other resources) are spent - with a lot of other code for UI, configuration, some parsing etc.
And you seem to be talking about everyday desktop applications: Browser, mail client, word processor, instant messaging client, and maybe even your IDE as a programmer. Those kind of apps don't have that kernel of hard computational work - their work is much more diffuse.
But a whole OS, with a browser, network stack, dozens of applcations, etc., it's pretty common to find hundreds or thousands of things which are all more or less equally sucking performance. So on one hand, it's a much harder problem. On the other hand, front-end people have this attitude that performance doesn't matter because hardware is fast enough and it's better to write in the most abstract language possible because changing diapers makes your hands smell gross.
To-the-point software is actually really fast, like gcc/clang does tons of optimizations compared to what compilers did a few decades ago, and I doubt you could realistically optimize it by a significant percentage in the general case. Same for databases. Browsers themselves are also pushing the boundaries of hardware, they can display static HTML/CSS ridiculously fast even though it is not ideal for layouting. Pretty much only the top of the abstractions “suck” sometimes, but there are really fast JS apps as well.
I have news for you, all code is difficult to reason about at some level. Sure, certain abstracts can be easier to reason about than others if you take certain examples that I can generally argue are reductionist, but any reasonably well developed piece of code (read, actual useful code) will require a certain level of sophistication and complexity that will eventually make it difficult to reason about.
I know this is a reach and a bit of a red-herring, but developers really really need to remember the allergy to "code being difficult to reason about" is a sometimes a useful intuition or heuristic to use, the heuristic being "easier to reason about is better," but it really is a heuristic at the end of the day. Abuse of this heuristic is often a motive behind all sorts of detrimental behaviors in tech, reinventing the wheel and NIH (I didn't write, it, so it's difficult to reason about!), rewrites from scratch (our "technical debt" is now hard to reason about, let's rewrite from scratch!) and the problem OP refers to (a solution that maps to the machine better is hard to reason about, let's use this less efficient solution instead!). And a big problem here is that these issues are often not issues to hackers because it just means more programming for them which they love! It just hurts users and customers.
Yeah - it also doesn't crash my PC when an app segfaults, hell even video driver failing these days won't bring my system down. I don't need to restart my PC when I plug in a device. That's not really compatible with your childhood DOS games that enter kernel mode and talk to HW directly - you pay for abstractions - and it's a good tradeoff.
I can also have files with more than 5 character file names, and my FS can revert file changes and create snapshots.
These "good old days" people are just unrealistic, VIM is fast even today - but I prefer having a graphical representation of file tree with fancy icons and search. But that's just me and 99% of people out there.
Performance is a feature - and it's not a feature most people rank high untill it becomes an issue. As much as a minority of people here are crying about electron and HTML - a huge number of developers use VSCode daily and enjoy the approachability of a JS/DOM plugin ecosystem. VIM is free and available for decades now, and it has such amazing UX that the one of the most upvoted questions on SO was how to exit from it...
Slack is dog slow - and IRC has been freely available for decades, most people here use Slack daily. Who uses IRC anymore ?
So keep using your Emacs integrated window manager and stop bothering the rest of us :D
Still I don't know what bearing of all this has on the point I made above.
As you alluded to, a lot of programmers are happy to tweak and refine for ages so it's probably not that.
Is it just that programmers are lazy or need to care more or be better educated about optimization?
Honestly I really doubt it. I fully agree with the other takes that companies have incentives to ship working software quickly. Features sell subscriptions not performance. If it comes down to choosing between functionality and performance leaders will ship the slow feature every time.
I'm not even sure they're necessarily wrong most of the time. If you know where to look the internet is a graveyard of carefully engineered products that were beat to market and obliterated by three people with a shitty electron app.
There are cases at the tail where this will make or break a project, but those are pretty rare.
It’s worth challenging this assumption and asking whether it’s really true, or whether this belief is old and outdated like the old languages. I’m certain that providers of compute infrastructure like Amazon, Microsoft, Google and others completely disagree that performance doesn’t matter. Letting slow code run literally costs them money because it prevents multiple people from being able to use the same hardware resource. Even on your own machine, slow processes hog resources that other processes could use. The reason my laptop fans keep spinning up in Windows while my machine is basically idle is due to hundreds of processes written by people who erroneously believe the performance of 99% of the code doesn’t matter. We’re also at a point where we’ve started to measure the energy efficiency and environmental impact of wasting compute cycles. So, performance always matters.
> For the 1%, we can think about cache misses [...] but that has the enormous downside of making code hard to reason about.
You’re agreeing with the author here. Maybe you missed the explicit point in the in the article that is calling for a language that makes basic cache awareness a first class citizen, precisely so that it’s easier to reason about?
“This is exactly the type of problem that should be solved via the features of our programming languages, since writing these custom sorting functions is very tedious. What if a programming language would provide us with a structure that would act like an array of structs, but internally it would really behave like a struct of arrays? We could program in the typical object oriented way that is convenient for humans, while still enjoying in great performance due to playing nice with the hardware.”
> You are also optimizing for today’s hardware, which means that it might not run that great on new hardware in 20 years.
For the last 20 years, the need to be cache-aware has increased in importance. For the next 20 years, the gap between memory and compute will get larger still. There’s no scenario on the horizon where we don’t have to become more memory conscious as programmers or use more memory conscious languages and tools. There aren’t many performance tricks and concepts that worked 20 years ago and don’t work today. Using a structure of arrays (SoA) instead of an array of structures (AoS) has long been and will continue to be a good idea.
There are rare cases where there is no one point that is slow, that’s a bad situation to find oneself in — it usually means that the base architecture/abstraction sucks and needs a big refactor.
But even in so low-level code as the linux kernel, you see the use of linked lists all the time. Because it simply doesn’t matter if you are not in a hot loop and iterate slowly over 3 elements.
Use of linked lists in the linux kernel is also not very relevant, and somewhat of a red herring. Kernel-level (as well as games & embedded programming) linked lists are frequently allocated and used very differently from typical high level language linked lists. Scripting languages are allocating individual linked list nodes by default. Linux uses linked lists for the memory allocator itself, for example, and allocates nodes at a lower level in blocks. There are no particularly good alternatives, and no great ways for the allocator itself to be cache aware, though that is ongoing research. The main perf problem with using linked lists in high level languages is not the fact that they cause cache-incoherent access, it's worse, it's the fact that they allocate and deallocate memory constantly which is much much slower. This is one reason why linked lists don't get used as much as arrays in high level languages, when arrays are a reasonable alternative.
Anyway, it's somewhat of a straw man to bring up linked lists when the author's valid point was SoAs are better than AoSs from a cache perspective, and that our languages aren't yet doing much to help us code that way, but they could in the future if we decide to want them to.
I just meant to point out that SoA vs AoS or an even slower data structure is not necessarily relevant for all programs.
> The main perf problem with using linked lists in high level languages is not the fact that they cause cache-incoherent access, it's worse, it's the fact that they allocate and deallocate memory constantly which is much much slower.
This is not true though; at least the JVM most definitely doesn’t do that, and I’m sure other state-of-the-art GCs neither, like V8.
First of all, allocation is basically free, much faster than malloc and friends. It’s only a pointer bump. And the cost of deallocation is amortized/may not even happen. With plenty of memory available, GC doesn’t really happen, and for long-lived objects in case of a generational GC, they may not necessarily get checked at all after a time.
So the usual linked list gets appended a few times is close to optimal, iteration has the same cost as in lower level languages with pointer chasing (usually a bit more because objects usually have headers that take quite some space, less of them potentially fitting into cache). It even has some advantage in that a modern compacting GC may move the objects close to each other.
I wonder why array-backed linked lists aren't more of a thing. I've used those before in C in hobby projects. You allocate an extendable array of nodes and the nodes next and/or prev pointers become indices in the array (although pointers would still work). Added nodes go on the end, deleted nodes stay where they are and just aren't referenced. If there is high churn, do something to keep track of inactive nodes to fill next. You're still spamming around an area of memory when iterating the list, but it's at least confined to one guaranteed contiguous block and you're only allocating to extend the array and not doing a lot of allocating and freeing of memory. You still have the memory overhead of the next/previous indices, though, so fewer nodes fit in cache than otherwise would.
A closely related concept to what you're describing is internal storage linked lists [1], meaning the class/node contains the link(s) and is specialized for one particular kind of data, which often makes it more compact, and easier to allocate a block of nodes in a single call.
Anyway, I'd guess a big reason it's harder to find array backed lists when searching for linked list implementations is just because it's more advanced and may introduce memory management topics into articles and blog posts that the author doesn't want to cover. But I agree, once you use array-backed lists, the common examples of linked lists you can find online where new is used twice per node, once for a generic node, and once for the payload, seems ridiculously wasteful. This is the kind of thing I meant earlier - best practice would be to avoid a naive bad linked list even in code that isn't a hot spot. Time doesn't have to be spent optimizing it, and it doesn't need to appear as a hotspot, to know it should be avoided.
[1] https://en.wikipedia.org/wiki/Linked_list#Internal_and_exter...
Which implementation are we talking about here? Do you mean JVM internal linked lists, or user allocations of linked list nodes? I was talking about implementations such as (first Google result for "Java linked list implementation") https://www.geeksforgeeks.org/implementing-a-linked-list-in-...
> First of all, allocation is basically free
No, definitely not true. You're claiming that memory allocation in a tight loop is not a bottleneck. I completely disagree, having written several high performance web apps with thousands to millions of users, the single most important performance "best practices" strategy in any tight-loop Javascript is to avoid memory allocation in the hot spot. Also true in Python, true in C++, true in every language I've ever used, and has been true for decades and is still true despite significant performance improvements in V8 and other engines. This is the whole reason there are high performance libraries in JavaScript like https://glmatrix.net/ that specifically structure the API so that the user has complete control over allocations, precisely because batching and re-using allocations is faster than not.
Not too familiar with JS vm internals, but OpenJDK has quite a good escape analysis.
Yes, agreed. This is the point the author was making, and noting that we have room to actually improve the languages, and make some basic optimizations easier to think about and code explicitly.
> object reuse via pools may be even worse than creating non-escaping objects in a tight loop.
When does this happen in practice, and how likely is it? What does it imply about object lifetime if your objects don't escape, and why wouldn't you be re-using an object on the stack or local scope instead? How often would replacing a non-escaping object creating with a re-use that escapes be a realistic alternative? I'm sure a contrived case can be setup, but this seems like an unlikely explanation to me as general advice. It's bad practice to hope your objects don't escape, and difficult to ensure. In general, if you want higher performance, then avoid using new in hot spots. The difference is often an order of magnitude in an flat O(n) loop.
My point was more along the way that sometimes very non-intuitive (from a low-level perspective) solutions may optimize better, because that’s what the compiler writers optimized for.
And object pools at least in Java is not necessarily good. The first go at it should be the plainest implementation one can imagine (on a micro level. Of course architecturally it is important to think about optimization, like whether it will be a class or an array, etc.)
This brings two rhetorical questions, How much more effective compute would we have if all code running today were perfectly optimized, and how much less energy would be needed under the same circumstances?
The computer gains could only be used in networks where unused compute can be scheduled / sold to other tasks. The energy gains could only be reaped if processors had proper low power modes they could fall into or some other method of conservation.
You wouldn’t say the plan for a safer building is harder to read.
People excusing their poorly written code as “highly-optimized” is the oldest cop-out we’ve got as programmers.
Some of the future costs can be mitigated by using a few techniques from the start. One good idea is to keep an vanilla unoptimized version of the part that you are optimizing around as a reference implementation. This will help understanding what the optimized code does and can be used as a stopgap to support new hardware. It can also be a nice baseline to compare optimizations against.
The beauty of compilers is that they do a decent job of optimizing the 99% of code that would otherwise never be optimized, at 0 cost. This is a different sweet spot on the cost benefit curve.