News for hackers. Stuff that matters.
Re:Reducing Slack’s memory footprint by bborud on Saturday March 04, 2017 @05:42AM (#13790298)
Slack has to be one of the worst pieces of software I have seen based on what it does and how it behaves. It gobbles CPU and memory to do something that should require neither. It is offensively wasteful.
Re:Reducing Slack’s memory footprint by frik on Saturday March 04, 2017 @11:54AM (#13791672)
it reminds me how lean common IRC clients are (written in C like mIRC). 15MB RAM and less than 1% CPU is enough for 2 networks and several channels (even with hundreds persons in a channel). Incidently it was also mIRC that showed that Intel removed some CPU features from the first Intel Atom CPU gen. Processing lot's of text isn't within the capability of Atom CPU, something even a Pentium 3 had no problem.
Re:Reducing Slack’s memory footprint by bborud on Sunday March 05, 2017 @11:12AM (#13796579)
I used ircII under various UNIXen and the resource footprint was even smaller.

That being said it had a negligible UI footprint since it was a curses-based text UI, doesn't embed media content or render text in a GUI, and it doesn't really do much.

Slack does a lot more, but it shouldn't need to burn CPU while doing nothing.

But it is an illustration of how little resources you actually need for IO, simple state management and dealing with the protocol layers. You'd accept bloat in all of these areas to get more functionality, but not by orders of magnitude.

Out of curiosity I downloaded ircII-latest, ran `configure` and `make`, and then started it up (about 30 seconds from clicking the link to starting the compiled binary btw). Initial memory footprint on startup was 465kB. Now it is up to a whopping 604kB. As it is mostly idling (very little chat traffic on the channel) the CPU use is at 0.0%.

Re:Reducing Slack’s memory footprint by ris on Saturday March 04, 2017 @12:16PM (#13791810)
Indeed - my frequent complaint is how opening slack and trello will use up half the memory on my small laptop.
Re:Reducing Slack’s memory footprint by mordant on Saturday March 04, 2017 @05:48AM (#13790309)
And yet Slack runs just fine on my 2016 12" MacBook, iPhone 7 Plus, and iPad Mini 4.
Re:Reducing Slack’s memory footprint by coldtea on Saturday March 04, 2017 @05:58AM (#13790338)
"...with a single team of 5 and 2 channels".

Seriously, it's not about "running just fine", it's about how many resources it uses. Even the startup time is awful for what it does.

And the Slack team itself recognizes that there are issues -- as seen in TFA:

"However, these capabilities come at a cost: the desktop client can use a lot of memory. This memory footprint increases as the user signs into more teams, as each team runs in its own webview. More memory usage means worse performance, degrading our customer’s experience of Slack and their other applications."

and:

"At the time of writing, about 36% of active Slack users are signed into more than one team in the desktop app. 17% are signed into 3 or more teams, and 5% are signed into 5 or more teams. Each of these teams is running the full JavaScript web application and maintaining and updating the DOM tree as the state of the team changes. This application can consume between ~130MB (p10) and ~960MB (p99) of memory depending on team activity and UI state."

Re:Reducing Slack’s memory footprint by bborud on Saturday March 04, 2017 @06:14AM (#13790378)
I suppose you may not know any better. And I am not saying that to be mean, I'm merely pointing out that your frame of reference may not be the best to judge how well the desktop Slack client was done with regard to resource use. You may not understand just how few resources the desktop client could reasonably be expected to consume.

And you would not be alone. Over the past 7-8 years I'd had to teach people how to benchmark things. I see people measure things and then use their first result as the performance reference. Sure, they can see relative changes, but if you are orders of magnitude off a reasonable result, a relative reduction of a few tens of percent is still orders of magnitude off.

As a developer you should have an idea what a _reasonable_ number ought to be. For CPU, for memory, for latency, for anything you care to measure.

"It runs just fine" isn't data. It isn't a useful metric for performance.

Re:Reducing Slack’s memory footprint by viraptor on Saturday March 04, 2017 @03:21PM (#13792754)
I see a lot of it lately. For example I looked for some kind of status bar GitHub notification tool. The most popular open source one includes its own electron environment... to display one icon and a list. That's a >100MB app for a completely trivial task.
Re:Reducing Slack’s memory footprint by joekim on Saturday March 04, 2017 @06:59AM (#13790516)
> "It runs just fine" isn't data. It isn't a useful metric for performance.

I don't see any claims from "mordant" that their anecdote is a useful performance metric. If anything it'd be a data point that could be used as a user experience metric. ie. X% of users in the target demographic agree with this statement, "works fine for me."

The point was a counter to your claim of Slack being, "offensively wasteful". Further, isn't "offensively wasteful" equally useful as a performance metric? If you're experienced in performance perhaps you could share some of your "reason numbers" and contrast that to measurements your made against the Slack client.

Re:Reducing Slack’s memory footprint by bborud on Sunday March 05, 2017 @10:00AM (#13796198)
By "offensively wasteful" I mean off by orders of magnitude. At which point the problem isn't one of mere programming but of actually having some idea of what you are doing.

Recovery from this state of affairs doesn't happen by telling people how to fix concrete, individual things, but teaching them to think about what they are doing and to analyze things. In order to know when you are off by orders of magnitude you need to have some idea of what is reasonable before you measure. If the reality you achieve is way off what is reasonable, you either need to change your perception of what "reasonable" is or you have to face the possibility that you may not have done a good job.

A chat client that deals with perhaps 3-4 teams and perhaps 200-300 people should not require 1Gb of memory and it should consume a negligible amount of CPU. Any way you slice it that's a LOT of resources per user, per team or per communication volume. So Slack miserably fails even cursory glance at workload/resource usage.

You can start off with the lower layers: speaking a chat protocol. How much resources should that eat up? For a chat application: too little to measure on a modern machine. (I spent most of the early 2000s working on web-scale web crawlers on much, much weaker server machines than even cheap desktop machine is today). The IO should have trivial cost, the protocol parsing should likewise have trivial cost or you have messed up badly.

Then move on to the internal state. Draw it on paper. What are the main structures you will be needing. How much data do you need to keep in memory, how much space does it need to take, how do access it and manipulate state? What is a reasonable CPU expenditure?

Next is the UI. How do you make it responsive. How do you make it not gobble up tons of memory. Are there tradeoffs you can make? How do other applications do things? Where are the resources wasted?

One doesn't produce a resource hog like Slack by doing one or two things in a wasteful and sloppy manner. One does it by having the wrong attitude and systematically doing a bad job.

Re:Reducing Slack’s memory footprint by joekim on Sunday March 05, 2017 @06:30PM (#13799543)
Thanks for your response. I generally agree with the technical approach, but not with your assessment of Slack's products.

> By "offensively wasteful" I mean off by orders of magnitude. At which point the problem isn't one of mere programming but of actually having some idea of what you are doing.

That would imply that Slack, doesn't know what they're doing. Slack is an extremely successful company. I myself use it for several teams simultaneously. From a user and business perspective it generates a lot of value and that ultimately trumps criticisms of their implementation.

> One doesn't produce a resource hog like Slack by doing one or two things in a wasteful and sloppy manner. One does it by having the wrong attitude and systematically doing a bad job.

Perhaps what you're taking into consideration isn't broad enough. Going back to the original comment, if they did such a bad job, then why am I able to get a lot of value from it? Their implementation seems "good enough".

What's the negative consequences of their "orders of magnitude" performance problems? Does it destroy it's business value? What would you do if you were the CEO? Cease all active development and re-build everything in scratch in c++?

Re:Reducing Slack’s memory footprint by bborud on Monday March 06, 2017 @12:19PM (#13804181)
I'm not denying their success. But I'm also not attributing it to the quality of their product.
Re:Reducing Slack’s memory footprint by ris on Saturday March 04, 2017 @12:20PM (#13791829)
And yet I can do internet chat just fine on my Names three outrageously expensive and highly powered devices...
Re:Reducing Slack’s memory footprint by pdog on Friday March 03, 2017 @09:18PM (#13788752)
Cool technique, but you hint at the best solution: "running all teams in a shared context to eliminate the overhead of one webview per team."
Re:Reducing Slack’s memory footprint by brianwawok on Friday March 03, 2017 @10:55PM (#13789136)
Wouldn't the best solution be write it in C and have the entire app take 20MB of ram?
Re:Reducing Slack’s memory footprint by CapacitorSet on Saturday March 04, 2017 @04:49AM (#13790170)
I think Slack is focusing on code reuse, eg. using the same HTML/CSS/JS codebase across all platforms - hence it would be preferrable to avoid having to compile for every platform.
Re:Reducing Slack’s memory footprint by kalleboo on Saturday March 04, 2017 @05:25AM (#13790255)
I don't think compliation is the issue, it's supporting the diverse platform APIs and UI toolkits.
Re:Reducing Slack’s memory footprint by coldtea on Saturday March 04, 2017 @06:02AM (#13790349)
They use like 5 widgets (styled text, textfield, buttons, radios, etc). They could create their own UI toolkit with 1/10 the resources they have.
Re:Reducing Slack’s memory footprint by joekim on Saturday March 04, 2017 @07:12AM (#13790563)
If this is true, then why are even bigger players with more resources like Facebook and Microsoft building on top of Electron instead of building natively?

I surmise that you are not alone in believing that Slack would be better built as a native app. But if it's such a bad direction why are so many going down this road and what are they missing?

Re:Reducing Slack’s memory footprint by coldtea on Saturday March 04, 2017 @09:51AM (#13791105)
>If this is true, then why are even bigger players with more resources like Facebook and Microsoft building on top of Electron instead of building natively?

Because software is a pop culture, and the latest fad always prevails at most shops. Also companies care more about cutting corners and shipping sooner than about the long haul.

That said, it's not like Facebook particularly enjoyed web based apps: "Mark Zuckerberg: Our Biggest Mistake Was Betting Too Much On HTML5"

https://techcrunch.com/2012/09/11/mark-zuckerberg-our-bigges...

Re:Reducing Slack’s memory footprint by joekim on Saturday March 04, 2017 @03:51PM (#13792870)
The mistake there was one of timing and approach and they learned from it. The promise of cross-platform, web-based technology is still there, that's why they created React Native.
Re:Reducing Slack’s memory footprint by coldtea on Saturday March 04, 2017 @05:33PM (#13793269)
React Native is not web-based. It just uses Javascript.
Re:Reducing Slack’s memory footprint by joekim on Sunday March 05, 2017 @06:14PM (#13799461)
The technology originated from web. It uses javascript, React, babel, flexbox, etc.
Re:Reducing Slack’s memory footprint by pjmlp on Saturday March 04, 2017 @09:38AM (#13791048)
> ....Facebook and Microsoft building on top of Electron instead of building natively?

JavaScript devs are usually cheaper to hire than C++, Java, Objective-C, Swift, .NET ones.

Re:Reducing Slack’s memory footprint by joekim on Saturday March 04, 2017 @03:49PM (#13792857)
I think VSCode by Microsoft is a good example of Electron done well.

In their case, I don't think it's because their web technology devs are cheaper. I wouldn't be surprised if that team knew multiple platforms and languages. I think it's all about finding the balance between dev costs, making a great product, and getting it out fast on multiple platforms.

Re:Reducing Slack’s memory footprint by pjmlp on Saturday March 04, 2017 @04:49PM (#13793098)
On their case, they want to get a few notches up in the SV popchart culture, top of the pops.
Re:Reducing Slack’s memory footprint by joekim on Sunday March 05, 2017 @06:15PM (#13799469)
Being cool in SV is only so valuable.
Re:Reducing Slack’s memory footprint by pjmlp on Saturday March 04, 2017 @06:36AM (#13790438)
Apparently for the web dev generation, writing modular native code across OSes is a lost art, attained by us old druids not worth listening to.
Re:Reducing Slack’s memory footprint by coldtea on Saturday March 04, 2017 @06:46AM (#13790478)
Yeah, and it's especially puzzling since they've given up one of the major hassles of going natively cross-platform anyway: having to support a native look and feel.

Which makes creating a similar (uniform looking across platforms) native UI lib even easier than catering to the intricacies of each platform.

Re:Reducing Slack’s memory footprint by pjmlp on Saturday March 04, 2017 @03:14AM (#13789922)
Yes, it is incredible the amount of gimmicks web devs do instead of using the right tools for native development.
Re:Reducing Slack’s memory footprint by joekim on Saturday March 04, 2017 @07:23AM (#13790587)
Could you elaborate on what you think the elaborate on what the "right" tools would be?
Re:Reducing Slack’s memory footprint by pjmlp on Saturday March 04, 2017 @08:26AM (#13790795)
Write in native languages, common to all platforms, using the OS specific APIs in each platform with the help on an abstraction layer.

Lets pick C++ as an example of native language.

Design the application architecture with the abstract ideas of what should be available across all platforms, but keeping in mind the native idioms.

For each of such concepts, provide an interface (pure abstract classes in C++ speak) with an implementation for each OS.

The actual cpp files are selected during the build process according to the OS being compiled, without any kind of #ifdefs.

Then showing a notification to the user using this architecture is as simple as.

    auto notification = NotificationService.instance();
    notification->show("New email available!");
Or opening a file, using the platform specific file dialogs:

    auto filePicker = Pickers.file_instance()
    auto filename = filePicker->open_file();
This are just two basic examples, the idea is not to replicate Qt, wxWidgets, JUCE,... rather pick just the concepts required for the application, not to build a full blown framework.

This is the approach taken by quite a few companies.

Of course, there is also the shortcut of using something like Qt, but then you need to live with the problem that the widgets aren't native, rather emulated.

On the other hand, still outperforms any browser based application.

The craziness with Electron based apps isn't nothing new to us oldies.

Since the early days of the web, OSes have supported web widgets like MSHTML on Windows, Windows packaged Web Applications introduced on Windows Active Desktop, Mozilla's XUL and many others.

They were never a match to native UI/UX.

Re:Reducing Slack’s memory footprint by mdekkers on Saturday March 04, 2017 @04:38AM (#13790137)
why not use the right tool for the job instead of doing Olympic-level gymnastics to use a hammer as a screwdriver?
Re:Reducing Slack’s memory footprint by jjnoakes on Saturday March 04, 2017 @09:32AM (#13791020)
Because it is faster and cheaper to get a junior JavaScript developer to bang out a cross platform GUI in electron than it is to get one or more developers together to write a proper cross-platform native application.

They cared more about cost and time to market than performance and technical prowess, and in a business context, that is often a correct trade-off.

I don't like it because I prefer small fast clean applications, but I know I'm in the minority and I accept that that's how the business world works.

Re:Reducing Slack’s memory footprint by mdekkers on Saturday March 04, 2017 @11:15AM (#13791474)
They cared more about cost and time to market than performance and technical prowess, and in a business context, that is often a correct trade-off.

Cool when you are starting up and all, but surely they must have a bit more money now, and would be able to do it right?

Re:Reducing Slack’s memory footprint by nemothekid on Saturday March 04, 2017 @12:12PM (#13791785)
When Facebook was hitting performance issues with PHP, they decided to rewrite the entire runtime instead of rewriting in a different language.

If Facebook with its billions decided it was cheaper to rewrite PHP, the cost/benefit analysis must favor optimizing current code than doing a rewrite

Re:Reducing Slack’s memory footprint by mdekkers on Saturday March 04, 2017 @12:43PM (#13791941)
Facebook is Facebook. Slack isn't Facebook, and neither is every other business besides Facebook Facebook decided to rewrite PHP because the CBA worked out for Facebook - that doesn't mean the argument holds true by default for any other situation.
Re:Reducing Slack’s memory footprint by jjnoakes on Saturday March 04, 2017 @11:36AM (#13791571)
But why spend the money to do it from scratch when you can spend less focusing on the pain points?

If your users pay you either way, the cheaper option (from a business point of view) is better.

Re:Reducing Slack’s memory footprint by coldtea on Saturday March 04, 2017 @05:56AM (#13790330)
Because Electron (or whatever they use) is web-scale.
Re:Reducing Slack’s memory footprint by mdekkers on Saturday March 04, 2017 @11:14AM (#13791472)
Because Electron (or whatever they use) is web-scale.

Web-scale? What's web scale? why does my desktop client need to be "web-scale"?

Re:Reducing Slack’s memory footprint by coldtea on Saturday March 04, 2017 @11:57AM (#13791700)
Everything needs to be web-scale:

https://www.youtube.com/watch?v=b2F-DItXtZs

Re:Reducing Slack’s memory footprint by paulddraper on Saturday March 04, 2017 @06:08PM (#13793408)
Re:Reducing Slack’s memory footprint by mdekkers on Saturday March 04, 2017 @12:50PM (#13791977)
That's hilarious
Re:Reducing Slack’s memory footprint by thunder-ltu on Saturday March 04, 2017 @07:18AM (#13790576)
This.