News for hackers. Stuff that matters.
Re:Engineering for Slow Internet by bayindirh on Friday May 31, 2024 @04:49AM (#40532760)
However, getting 6.4KB of data (just tested on my blog) or 60KB of data (a git.sr.ht repository with a README.md and a PNG) is way better than getting 20MB of frameworks in the first place.
Re:Engineering for Slow Internet by mike_hearn on Friday May 31, 2024 @07:32AM (#40533721)
Yeah, but your blog is not a full featured chat system with integrated audio and video calling, strapped on top of a document format.

There are a few architectural/policy problems in web browsers that cause this kind of expansion:

1. Browsers can update large binaries asynchronously (=instant from the user's perspective) but this feature is only very recently available to web apps via obscure caching headers and most people don't know it exists yet/frameworks don't use it.

2. Large download sizes tend to come from frameworks that are featureful and thus widely used. Browsers could allow them to be cached but don't because they're over-aggressive at shutting down theoretical privacy problems, i.e. the browser is afraid that if one site learns you used another site that uses React, that's a privacy leak. A reasonable solution would be to let HTTP responses opt in to being put in the global cache rather than a partitioned cache, that way sites could share frameworks and they'd stay hot in the cache and not have to be downloaded. But browsers compete to satisfy a very noisy minority of people obsessed with "privacy" in the abstract, and don't want to do anything that could kick up a fuss. So every site gets a partitioned cache and things are slow.

3. Browsers often ignore trends in web development. React style vdom diffing could be offered by browsers themselves, where it'd be faster and shipped with browser updates, but it isn't so lots of websites ship it themselves over and over. I think the SCIter embedded browser actually does do this. CSS is a very inefficient way to represent styling logic which is why web devs write dialects like sass that are more compact, but browsers don't adopt it.

I think at some pretty foundational level the way this stuff works architecturally is wrong. The web needs a much more modular approach and most JS libraries should be handled more like libraries are in desktop apps. The browser is basically an OS already anyway.

Re:Engineering for Slow Internet by Sesse__ on Friday May 31, 2024 @11:12AM (#40536223)
> CSS is a very inefficient way to represent styling logic which is why web devs write dialects like sass that are more compact, but browsers don't adopt it.

I don't know exactly which features you are referring to, but you may have noticed that CSS has adopted native nesting, very similarly to Sass, but few sites actually use it. Functions and mixins are similar compactness/convenience topics being worked on by the CSSWG.

(Disclosure: I work on style in a browser team)

Re:Engineering for Slow Internet by mike_hearn on Friday May 31, 2024 @12:19PM (#40537007)
I hadn't noticed and I guess this is part of the problem. Sorry this post turned into a bit of a rant but I wrote it now.

When it was decided that HTML shouldn't be versioned anymore it became impossible for anyone who isn't a full time and very conscientious web dev to keep up. Versions are a signal, they say "pay attention please, here is a nice blog post telling you the most important things you need to know". If once a year there was a new version of HTML I could take the time to spend thirty minutes reading what's new and feel like I'm at least aware of what I should learn next. But I'm not a full time web dev, the web platform changes constantly, sometimes changes appear and then get rolled back, and everyone has long since plastered over the core with transpilers and other layers anyway. Additionally there doesn't seem to be any concept of deprecating stuff, so it all just piles up like a mound of high school homework that never shrinks.

It's one of the reasons I've come to really dislike CSS and HTML in general (no offense to your work, it's not the browser implementations that are painful). Every time I try to work out how to get a particular effect it turns out that there's now five different alternatives, and because HTML isn't versioned and web pages / search results aren't strongly dated, it can be tough to even figure out what the modern way to do it is at all. Dev tools just make you even more confused because you start typing what you think you remember and now discover there are a dozen properties with very similar names, none of which seem to have any effect. Mistakes don't yield errors, it just silently does either nothing or the wrong thing. Everything turns into trial-and-error, plus fixing mobile always seems to break desktop or vice-versa for reasons that are hard to understand.

Oh and then there's magic like Tailwind. Gah.

I've been writing HTML since before CSS existed, but feel like CSS has become basically non-discoverable by this point. It's understandable why neither Jetpack Compose nor SwiftUI decided to adopt it, even whilst being heavily inspired by React. The CSS dialect in JavaFX I find much easier to understand than web CSS, partly because it's smaller and partly because it doesn't try to handle layout. The way it interacts with components is also more logical.

Re:Engineering for Slow Internet by Sesse__ on Friday May 31, 2024 @01:58PM (#40538326)
You may be interested in the Baseline initiative, then. (https://web.dev/baseline/2024)
Re:Engineering for Slow Internet by mike_hearn on Friday May 31, 2024 @02:13PM (#40538523)
That does look useful, thanks!
Re:Engineering for Slow Internet by arvinsim on Saturday June 01, 2024 @01:45AM (#40543203)
> Oh and then there's magic like Tailwind. Gah.

I'm not sure why Tailwind is magic. It's just a bunch of predefined classes at its core.

Re:Engineering for Slow Internet by semi on Friday May 31, 2024 @09:16PM (#40541995)
The privacy issues aren't just hypothetical, but that aside, that caching model unfortunately doesn't mesh well with modern webdev. It requires all dependencies to be shipped in full, no tree shaking to only include the needed functions. And separately as individual files.. and for people to actually stick to the same versions of dependencies
Re:Engineering for Slow Internet by mike_hearn on Saturday June 01, 2024 @04:49AM (#40544015)
Can you show some real sites that were mounting such attacks using libraries?
Re:Engineering for Slow Internet by RGamma on Friday May 31, 2024 @06:14AM (#40533231)
Also wonder how many savings are still possible with a more efficient HTML/CSS/JS binary representation. Text is low tech and all but it still hurts to waste so many octets for such a relatively low amount of possible symbols.

Applies to all formal languages actually. 2^(8x20x10^6) ~= 2x10^48164799 is such a ridiculously large space...

Re:Engineering for Slow Internet by chris_pie on Friday May 31, 2024 @06:48AM (#40533418)
Re:Engineering for Slow Internet by jiggawatts on Friday May 31, 2024 @07:02AM (#40533494)
The generalisation of this concept is what I like call the "kilobyte" rule.

A typical web page of text on a screen is about a kilobyte. Sure, you can pack more in with fine print, and obviously additional data is required to represent the styling, but the actual text is about 1 kb.

If you've sent 20 MB, then that is 20,000x more data than what was displayed on the screen.

Worse still, an uncompressed 4K still image is only 23.7 megabytes. At some point you might be better off doing "server side rendering" with a GPU instead of sending more JavaScript!

Re:Engineering for Slow Internet by Lex-2008 on Friday May 31, 2024 @09:23AM (#40534816)
> "server side rendering" with a GPU instead of sending more JavaScript

Some 7~10 years ago I remember I saw somewhere (maybe here on HN) a website which did exactly this: you gave it an URL - it downloaded a webpage with all its resources, rendered and screenshot'ed it (probably in headless Chrome or something), and compared size of png screenshot versus size of webpage with all its resources.

For many popular websites, png screenshot of a page indeed was several times less than webpage itself!

Re:Engineering for Slow Internet by jcgrillo on Friday May 31, 2024 @09:56AM (#40535232)
If your server renders the image as text we'll be right back down towards a kilobyte again. See https://www.brow.sh/
Re:Engineering for Slow Internet by skydhash on Friday May 31, 2024 @09:59AM (#40535271)
I read epubs, and they’re mostly html and css files zipped. The whole book usually comes under a MB if there’s not a lot of big pictures. Then you come across a website and for just an article you have to download tens of MBs. Disable JavaScript and the website is broken.
Re:Engineering for Slow Internet by RGamma on Friday May 31, 2024 @03:39PM (#40539377)
Soo.. there should be a standardized web API for page content. And suddenly... gopher (with embedded media/widgets).
Re:Engineering for Slow Internet by pgraf on Friday May 31, 2024 @06:42AM (#40533383)
Shouldn’t HTTP compression reap most of the benefits of this approach for bigger pages?
Re:Engineering for Slow Internet by LtWorf on Friday May 31, 2024 @10:36AM (#40535759)
Surely you're aware of gzip encoding on the wire for http right?
Re:Engineering for Slow Internet by RGamma on Friday May 31, 2024 @11:45AM (#40536612)
Sure, would be interesting to know how it would fare against purpose-made compression under real world conditions still...
Re:Engineering for Slow Internet by LtWorf on Friday May 31, 2024 @09:24PM (#40542058)
gzip is fast. And it was made for real world conditions.
Re:Engineering for Slow Internet by floam on Saturday June 01, 2024 @01:28PM (#40547544)
Brotli is a better example, as it was literally purpose made for HTML/CSS/JS. It is now supported basically everywhere for HTTP compression, and uses a huge custom dictionary (about 120KB) that was trained on simulated web traffic.

You can even swap in your own shared dictionary with a HTTP header, but trying to make your own dictionary specific to your content is a fool’s errand, you’ll never amortize the cost in total bits.

What you CAN do with shared dictionaries though, is delta updates.

https://developer.chrome.com/blog/shared-dictionary-compress... https://news.ycombinator.com/item?id=39615198

Re:Engineering for Slow Internet by LtWorf on Sunday June 02, 2024 @05:52PM (#40557441)
Is it supported by curl or any browser or no?
Re:Engineering for Slow Internet by floam on Monday June 03, 2024 @12:13AM (#40559261)
Every browser, curl yes.
Re:Engineering for Slow Internet by andrepd on Friday May 31, 2024 @05:09AM (#40532867)
Yes. It's inexcusable that text and images and video pulls in megabytes of dependencies from dozens of domains. It's wasteful on every front: network, battery, and it's also SLOW.
Re:Engineering for Slow Internet by LtWorf on Friday May 31, 2024 @10:42AM (#40535846)
The crap is that even themes for static site generators like mkdocs link resources from cloudflare rather than including them in the theme.

For typedload I've had to use wget+sed to get rid of that crap after recompiling the website.

https://codeberg.org/ltworf/typedload/src/branch/master/Make...

Re:Engineering for Slow Internet by mwcampbell on Friday May 31, 2024 @05:06AM (#40532845)
False dichotomy, with what is likely extreme hyperbole on the JS side. Are there actual sites that ship 20 MB, or even 5 MB or more, of frameworks? One can fit a lot of useful functionality in 100 KB or less of JS< especially minified and gzipped.
Re:Engineering for Slow Internet by tecleandor on Friday May 31, 2024 @05:40AM (#40533052)
Well, I'm working right now so let me check our daily "productivity" sites (with an adblocker installed):

  - Google Mail: Inbox is ~18MB (~6MB Compressed). Of that, 2.5MB is CSS (!) and the rest is mostly JS
  - Google Calendar: 30% lower, but more or less the same proportions
  - Confluence: Home is ~32MB (~5MB Comp.). There's easily 20MB of Javascript and at least 5MB of JSON. 
  - Jira: Home is ~35MB (~7MB compressed). I see more than 25MB of Javascript
  - Google Cloud Console: 30MB (~7MB Comp.). I see at least 16MB of JS
  - AWS Console: 18MB (~4MB Comp.). I think it's at least 12MB of JS
  - New Relic: 14MB (~3MB Comp.). 11MB of JS.
    This is funny because even being way more data heavy than the rest, its weight is way lower.
  - Wiz: 23MB (~6MB Comp.) 10MB of JS and 10MB of CSS
  - Slack: 60MB (~13MB Compressed). Of that, 48MB of JS. No joke.
Re:Engineering for Slow Internet by tonyhart7 on Friday May 31, 2024 @06:32AM (#40533323)
too much crap holy and this is worse case scenario with adblock
Re:Engineering for Slow Internet by jerf on Friday May 31, 2024 @09:29AM (#40534880)
I sometimes wish I could spare the time just to tear into something like that Slack number and figure out what it is all doing in there.

Javascript should even generally be fairly efficient in terms of bytes/capability. Run a basic minimizer on it and compress it and you should be looking at something approaching optimal for what is being done. For instance, a variable reference can amortize down to less than one byte, unlike compiled code where it ends up 8 bytes (64 bits) at the drop of a hat. Imagine how much assembler "a.b=c.d(e)" can compile into to, in what is likely represented in less compressed space than a single 64-bit integer in a compiled language.

Yet it still seems like we need 3 megabytes of minified, compressed Javascript on the modern web just to clear our throats. It's kind of bizarre, really.

Re:Engineering for Slow Internet by LtWorf on Friday May 31, 2024 @10:48AM (#40535924)
js developers had this idea of "1 function = 1 library" for a really long time, and "NEVER REIMPLEMENT ANYTHING". So they will go and import a library instead of writing a 5 line function, because that's somehow more maintainable in their mind.

Then of course every library is allowed to pin its own dependencies. So you can have 15 different versions of the same thing, so they can change API at will.

I poked around some electron applications.

I've found .h files from openssl, executables for other operating systems, megabytes of large image files that were for some example webpage, in the documentation of one project. They literally have no idea what's in there at all.

Re:Engineering for Slow Internet by mike_hearn on Friday May 31, 2024 @01:23PM (#40537861)
That's a good question. I just launched Slack and took a look. Basically: it's doing everything. There's no specialization whatsoever. It's like a desktop app you redownload on every "boot".

You talk about minification. The JS isn't minified much. Variable names are single letter, but property names and more aren't renamed, formatting isn't removed. I guess the minifier can't touch property names because it doesn't know what might get turned into JSON or not.

There's plenty of logging and span tracing strings as well. Lots of code like this:

            _n.meta = {
                name: "createThunk",
                key: "createThunkaddEphemeralMessageSideEffectHandler",
                description: "addEphemeralMessageSideEffect side effect handler"
            };
The JS is completely generic. In many places there are if statements that branch on all languages Slack was translated into. I see checks in there for whether localStorage exists, even though the browser told the server what version it is when the page was loaded. There are many checks and branches for experiments, whether the company is in trial mode, whether the code is executing in Electron, whether this is GovSlack. These combinations could have been compiled server side to a more minimal set of modules but perhaps it's too hard to do that with their JS setup.

Everything appears compiled using a coroutines framework, which adds some bloat. Not sure why they aren't using native async/await but maybe it's related to not being specialized based on execution environment.

Shooting from the hip, the learnings I'd take from this are:

1. There's a ton of low hanging fruit. A language toolchain that was more static and had more insight into what was being done where could minify much more aggressively.

2. Frameworks that could compile and optimize with way more server-side constants would strip away a lot of stuff.

3. Encoding logs/span labels as message numbers+interpolated strings would help a lot. Of course the code has to be debuggable but hopefully, not on every single user's computer.

4. Demand loading of features could surely be more aggressive.

But Slack is very popular and successful without all that, so they're probably right not to over-focus on this stuff. Especially for corporate users on corporate networks does anyone really care? Their competition is Teams after all.

Re:Engineering for Slow Internet by taeric on Friday May 31, 2024 @10:24AM (#40535600)
This is mind blowing to me. I expect that the majority of any application will be the assets and content. And megabytes of CSS is something I can't imagine. Not the least for what it implies about the DOM structure of the site. Just, what!? Wow.
Re:Engineering for Slow Internet by bayindirh on Friday May 31, 2024 @05:38AM (#40533045)
I just tried some websites:

    - https://web.whatsapp.com 11.12MB compressed / 26.17MB real.
    - https://www.arstechnica.com 8.82MB compressed / 16.92MB real.
    - httsp://www.reddit.com 2.33MB compressed / 5.22 MB real.
    - https://www.trello.com (logged in) 2.50MB compressed / 10.40MB real.
    - https://www.notion.so (logged out) 5.20MB compressed / 11.65MB real.
    - https://www.notion.so (logged in) 19.21MB compressed / 34.97MB real.
Re:Engineering for Slow Internet by SpaceNugget on Friday May 31, 2024 @05:33AM (#40533014)
Well, in TFA, if you re-read the section labeled "Detailed, Real-world Example" yes, that is exactly what the person was encountering. So no hyperbole at all actually.
Re:Engineering for Slow Internet by b3orn on Friday May 31, 2024 @06:10AM (#40533203)
I'm getting almost 2MB (5MB uncompressed) just for a google search.
Re:Engineering for Slow Internet by ezequiel-garzon on Friday May 31, 2024 @08:44AM (#40534419)
I agree with adding very little JavaScript, say 1kB https://instant.page/ to make it snappier.