News for hackers. Stuff that matters.
Re:How fast are Linux pipes anyway? by jagrsw on Thursday June 02, 2022 @05:38AM (#31593035)
Something maybe a bit related.

I just had 25Gb/s internet installed (https://www.init7.net/en/internet/fiber7/), and at those speeds Chrome and Firefox (which is Chrome-based) pretty much die when using speedtest.net at around 10-12Gbps.

The symptoms are that the whole tab freezes, and the shown speed drops from those 10-12Gbps to <1Gbps and the page starts updating itself only every second or so.

IIRC Chrome-based browsers use some form of IPC with a separate networking process, which actually handles networking, I wonder if this might be the case that the local speed limit for socketpair/pipe under Linux was reached and that's why I'm seeing this.

Re:How fast are Linux pipes anyway? by reitanqild on Thursday June 02, 2022 @05:56AM (#31593116)
> and at those speeds Chrome and Firefox (which is Chrome-based)

AFAIK, Firefox is not Chrome-based anywhere.

On iOS it uses whatever iOS provides for webview - as does Chrome on iOS.

Firefox and Safari is now the only supported mainstream browsers that has their own rendering engines. Firefox is the only that has their own rendering engine and is cross platform. It is also open source.

Re:How fast are Linux pipes anyway? by SahAssar on Thursday June 02, 2022 @06:44AM (#31593357)
> Firefox is the only that has their own rendering engine and is cross platform.

Interestingly safaris rendering engine is open source and cross platform, but the browser is not. Lots of linux-focused browsers (konquerer, gnome web, surf) and most embedded browsers (nintendo ds & switch, playstation) use webkit. Also some user interfaces (like WebOS, which is running all of LG's TVs and smart refrigerators) use webkit as their renderer.

Re:How fast are Linux pipes anyway? by qwerty456127 on Thursday June 02, 2022 @06:49AM (#31593397)
WebKit itself is a fork of the Konqueror's original KHTML engine by the way.
Re:How fast are Linux pipes anyway? by tmccrary55 on Thursday June 02, 2022 @08:30AM (#31593979)
Browser Genealogy
Re:How fast are Linux pipes anyway? by cturtle on Thursday June 02, 2022 @08:49AM (#31594143)
Now I want to see the family tree!
Re:How fast are Linux pipes anyway? by capableweb on Thursday June 02, 2022 @08:55AM (#31594222)
Re:How fast are Linux pipes anyway? by tinus_hn on Thursday June 02, 2022 @07:49AM (#31593720)
IOS uses WebKit which is also what Chrome is based on.
Re:How fast are Linux pipes anyway? by Comevius on Thursday June 02, 2022 @08:11AM (#31593865)
Chrome uses Blink, which was forked from WebKit's WebCore in 2013. They replaced JavaScriptCore with V8.
Re:How fast are Linux pipes anyway? by yosamino on Thursday June 02, 2022 @06:29AM (#31593278)
> AFAIK, Firefox is not Chrome-based anywhere.

Not technically "Chrome-based", but Firefox draws graphics using Chrome's Skia graphics engine.

Firefox is not completely independent from Chrome.

Re:How fast are Linux pipes anyway? by SahAssar on Thursday June 02, 2022 @06:47AM (#31593381)
Skia started in 2004 independently of google and was then acquired by google. Calling it "Chrome's Skia graphics engine" makes it sound like it was built for chrome.
Re:How fast are Linux pipes anyway? by bawolff on Thursday June 02, 2022 @06:44AM (#31593358)
I feel like counting every library is silly.

In any case, i thought chrome used libnss which is a mozilla library, so you could say the reverse as well.

Re:How fast are Linux pipes anyway? by bayindirh on Thursday June 02, 2022 @06:13AM (#31593212)
Chrome fires many processes and creates an IPC based comm-network between them to isolate stuff. It's somewhat abusing your OS to get what its want in terms of isolation and whatnot.

(Which is similar to how K8S abuses ip-tables and makes it useless for other ends, and makes you install a dedicated firewall in front of your ingress path, but let's not digress).

On the other hand, Firefox is neither chromium based, nor is a cousin of it. It's a completely different codebase, inherited from Netscape days and evolved up to this point.

As another test point, Firefox doesn't even blink at a symmetric gigabit connection going at full speed (my network is capped by my NIC, the pipe is way fatter).

Re:How fast are Linux pipes anyway? by pjmlp on Thursday June 02, 2022 @07:16AM (#31593551)
It is using what OS processes where created in first place.

Unfortunately the security industry has proven the why threads are a bad ideas for applications when security is a top concern.

Same applies to dynamically loaded code as plugins, where the host application takes the blame for all instabilty and exploits they introduce.

Re:How fast are Linux pipes anyway? by bayindirh on Thursday June 02, 2022 @07:24AM (#31593594)
Yes, Firefox is also doing the same, however due to the nature of Firefox's processes, the OS doesn't lose much responsiveness or doesn't feel bogged down when I have 50+ tabs open due to some research.

If you need security, you need isolation. If you want hardware-level isolation, you need processes. That's normal.

My disagreement with Google's applications are how they're behaving like they're the only running processes on the system itself. I'm pretty aware that some of the most performant or secure things doesn't have the prettiest implementation on paper.

Re:How fast are Linux pipes anyway? by ReactiveJelly on Thursday June 02, 2022 @08:48AM (#31594140)
There used to be a setting to tweak Chrome's process behavior.

I believe the default behavior is "Coalesce tabs into the same content process if they're from the same trust domain".

Then you can make it more aggressive like "Don't coalesce tabs ever" or less aggressive like "Just have one content process". I think.

I'm not sure how Firefox decides when to spawn new processes. I know they have one GPU process and then multiple untrusted "content processes" that can touch untrusted data but can't touch the GPU.

I don't mind it. It's a trade-off between security and overhead. The IPC is pretty efficient and the page cache in both Windows and Linux _should_ mean that all the code pages are shared between all content processes.

Static pages actually feel light to me. I think crappy webapps make the web slow, not browser security.

(inb4 I'm replying to someone who works on the Firefox IPC team or something lol)

Re:How fast are Linux pipes anyway? by girvo on Thursday June 02, 2022 @09:21AM (#31594522)
> inb4 I'm replying to someone who works on the Firefox IPC team or something lol

The danger and joy of commenting on HN!

Re:How fast are Linux pipes anyway? by bayindirh on Thursday June 02, 2022 @09:30AM (#31594631)
I'm harmless, don't worry. :) Also you can find more information about me in my profile.

Even if I was working on Firefox/Chrome/whatever, I'd not be mad at someone who doesn't know something very well. Why should I? We're just conversing here.

Also, I've been very wrong here at times, and this improved my conversation / discussion skills a great deal.

So, don't worry, and comment away.

Re:How fast are Linux pipes anyway? by jagrsw on Thursday June 02, 2022 @06:29AM (#31593277)
> As another test point, Firefox doesn't even blink at a symmetric gigabit connection going at full speed (my network is capped by my NIC, the pipe is way fatter).

FWIW Firefox under Linux (Firefox Browser 100.0.2 (64-bit)) behaves pretty much the same as Chrome. The speed raises quickly to 5-8Gb/s, then the UI starts choking, and the shown speed drops to 500Mb/s. It could be that there's some scheduling limit or other bottleneck hit in the OS itself, assuming these are different codebases (are they?).

Re:How fast are Linux pipes anyway? by bayindirh on Thursday June 02, 2022 @06:39AM (#31593328)
I'd love to test and debug the path where it dies, but none of the systems we have firefox have pipes that fat (again NIC limited).

However, you can test the limits of Linux by installing CLI version of Speedtest and hitting a nearby server.

The bottleneck maybe in the browser itself, or in your graphics stack, too.

Linux can do pretty amazing things in the network department, otherwise 100Gbps Infiniband cards wouldn't be possible at Linux servers, yet we have them on our systems.

And yes, Chrome and Firefox are way different browsers. I can confidently say this, because I'm using Firefox since it's called Netscape 6.0 (and Mozilla in Knoppix).

Re:How fast are Linux pipes anyway? by jeffreygoesto on Thursday June 02, 2022 @07:04AM (#31593485)
From my experience long ago, all high performance networking under Linux was traditionally user space and pre-allocated pools (netmap, dpdk, pf-ring...). Did not follow, how much io_uring has been catching up for network stack usage... Maybe somebody else knows?
Re:How fast are Linux pipes anyway? by bayindirh on Thursday June 02, 2022 @07:10AM (#31593524)
While I'm not very knowledgeable in specifics, there are many paths for networking in Linux now. The usual kernel based one is there, also there's kernel-bypass [0] paths used by very high performance cards.

Also, Infiniband can directly RDMA to and from MPI processes for making "remote memory local", allowing very low latencies and high performance in HPC environments.

I also like this post from Cloudflare [1]. I've read it completely, but the specifics are lost on me since I'm not directly concerned with the network part of our system.

[0]: https://medium.com/@penberg/on-kernel-bypass-networking-and-...

[1]: https://blog.cloudflare.com/how-to-receive-a-million-packets...

Re:How fast are Linux pipes anyway? by sophacles on Thursday June 02, 2022 @11:47AM (#31596336)
I have a service that beats epoll with io_uring (it reads gre packets from one socket, and does some lookups/munging on the inner packet and re-encaps them to a different mechanism and writes them back to a different socket). General usage for io_uring vs epoll is pretty comparable IIUC. It wouldn't surprise me if streams (e.g. tcp) end up being faster via io_uring and buffer registration though.

Totally tangential - it looks like io_uring is evolving beyond just io and into an alternate syscall interface, which is pretty neat imho.

Re:How fast are Linux pipes anyway? by bawolff on Thursday June 02, 2022 @06:46AM (#31593377)
> I can confidently say this, because I'm using Firefox since it's called Netscape 6.0 (and Mozilla in Knoppix).

Mozilla suite/seamonkey isn't usually considered the same as firefox, although obviously related.

Re:How fast are Linux pipes anyway? by bayindirh on Thursday June 02, 2022 @06:50AM (#31593407)
I'm not talking about the version which evolved to Seamonkey. I'm talking about Mozilla/Firefox 0.8 which had a Mozilla logo as a "Spinner" instead of Netscape logo on the top right.
Re:How fast are Linux pipes anyway? by bawolff on Thursday June 02, 2022 @11:41AM (#31596244)
Netscape 6 was not firefox based https://en.m.wikipedia.org/wiki/Netscape_6

Firefox 0.8 did not have netscape branding http://theseblog.free.fr/firefox-0.8.jpg

Re:How fast are Linux pipes anyway? by bayindirh on Friday June 03, 2022 @12:58PM (#31610827)
> Netscape 6 was not Firefox based.

I know. Firefox was not even an idea when Netscape 6 was released. However, inverse is true. Firefox is based on Netscape. It's just branched off actually. It started as a pared down version of SeaMonkey apparently.

The thing I was remembering from Knoppix 3.x days was "Mozilla Navigator" of SeaMonkey/Mozilla Suite, which is even older than Firefox, and discontinued 3 years later. I just booted the CD to look at it.

At the end of the day, Firefox is just Netscape Navigator, evolved.

Re:How fast are Linux pipes anyway? by implying on Thursday June 02, 2022 @05:49AM (#31593085)
Firefox is not based on the chromium codebase, it is older.
Re:How fast are Linux pipes anyway? by formerly_proven on Thursday June 02, 2022 @06:22AM (#31593256)
Well if we're talking ancestors that's technically true, but not by that much - Firefox comes from Netscape, Chrome/Safari/... come from KHTML.
Re:How fast are Linux pipes anyway? by wodenokoto on Thursday June 02, 2022 @08:30AM (#31593972)
> ... on August 16, 1999 that [Lars Knoll] had checked in what amounted to a complete rewrite of the KHTML library—changing KHTML to use the standard W3C DOM as its internal document representation. https://en.wikipedia.org/wiki/KHTML#Re-write_and_improvement

> In March 1998, Netscape released most of the code base for its popular Netscape Communicator suite under an open source license. The name of the application developed from this would be Mozilla, coordinated by the newly created Mozilla Organization https://en.wikipedia.org/wiki/Mozilla_Application_Suite#Hist...

Netscape Communicator (or Netscape 4) was released in 1997, so If we are tracing lineage, I'd say Firefox has a 2 year head start.

Re:How fast are Linux pipes anyway? by elpescado on Thursday June 02, 2022 @06:48AM (#31593394)
AFAIR, KHTML was/is not related to Netscape/Gecko in any way.
Re:How fast are Linux pipes anyway? by merightnow on Thursday June 02, 2022 @06:47AM (#31593391)
Unrelated question, what hardware do you use to setup your network for 25Gb/s? I've been looking at init7 for a while, but gave up and stayed with Salt after trying to find the right hardware for the job.
Re:How fast are Linux pipes anyway? by jagrsw on Thursday June 02, 2022 @06:52AM (#31593420)
NIC: Intel E810-XXVDA2

Optics: To ISP: Flexoptics (https://www.flexoptix.net/de/p-b1625g-10-ad.html?co10426=972...), Router-PC: https://mikrotik.com/product/S-3553LC20D

Router: Mikrotik CCR-2004 - https://mikrotik.com/product/ccr2004_1g_12s_2xs - warning: it's good to up to ~20Gb/s one way. It can handle ~25Gb/s down, but only ~18Gb/s up, and with IPv6 the max seems to be ~10Gb/s any direction.

If Mikrotik is something you're comfortable using you can also take a look at https://mikrotik.com/product/ccr2216_1g_12xs_2xq - it's more expensive (~2500EUR), but should handle 25Gb/s easily.

Re:How fast are Linux pipes anyway? by zrail on Thursday June 02, 2022 @07:29AM (#31593623)
IIRC most Mikrotik products lack hardware IPv6 offload which is probably why you're seeing lower speeds.
Re:How fast are Linux pipes anyway? by BenjiWiebe on Thursday June 02, 2022 @09:42AM (#31594781)
In that case 10Gb/s sounds actually pretty good, if that's without hardware offload.
Re:How fast are Linux pipes anyway? by jcims on Thursday June 02, 2022 @06:01AM (#31593153)
Speedtest does have a CLI as well, might be interesting to compare them.
Re:How fast are Linux pipes anyway? by zrail on Thursday June 02, 2022 @07:38AM (#31593663)
Thing to note: the open source version on GitHub, installable by homebrew and native package managers, is not the same version as Ookla distributes from their website and is not accurate at all.
Re:How fast are Linux pipes anyway? by jagrsw on Thursday June 02, 2022 @06:18AM (#31593241)
Re:How fast are Linux pipes anyway? by sph on Thursday June 02, 2022 @05:58AM (#31593136)
This makes me wonder... does anyone offer an iperf-based speedtest service on the Internet?
Re:How fast are Linux pipes anyway? by jagrsw on Thursday June 02, 2022 @06:45AM (#31593368)
Ha.. my ISP does :) I can hit those 25Gb/s when connecting directly (bypassing the router as it barely handles those 25Gb/s).

With it in the way I get ~15-20Gb/s

  $ iperf3 -l 1M --window 64M -P10 -c speedtest.init7.net
  ..
  [SUM]   0.00-1.00   sec  1.87 GBytes  16.0 Gbits/sec  181406

  $ iperf3 -R -l 1M --window 64M -P10 -c speedtest.init7.net
  ..
  [SUM]   0.00-1.00   sec  2.29 GBytes  19.6 Gbits/sec
Re:How fast are Linux pipes anyway? by scoopr on Thursday June 02, 2022 @06:36AM (#31593315)
Well there are some public iperf servers listed here: https://iperf.fr/iperf-servers.php
Re:How fast are Linux pipes anyway? by r3dey3 on Thursday June 02, 2022 @10:31PM (#31603549)
https://github.com/R0GGER/public-iperf3-servers is also a good list - a couple more US servers in various datacenters
Re:How fast are Linux pipes anyway? by pca006132 on Thursday June 02, 2022 @06:54AM (#31593428)
Is it only affecting the browser or the entire system? It might be possible that the CPU is busy handling interrupts from the ethernet controller, although in general these controllers should use DMA and should not send interrupts frequently.
Re:How fast are Linux pipes anyway? by jagrsw on Thursday June 02, 2022 @06:54AM (#31593435)
Only browser(s), the OS is capable of 25Gb/s - checked with iperf and also speedtest-cli - https://www.speedtest.net/result/c/e9104814-294f-4927-af9f-d...
Re:How fast are Linux pipes anyway? by Spooky23 on Thursday June 02, 2022 @08:09AM (#31593850)
I ran into this with a VDI environment in a data center. We had initially delivered 10Gb Ethernet to the VMs, because why not.

Turned out windows 7 or the NICs needed a lot of tuning to work well. There was alot of freezing and other fail.

Re:How fast are Linux pipes anyway? by jcranberry on Thursday June 02, 2022 @10:20AM (#31595234)
Sounds like a hard drive cache filling up.
Re:How fast are Linux pipes anyway? by megous on Thursday June 02, 2022 @10:57AM (#31595682)
One would assume speed testing website would use `Cache-Control: no-store`...

But alas, they do not, lol. They just use no-cache on the query which will not prevent the browser from storing the data.

https://megous.com/dl/tmp/8112dd9346dd66e8.png

Re:How fast are Linux pipes anyway? by def- on Thursday June 02, 2022 @05:50AM (#31593087)
Firefox is only Chrome-based on iOS.
Re:How fast are Linux pipes anyway? by karamanolev on Thursday June 02, 2022 @06:09AM (#31593192)
It's Safari-based, which is Webkit-based. Chrome is also Safari-based on iOS, because all the browsers must be. There's no actual Chrome (as in Blink, the browser engine) on iOS, at least in Play Store.
Re:How fast are Linux pipes anyway? by Izkata on Thursday June 02, 2022 @06:50AM (#31593405)
> It's Safari-based, which is Webkit-based.

Firefox only uses Webkit on iOS, due to Apple requirements. It uses Gecko everywhere else. And I don't think it's ever been Safari-based anywhere.

Re:How fast are Linux pipes anyway? by karamanolev on Friday June 03, 2022 @06:15PM (#31614351)
Replying to

> Firefox is only Chrome-based on iOS.

So I'm talking only about iOS. When I said it's Safari-based, I meant Webkit based, but I thought Firefox/Chrome actually pull parts of Safari on iOS. Quick research says that's wrong and they just use Webkit. Not an iOS dev, so someone can point out better sources for the 100% correct terminology.

Re:How fast are Linux pipes anyway? by rwaksmunski on Thursday June 02, 2022 @06:00AM (#31593144)
You mean WebKit.
Re:How fast are Linux pipes anyway? by jve on Thursday June 02, 2022 @05:56AM (#31593114)
Do you actually mean Gbit/s? 25Gb/s would translate to 200Gbit/s ...
Re:How fast are Linux pipes anyway? by Denvercoder9 on Thursday June 02, 2022 @05:57AM (#31593119)
The small "b" is customarily used to refer to bits, with the large "B" used to refer to bytes. So 25 Gb/s would be 25 Gbit/s, while 25 GB/s would be 200 Gbit/s.
Re:How fast are Linux pipes anyway? by karamanolev on Thursday June 02, 2022 @05:58AM (#31593133)
Gb != GB. Per Wikipedia, which aligns with my understanding,

"The gigabit has the unit symbol Gbit or Gb."

25GB/s would translate to 200Gbit/s and also 200Gb/s.