News for hackers. Stuff that matters.
Re:Golang is evil on shitty networks by sneak on Thursday December 29, 2022 @07:32PM (#34180075)
> I'd imagine most Go software is deployed in datacentres where the network is high quality

The problem is that those datacenters are plugged into the Internet, where the network is not always high quality. TFA mentions the Caddy webserver - this is "datacenter" software designed to talk to diverse clients all over the internet. The stdlib should not tamper with the OS defaults unless the OS defaults are pathological.

Re:Golang is evil on shitty networks by tptacek on Thursday December 29, 2022 @07:47PM (#34180198)
That doesn't make much sense. There are all sorts of socket and file descriptor parameters with defaults that are situational; NDELAY is one of them, as is buffer size, nonblockingness, address reuse, &c. Maybe disabling Nagle is a bad default, maybe it isn't, but the appeal to "OS defaults" is a red herring.
Re:Golang is evil on shitty networks by Terretta on Thursday December 29, 2022 @08:38PM (#34180639)
You guys had this convo before, in 2015, on what the interaction of the two settings should be doing:

https://news.ycombinator.com/item?id=34180239

Re:Golang is evil on shitty networks by fragmede on Thursday December 29, 2022 @09:18PM (#34180996)
Re:Golang is evil on shitty networks by mdolah on Thursday December 29, 2022 @09:23PM (#34181030)
Love this historical find. How did you find this relevant conversation? Was this something with HN search?
Re:Golang is evil on shitty networks by subroutine on Friday December 30, 2022 @05:39AM (#34183782)
You haven't committed to memory every hn comment from Ptacek over the last decade?
Re:Golang is evil on shitty networks by Teckla on Thursday December 29, 2022 @08:04PM (#34180327)
In my opinion, the Principle of Least Surprise applies here.

Go is defaulting to surprising (unexpected) behavior.

Re:Golang is evil on shitty networks by giovannibajo1 on Thursday December 29, 2022 @08:24PM (#34180506)
I think also “least surprise” depends on your background. In Go, also files don’t buffer by default, contrary to many languages including C. If you call Write() 100 times, you run exactly 100 syscalls. Intermediate Go programmers learn this and that they must explicitly manage buffering (eg: via bufio).

I don’t think it’s wrong that sockets follow the same design. It gives me less surprise.

Re:Golang is evil on shitty networks by incompatible on Thursday December 29, 2022 @08:38PM (#34180642)
I think my C is getting rusty, but "write" operates on a file descriptor, doesn't it? It's unbuffered. The buffered versions are things like printf and puts.
Re:Golang is evil on shitty networks by tptacek on Thursday December 29, 2022 @08:47PM (#34180735)
That's POSIX; C's equivalent is a FILE, which generally is buffered.
Re:Golang is evil on shitty networks by Kab1r on Thursday December 29, 2022 @10:43PM (#34181528)
I thought Linux does in kernel buffering with `write`
Re:Golang is evil on shitty networks by blibble on Thursday December 29, 2022 @08:49PM (#34180760)
that Write() doesn't call fsync() though, does it?

so there's no buffering going on in the application, but the bytes almost certainly don't hit the disk before Write() returns

they've just been staged into an OS buffer, with the OS promising to write them out to the disk at a later time (probably, maybe...? hopefully!)

which is exactly the same as a regular TCP socket (with Nagle disabled, i.e. the default, non Go way)

Re:Golang is evil on shitty networks by Ferret7446 on Friday December 30, 2022 @04:53AM (#34183570)
For userland programming, what matters is the syscall level, as that is expensive (and also the API you have for the kernel). Whether the kernel then does internal buffering is irrelevant and uncontrollable beyond any other syscalls which may or may not be implemented (maybe you're running on a custom kernel that doesn't buffer disk writes?).

One write == one syscall, easy. If you want buffering, you add it.

Re:Golang is evil on shitty networks by blibble on Friday December 30, 2022 @12:16PM (#34187127)
> For userland programming, what matters is the syscall level, as that is expensive

which is why pretty much every programming language buffers file output by default

even C

(other than Go, obviously)

> Whether the kernel then does internal buffering is irrelevant

everyone that's attempted to write reliable software that cares about what ends up on disk, or the other side of the socket will disagree

Re:Golang is evil on shitty networks by pclmulqdq on Thursday December 29, 2022 @08:19PM (#34180468)
In my experience, you usually don't want to be Nagling in code that lives in a datacenter. Go's default is likely set up around that idea.
Re:Golang is evil on shitty networks by Xorlev on Thursday December 29, 2022 @08:25PM (#34180526)
Have you ever had to chase down strange latency issues? Arguably, this behavior is the least surprising for Go's typical deployment environment.
Re:Golang is evil on shitty networks by lanstin on Thursday December 29, 2022 @11:17PM (#34181722)
Twice I have run into this behavior having known and forgotten it. Chatty non http protocols with a few small messages doing with before bulk data flow. Pissed me off and surprised me. Now I make sure my defaults for any framework I am using are no delay, and I make sure to plug my computing device into Ethernet whenever possible.
Re:Golang is evil on shitty networks by squeaky-clean on Thursday December 29, 2022 @11:31PM (#34181812)
Only if you know that Nagle's algorithm exists and is used everywhere. For anyone with networking experience it's unexpected, but I still remember learning about Nagle's algorithm when trying to fix latency on a game server I was hosting as a teen. That was surprising behavior to me at the time.
Re:Golang is evil on shitty networks by jrockway on Tuesday January 03, 2023 @02:05PM (#34236316)
Articles with titles like "How I spent 3 weeks discovering that Nagle's Algorithm exists" are a HN staple. Turning off Nagle follows the principle of least surprise. This article is the first time anyone has ever written about being surprised by Nagle's Algorithm being off.
Re:Golang is evil on shitty networks by drpixie on Friday December 30, 2022 @12:17AM (#34182081)
Also, for small packets, disabling consolidation means adding LOTS of packet overhead. You're not sending 1 million * 50 bytes of data, you're sending 1 million * (50 bytes of data + about 80 bytes of TCP+ethernet header).

Disabling Nagle makes sense for tiny request/replys (like RPC calls) but it's counterproductive for bulk transfers.

I'm not the only one who don't like the thought of a standard library quietly changing standard system behaviour ... so know I have to know the standard routines and their behaviour AND I have to know which platforms/libraries silently reverse things :(

Re:Golang is evil on shitty networks by boring_twenties on Friday December 30, 2022 @01:42AM (#34182541)
Bulk transfer applications should just use larger buffers
Re:Golang is evil on shitty networks by josefx on Friday December 30, 2022 @01:55AM (#34182606)
Wouldn't that still end up with sub optimal network messages unless those large buffers are an exact multiple of the MTU on the network?
Re:Golang is evil on shitty networks by boring_twenties on Tuesday January 03, 2023 @09:49AM (#34232007)
Hm, I'm not 100% sure about this. If your first buffer is big enough, your next write should be issued before the OS has managed to write it all.