Beating C with 70 lines of Go

Re:Beating C with 70 lines of Go by manningthegoose on Wednesday November 20, 2019 @02:41PM (#21587638)

I actually disagree. If you look at the code written in the article, it's hardly unreadable or very code golf-y as you suggest. I think the author is really just trying to say that programmers shouldn't dismiss Go as a possible systems language in favor of C just because C has a reputation of being faster in all cases.

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by dijit on Wednesday November 20, 2019 @03:05PM (#21587895)

Are we really saying a garbage collected language should be considered for a systems language? I know the GC is fast but surely memory managed languages are going to have less issues with pausing during execution, no?

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by zenhack on Wednesday November 20, 2019 @03:13PM (#21587984)

Re: pause times, Go's garbage collector is actually really good here; you're looking at 10s of microseconds.

I'm convinced the term "systems language" doesn't have a coherent meaning at this point. See:

https://zenhack.net/2018/07/14/three-funerals-in-the-name-of...

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by __d on Wednesday November 20, 2019 @05:11PM (#21589142)

Right, so when I’m trying to process a network packet in about a microsecond (a reasonable target for financial trading software), Go’s GC rules it out. Many low-level, high-performance systems language tasks aren’t feasible in Go for similar reasons.

That doesn’t make it a bad language, but it’s not universally applicable either.

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by zenhack on Thursday November 21, 2019 @01:21AM (#21591866)

> but it’s not universally applicable either.

Nothing is universally applicable.

But yeah, I certainly wouldn't use it for hard real-time tasks. If you can't tolerate missing deadlines ever, there is a very short list of acceptable tools.

But (and correct me if I'm wrong; it's not my area) HFT doesn't strike me as hard real-time? See also a sibling comment that asks about Jane Street's OCaml use.

It's not like GC pauses are happening constantly; Go programs don't allocate that much, and a well tuned program can go a long time between collections. It likely is appropriate for many soft or firm real time systems. And you can shut the GC off if there are sections where GC really must not happen:

https://golang.org/pkg/runtime/debug/#SetGCPercent

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by __d on Thursday November 28, 2019 @12:16AM (#21654766)

That's how those who use Java do it: write their code to minimize allocations, and then configure the JVM with a GC threshold that's bigger than their whole day's allocations, and manually GC at a suitable time.

Last I looked (a while ago) it seemed like there wasn't quite enough control on the Go GC to do this effectively. It's very likely that's been fixed by now.

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by crematoria on Wednesday November 20, 2019 @11:23PM (#21591445)

Strange. I know OCaml is used in HFT (thanks to all those Jane Street ads) and OCaml is also GC'd. What does OCaml have here that Go doesn't?

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by pjmlp on Thursday November 21, 2019 @02:35AM (#21592126)

Just like in hard real time systems using malloc() rules C out, and its use is explicitly forbidden in certification processes.

I guess C isn't universally applicable either.

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by Gibbon1 on Thursday November 21, 2019 @03:52AM (#21592436)

The difference is, you can write perfectly usable programs in C without littering the codebase with malloc() and free().

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by pjmlp on Thursday November 21, 2019 @03:58AM (#21592457)

Just like you can in Go, as proven by TinyGo.

Or in dozen of other GC enabled systems programming languages for that matter.

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by Gibbon1 on Thursday November 21, 2019 @04:26AM (#21592565)

Tip: Give up dude.

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by pjmlp on Thursday November 21, 2019 @06:05AM (#21593021)

Exactly my tip to the anti-GC luddites.

The difference being evolution, takes care of sorting that out, like in all anti-technology groups throughout mankind history.

[ Parent | on HN ]

[dead]

Re:Beating C with 70 lines of Go by pjmlp on Friday November 22, 2019 @11:34AM (#21607133)

To the extent that my work is open to memory corruption security exploits, with possible high monetary costs, caused by anti-GC luddites.

https://googleprojectzero.blogspot.com/2019/11/bad-binder-an...

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by Gibbon1 on Friday November 22, 2019 @02:36PM (#21609198)

Great we can have system software that's as shitty as user facing now.

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by pjmlp on Saturday November 23, 2019 @02:17AM (#21612895)

"The Singularity research codebase and design evolved to become the Midori advanced-development OS project. While never reaching commercial release, at one time Midori powered all of Microsoft’s natural language search service for the West Coast and Asia."

https://www.microsoft.com/en-us/research/project/singularity...

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by pleasecalllater on Wednesday November 20, 2019 @04:58PM (#21589025)

I always thought the GC pause depends on the heap size... a little searching... http://big-elephants.com/2018-09/unexpected-gc-pauses/.

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by naikrovek on Wednesday November 20, 2019 @07:52PM (#21590383)

It depends on how many things need to be collected, which is often a function of heap size.

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by pjmlp on Wednesday November 20, 2019 @05:35PM (#21589367)

Yes, as proven multiple times, and being pushed forward by the likes of Apple, Google and Microsoft.

Regarding using Go as a real systems language (in the same meaning as C):

- gVisor hypervisor on Google Cloud and Linux sandbox on Chromebooks

- Android GPGPU debugger

- Fuchsia TCP/IP stack and volume management

- Baremetal TinyGo on Arduino Nano33 IoT, Adafruit Circuit Playground Express, BBC micro:bit among many others

- Coreboot firmware

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by dijit on Sunday November 24, 2019 @07:00AM (#21619873)

Coreboot is C/ASM though.

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by agumonkey on Thursday November 21, 2019 @03:31AM (#21592356)

I think there are already a few gc/jitted components in mainstream kernels (bpf, lua drivers). So in some points, it's not a crazy idea. Now I don't advocate for Java drivers.

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by IshKebab on Wednesday November 20, 2019 @03:36PM (#21588206)

In the sense that the Go authors used "systems language", i.e. for writing things like servers and command line tools, sure why not? Go's GC pauses are extremely short (under 1ms). That shouldn't really affect much.

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by crematoria on Wednesday November 20, 2019 @11:26PM (#21591461)

Source? People write CLI tools in Ruby and Node.js these days, so I really doubt that anybody would use the term "systems language" to refer to languages well-suited to building CLI tools.

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by IshKebab on Thursday November 21, 2019 @03:40AM (#21592393)

https://www.wired.com/2009/11/google-announces-a-new-program...

> As a systems language, Go is intended to be used for developer applications like, for example, web servers.

Couldn't find a citation for CLI tools, but that should satisfy you (since people write web servers in Ruby and Node.js too).

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by weberc2 on Wednesday November 20, 2019 @03:58PM (#21588430)

As others mentioned "systems language" isn't well defined. If you want to define a "systems program" as one with hard real-time requirements, then no, Go isn't a very good choice; however, the article defines it differently (such that `wc` satisfies), and demonstrates that Go satisfies that definition.

That said, it's better to use terms that map better to a set of requirements, such as "hard-realtime" or "soft-realtime".

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by rurban on Thursday November 21, 2019 @01:53AM (#21591971)

Esp. as a systems language. Safety first.

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by saghm on Wednesday November 20, 2019 @10:02PM (#21591072)

I'm not sure this proves that Go is a possible systems language, since it could be equally like that the explanation is just "wc doesn't need to be written in a systems language to be fast". For an executable that runs, processes some stuff, and then terminates in the span of a second or so, you're not going to incur much slowdown due to a GC counting references and cleaning stuff up, but for something long-running, it could make more of a difference.

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by zopppo on Wednesday November 20, 2019 @02:38PM (#21587609)

I could be wrong, but it seems like it wouldn't be difficult to outperform these implementations with a similarly hacky C implementation. I expect that blog post in the coming days.

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by ajeetdsouza on Wednesday November 20, 2019 @02:52PM (#21587752)

Hey, author here. You are absolutely right - Go would have a hard time outperforming finely tuned C. Rather, the article was more focused on exploring concurrency mechanisms in Go. The (admittedly clickbait) title was more of a reference to the articles before it than anything else.

That said, I think you'll find this SIMD-enhanced wc interesting: https://github.com/expr-fi/fastlwc/

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by barbegal on Wednesday November 20, 2019 @02:43PM (#21587657)

It's mentioned in the post https://github.com/expr-fi/fastlwc/

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by zopppo on Wednesday November 20, 2019 @02:47PM (#21587695)

Ah, I didn't see that. Thanks!

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by phaemon on Wednesday November 20, 2019 @02:41PM (#21587632)

Why wait?! Do your "Beating C with C" blog post and enjoy your HN fame!

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by ajeetdsouza on Wednesday November 20, 2019 @02:59PM (#21587841)

Author here. Why do you feel it is code golf? My primary focus when writing it was readability - I'm sure I could do it in much less than 70 lines if I had to. If you have any suggestions for improving the readability of my implementation, do let me know!

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by cjensen on Wednesday November 20, 2019 @03:26PM (#21588108)

The version you are comparing against is ancient and assumes a low-memory footprint. It only uses something like 10KB of data. It also has a ton more features -- like command line parsing, the ability to read from a file on the command line or stdin, and locale-specific recognition of whitespace.

It's really not solving the same problem.

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by ajeetdsouza on Wednesday November 20, 2019 @03:53PM (#21588387)

Argument parsing is hardly the bottleneck when you're scanning 1 GB of data. The reason I didn't implement the rest of the features was to keep the implementation light and readable, and I don't think a fully compliant wc implemented in Go with these patterns would be significantly slower.

As for having a low-memory footprint, 2 of the 4 implementations I mentioned (one single core, one multi-core) consume less memory than wc, and still run much faster.

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by aikah on Wednesday November 20, 2019 @03:45PM (#21588300)

How much memory does your program uses VS the C program? Did you profile that?

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by strange_aeons on Wednesday November 20, 2019 @03:58PM (#21588440)

Yes, it's the last column in all the result tables.

[ Parent | on HN ]

Re:Beating C with 70 lines of Go by Someone on Wednesday November 20, 2019 @03:55PM (#21588405)

FTA: ”Our parallelized implementation runs at more than 3.5x the speed of wc, while matching its memory consumption”

[ Parent | on HN ]