News for hackers. Stuff that matters.
Re:Why Discord is switching from Go to Rust by _ph_ on Tuesday February 04, 2020 @02:19PM (#22239786)
With recent Go releases, GC pauses have become neglible for most applications. So this should not get into your way. However, it can easily tweaked, if needed. There is runtime.ForceGCPeriod, which is a pointer to the forcegcperiod variable. A Go program, which really needs to change this, can do it, but most programs shouldn't require this.

Also, it is almost trivial to edit the Go sources (they are included in the distribution) and rebuild it, which usually takes just a minute. So Go is really suited for your own experiments - especially, as Go is implemented in Go.

Re:Why Discord is switching from Go to Rust by nemo1618 on Tuesday February 04, 2020 @02:25PM (#22239858)
runtime.ForceGCPeriod is only exported in testing, so you wouldn't be able to use it in production. But as you said, the distribution could easily be modified to fit their needs.
Re:Why Discord is switching from Go to Rust by _ph_ on Tuesday February 04, 2020 @02:33PM (#22239950)
Thanks, didn't catch that this is for testing only.
Re:Why Discord is switching from Go to Rust by calcifer on Tuesday February 04, 2020 @02:56PM (#22240223)
> especially, as Go is implemented in Go.

Well, parts of it. You can't implement "make" or "new" in Go yourself, for example.

Re:Why Discord is switching from Go to Rust by _ph_ on Tuesday February 04, 2020 @03:13PM (#22240399)
You have to distinguish between the features available to a Go program as the user writes it and the implementation of the language. The immplementation is completely written in Go (plus a bit of low-level assembly). Even if the internals of e.g. the GC are not visible to a Go program, the GC itself is implemented in Go and thus easily readeable and hackeable for experienced Go programmers. And you can quickly rebuild the whole Go stack.
Re:Why Discord is switching from Go to Rust by calcifer on Tuesday February 04, 2020 @05:10PM (#22241575)
> You have to distinguish between the features available to a Go program as the user writes it and the implementation of the language.

I do, I'm just objecting to "Go is implemented in Go".

Re:Why Discord is switching from Go to Rust by dodobirdlord on Tuesday February 04, 2020 @11:35PM (#22244060)
This reminds me of the ongoing saga of RUSTC_BOOTSTRAP[0][1]

The stable compiler is permitted to use unstable features in stable builds, but only for compiling the compiler. In essence, there are some Rust features that are supported by the compiler but only permitted to be used by the compiler. Unsurprisingly, various non-compiler users of Rust have decided that they want those features and begun setting the RUSTC_BOOTSTRAP envvar to build things other than the compiler, prompting consternation from the compiler team.

[0] https://github.com/rust-lang/cargo/issues/6627 [1] https://github.com/rust-lang/cargo/issues/7088

Re:Why Discord is switching from Go to Rust by estebank on Wednesday February 05, 2020 @01:15AM (#22244429)
This is not entirely correct. These things that "can only be used by the compiler" are nightly features that haven't been stabilized yet. Some of them might never be stabilized, but you could always use them in a nightly conpiler, stability assurances just fly out the window then. This is also why using that environment variable is highly discouraged: it breaks the stability guarantees of the language and you're effectively using a pinned nightly. This is reasonable only in a very small handful of cases.
Re:Why Discord is switching from Go to Rust by giornogiovanna on Wednesday February 05, 2020 @01:53AM (#22244596)
Yep. Beyond that, there is at least one place[0] where the standard library uses undefined behavior "based on its privileged knowledge of rustc internals".

[0]: https://doc.rust-lang.org/src/std/io/mod.rs.html#379

Re:Why Discord is switching from Go to Rust by dodobirdlord on Wednesday February 05, 2020 @02:25AM (#22244718)
I don't see what is incorrect? Perhaps I was insufficiently clear that when I said "the compiler" I meant "the stable compiler" as opposed to more generally all possible versions of rustc. The stable compiler is permitted to use unstable features for its own bootstrap, example being the limited use of const generics to compile parts of the standard library.
Re:Why Discord is switching from Go to Rust by _ph_ on Wednesday February 05, 2020 @02:01AM (#22244639)
But on what basis? What part of Go isn't implemented in Go?
Re:Why Discord is switching from Go to Rust by calcifer on Wednesday February 05, 2020 @04:14AM (#22245206)
I gave an example a bit further up, here [1].

[1] https://news.ycombinator.com/item?id=22240223

Re:Why Discord is switching from Go to Rust by _ph_ on Wednesday February 05, 2020 @04:43AM (#22245384)
But this isn't a contradiction to the statement, that Go is implemented in Go. If you look at the sources of the Go implementation, the source code is 99% Go, with a few assembly functions (most for optimizations not performed by the compiler) and no other programming language used.
Re:Why Discord is switching from Go to Rust by calcifer on Wednesday February 05, 2020 @07:50AM (#22246286)
That's not correct. The implementation of "make", for example, looks like Go but isn't - it relies on internal details of the gc compiler that isn't part of the spec [1]. That's why a Go user can't implement "make" in Go.

[1] https://golang.org/ref/spec

Re:Why Discord is switching from Go to Rust by _ph_ on Wednesday February 05, 2020 @08:11AM (#22246406)
In which language do you think is "make" implemented?
Re:Why Discord is switching from Go to Rust by scott_s on Wednesday February 05, 2020 @10:38AM (#22247651)
If I may interject: I believe you are both trying to make orthogonal points. calcifer's is trying to say that some features of Go are compiler intrinsics, and cannot be implemented as a library. You are making a different point, which is that those intrinsics are implemented in Go, the host language. Both statement can be true at the same time, but I agree that the terms were not used entirely accurately, causing confusion.
Re:Why Discord is switching from Go to Rust by slrz on Tuesday February 04, 2020 @04:32PM (#22241212)
And yet, maps and slices are implemented in Go.

https://golang.org/src/runtime/map.go

https://golang.org/src/runtime/slice.go

I don't see why you couldn't do something similar in your own Go code. It just won't be as convenient to use as the compiler wouldn't fill in the type information (element size, suitable hash function, etc.) for you. You'd have to pass that yourself or provide type-specific wrappers invoking the unsafe base implementation. More or less like you would do in C, with some extra care to abide by the rules required for unsafe Go code.

Re:Why Discord is switching from Go to Rust by calcifer on Tuesday February 04, 2020 @05:12PM (#22241591)
Nothing you wrote contradicts what I said. You can't implement "make" in Go. The fact that you can implement some approximation of it with a worse signature and worse runtime behaviour (since it won't be compiler assisted) doesn't make it "make".
Re:Why Discord is switching from Go to Rust by robocat on Tuesday February 04, 2020 @04:06PM (#22240965)
You still may have significant CPU overhead from the GC e.g. the twitch article (mentioned elsewhere in comments) measured 30% CPU used for GC for one program (Go 1.5 I think).

Obviously they consider spending 50% more on hardware is a worthwhile compromise for the gains they get (e.g. reduction of developer hours and reduced risk of security flaws or avoiding other effects of invalid pointers).

Re:Why Discord is switching from Go to Rust by _ph_ on Tuesday February 04, 2020 @04:15PM (#22241062)
In this case, as they were running into the automatic GC interval, their program did not create much, if any garbage. So the CPU overhead for the GC would have been quite small.

If you do a lot of allocations, the GC overhead rises of course, but also would the effort of doing allocations/deallocations with a manual managing scheme. In the end it is a bit trade-off, what fits the problem at hand best. The nice thing about Rust is, that "manual" memory management doesn't come at the price of program correctness.

Re:Why Discord is switching from Go to Rust by pkolaczk on Wednesday February 05, 2020 @05:01AM (#22245455)
Languages that have GC frequently rely on heap allocation by default and make plenty of allocations. Languages with good manual memory management frequently rely on stack allocation and give plenty of tools to work with data on the stack. Automatic allocation on the stack is almost always faster than the best GC.
Re:Why Discord is switching from Go to Rust by _ph_ on Wednesday February 05, 2020 @07:10AM (#22246055)
GC languages often do and also often do not. Most modern GC languages have escape analysis. So if the compiler can deduct that an object does not escape the current scope, it is stack allocated instead of heap allocated. Modern JVMs do this and Go does this also. Furthermore, Go is way more allocation friendly than e.g. Java. In Go an array of structs is a single item on the heap (or stack). In Java, you would have an array of pointers to separately allocate on the heap (Java is just now trying to rectify this with the "record" types). Also, structs are passed by value instead of reference.

As a consequence, the heap pressure of a Go program is not necessarily significantly larger than that of an equivalent C or Rust program.

Re:Why Discord is switching from Go to Rust by pkolaczk on Wednesday February 05, 2020 @01:06PM (#22249438)
Escape analysis is very limited and what I found in practice, it often doesn't work in real code, where not all the things are inlined. If a method allocates an object and returns it 10 layers up, EA can't do anything.

In contrary, in e.g. C I can wrap two 32-bit fields in a struct and freely pass then anywhere with zero heap allocations.

Also, record types are not going to fix the pointer chasing problem with arrays. This is promised by Valhalla, but I'be been hearing about it for 3 years or more now.

Re:Why Discord is switching from Go to Rust by andoriyu on Tuesday February 04, 2020 @04:16PM (#22241071)
> Also, it is almost trivial to edit the Go sources (they are included in the distribution) and rebuild it, which usually takes just a minute. So Go is really suited for your own experiments - especially, as Go is implemented in Go.

Ruby 1.8.x wants to say "Hello"