News for hackers. Stuff that matters.
Re:OCaml for the Masses (2011) by christophilus on Monday November 26, 2018 @10:48AM (#18534113)
Man, I agree 100%. I wonder if it would be possible to get this by compiling an Elm/SML-like language to Go?

[Edit: I'd add a few things to your feature list: really fast, OCaml-speed compilation, along with a utop-like REPL]

Re:OCaml for the Masses (2011) by weberc2 on Monday November 26, 2018 @11:11AM (#18534307)
+1 regarding your additions to my list! With respect to compiling an SML-like language to Go (I've thought about this a lot), it would probably get you a long way, but the performance would suffer because FP langs allocate like crazy and Go's GC is not optimized for lots of fine-grained allocations. And there would still be a long tail of things that didn't work quite right: stack traces would need to be modified to map to lines in your source program (instead of the emitted Go code), documentation generation and linking would need to be patched, dependency fetching would need to understand and distinguish between packages in $MYLANG and Go packages (so it invokes the $MYLANG->Go compiler appropriately), and so on. That said, it would certainly be a great proof of concept!
Re:OCaml for the Masses (2011) by vram22 on Monday November 26, 2018 @01:56PM (#18535825)
>Go's GC is not optimized for lots of fine-grained allocations

Why is it not? Hadn't heard of this point before. Is it because of catering to the most common use cases, which are not fine-grained, maybe?

Re:OCaml for the Masses (2011) by weberc2 on Monday November 26, 2018 @03:11PM (#18536386)
Go has value types and they're used pervasively. Where an OCaml or Python or Haskell list is O(n) allocations, a Go slice is 1 allocation (values are laid out side-by-side in memory). This allows Go to have a simple, concurrent GC with pause times on the order of 1us (compared to 10-100ms, as is common for compacting generational collectors) as well as great (CPU) cache properties (things are more likely to be collocated in memory, so you're more likely to have cache hits).
Re:OCaml for the Masses (2011) by vram22 on Monday November 26, 2018 @03:35PM (#18536561)
Got it, thanks. The Go team has been working on improving Go's performance, over time, I've read. One other such example I read about is that for compilation, where package A depends on B and B depends on C, the metadata for C is stored in the header of B, so when A is being compiled, it only needs to read B, not C - and there are other related optimizations on the compilation front, IIRC. I thought that was a good optimization.

I wish the EXE/binary sizes of Go programs were smaller, though I know they've done some work on that too. Coming from a C background originally, I was a bit surprised when I first saw that the size of a simple hello-world-style Go binary was ~1 MB (some years ago). Equivalent Free Pascal and D binaries are a lot smaller, IIRC, from what I tested at the time (~100 KB or less). Heck, the Red interpreter / compiler (and the Rebol interpreter before it) fit in ~1 MB. Don't mean to denigrate the efforts of the Go team at all (even apart from the fact that as a long-time Unix guy, Rob Pike, Ken Thompson, et al are among the people I look up to); I know language translation technology is a hard field, plus the scope of Go may be much more than that of Rebol / Red; just speaking from the POV of a user, hoping for smaller binary sizes.

Re:OCaml for the Masses (2011) by weberc2 on Monday November 26, 2018 @03:51PM (#18536690)
> Got it, thanks. The Go team has been working on improving Go's performance, over time, I've read. One other such example I read about is that for compilation, where package A depends on B and B depends on C, the metadata for C is stored in the header of B, so when A is being compiled, it only needs to read B, not C - and there are other related optimizations on the compilation front, IIRC. I thought that was a good optimization.

AFAICT that's a compiler optimization, not a runtime optimization. But yes, generally the Go team is working to improve performance.

> I wish the EXE/binary sizes of Go programs were smaller, though I know they've done some work on that too. Coming from a C background originally, I was a bit surprised when I first saw that the size of a simple hello-world-style Go binary was ~1 MB (some years ago). Equivalent Free Pascal and D binaries are a lot smaller, IIRC, from what I tested at the time (~100 KB or less). Heck, the Red interpreter / compiler (and the Rebol interpreter before it) fit in ~1 MB. Don't mean to denigrate the efforts of the Go team at all (even apart from the fact that as a long-time Unix guy, Rob Pike, Ken Thompson, et al are among the people I look up to); I know language translation technology is a hard field, plus the scope of Go may be much more than that of Rebol / Red; just speaking from the POV of a user, hoping for smaller binary sizes.

Go binaries contain _everything_, while C programs (usually) statically link against libc (glibc is on the order of 10mb by itself). Go's runtime is also quite a lot more complex than C's--it comes with a scheduler, a reflection system, and a garbage collector. These are one-time costs--the binary for a program 10 times the size of hello world is pretty much the same size as the hello world binary. If you really want, you can dynamically link a Go program, but I can't imagine a scenario where this is practical.

Re:OCaml for the Masses (2011) by vram22 on Tuesday November 27, 2018 @06:50AM (#18541519)
>AFAICT that's a compiler optimization, not a runtime optimization. But yes, generally the Go team is working to improve performance.

Right, it is a compile time one.

>Go binaries contain _everything_, while C programs (usually) statically link against libc (glibc is on the order of 10mb by itself).

Did you mean "dynamically" (for C) [1]? Also, I'm not too sure if Go links all dependencies statically by default, if that is what you implied above. IIRC I read somewhere recently that some things are linked statically, and some dynamically - I need to check that.

[1] I need to check it for C too, because things have changed since I last used C a lot.

>Go's runtime is also quite a lot more complex than C's--it comes with a scheduler, a reflection system, and a garbage collector. These are one-time costs--the binary for a program 10 times the size of hello world is pretty much the same size as the hello world binary.

Agreed on the points about the other stuff Go has that C does not, like a GC, reflection, etc. And that those are one-time costs. But they are still costs, and if you create lots of small binaries, that cost adds up. Disk space is cheap nowadays, but depending on the use case and your environment (e.g. small or embedded system, cost constraints if shipping many copies of a h/w + s/w product or appliance, etc., that cost might be significant. Whereas with some of those other languages that I mentioned above, those costs can be quite a bit less, times the number of copies of binary or appliance that you ship. That's all that I meant. That's not to say that we should not use Go, though, of course. As always, the overall picture and all factors need to be taken into account, e.g. the benefits of Go's language features vs. some of those others, team knowledge of the various language contenders, even availability of needed libraries, time to market, etc. etc.

Re:OCaml for the Masses (2011) by weberc2 on Tuesday November 27, 2018 @10:59AM (#18543382)
> Did you mean "dynamically" (for C) [1]?

Yeah, I did. Good catch.

> Also, I'm not too sure if Go links all dependencies statically by default, if that is what you implied above. IIRC I read somewhere recently that some things are linked statically, and some dynamically - I need to check that.

By default, Go does dynamically link to some things--it will prefer to use the system DNS resolver if one is available, for example. But you can disable this by setting `CGO_ENABLED=0` at compile time. If you do this, a Go program needs nothing more than a linux kernel (no system libraries or even libc) and whatever other files it requires (certificates, for SSL for example).

> I need to check it for C too, because things have changed since I last used C a lot.

I'm sure you can make statically-linked C programs which are smaller. For example, the musl libc implementation is quite small and designed for static linkage. But lots of libraries break (often in bizarre and hard-to-debug ways) if they don't link against glibc specifically, so it's not a panacea.

> But they are still costs, and if you create lots of small binaries, that cost adds up. Disk space is cheap nowadays, but depending on the use case and your environment (e.g. small or embedded system, cost constraints if shipping many copies of a h/w + s/w product or appliance, etc., that cost might be significant.

Agreed, but again, Go does allow for dynamic linkage in those cases so you can conserve disk space. There's no doubt that if you're golfing, C allows for smaller binaries than Go, but Go can meet 99.9% of real-world requirements when it comes to size (if you're at the extreme end of what Go can support, you've probably already ruled out Go for other reasons).

There are still many reasons to pick other languages besides Go--if you're working on a hard real-time system, if you're working with a team of people who all prefer a different language, if you're working in a domain (iOS app dev, for example) for which there are no good Go libraries, if you're willing to trade all else for extreme performance or extreme correctness, etc. But Go is actually a pretty good little language for most things that aren't at those extremes.

Re:OCaml for the Masses (2011) by vram22 on Tuesday November 27, 2018 @12:27PM (#18544387)
>By default, Go does dynamically link to some things--it will prefer to use the system DNS resolver if one is available, for example. But you can disable this by setting `CGO_ENABLED=0` at compile time. If you do this, a Go program needs nothing more than a linux kernel (no system libraries or even libc) and whatever other files it requires (certificates, for SSL for example).

Ah, sounds similar to what it says in the osso.nl link I posted below. Good to know.

>I'm sure you can make statically-linked C programs which are smaller. For example, the musl libc implementation is quite small and designed for static linkage. But lots of libraries break (often in bizarre and hard-to-debug ways) if they don't link against glibc specifically, so it's not a panacea.

Didn't know this, interesting. But actually, I didn't mean making statically-linked C programs smaller by using leaner libraries like musl. I just meant that for equivalent code, the resulting C binary (even when using the default libc) was smaller by a lot than the Go binary. And so were the Free Pascal and D binaries smaller than the Go binary, but they were both slightly larger than the C binary. But it was only an anecdotal observation based on writing a few small programs in C, Go, Free Pascal, and D, and comparing the resulting binary sizes. Not a thorough scientific study.

>Agreed, but again, Go does allow for dynamic linkage in those cases so you can conserve disk space.

Got it.

>>There's no doubt that if you're golfing, C allows for smaller binaries than Go, but Go can meet 99.9% of real-world requirements when it comes to size (if you're at the extreme end of what Go can support, you've probably already ruled out Go for other reasons).

It wasn't about golfing, it was about equivalent code in the two languages, and the relative sizes of the resulting binaries. But good last point (about 'other reasons').

>There are still many reasons to pick other languages besides Go--if you're working on a hard real-time system, if you're working with a team of people who all prefer a different language, if you're working in a domain (iOS app dev, for example) for which there are no good Go libraries, if you're willing to trade all else for extreme performance or extreme correctness, etc. But Go is actually a pretty good little language for most things that aren't at those extremes.

Agreed. In that sense it is somewhat like Python, which is sometimes described as "the second-best language for almost any domain", an interesting description that I read only somewhat recently. I wouldn't necessarily fully agree with that description, it is probably the best language or at least equally the best (with some other language) for certain areas. Similarly, there are probably areas where Go is, if not the best language, one of the best, for certain kinds of tasks, such as the command-line utilities, web and network server applications that it is often used for.

Re:OCaml for the Masses (2011) by vram22 on Tuesday November 27, 2018 @07:17AM (#18541678)
I googled briefly about dynamic vs. static linking of Go binaries, didn't find much useful info; will try more again later and post again here if I find anything. Will also do some experiments with compiling and linking and see if that gives the needed info.

Update: Found this page which seems to make sense:

https://www.osso.nl/blog/golang-statically-linked/

Re:OCaml for the Masses (2011) by pjmlp on Tuesday November 27, 2018 @02:29AM (#18540261)
Haskell can do value types allocations, not everything is a list.
Re:OCaml for the Masses (2011) by weberc2 on Tuesday November 27, 2018 @11:04AM (#18543438)
Unless 90% of Haskell allocations are value types on the stack, this is irrelevant pedantry.
Re:OCaml for the Masses (2011) by pjmlp on Tuesday November 27, 2018 @11:24AM (#18543656)
Do not optimize what isn't proven by an optimizer.

The features are there, 90% of the cases it doesn't matter for the typical use cases.

When it does matter, they are in reach to any savy Haskell developer.