Note that you still shouldn't use Go as a functional language (using map/reduce/filter); it's not optimized for functional programming, and the syntax would be pretty gnarly.

A for-loop has what's called "mechanical sympathy", and will go through things in the order that it is in memory. It'll allow the compiler and runtime to manage things quickly through memory, CPU registers, and CPU pipelines.

A great post on the subject is "Why Go Getting Generics Will Not Change Idiomatic Go": http://www.jerf.org/iri/post/2955

The bit about "mechanical sympathy" is wrong, though. Plenty of languages inline map/reduce or iterator type code down to very efficient cache-friendly traversal of memory, most notably Rust.

Go's compiler is just particularly naive.

Actually I think while the compiler is a bit naive, the compiler writers are not. They are striving for simplicity, maintainability, speed, etc., vs trying to optimize high-level FP-esque idioms into fast machine code.

The compiler has definitely gotten more optimal over the years, but it's a different beastie than the rust one. Because go is not rust.

It should be noted, though, that rustc does not really have any particular tricks up its sleeve to optimize functional patterns, or indeed much of anything. A lot of the LLVM IR that rustc generates is astonishingly naive and verbose – largely by design! rustc totally depends on LLVM's optimization passes to throw out all the boilerplate and output lean and mean optimized machine code.

Right but that just means it's LLVM being clever, not rustc.

The main compiler is the self-hosting gc go compiler. Iirc ~~it's the reference implementation~~ erm I guess it's determined by spec, not reference implemenation. Gc is definitely "the default" though.

Then There's the gofrontend, which is a compiler frontend to other compilers, which can work with gcc and llvm. I am not aware of the state of the art for these, but I wonder if there are performance differences.

Go is not Rust, but there is a lot it could learn from Rust without sacrifice: sum types, Result for error handling, iterators with for loop support, [edit] non-nullable types etc.

I've wanted a Result type I could map over in Go as soon as I saw the idea. But someone recently pointed out, without pattern matching and early return, it's kind of nerfed. Not sure I totally agree (I think it'd still be awesome to have Result, if only for .map()), but I see their point.

I disagree that it would be without sacrifice. That's kind of the whole point.

Unless I'm missing something, go has iterators with for loops.

Go switches with exhaustion checks like in [1] would go a long way. An early return construct would be important as well, though, you're right.

Go's for loops can only iterate over builtin types: slices, maps, channels, etc. You can't produce a custom type that is to be iterated over.

1. https://github.com/BurntSushi/go-sumtype

Not as simple as it seems. For instance, what is the zero value of a sum type? Let's say, the zero value of a Result<int, error>?

Perfect example of a caveat to "without sacrifice". For context, go really cares about having "zero values" - every initialized variable of some type must have a zero value.

The zero of Error is nil, so logically this would be Result(0, nil). But what do you set the internal state? It's not Ok, cause that int isn't used, and it's not Err either.

Option would work fine though and it would be amazing to map over options instead of nil checks.

I think you'd need to build Result on top of Option, it's naturally kind of ternary - "I have value" "I have error" "idk", either internally or externally.

What might be awesome would be if the Error were a slice type, nil would be "uninitialized" and empty slice means "initialized". still not...pleasant.

Zero values are definitely an issue, and it'd be hard to get rid of them while sticking to Go's imperative/statement-based style. Still, it'd nice to use an Option type instead of pointers.

IIRC in Rust it has to be either of the Ok or Err variants and the enclosed type can then provide its own "zero value" via the standard library "Default" [0] trait or your own implementation for the particular type. That trait is also implemented not for "Result" but for "Option" giving it a default value of "None".

[0]: https://doc.rust-lang.org/std/default/trait.Default.html

Yes, for Option the default value is quite obvious. I'm talking about Result, because it's the second best-known sum type, and it doesn't have an obvious default value.

Bindings in Rust have to initialized with an explicit value if the first use comes before the first modification/assignment otherwise [0] you get an error message with error E0381 [1]. Even if that explicit assignment is just through the use of the type's default method [2], if available.

Since it's implemented for Option<T> you can do it the same way [3] for that, but that's not available for Result<T,U> since the semantics are different from Option which is basically just Rust's alternative for 'null'/'nil' (but verifiable by the compiler).

Thus you'd have to choose what variant of the enum you want to have at initialization, after which you can use the Default implementation of the enclosed type, if available (and so on, if there are more nested types).

Though honestly I can't recall using the kind of pattern in Rust that would "need" something like that. Through the handling of code blocks as expressions, early returns etc. you for example don't need to initialize a variable before having an if-/match-/for-/while-/etc. block, you can simply use that block as an expression on the right side of the assignment [4].

[0]: https://play.rust-lang.org/?version=stable&mode=debug&editio...

[1]: https://doc.rust-lang.org/stable/error-index.html#E0381

[2]: https://play.rust-lang.org/?version=stable&mode=debug&editio...

[3]: https://play.rust-lang.org/?version=stable&mode=debug&editio...

[4]: https://play.rust-lang.org/?version=stable&mode=debug&editio...

In Go, every type has a sensible built-in zero value. In Rust there is no need to worry about what the 'default' value of a Result type should be. So choosing a sensible default value for result would be a problem in Go (but not in Rust).

The zero value of a sum type can be the zero value of the first (or chosen) constituent. In this case, it could be Result<0>.

I'm not sure that would be any better, in practice, than doing return 0, nil.

It would allow functions to be treated generically as returning error-able types. Since some functions even return 3 arguments where the third is the error, this isn't possible today.

It would also allow defer-like syntax to be used on the value a function returns, to return early from the calling function. Rust's ? is an excellent example of this, allowing explicit delegation of error handling that is still very lightweight syntactically.

Function composition becomes a lot easier with Result/Option, because you can write functions in terms of assuming a "good" value (not nil/err or otherwise "bad") and then map over it. Or if we really wanna be fancy, flatmap/bind over it.

Foo, err requires either that all receiving functions accept (foo, err), or manually nil-checking at every stage.

Try/catch is the least composable, because deeper error types can "jump out" from any layer. Which means you have to keep wrapping catches, until you hit some base layer you can recover from. You get stack traces, but you lose type information along the way.

Sum (result) and product (foo, nil) types preserve type information. Meaning you can operate more generically over it. Meaning less code duplication.

Speed and maintainability vs trying to optimize high-level FP-esque idioms into fast machine code is a false dichotomy. OCaml has a really fast compiler, it has been maintained by a small team for 25 years, and it produces fast machine code. A big caveat in comparing OCaml and Go would be multicore support, but it's coming in OCaml, without breaking existing programs and with only a few percent of performance lost.

> Plenty of languages inline map/reduce or iterator type code down to very efficient cache-friendly traversal of memory

Very few languages outside of Rust and Haskell actually do this reliably and that's including languages like Ocaml and F#. Rust and Haskell are certainly the only even remotely popular languages that do it.

Does Haskell actually succeed here?

This is perhaps not the right way to measure, but: When I last checked the programming language benchmark game [0], the Haskell submissions were basically as fast as C/C++/Rust. But, this came at the cost of writing non-idiomatically/non-functionally – essentially abusing the language to write C in Haskell. I would be curious how well "idiomatic" Haskell compares to C++/Rust, in terms of the compiler optimizing away the functional abstractions.

[0] https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

The benchmarks game isn't limited to one Haskell program per task.

Look at both

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

and

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

Thanks, that's pretty convincing.

I think OCaml can do it with flamba, at least that's what this library readme says: https://github.com/c-cube/iter

> with flambda under sufficiently strong optimization flags, such compositions of operators should be compiled to an actual loop with no overhead!

Does the elephant-in-the-room of C++ do it? I thought it does.

I can't bet on what Java and JavaScript JITs make of map / forEach / filter / reduce, but I suspect that they can be pretty efficient, given the clear delineation and the repeated patterns.

(The key phrase is “stream fusion”.)

I love python, and I don't blame you. I can't stand the Django way of doing things. Everything is so opinionated, dynamic, and magical in the bad way. Pytest has a similar vibe and it drives me nuts. Not a fan of OOP, either, I much prefer immutable objects that are either pure data, or convenience methods that return a copy. No more setters, state mutation methods, setattr, unless it's deeply abstracted away.

Nowadays I use Fastapi, pydantic, Result, and static typing everywhere. Protocol interfaces are great.

This!!! I’ve done a fair bit of python api development mostly for ML services. We used flask/gunicorn and I’m considering moving to fastapi. I’m sure Django is awesome for what it does well but python has a lot more options.

Fwiw - I do dabble in Go. I’ve used it for some very targeted systems work where I don’t have to install a python interpreter on the target machine and hacking some k8s operators. I really like Go - but the ML ecosystem is pretty heavily python.

> We used flask/gunicorn and I’m considering moving to fastapi.

Do it, do it nao! And by that, I mean at your earliest convenience. It's such a step up from Flask. All the serde is handled for you via Pydantic. No more manually vetting data. Just write constraints.

> Package management

Are you seriously saying that github.com/somelib is a better package management? Of course nobody forces to use that, but it's what actually been used in most go projects. Even NPM looks better in comparison.

The best quality package management I've seen is Maven, followed by Cargo.

Go has had built-in package management for a while now. It's extremely fast and generates reproducible builds effortlessly in its default behavior in a way I've not experienced before from any other tool.

As somebody currently working on something that has a mix of Node and Go, I definitely prefer github.com/somelib. In that case there's no worry about package name conflicts (everybody has equally terrible names), so you can always end up doing imports with reasonable names because only the last bit matters. With a single namespace, people can (and do) camp on the obvious names, and often they move on and replacement libraries have to have cute names.

This doesn't mean people _will_ name their things sensibly of course. I've seen golang projects with packages named after gardening, meteorology, and aviation… and each is totally confusing when you're new to it, especially when the naming theme is completely unrelated to what the project actually does.

Here maven would be acceptable (similar to go, there is a domain identifier embedded) whereas cargo (bare words) wouldn't be great.

I disagree

for private Go packages all I need is to point it to another git repo. for nuget/maven etc - I need an artifact registry, with it's own bells and whistles

> Are you seriously saying that github.com/somelib ..

Well, yeah, I mean if one can say Maven is best package management then I think I can get away by saying Earth is flat or stationary or some such interesting facts.

To be honest, maven is not the best, but I would take it any day above npm, pip and composer. It's definitely orders of magnitude better than many 'modern' package managers out there.

npm is just another layer on top of git. go uses git directly. how is that any worse?

From someone who briefly used 'bower', which was JS dependency management using git, relying on git is a lot worse.

Github repos can vanish, NPM will only let repos be deleted if they have no direct dependencies also on NPM. So you have some guarantees your dependencies wont vanish.

If the repo was used at least once, it will be cached by the default proxy. At that point, deleting it will have no effect on builds

Does go have some intermediate cache repository between github and enduser?

Yes, https://proxy.golang.org/

You can also configure go to use your own for internal projects or if you just want to avoid the global proxy.

I want to second this. With quick compile times, fat binaries and first class support for crosd compiling, go is suitable for pretty much everything python would be used for. My only use cases for python are as a replacement to bash for shell scripting.

Eh, not really in the data wrangling and machine learning space, at least not quite yet. It's not really amenable to writing libraries for numerical processing - it ends up being super tedious to write something like numpy (you have to implement for every data type, there's at least a dozen+) and nigh-impossible to write an API like pandas, because dataframes are just too dynamic. You could do these things, but it's just super unpleasant.

I'm hoping generics makes it easier to write things like numpy, without doing `meao_u8`, `mean_u16` etc.

+ there's bool, int/uint8,16,32,64, float32/64, complex64/128, that's 13 right there, plus object, a bunch of extended sizes on some architectures, and c-alias types.

That's a fair point - I don't work in the analysis or ml spave so have no understanding of it!

Other people point out ML is a use case where Go isn't suitable. And I agree.

But as someone who used to write a LOT of Python, Go has replaced it for anything service or business-logic oriented, and while I do very little data wrangling and machine learning, Julia would be my go to for that. Julia and Go are in, many ways, complementary opposites. If you need to use PyTorch/TensorFlow, okay, fine Python (I'm not going to seriously advocate for just wrapping it all in PyCall.jl unless it's just incidental usage), but Julia is a really strong language for this stuff nowadays.

Sorry to break it to you, interfaces are OOP and trace back to Objective-C and Common Lisp Object System protocols or BETA patterns.

I'm talking about Go vs Python, not about the origins of OOP.

> I'm done with OOP and dynamic typing.

It is hard to be done with OOP when Go also does OOP, even if in a different way than Python does it.

I like Python the language just fine. Static typing needs a good bit of work, but the real pain points for me have always been performance and tooling (especially package management).

But Go has solid answers for these problems and an excellent static typing story (generics are gravy, but sum types would be nice). I’ve been very happy with Go on balance.

Except when you need to use struct tags, then everything is flimsy. I would rather have C#'s attribute over strings only.

I haven’t had any problem with struct tags personally. I wish Python had some sane way to reliably auto marshal objects to JSON in the stdlib.

I have also left Python for projects where I can chose the tools, but I have not stuck with Go. I went to Racket which has much like Python a "Batteries included" feeling, it is a very easy language with a lot of power and a really good JIT compiler. For me, it is perfect for exploration and algorithm design. For example, I could develop an algorithm in Racket and easily port it to C++11 which was what the client / employer ultimately wanted - still having a much quicker total development time.

For things that are performance-critical or require high reliability, I have successfully used Rust, even if I am still much slower writing it. One thing I like is that the resulting code is not more verbose. Which also means that it is easy to read even after months of not touching it.

That is a strange sentiment, based on the reasons you've given. Package management on Python with pipenv or poetry have not been an issue in my experience. The other half is kinda hard to pinpoint since "feel" doesn't say much, but is surprising given how clear and powerful the syntax is. I've also taken to use type annotation in Python to ease some of the typing obscurantism of pure dynamic typing.

And python does have map / reduce and much more... so I'm not sure what you mean there.

OTOH, I feel Go and Python are mostly used in quite different domains. To me, Python is the new bash. Go is more for those who dislike C++ and Rust (or Haskell or...) and prefer a certain kind of simplicity.

If you don't like Django then you should really try out Flask. It is everything that is right which is wrong about Django and vice versa.

fwiw, I think your experience speaks more to Djanjo and other OOP-inspired packages. Python allows for many different styles of packages to be written... and it's not always a good feature to have.