News for hackers. Stuff that matters.
Re:Half a million lines of Go by byronr on Sunday May 16, 2021 @07:32PM (#27178269)
I have written hundreds of thousands of lines of plain old C, as well hundreds of thousands of lines of Go over a career spanning three decades. I've used Go in earnest since 2013.

I would pick Go for any large project in a heartbeat.

I have to concur that the haters who post to HN probably have never used this language in earnest.

Here's what you get out of the box:

* Extensive standard library * Cheap concurrency * Fast compilation * Built-in test/benchmark framework, including coverage and race detection * Built-in heap and CPU profiling

Of all the complaints the one I understand the least is the reservations about performance. It is within 10-20% of C, and callouts to C and assembly are possible. (My 8-way multiplexed MD5 routine was written in AVX assembler and lives on in minio.) Extracting that last 10-20% is two or three sigmas away from the norm of how programming languages are used these days.

The objection to generics is similar. The lack of generics shows up -- once in a long while. It doesn't prevent me from being immensely productive with this language.

Looking back at the last few years I've wondered if I could have accomplished what I did in Go by using a different language (C in particular) and the answer, for me, is "not a chance".

Re:Half a million lines of Go by ferdowsi on Sunday May 16, 2021 @10:10PM (#27179394)
I recently spearheaded the creation of a new Go service at my company. If it was Node or Python I could easily imagine a team getting sucked into weeks of bikeshedding the API framework, the test framework, linting tools, formatting tools, the TLS stack, research spikes on packaging and distribution, etc.

With Go, all of these problems were basically solved by the strength of the standard library. We could not be happier.

Re:Half a million lines of Go by Abishek_Muthian on Sunday May 16, 2021 @10:43PM (#27179589)
> the TLS stack

Networking prowess of Go is often underrated in these discussions. Building an web application with server, router, mux, auth etc. all from standard library and just deploying the binary blob on the cloud (irrespective of the architecture) to get your application running with less effort than competing programming languages/frameworks is itself enough for most use cases.

Re:Half a million lines of Go by theshrike79 on Monday May 17, 2021 @07:20AM (#27182223)
Did the programming on macOS, build machines were Linux and the customer ran the code on Windows.

Zero Go-related issues during the whole project. Just drop in one executable and one config file and it just works.

Re:Half a million lines of Go by takeda on Monday May 17, 2021 @03:07AM (#27180768)
I think having options is just sign of more mature language.

In python you could also implement the API framework, do unit testing, TLS, packaging using standard library. And similarly there are 3rd party frameworks for Go as well.

Re:Half a million lines of Go by loopz on Monday May 17, 2021 @03:32AM (#27180890)
There's always tradeoffs, to be sure. What many are trying to convey is that Go's defaults tend to be the stronger and simpler options.

However, for a lone wolf project, I'd lean more on something like gorm, than just hand-craft and maintain lots of sql.

Another misconception is that it's all for inexperienced programmers. But you need decades of experience to appreciate all the opinionated tradeoffs already made in Go.

It's just one more tool though, and nowhere near FP, which is a different beast altogether. Lots of features around Go and ecosystem is clearly inspired from Haskell. That might be another good tool from the other end of the spectrum.

Re:Half a million lines of Go by takeda on Monday May 17, 2021 @11:58AM (#27184937)
Go is just new, so it had opportunity to build what we already know. Python is older than Java. When it was first released there was no World Wide Web even, not mentioning REST.
Re:Half a million lines of Go by boyter on Sunday May 16, 2021 @08:22PM (#27178668)
Agreed.

I used to use Java because it was fast enough, have good libraries and was productive enough. Go is almost as fast, the standard library has most of what you need and its ability to compile to a single binary makes deployments simple.

I also look back and think, could I have done all that in Java? Probably... but I also feel it would have required more effort.

Re:Half a million lines of Go by stickfigure on Sunday May 16, 2021 @11:01PM (#27179669)
Java has come a long way. It's pretty easy to program in a functional style in Java these days. Go seems to go out of its way to make that hard. Maybe generics will help.
Re:Half a million lines of Go by loopz on Monday May 17, 2021 @03:37AM (#27180912)
Go is more about explicit imperative style, defaulting to good/known performance characteristics. Would advice against trying FP with it, unless just for testing it out.
Re:Half a million lines of Go by byronr on Sunday May 16, 2021 @08:42PM (#27178779)
Go and Java are close cousins.

The difference is of course the JIT -- with Go the single binary means you can disassemble it to find out what it's trying to do. JIT adds a layer of mystery.

Also, with Go it's largely possible to avoid heap allocations in performance hotspots. With Java I'm not so sure.

Re:Half a million lines of Go by ta988 on Sunday May 16, 2021 @09:32PM (#27179158)
But the tools that are available to profile performance, memory etc on the JDK are wonderful.
Re:Half a million lines of Go by tptacek on Sunday May 16, 2021 @08:59PM (#27178896)
It is much easier to reverse engineer compiled Java than Go.

Later

But see below, I think I misconstrued this comment.

Re:Half a million lines of Go by unscaled on Sunday May 16, 2021 @09:37PM (#27179186)
I agree in general, but in this particular case (escape analysis [1]), you won't see what's happening in the ByteCode output. That being said, it's still very easy to see how the code will be JITed by running it through JMH (or just making sure it runs enough time to be optimized). The JVM and IDEs like IntelliJ contain tons of tools to review that.

To get back to the issue at hand, from my personal experience, if you have many small varibales, your app will perform better in Java than Go. The escape analysis in Java is probably more advanced than the one in Go, and in Java you could also use a generational GC if you prefer throughput to latency.

[1] To be more accurate, the actual optimization the JVM does is called Scalar Replacement, and it's not even about allocating variables on the stack, but rather about inlining primitives: https://shipilev.net/jvm/anatomy-quarks/18-scalar-replacemen...

Re:Half a million lines of Go by facorreia on Sunday May 16, 2021 @09:20PM (#27179071)
The argument above is being able to reverse engineer the machine code that will be executed by the processor. Java bytecode isn't that. That's why he said "the JIT adds a layer of mystery".
Re:Half a million lines of Go by tptacek on Sunday May 16, 2021 @09:24PM (#27179103)
Ah, that makes sense.
Re:Half a million lines of Go by sangnoir on Sunday May 16, 2021 @09:31PM (#27179152)
I suspect parent may be better able to divine what the Go compiler intended by looking at the produced assembly, vs. knowing or predicting how the JVM will behave by looking at bytecode/decompiled Java code. Not that it is impossible, it's an additional skill that's bourne of familiarity with Hotspot/Graal/$JVM
Re:Half a million lines of Go by byronr on Sunday May 16, 2021 @09:48PM (#27179250)
It's awesome to see all this Java expertise on this thread but at its essence -- a jar file doesn't define the program that's going to run. It's the union of a jar and the JVM it runs on. With Go, the binary is the whole story.

So even if I can predict how a particular JVM will treat a given jar, I'm still stuck if my code might be deployed across a range of JVMs.

I say all this having written orders of magnitude more Go, with apologies to the Java programmers.

Re:Half a million lines of Go by jolux on Sunday May 16, 2021 @11:18PM (#27179764)
>So even if I can predict how a particular JVM will treat a given jar, I'm still stuck if my code might be deployed across a range of JVMs.

This seems like an odd objection to me. I'm not the biggest Java expert, and I don't currently use it professionally, but I've never written Java that was meant to run on a runtime I not only didn't control but did not know the version of. In theory this was supposed to be a big feature of Java, in practice the community is moving towards bundling the JVM when distributing apps, precisely because versioning is such a fiasco.

Re:Half a million lines of Go by mousepilot on Monday May 17, 2021 @12:45PM (#27185474)
Why not just ask the current jvm what version it is and act accordingly? I mean surely you can advise the user / customer what JVM's your code is compatible with, which really translates to what JVM's you have tested your programming on, right?

I'm obviously surrounded by folks far smarter than me, so I'm posting something probably grossly uninformed to see what the experts think!

Re:Half a million lines of Go by jolux on Monday May 17, 2021 @04:33PM (#27188092)
>Why not just ask the current jvm what version it is and act accordingly?

I would! That's more or less what I was saying. But earlier on in the life of Java, there were technologies like Web Start and applets that could be embedded in webpages and the idea was that they would run on the client much like JavaScript does today, and you would have little idea of what the client environment was like, other than "it's a JVM." That turned out to be a bad idea for a number of reasons, most famously security, and the situations in which you can't just bundle your JVM are dwindling.

Re:Half a million lines of Go by mousepilot on Monday May 17, 2021 @04:53PM (#27188312)
Fair enough! Still, I'm thinking many javascript pages query to find out the browser environment and also act accordingly. I do agree with the security complaint, however I also like web apps.

I do think that you are sort of implying that web applets can't determine which jvm they're running under, is this really the case? Was there any reason that the java webapp folks prevented the applet from inquiring of the java version?

This page chats about the version stuff for applets, it seems even with applets there was some options for picking the java version:

https://docs.oracle.com/javase/8/docs/technotes/guides/deplo...

You've made a nice argument for bundling the jvm so maybe I should change my inclination there, especially since I do like avoiding depencencies.

Re:Half a million lines of Go by jolux on Monday May 17, 2021 @05:29PM (#27188675)
> I do agree with the security complaint, however I also like web apps.

Java applets are basically dead at this point, it’s all JS.

> I do think that you are sort of implying that web applets can't determine which jvm they're running under, is this really the case? Was there any reason that the java webapp folks prevented the applet from inquiring of the java version?

I never wrote applets but the point I was trying to make is that your environment is fundamentally unknown in that context. You could query for different versions and so on, but you would probably never know for sure because you don’t control the runtime environment (everything from the JVM down to the OS). This was supposed to be a big selling point of Java and it has for the most part turned out to be more trouble than it’s worth. That’s all.

Re:Half a million lines of Go by sangnoir on Sunday May 16, 2021 @10:08PM (#27179377)
>... a jar file doesn't define the program that's going to run. It's the union of a jar and the JVM it runs on. With Go, the binary is the whole story.

I was saying the same thing - albeit poorly perhaps. I'm no Java expert either, but once had to download 'jad' off sourceforge to decompile a buggy 3rd party lib :-).

Re:Half a million lines of Go by vips7L on Monday May 17, 2021 @07:49AM (#27182411)
> I'm still stuck if my code might be deployed across a range of JVMs.

Modern Java is meant to be packaged with the JVM via jpackage. You will know exactly what you're running on.

Re:Half a million lines of Go by prerok on Monday May 17, 2021 @01:12AM (#27180259)
> Go is almost as fast

The performance measurements I've seen point to Go being faster. To me it seems also to make more sense since Go is compiled to native code, whereas Java still needs JVM to run the non-native binary.

Can you elaborate in what kind of cases Java had better results?

Re:Half a million lines of Go by boyter on Monday May 17, 2021 @05:15AM (#27181469)
Tight loops usually where it’s really dealing with math and not business logic. The JVM GC was tuned for throughout over latency which may be a factor.
Re:Half a million lines of Go by strken on Sunday May 16, 2021 @11:11PM (#27179731)
The problem with not having generics is the systems you build instead. They either throw away type safety and use reflection, which causes hard to debug issues, or they lean on code generation, which is slow and hard to debug.

I agree that the lack of generics only shows up once in a while and doesn't hamper productivity as much as might be expected, but I'm still very excited to see them added to the language.

Re:Half a million lines of Go by 1_player on Monday May 17, 2021 @04:17AM (#27181140)
> throw away type safety

In theory yes, in practice if your Sort array function takes a list of interface{} instead of a well defined type is not going to cause any type bugs. Might be slow, might be ugly, but lack of types on functional/generic stdlib functions doesn't make your code buggier if the original implementation is sound.

Re:Half a million lines of Go by theshrike79 on Monday May 17, 2021 @07:24AM (#27182252)
I'm cautiously optimistic about the addition of generics to Go.

If all goes well, we'll get some nice stdlib stuff and 3rd party functions for handling different kind of data with a common api.

But if the - you know what kind of people - discover it in earnest, we'll get a hellscape of C++ templating proportions when people go way overboard with it.

Time will tell.

Re:Half a million lines of Go by auxym on Sunday May 16, 2021 @07:56PM (#27178471)
Sincere question out of curiosity, since I have not been exposed to Go at all.

How do container types, like arrays or hashmaps work without generics? Do you basically have to declare and implement types/methods for each different contained type?

Re:Half a million lines of Go by SiVal on Sunday May 16, 2021 @09:03PM (#27178932)
I hardly ever need generics directly. If I have to write an algorithm for 2 types, I can copy and tweak as easily as I can deal with the added complexity of programming with abstract types.

BUT, I need generics all the time indirectly. I've worked in lots of languages over the decades, and I highly value languages with mature/debugged/optimized standard libraries of algorithms and data types (esp. "containers").

Go has an impoverished subset "built-in", and that's all it offers.

One of the best things about Go is the high quality of its standard libraries--where it has standard libraries--so there is a gaping hole where the container and algo libraries ought to be. Experts in these algorithms could take advantage of Go's concurrency for example to write library algorithms that provide a combination of performance and safety that I couldn't create quickly if at all. They could, that is, if they had generics in the language, but they don't, so I'm on my own (for now).

When people (repeatedly) claim that Go doesn't really need generics because they (the claimants) hardly ever need generics, I have to wonder whether their knowledge of DS and algos is especially high or especially low: that they could whip up a debugged and optimized algo as easily as calling it from a library or whether they aren't even aware of the DS/algo that they really ought to be using.

Re:Half a million lines of Go by catlifeonmars on Monday May 17, 2021 @01:57AM (#27180454)
Personally, it’s very rare that I need to reach for a specialized algorithm. I’m not sure if that’s primarily the problem space I work in (services providing user identity and access management), but I never find the need to go beyond a brute force search or need a data structure other than an array or built in hash map (sometimes a concurrent hash map). Most of the optimization I do involves eliminating network calls and implementing caching.
Re:Half a million lines of Go by void_mint on Sunday May 16, 2021 @08:56PM (#27178872)
Others have answered, but I just wanted to add, your question is usually where beginners/non Go users start to pearl-clutch. I would advise, if you're curious, to give it a shot for a while. As with all new things, try to not just write the language you're used to, but in the syntax of the language you're using, and instead embrace patterns that exist in a given language.

Lots of people find the things you're concerned about missing to not really be the end of the world.

Or you might hate it, and that's also okay.

Re:Half a million lines of Go by zeugmasyllepsis on Sunday May 16, 2021 @08:16PM (#27178628)
Go supports a handful of built-in generic types, like hashmaps. What Go currently lacks is user-defined generics, though that is actively being worked on.
Re:Half a million lines of Go by jimsmart on Sunday May 16, 2021 @08:05PM (#27178535)
Arrays/slices and maps are in fact provided as built-ins that function like generics, e.g. []foo and map[foo]bar work as expected, as do associated helper methods.

But those are the only container types with built-in support like that. (Channels can similarly be of any type)

Re:Half a million lines of Go by takeda on Monday May 17, 2021 @03:09AM (#27180784)
They use hack of "interface{}" which does type checking at runtime (slow). And you then cast it to a proper type.

If I remember correctly (haven't used it in a year) Go cheats a bit, and it has few functions in standard library that do have generics (like make), it just doesn't provide this functionality to the developer.

Re:Half a million lines of Go by loopz on Monday May 17, 2021 @03:57AM (#27181018)
If you can avoid interface{} you should, ie. by declaring interfaces to types and using them explicitly. You can then mix your own types, and still be protected by the type system.
Re:Half a million lines of Go by takeda on Monday May 17, 2021 @12:10PM (#27185083)
Yes, but that requires generics and without them code generators (and if you don't use them, a lot of copy & paste).
Re:Half a million lines of Go by tptacek on Sunday May 16, 2021 @08:30PM (#27178712)
Roughly the same way containers work in Python.
Re:Half a million lines of Go by dragonwriter on Sunday May 16, 2021 @08:52PM (#27178843)
> Roughly the same way containers work in Python.

That’s not really true, since built-in containers are special sui generis generics in Go, whereas in an untyped python they get by being untyped, and in python’s statically-checked type system (as implemented in pyright, mypy, etc.) they use general-purpose generics.

Re:Half a million lines of Go by tptacek on Sunday May 16, 2021 @08:54PM (#27178858)
That complaint is as old as the language --- the first piece about Go I ever read, after my friend Yan convinced me to pick up Go, called it out for having "generics for me and no generics for thee". I've never understood why I'm meant to care. I get, if you want to write your own generic containers, being irritated that there aren't universally available generics (yet). I don't get being upset that the language has some special-cased generics. It's not personal. They're not rubbing salt in your wounds. They just need a map type that works.

At any rate: the person asking the question is wondering how you manage to have performant containers with reasonable interfaces in a language without generics. That's a good question! If you've worked a lot in C, you might be imagining a horrible mess of void-stars or something. But no, the ergonomics are basically those of Python, with extra typing.

Re:Half a million lines of Go by joshuamorton on Sunday May 16, 2021 @09:15PM (#27179033)
Given that they stated "Do you basically have to declare and implement types/methods for each different contained type?" in the comment, I don't think they were wondering if you end up using void*/interface{}.

I really don't understand how the python comparison makes sense. Since those particular containers are special cased, the ergonomics are also the same as java (and I'd argue more like java than python). Other than ownership, I don't think go's maps are any more python like than C++s or Rusts either.

> reasonable interfaces in a language without generics

and with static typing. Python doesn't fall into that category. It either doesn't have static types, or does have generics. The answer (as others have mentioned) is that go does support generics for a few particular container types.

Re:Half a million lines of Go by dragonwriter on Sunday May 16, 2021 @10:39PM (#27179566)
> That complaint is as old as the language

I’m not making a complaint, I’m pointing out a fact without a value judgement.

> At any rate: the person asking the question is wondering how you manage to have performant containers with reasonable interfaces in a language without generics.

Uh, I don’t see either “performant” or “with reasonable interfaces” there, the question is: “How do container types, like arrays or hashmaps work without generics? Do you basically have to declare and implement types/methods for each different contained type?”

The answer is, for built-in containers, “no, because they use special-purpose generics”. For custom containers, its basically “yes”, but you probably only want a single contained type with an appropriate interface anyway. In neither case is “basically the same as Python” particularly accurate, though I suppose the former case is similar to typed python using generics in the same way as any language with generics would be, while the latter is loosely similar to bare Python or typed Python using protocols instead of generics. But only loosely.

Re:Half a million lines of Go by masklinn on Monday May 17, 2021 @04:42AM (#27181290)
> The answer is, for built-in containers, “no, because they use special-purpose generics”.

Unless you want to implement utility helpers working on them then it’s yes again, because Go doesn’t have generic functions either, so while it does have a few builtin generic containers, you can’t operate genetically over them.

Re:Half a million lines of Go by tptacek on Monday May 17, 2021 @12:18AM (#27180008)
Yeah, I'm choosing not to die on this hill. I'm just saying that Go has first-class table and resizable array ADTs.
Re:Half a million lines of Go by masklinn on Monday May 17, 2021 @04:45AM (#27181306)
They’re not first class though: you can not take in, return, or work with the ADTs, only with concrete instances of them.
Re:Half a million lines of Go by tptacek on Monday May 17, 2021 @03:30PM (#27187345)
Hey, there, I'm over on this other hill now, not dying on that one. :)
Re:Half a million lines of Go by YZF on Sunday May 16, 2021 @10:02PM (#27179340)
I've also been mostly using Go over the last 6-8 years. Before that it's mostly been C++, pretty "modern" towards the end.

Go would definitely be my preferred pick for distributed/server applications even though the previous large (mixed C and C++) project I was on was a large distributed application and we did just fine. In fact I'd say our productivity and quality in C++ in that other team was just as good as any I've seen (which is not a random/double blind sort of thing, who knows if that project started in Go from day 1).

I would say that in a large project the language choice has some impact but there are many other variables. A well managed C++ project with competent developers can work well. A less than well managed with so-so developers in Go can be disaster zone. There are lots of other moving pieces, testing, build automation, CI/CD etc.

There are certain areas where I would prefer not to use Go. Basically areas where you spend a lot of time up or down the abstractions. I would prefer not to implement a video codec or a signal processing system in Go. Firmware, motion control etc. would also probably not be a good fit. Going the other direction there are things I can do really quickly in Python (lessay grab some data from some services, do some manipulation, draw some graphs), generally smaller utility like pieces where I want to use data with various random types, use third party libraries. I wouldn't want to write a large project in Python because it's slow and is dynamically typed.

I would also beg to differ wrt/ 10%-20% performance difference. You tend to rely a lot more on GC and produce more "garbage" in Go and the optimizer isn't as good vs. the best C/C++ compilers, I'd say it's more like 50%-150%. But for a lot of stuff it doesn't matter. If you're just making database queries then most of your CPU is sitting there anyways and the delta on the caller side isn't a factor.

Go is pretty nice in it's zone. It's got rough edges but all in all it's fun to use. There are occasional annoyances (like those times where you implement another cache for the 10th time because you can't build a good generic cache or those times where you need a priority queue and interact with those clunky library implementations, so yeah, I'm in the generics fan club). But that hasn't stopped me from using and loving Go. I don't miss those C++ core dumps ;) or dealing with memory management (even with smart pointers and containers it can be a pain). Go's more dynamic aspects (reflection etc.) also make some things easier vs. C++. People can learn Go really quickly which is another bonus, learning C or C++ can take years.

Re:Half a million lines of Go by theshrike79 on Monday May 17, 2021 @07:30AM (#27182287)
If you can find a team of competent C++ programmers, most definitely you should go with C++ for all tasks.

But if you've got a pool of people who have never touched either C++ or Go, I'd pick Go for the project. It's orders of magnitude easier to teach generic programmers the basics of writing decent Go than C++.

Re:Half a million lines of Go by byronr on Sunday May 16, 2021 @10:29PM (#27179514)
100% on the team making the most difference. (An aside: a software project I've been admiring recently is the Apollo Guidance Computer's. To the moon and back on a 15-bit 1s complement machine and a few kb of memory. It's a case where the team was certainly better than the available tools.)

On the perf side: I'm curious to know where Go sits now relative to C assuming you elide away all heap allocations in your inner loops. In the early days Go's codegen was based on plan9's simple and simplistic C compiler's back end. Things have gotten much better since then and to my knowledge it's within striking distance of C. But I could be wrong.

Re:Half a million lines of Go by mohbk on Monday May 17, 2021 @07:31AM (#27182293)
there's a number of benchmarks you can checkout online.

number crunching wise, Go is on par with Java and C#: https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

for webdev work, the standard library might not be that fast, but it's fast enough, and if you need even more speed the fasthttp third party Implementation offers good speed: https://www.techempower.com/benchmarks/

Re:Half a million lines of Go by YZF on Monday May 17, 2021 @10:14PM (#27190616)
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

Sort of supports my pulled out of thin air numbers (50-150%). In C/C++ you got those lovely SSE intrinsics ;) But look at the binary tree entry as well, that sort of x13 win for C++ really shows you how C++ lets you get sheer power with decent abstractions.

Re:Half a million lines of Go by froh on Monday May 17, 2021 @02:06AM (#27180490)
should we add the amazing declarative marshalling and unmarshalling to/from json and xml and possibly others? or is that part of the interaction with the outside world, ffi and marshalling in one bag?
Re:Half a million lines of Go by masklinn on Monday May 17, 2021 @04:47AM (#27181320)
Isn’t it stringly typed, full of runtime introspection and rather slow?
Re:Half a million lines of Go by froh on Monday May 17, 2021 @10:40AM (#27183957)
sure it is. however isn't most xml or json marshalled communication I/O bound? so even though it could be faster it also is "fast enough"?
Re:Half a million lines of Go by masklinn on Monday May 17, 2021 @02:06PM (#27186265)
It depends a lot how you're getting it and what you're doing with it. The existence of projects like rapidjson, simdjson, … indicates that even without GTAV-style mistakes, JSON enc/dec can absolutely be your limiting factor. Even more so if you're on a local network (e.g. between machines in the same DC) where bandwidth is… essentally infinite, for all intents and purposes? When you've got 10Gb/s or even 100Gb/s between your racks, odds are IO is only the limiting factor for very short messages (lots of framing overhead).

As for XML, it's an absolute bear to encode and decode, documents are often large (dozens, hundreds, even thousands of GB), and the format is complicated enough that writing truly fast parsers is difficult. Being CPU-bound when XML is involved is routine, especially if you need a DOM.

Re:Half a million lines of Go by Siira on Sunday May 16, 2021 @11:49PM (#27179886)
It’s not that Go is not a great language; The problem is that it’s a very conservative language that is not picking up low-hanging fruit. Their maintainers also have a nasty habit of being condescending to other people’s use cases; E.g., f-strings, unix shebang as comment, ... .