Garbage collection has become very fast in recent versions of the language. It's quite painless now, and we did struggle with it in the past. I'd take the current GC over dealing with other strategies unless you are a very specific, >60fps kind of situation. We literally don't think about the GC anymore.
Random stats from one of our busier nodes, over the past 20 min: 40GB currently in use, 391GB allocated (and freed) in the last 20min average of 2.6ms GC pauses every 70.6s 95th percentile on GC time is 7.26ms
For dependency management, there's a lot of tools that make things more automated and I think probably offer people what they are expecting/used to from other environments. This has been a sticky point for sure. I've settled on a bare bones solution that seems to work well: just using git submodules in the vendor directory within the project repo. Does everything I need (pin to version of a fork, build immediately after a clone and a submodule update --init). As of Go1.5, the vendor directory is supported by the compiler, and it will look there first to resolve dependencies. While this does work with recursive dependencies, you will want to flatten the dependency graph as much as possible if types are being used across multiple deps because types are distinguished by their full import paths. This encourages breaking things into modules so I'm ok with it.
First-tier support is a problem I have no doubt, but not one we've suffered from personally.
I'd say that debugging comes up a lot too, and Go doesn't have an official, reliable option for debugging. The Delve project is getting close to that though it looks like. I personally debug with the stdlib pprof package (provides an http server that prints stack traces for all goroutines currently active, allows cpu/memory/blocking profiles to be requested), print statements, general testing/experimentation and newrelic.
Edit: other comments below have mentioned some of the tools available to manage your vendor/ tree. Using git submodules manually can be crude, especially when adding dependencies that themselves have other dependencies.
We're (I'm one of the contributors) working on them.
Check them out here: https://github.com/GoogleCloudPlatform/gcloud-golang
Datastore docs: http://godoc.org/google.golang.org/cloud/datastore
Nice business model.
Go is certainly a language that is used at Google, but AFAIK a lot of "Googlers" don't really like it and don't use it. It certainly not the "official language at Google", given the weight of C++ and Java there. But that's the consequence of being opinionated. Using Go means having Rob Pike over your shoulder telling you how to write code. And he made sure you can't escape that fact since there is no place for "ninja coding" with Go.
Go to me feels like they implemented the Google Java and C++ style guides as a language.
Python as well.
>> how little Google seems to care about the language and ecosystem
Let's compare with Microsoft. The top four out of five users at StackOverFlow have top tags in C#, I guess Google have a long way to Go.
Why do you think that WinRT is basically the return of .NET the Ext-VOS project origins, but with .NET metadata instead of COM type libraries?
.NET Native is what Ext-VOS was going to be, if it wasn't for the decision to create the CLR instead.
Where on earth did you hear that? It's simply not true.
(googler)
The archetypal form of cowbody coding is the copy-and-paste: faster than any code reuse technique, but a booby trap for the future.
"Ninja coding", on the other side, refers to coding for cleverness' sake, at the cost of legibility and ease of use. No self-respecting ninja would simply copy-and-paste.
And a completely opposite philosophy from Go when it comes to error handling. Erlang/Elixir embraces failure (and immediately logs and restarts the process); Go seems to ignore it (unless you explicitly check, which to me seems... insane. For reasons why failing fast is better than failing silently, read https://blog.codinghorror.com/whats-worse-than-crashing/ .)
Elixir/Erlang at least have pattern-matching which makes checks like that (from functions that return a non-OK value on errors) basically inlined:
`{:ok, val} = call_some_func(with_args)`
If it doesn't match on the :ok (i.e., an error occurred), you get a match error which gets logged, and the process typically gets killed and restarted by a supervisor process in a millisecond.
Is it that bad now in 1.6 with vendor support? And a tool like `govendor` makes it easy to stick things inside of vendor.
> it just really surprises me how little Google seems to care about the language and ecosystem
To be blunt, google's priority is google, not the open source community or other companies using golang. Dependency management wasn't a priority because of mono repo. Having said that, they are pretty good about improving things for everyone, but it will never be like a company such as Typesafe whose product is the language and tooling itself.
What I would really like to be able to do though is be able to write unmanaged blocks when I need them and know better than the compiler rather than having to "run my own heap" on top of a buffer like I typically have to do in managed languages that don't have that opt out.
That is, you do this if you want a block to run without the GC interrupting it. That is, it's a really high-priority (realtime, or close) block. When it needs to run, it's the most important use of the CPU. But if you demand that the GC makes progress, you're saying that it also is important. And it is, but it's less important than the critical block. If you don't say that, then you wind up with "everything is important", and that way lies madness.
But you may be able to do it. You need enough CPU bandwidth that you can run your critical sections and spend enough time outside them that the garbage collector keeps up. (And then, every time you add to your code, you need to make sure that the CPU still has enough time...)
Do note that it triggers a GC when you call this, so it's not for all situations, but can be useful.
Are you sure? Google offers 2 Google Cloud APIs for Go:
- Google API Client Library for Go, which is auto-generated like you wrote;
- Google Cloud Platform Client Library for Go, which is intended to be idiomatically-designed.
Are idiomatic API libraries really a good thing? If an API is used heavily throughout your application, doesn't that mean it's time to wrap that part of the API in some idiomatic wrapper code that is meaningful in your domain?
Functions are idiomatic in all languages. Personally, I'd rather APIs mostly stick to simple functions, and let the application developer build idioms around them if they like. And if the API call involves a web request, I am happy to just make the HTTP call myself. Half the time you need to understand the interchange format to debug your code anyway. My experience is that wrappers rarely protect you from that.
I guess if your application is 90% calls to an API, and you use a huge range of different functions in that API then you'd want a good, complete library. But how common is that?
Also, have you actually seen the auto-generated APIs he's referring to?
Here are the Godocs for the BigQuery API: https://godoc.org/google.golang.org/api/bigquery/v2
Here's some example code demonstrating the API's bad habits -- such as deeply nested struct pointers, and Do() methods everywhere: https://github.com/google/google-api-go-client/blob/master/e...
It's still labeled "experimental" though, which makes me a bit wary to use it just yet.
If you're OK with changing your code sometime in the future, then I'd recommend giving it a try for this or your next project. Changes will likely be minimal.
Use something like git-vendor: https://github.com/brettlangdon/git-vendor
Idiomatic APIs can be a real PITA, especially when you call them from a different language. KISS should reign supreme. (With bad 80's special effects in a made for TV movie filmed at a carnival.) If you leave it up to the client developer to build the idiomatic facade/interface, then the API is simpler, the code is more properly idiomatic in the end, and everyone is happier.
Source: I've written many servers in both C and Java and given the choice will never use a GC language again for a server. Holding out some hope for Rust..
However I would still favor java instead of go
Could the problem have been the "global interpreter lock"? The GIL makes it impossible for multiple threads to simultaneously perform certain basic operations, effectively making the system single-threaded.
- The Go compiler certainly isn't as sophisticated as the better C compilers. It looks that the code produces by Go 1.7 is quite improves vs. 1.6 with the SSA compiler and I would expect further gains in future releases.
- The Go compiler does not offer optimization levels and overall is optimized for compilation speed to, so these tradeoffs could be limiting.
- While the Go GC is very good and improving, so that the pauses are very low, it does use CPU, also while running in parallel to the program. How much exactly very much depends on the allocation behavior of the program, but should be accounted for.
- 1x benchmark - C / static C++
- 2x slower - pure virtual C++ / objective-c
- 3x slower - statically typed GC languages (java,go)
- +6x slower - dynamically typed GC languages (js, python, etc)
After a well optimized implementation, speed comes down to manual vs GC memory management, static vs virtual function dispatch, dynamic vs static typing and heap vs stack allocation.
Compile speed is usually slowed down by any sort of type inference or templating/generics, which is probably one reason why go resists implementing generics or subclassing
Your rules of thumb are probably mostly related to CPU bound tasks. If database i/o is the bottleneck, then the differences across language probably don't matter much.
And, runtime performance isn't always the most important thing. In many cases, "time to market" or "cost of development" might be a higher order priority. In those cases, a higher level language might be more attractive.
That was the good. The bad? Garbage collection, dependency management, and lack of first-tier support in various libraries. Garbage collection makes the otherwise lightweight and speedy language a memory hog under heavy loads. Not too bad, but I have to kick the memory up on my servers. Dependency management is a nightmare; honestly the worst part about it. The lack of first-tier support in various libraries is a close second. AWS's API libraries had relentless, undocumented breaking changes when we were using them, all on the master branch of their one repo (breaking Golang's guidelines for dependencies). Google itself doesn't actually have any real API libraries for their cloud services. They autogenerate all API libraries for golang, which means they're not idiomatic, are convoluted to use, and the documentation is a jungle.
We continue to use Go because of its strengths, but it just really surprises me how little Google seems to care about the language and ecosystem.