News for hackers. Stuff that matters.
Re:Program your next server in Go by fpgaminer on Tuesday June 07, 2016 @02:12PM (#11856412)
All of the server backends at my company are written in Go. This was a result of me writing a couple servers in Python a few years back, ending up with lots of problems related to hanging connections, timeouts, etc. I tried a couple different server libraries on Python but they all seemed to struggle with even tiny loads. Not sure what was up with that, but ultimately I gave Go a swing, having heard that it was good for server applications, and I haven't looked back. It has been bullet proof from day one and I am overall happy with the development experience.

That was the good. The bad? Garbage collection, dependency management, and lack of first-tier support in various libraries. Garbage collection makes the otherwise lightweight and speedy language a memory hog under heavy loads. Not too bad, but I have to kick the memory up on my servers. Dependency management is a nightmare; honestly the worst part about it. The lack of first-tier support in various libraries is a close second. AWS's API libraries had relentless, undocumented breaking changes when we were using them, all on the master branch of their one repo (breaking Golang's guidelines for dependencies). Google itself doesn't actually have any real API libraries for their cloud services. They autogenerate all API libraries for golang, which means they're not idiomatic, are convoluted to use, and the documentation is a jungle.

We continue to use Go because of its strengths, but it just really surprises me how little Google seems to care about the language and ecosystem.

Re:Program your next server in Go by courtf on Tuesday June 07, 2016 @02:55PM (#11856809)
We run a cluster of P2P, GPU heavy machines that use Go to ingest byte streams of raw radar data, store that info in btrees, and render, cache & serve map tiles that are drawn on the fly in response to http requests. We are not using much outside the stdlib (opengl and gdal bindings).

Garbage collection has become very fast in recent versions of the language. It's quite painless now, and we did struggle with it in the past. I'd take the current GC over dealing with other strategies unless you are a very specific, >60fps kind of situation. We literally don't think about the GC anymore.

Random stats from one of our busier nodes, over the past 20 min: 40GB currently in use, 391GB allocated (and freed) in the last 20min average of 2.6ms GC pauses every 70.6s 95th percentile on GC time is 7.26ms

For dependency management, there's a lot of tools that make things more automated and I think probably offer people what they are expecting/used to from other environments. This has been a sticky point for sure. I've settled on a bare bones solution that seems to work well: just using git submodules in the vendor directory within the project repo. Does everything I need (pin to version of a fork, build immediately after a clone and a submodule update --init). As of Go1.5, the vendor directory is supported by the compiler, and it will look there first to resolve dependencies. While this does work with recursive dependencies, you will want to flatten the dependency graph as much as possible if types are being used across multiple deps because types are distinguished by their full import paths. This encourages breaking things into modules so I'm ok with it.

First-tier support is a problem I have no doubt, but not one we've suffered from personally.

I'd say that debugging comes up a lot too, and Go doesn't have an official, reliable option for debugging. The Delve project is getting close to that though it looks like. I personally debug with the stdlib pprof package (provides an http server that prints stack traces for all goroutines currently active, allows cpu/memory/blocking profiles to be requested), print statements, general testing/experimentation and newrelic.

Re:Program your next server in Go by jre on Wednesday June 08, 2016 @03:42AM (#11860658)
What company are you working at ? Looks like you do some interesting geospatial/map rendering stuff. Are you hiring ? :-)
Re:Program your next server in Go by dalailambda on Tuesday June 07, 2016 @04:42PM (#11857642)
How do you find the cgo overhead? Specifically with something like OpenGL, it seems like it would be a big pain point for rendering.
Re:Program your next server in Go by courtf on Tuesday June 07, 2016 @05:33PM (#11858075)
I don't have great measurements for this, but we have done optimizations to reduce the data flowing over the Go<->C interface. One of our key measurements that does make a big impact on performance is how often we need to upload data (that's not already over there) to the GPUs. So that's something we have worked on reducing (buffer reuse, compression). We also have a series of caches on the other side, so we aren't drawing more than we need to. It's hard for me to tease apart how much of these optimizations (and others) are ultimately aimed at addressing the cgo overhead, and how many are just typical stuff. The data we work with is cumbersome and my intuition is that there's probably a lot of room for optimization in our drawing even still, regardless of cgo. I wouldn't be surprised if a direct port C/C++ implementation of the rendering pipeline was significantly faster than ours in getting data into and out of the GPUs, but a big part of the project is data storage/networking/serving/caching as well and Go has bridged the gap for us (a small team that needs to build reasonably fast things reasonably quickly :)).
Re:Program your next server in Go by dalailambda on Tuesday June 07, 2016 @06:43PM (#11858596)
That's interesting. The cgo overhead was the only thing holding me back from considering it for games, since I didn't want to write a lot of C wrappers around the C libraries I want to use just to have them be more efficient, which is a shame, since Go is pretty nice, barring the C interop in some cases.
Re:Program your next server in Go by pjmlp on Wednesday June 08, 2016 @05:18AM (#11860985)
When I used to frequent gonuts, I raised the issue why they didn't went the FFI way as D, Rust, .NET, Delphi, FreePascal do, but sadly they rather use cgo as solution.
Re:Program your next server in Go by dalailambda on Wednesday June 08, 2016 @05:27AM (#11861024)
I'm pretty sure the problem is Go's use of segmented stacks and runtime, which makes it hard to properly interface with C efficiently.
Re:Program your next server in Go by pjmlp on Wednesday June 08, 2016 @06:01AM (#11861148)
Sure, but the compiler could generate that FFI code, why the need to rely on an external tool (cgo) and a C compiler?
Re:Program your next server in Go by slimsag on Tuesday June 07, 2016 @09:49PM (#11859468)
FWIW I did measure the CGO call overhead a while back: https://github.com/slimsag/cgo-batching
Re:Program your next server in Go by mbgaxyz on Tuesday June 07, 2016 @06:54PM (#11858663)
Would the vendor management support deterministic builds?
Re:Program your next server in Go by ereyes01 on Tuesday June 07, 2016 @07:00PM (#11858707)
Yes, that is the purpose of the vendor directory. You copy the source of each of your dependencies into vendor/ and commit it to your own source control so it can't change unless you do so yourself. I personally use git submodules to manage the contents of the vendor/ tree, but the go compiler/toolchain itself doesn't really care how you get the files into there.

Edit: other comments below have mentioned some of the tools available to manage your vendor/ tree. Using git submodules manually can be crude, especially when adding dependencies that themselves have other dependencies.

Re:Program your next server in Go by aj0strow on Tuesday June 07, 2016 @07:04PM (#11858730)
Yes. I check my vendor into git, so it's as deterministic as source code.
Re:Program your next server in Go by chrisbroadfoot on Tuesday June 07, 2016 @03:03PM (#11856883)
> Google itself doesn't actually have any real API libraries for their cloud services.

We're (I'm one of the contributors) working on them.

Check them out here: https://github.com/GoogleCloudPlatform/gcloud-golang

Datastore docs: http://godoc.org/google.golang.org/cloud/datastore

Re:Program your next server in Go by mattetti on Tuesday June 07, 2016 @02:55PM (#11856816)
We've been running splice.com on Go for 3 years now and handle 5TB of audio/binary data per day. Our memory usage is around 10-15MB per server and the GC pause time has been really low. You do need to stream your IOs instead of reading everything in memory. In regards to dependency management, we honestly had no issues and now with vendoring, it's even easier. We do use a main repo with lots of smaller packages and only a few 3rd party dependencies that are vendored or available via private git repos. We don't use Google cloud but I heard they have 2 repos, one that has auto generated code and one that has hand written code (but less complete).
Re:Program your next server in Go by kkirsche on Tuesday June 07, 2016 @06:31PM (#11858522)
Could you elaborate on what you mean by stream your IO? Is this just using buffered io it is there a concept I could read up on here?
Re:Program your next server in Go by mattetti on Tuesday June 07, 2016 @07:22PM (#11858843)
yes buffered io, you use readers to read/write small chunks at a time instead of loading everything at once. Go offers way to do both since in some cases, such as loading an entire file in memory can be fine/better/faster.
Re:Program your next server in Go by p0nce on Wednesday June 08, 2016 @03:09AM (#11860557)
OT about splice.com: how do you version control DAW projects, you have a custom parser + text diff? Would you support Orion and Reaper? :)
Re:Program your next server in Go by cheez on Tuesday June 07, 2016 @03:41PM (#11857176)
Your tagline should be: splice.com - no more sample packs. Haha.

Nice business model.

Re:Program your next server in Go by spriggan3 on Tuesday June 07, 2016 @02:19PM (#11856490)
> We continue to use Go because of its strengths, but it just really surprises me how little Google seems to care about the language and ecosystem.

Go is certainly a language that is used at Google, but AFAIK a lot of "Googlers" don't really like it and don't use it. It certainly not the "official language at Google", given the weight of C++ and Java there. But that's the consequence of being opinionated. Using Go means having Rob Pike over your shoulder telling you how to write code. And he made sure you can't escape that fact since there is no place for "ninja coding" with Go.

Re:Program your next server in Go by kyrra on Tuesday June 07, 2016 @11:13PM (#11859738)
Google has some fairly specific style guides for all languages used: https://github.com/google/styleguide/blob/gh-pages/README.md

Go to me feels like they implemented the Google Java and C++ style guides as a language.

Re:Program your next server in Go by dtamhk on Tuesday June 07, 2016 @04:35PM (#11857569)
>> given the weight of C++ and Java there.

Python as well.

>> how little Google seems to care about the language and ecosystem

Let's compare with Microsoft. The top four out of five users at StackOverFlow have top tags in C#, I guess Google have a long way to Go.

Re:Program your next server in Go by rcarmo on Wednesday June 08, 2016 @04:03AM (#11860738)
Erm. StackOverflow was built by folk who used C# heavily, and cargo-culting is amazingly prevalent on both the C# and Java "traditional IT" communities, so _of course_ StackOverflow is biased towards those runtimes...
Re:Program your next server in Go by Pxtl on Tuesday June 07, 2016 @08:00PM (#11859034)
And yet the os group at ms is famous for their naked derision of the .net Framework.
Re:Program your next server in Go by pjmlp on Wednesday June 08, 2016 @07:45AM (#11861530)
Because of political wars that lead the .NET team to base the Framework on the CLR instead of the COM+ Runtime that was being prototyped.

Why do you think that WinRT is basically the return of .NET the Ext-VOS project origins, but with .NET metadata instead of COM type libraries?

.NET Native is what Ext-VOS was going to be, if it wasn't for the decision to create the CLR instead.

Re:Program your next server in Go by dtamhk on Wednesday June 08, 2016 @12:04AM (#11859928)
Hum .Net is their own dog food .. did you mean Android/iOS? :-D
Re:Program your next server in Go by dtamhk on Wednesday June 08, 2016 @04:09AM (#11860750)
What I mean was that M$ have competing offer to Android/iOS but it is not getting the upper hand even within their own organization.
Re:Program your next server in Go by usmannk on Wednesday June 08, 2016 @03:00AM (#11860525)
Python isn't nearly at the level of C++ or Java there, at least not any more.
Re:Program your next server in Go by skybrian on Tuesday June 07, 2016 @02:27PM (#11856574)
It is one of the official languages. Google is too big a company to have a single official language.
Re:Program your next server in Go by nickbauman on Wednesday June 08, 2016 @12:21AM (#11859995)
This slideshow clinches it for me. Go has some specific strengths that match low level, network infra types of problems. Beyond that, Go is not a good fit. This sounds like a criticism, but I don't think so. It's a compliment: it's a sharp tool for a specific kind of cutting. It's not trying to be some all-singing all-dancing language, which has gotten us all in to quite a bit of trouble.
Re:Program your next server in Go by amaks on Wednesday June 08, 2016 @12:49AM (#11860103)
That's exactly right, it looks like Go has found a niche of the network servers. Here is good overview by Andrei Alexandrescu: https://www.quora.com/Which-language-has-the-brightest-futur...
Re:Program your next server in Go by skj on Tuesday June 07, 2016 @09:34PM (#11859404)
> but AFAIK a lot of "Googlers" don't really like it and don't use it

Where on earth did you hear that? It's simply not true.

(googler)

Re:Program your next server in Go by rpgmaker on Tuesday June 07, 2016 @11:16PM (#11859751)
Wow. Is Google so homogeneous that you both can be so sure about your respective affirmations?
Re:Program your next server in Go by matthewmacleod on Wednesday June 08, 2016 @07:39AM (#11861516)
I must admit that I heard exactly the same from multiple Google engineers, who were all quite dismissive of it.
Re:Program your next server in Go by mseepgood on Tuesday June 07, 2016 @02:23PM (#11856539)
What is "ninja coding"? Sounds stupid.
Re:Program your next server in Go by nameless912 on Tuesday June 07, 2016 @02:29PM (#11856593)
I think it refers to the Perl-era idea that good code should be somehow "clever" rather than maintainable. It's how bad programmers who spend hours agonizing over how to reduce their line count (presumably to save disk space?) justify their behavior.
Re:Program your next server in Go by spinlock on Tuesday June 07, 2016 @04:56PM (#11857752)
Personally, I think "cowboy coding" is the best derogatory name for this. IMO, what all of these ninjas have in common is that they don't realize that software development is a team exercise. Berkeley did a study on BSD and found that a file was opened 10x more often for reading that writing (i.e. people read code 10 times for every time they make a change). To my mind, "cowboy" conveys the proper amount of ignorance of the other people on your "team."
Re:Program your next server in Go by piaste on Wednesday June 08, 2016 @05:11AM (#11860954)
"Cowboy coding" already has a distinct meaning - it refers to writing code as fast as possible without concerns for technical debt.

The archetypal form of cowbody coding is the copy-and-paste: faster than any code reuse technique, but a booby trap for the future.

"Ninja coding", on the other side, refers to coding for cleverness' sake, at the cost of legibility and ease of use. No self-respecting ninja would simply copy-and-paste.

Re:Program your next server in Go by moriox on Tuesday June 07, 2016 @07:20PM (#11858832)
I totally agree with this sentiment. I think it's pretty well established at this point that in many cases clarity can trump optimization. Even more so when the optimization isn't performance based but rather line count based or "code golf" based. Also, do you happen to have a link to that berkley study? I can't seem to find it with a brief search.
Re:Program your next server in Go by vegabook on Wednesday June 08, 2016 @04:02AM (#11860734)
Or, it is how some programmers continue to find motivation and fulfillment in the activity as an intellectual pursuit, yes at the cost of maintainability, but as opposed to being good little corporate drones writing a zillion dull lines of kindergarten-obvious code in Golang and Java that even the lowest decile of programming dunce can understand. Which is what Golang is designed to do. That's what they mean by optimization for large code bases: dumbing down. That's not a criticism by the way because that's optimal for large teams in large companies with large code bases and no room for too much creativity. But programming as art/fun, where appropriate, is also not (yet) completely to be dismissed as stupid, IMO.
Re:Program your next server in Go by wyager on Tuesday June 07, 2016 @04:37PM (#11857581)
For optimal code maintainability, there is a happy compromise between excessively terse and excessively verbose code. There is a lot of code in the wild that's too spread out.
Re:Program your next server in Go by nameless912 on Tuesday June 07, 2016 @07:05PM (#11858737)
Well sure, but erring on the side of verbosity at least ensures that someone can follow your thinking as long as the code is well structured. I would prefer to read 25 lines of decent Python over 3 lines of regexes in Perl.
Re:Program your next server in Go by Drdrdrq on Wednesday June 08, 2016 @04:14AM (#11860765)
Sure, but at least I would prefer 25 lines of decent Python over 125 lines of overly verbose Java. Nothing is black and white, it is easy to make code too verbose and thus very taxing for the reader.
Re:Program your next server in Go by pmarreck on Tuesday June 07, 2016 @09:44PM (#11859447)
Have a look at Elixir and Phoenix. GC is per-process so no global slowdown. Extremely fast response times and uptimes. Many other features that mesh nicely with webserving (some courtesy of Erlang's VM).

And a completely opposite philosophy from Go when it comes to error handling. Erlang/Elixir embraces failure (and immediately logs and restarts the process); Go seems to ignore it (unless you explicitly check, which to me seems... insane. For reasons why failing fast is better than failing silently, read https://blog.codinghorror.com/whats-worse-than-crashing/ .)

Re:Program your next server in Go by zalmoxes on Tuesday June 07, 2016 @10:05PM (#11859527)
To any Go programmers ignoring errors is insane. I haven't seen code outside of simple examples ignore errors, and even there it's discouraged.
Re:Program your next server in Go by pmarreck on Thursday June 09, 2016 @03:44PM (#11871795)
But you still have to check for errors after every line of code where they are possible (if you're covering all the bases), no? Due to lack of a traditional exception model?

Elixir/Erlang at least have pattern-matching which makes checks like that (from functions that return a non-OK value on errors) basically inlined:

`{:ok, val} = call_some_func(with_args)`

If it doesn't match on the :ok (i.e., an error occurred), you get a match error which gets logged, and the process typically gets killed and restarted by a supervisor process in a millisecond.

Re:Program your next server in Go by nvarsj on Tuesday June 07, 2016 @03:22PM (#11857026)
> Dependency management is a nightmare

Is it that bad now in 1.6 with vendor support? And a tool like `govendor` makes it easy to stick things inside of vendor.

> it just really surprises me how little Google seems to care about the language and ecosystem

To be blunt, google's priority is google, not the open source community or other companies using golang. Dependency management wasn't a priority because of mono repo. Having said that, they are pretty good about improving things for everyone, but it will never be like a company such as Typesafe whose product is the language and tooling itself.

Re:Program your next server in Go by fideloper on Tuesday June 07, 2016 @03:36PM (#11857134)
is godep relevant still, or is 1.6 making something like govendor a "better" way to go?
Re:Program your next server in Go by rosadoft on Tuesday June 07, 2016 @04:47PM (#11857672)
I switched to Glide, a vendor-focused package manager, on 1.5. I think the community has already decided in favor of this approach. Post 1.5, Godeps' workflow just feels weird and unnecessary.
Re:Program your next server in Go by themartorana on Tuesday June 07, 2016 @05:15PM (#11857917)
Use godep everywhere. It supports the vendor folder, and I find it's still a great solution (and saves me using submodules).
Re:Program your next server in Go by sdboyer on Friday June 10, 2016 @01:03PM (#11877818)
it is, and works well enough for individuals, but doesn't make for the friendliest ecosystem.
Re:Program your next server in Go by inglor on Tuesday June 07, 2016 @02:19PM (#11856489)
To be fair, Go's GC got A LOT better over the past few versions and it's getting much better still.

What I would really like to be able to do though is be able to write unmanaged blocks when I need them and know better than the compiler rather than having to "run my own heap" on top of a buffer like I typically have to do in managed languages that don't have that opt out.

Re:Program your next server in Go by weberc2 on Tuesday June 07, 2016 @04:12PM (#11857413)
Have you considered pausing the GC for those blocks? Or am I misunderstanding your use case?
Re:Program your next server in Go by infogulch on Tuesday June 07, 2016 @04:30PM (#11857534)
How do you do this? I didn't know there was a way to temporarily disable the GC for specific blocks.
Re:Program your next server in Go by weberc2 on Wednesday June 08, 2016 @12:13AM (#11859962)
You can toggle the GC with https://golang.org/pkg/runtime/debug/#SetGCPercent. Turn it off before you enter your block and enable it again at the end of your block.
Re:Program your next server in Go by amelius on Wednesday June 08, 2016 @05:20AM (#11860991)
But can you guarantee that the GC actually makes progress that way?
Re:Program your next server in Go by AnimalMuppet on Wednesday June 08, 2016 @03:55PM (#11865122)
Maybe you can, but maybe you shouldn't.

That is, you do this if you want a block to run without the GC interrupting it. That is, it's a really high-priority (realtime, or close) block. When it needs to run, it's the most important use of the CPU. But if you demand that the GC makes progress, you're saying that it also is important. And it is, but it's less important than the critical block. If you don't say that, then you wind up with "everything is important", and that way lies madness.

But you may be able to do it. You need enough CPU bandwidth that you can run your critical sections and spend enough time outside them that the garbage collector keeps up. (And then, every time you add to your code, you need to make sure that the CPU still has enough time...)

Re:Program your next server in Go by weberc2 on Wednesday June 08, 2016 @01:08PM (#11863766)
What does it mean for a GC to "make progress"? My understanding was that the OP wanted to stop the GC for a particular code block, so it would seem that any "progress making" would be undesirable.
Re:Program your next server in Go by kyrra on Wednesday June 08, 2016 @12:05AM (#11859936)
https://golang.org/pkg/runtime/debug/#SetGCPercent

Do note that it triggers a GC when you call this, so it's not for all situations, but can be useful.

Re:Program your next server in Go by yxhuvud on Tuesday June 07, 2016 @03:04PM (#11856892)
Crystal might be worth checking out if you have that kind of need.
Re:Program your next server in Go by ngrilly on Tuesday June 07, 2016 @03:31PM (#11857097)
> Google itself doesn't actually have any real API libraries for their cloud services. They autogenerate all API libraries for golang, which means they're not idiomatic, are convoluted to use, and the documentation is a jungle.

Are you sure? Google offers 2 Google Cloud APIs for Go:

- Google API Client Library for Go, which is auto-generated like you wrote;

- Google Cloud Platform Client Library for Go, which is intended to be idiomatically-designed.

See https://cloud.google.com/go/apis.

Re:Program your next server in Go by erikpukinskis on Tuesday June 07, 2016 @02:37PM (#11856663)
> They autogenerate all API libraries for golang, which means they're not idiomatic, are convoluted to use, and the documentation is a jungle.

Are idiomatic API libraries really a good thing? If an API is used heavily throughout your application, doesn't that mean it's time to wrap that part of the API in some idiomatic wrapper code that is meaningful in your domain?

Functions are idiomatic in all languages. Personally, I'd rather APIs mostly stick to simple functions, and let the application developer build idioms around them if they like. And if the API call involves a web request, I am happy to just make the HTTP call myself. Half the time you need to understand the interchange format to debug your code anyway. My experience is that wrappers rarely protect you from that.

I guess if your application is 90% calls to an API, and you use a huge range of different functions in that API then you'd want a good, complete library. But how common is that?

Re:Program your next server in Go by akavlie on Tuesday June 07, 2016 @03:02PM (#11856869)
You're arguing that it's good to (intentionally) break with common code idioms for certain classes of libraries??

Also, have you actually seen the auto-generated APIs he's referring to?

Here are the Godocs for the BigQuery API: https://godoc.org/google.golang.org/api/bigquery/v2

Here's some example code demonstrating the API's bad habits -- such as deeply nested struct pointers, and Do() methods everywhere: https://github.com/google/google-api-go-client/blob/master/e...

Re:Program your next server in Go by chrisbroadfoot on Tuesday June 07, 2016 @03:05PM (#11856898)
Here's the idiomatic library for bigquery: https://godoc.org/google.golang.org/cloud/bigquery
Re:Program your next server in Go by akavlie on Tuesday June 07, 2016 @03:14PM (#11856963)
Thanks for that link, looks like that API was created after I started the project using BigQuery.

It's still labeled "experimental" though, which makes me a bit wary to use it just yet.

Re:Program your next server in Go by chrisbroadfoot on Tuesday June 07, 2016 @03:17PM (#11856994)
That's right. It's production-quality, but the API surface might change.

If you're OK with changing your code sometime in the future, then I'd recommend giving it a try for this or your next project. Changes will likely be minimal.

Re:Program your next server in Go by voltagex_ on Wednesday June 08, 2016 @02:19AM (#11860364)
For a production app, wouldn't you "pin" your dependency to a particular version? I know I wouldn't be allowed to track master in my workplace.
Re:Program your next server in Go by chrisbroadfoot on Wednesday June 08, 2016 @12:52PM (#11863625)
Yes! Please do that.

Use something like git-vendor: https://github.com/brettlangdon/git-vendor

Re:Program your next server in Go by stcredzero on Tuesday June 07, 2016 @02:45PM (#11856727)
Functions are idiomatic in all languages. Personally, I'd rather APIs mostly stick to simple functions, and let the application developer build idioms around them if they like.

Idiomatic APIs can be a real PITA, especially when you call them from a different language. KISS should reign supreme. (With bad 80's special effects in a made for TV movie filmed at a carnival.) If you leave it up to the client developer to build the idiomatic facade/interface, then the API is simpler, the code is more properly idiomatic in the end, and everyone is happier.

Re:Program your next server in Go by etatoby on Wednesday June 08, 2016 @04:31AM (#11860827)
Just a quick note about dependency management: we use "go get" to download / update dependencies, integrate / test, and then just include them all in the project's git repo. Never had an issue.
Re:Program your next server in Go by IndianAstronaut on Tuesday June 07, 2016 @05:52PM (#11858245)
Another major issue I have had with Go is database connectivity. The db drivers are really lackijg for Go. Makes it tough for those of us who use Teradata or Hive.
Re:Program your next server in Go by geodel on Wednesday June 08, 2016 @07:07AM (#11861400)
Well drivers are mostly written by database vendors or respective community in case of open source db. There is not much Go can do there.
Re:Program your next server in Go by devsquid on Tuesday June 07, 2016 @02:16PM (#11856458)
You would like to write an entire server without using a GC?
Re:Program your next server in Go by dboreham on Tuesday June 07, 2016 @04:38PM (#11857597)
I would not like to write a server with GC.

Source: I've written many servers in both C and Java and given the choice will never use a GC language again for a server. Holding out some hope for Rust..

Re:Program your next server in Go by dikaiosune on Tuesday June 07, 2016 @05:08PM (#11857869)
There are definitely some pain points building a server in Rust, but I think on the whole the tradeoff is worth it, and will become moreso as the ecosystem matures.
Re:Program your next server in Go by netheril96 on Tuesday June 07, 2016 @09:06PM (#11859302)
Care to elaborate?
Re:Program your next server in Go by asolove on Tuesday June 07, 2016 @02:18PM (#11856477)
Presumably he is complaining about the GC implementation's characteristics, not its existence. You can read a description of the team's design decisions here: https://blog.golang.org/go15gc but a simple summary would be: memory is cheap and getting cheaper, so focus on making GC fast rather than small memory footprints.
Re:Program your next server in Go by stincity on Tuesday June 07, 2016 @02:22PM (#11856528)
Hasn't go been improving the GC with it's rounds of updates to the language?
Re:Program your next server in Go by kasey_junk on Tuesday June 07, 2016 @02:28PM (#11856583)
Yes, with an emphasis on speed not memory.
Re:Program your next server in Go by merb on Tuesday June 07, 2016 @02:50PM (#11856773)
actually thats what the Java G1GC (which will be the default in Java 9) does aswell.

However I would still favor java instead of go

Re:Program your next server in Go by nameless912 on Tuesday June 07, 2016 @02:31PM (#11856608)
And in my opinion, this is a great tradeoff. Ultimately, adding more RAM to a server (or clicking a button on the AWS console) is a very easy fix. However, reducing latency is almost never easy. If they can prevent GC stuttering, I'll take the higher memory overhead every time.
Re:Program your next server in Go by tracker1 on Tuesday June 07, 2016 @06:22PM (#11858467)
GC has gotten significantly better the last few releases, I don't think most applications will notice.
Re:Program your next server in Go by haberman on Tuesday June 07, 2016 @02:38PM (#11856672)
People do it all the time. For example, in C++ with modern owning pointers like std::unique_ptr<> and std::shared_ptr<> it's really not that bad.
Re:Program your next server in Go by amelius on Wednesday June 08, 2016 @05:17AM (#11860984)
> I tried a couple different server libraries on Python but they all seemed to struggle with even tiny loads. Not sure what was up with that

Could the problem have been the "global interpreter lock"? The GIL makes it impossible for multiple threads to simultaneously perform certain basic operations, effectively making the system single-threaded.

Re:Program your next server in Go by tomcam on Tuesday June 07, 2016 @02:17PM (#11856472)
Invaluable perspective from the trenches--thanks.
Re:Program your next server in Go by pvelagal on Tuesday June 07, 2016 @03:04PM (#11856889)
Slide 13 (https://talks.golang.org/2016/applicative.slide#13) is interesting. I was expecting Go to be very close to C/C++ on the X axis (fast/efficient) as it doesn't use VM, but it is more close to Java ?
Re:Program your next server in Go by _ph_ on Tuesday June 07, 2016 @06:33PM (#11858532)
Ideally, Go should be pretty close to C, but currently there are reasons for it lagging behind (though there are also Go programs which perform better than their C counterparts):

- The Go compiler certainly isn't as sophisticated as the better C compilers. It looks that the code produces by Go 1.7 is quite improves vs. 1.6 with the SSA compiler and I would expect further gains in future releases.

- The Go compiler does not offer optimization levels and overall is optimized for compilation speed to, so these tradeoffs could be limiting.

- While the Go GC is very good and improving, so that the pauses are very low, it does use CPU, also while running in parallel to the program. How much exactly very much depends on the allocation behavior of the program, but should be accounted for.

Re:Program your next server in Go by eudoxus on Tuesday June 07, 2016 @03:30PM (#11857092)
It's true that Go compiles down to an executable and doesn't run on a VM like Java, but it does have it own runtime compiled into it, which manages the GC and goroutine scheduler to name two, so there is definitely some more overhead to Go then just straight C/C++
Re:Program your next server in Go by pcwalton on Tuesday June 07, 2016 @03:33PM (#11857116)
In addition, the language isn't designed for zero-cost abstractions like C++ is. You see this in the design of things like defer, which requires allocating records on the heap in certain cases, as compared to exceptions in C++ which can be implemented in a zero-cost manner.
Re:Program your next server in Go by weberc2 on Tuesday June 07, 2016 @04:16PM (#11857439)
I think interfaces are a better example, personally.
Re:Program your next server in Go by mahyarm on Tuesday June 07, 2016 @11:23PM (#11859776)
In my experience programming speed goes like this:

- 1x benchmark - C / static C++

- 2x slower - pure virtual C++ / objective-c

- 3x slower - statically typed GC languages (java,go)

- +6x slower - dynamically typed GC languages (js, python, etc)

After a well optimized implementation, speed comes down to manual vs GC memory management, static vs virtual function dispatch, dynamic vs static typing and heap vs stack allocation.

Compile speed is usually slowed down by any sort of type inference or templating/generics, which is probably one reason why go resists implementing generics or subclassing

Re:Program your next server in Go by snovv_crash on Wednesday June 08, 2016 @01:45AM (#11860271)
That 6x is for when you have a JIT. Without one (I'm looking at you, CPython) it is 20-30x, with perhaps the exception of Lua.
Re:Program your next server in Go by shpx on Wednesday June 08, 2016 @09:29AM (#11862119)
Re:Program your next server in Go by tyingq on Wednesday June 08, 2016 @08:59AM (#11861922)
Two considerations...

Your rules of thumb are probably mostly related to CPU bound tasks. If database i/o is the bottleneck, then the differences across language probably don't matter much.

And, runtime performance isn't always the most important thing. In many cases, "time to market" or "cost of development" might be a higher order priority. In those cases, a higher level language might be more attractive.

Re:Program your next server in Go by slmyers on Wednesday June 08, 2016 @03:02AM (#11860532)
Sorry, but I feel like I'm a little confused by your post. How can it be 6 times slower to program in JS or Python than C? Isn't this very much contrary to the conventional wisdom?
Re:Program your next server in Go by pmontra on Wednesday June 08, 2016 @05:41AM (#11861074)
I think OP wrote programming speed but meant running speed.
Re:Program your next server in Go by mahyarm on Wednesday June 08, 2016 @01:39PM (#11864012)
Yup. "Speed of the programming language"
Re:Program your next server in Go by trustyhank on Friday June 10, 2016 @11:32AM (#11877073)
Im surprised to hear your issues with Python, even Django can handle decent load depending what youre asking of it.
Re:Program your next server in Go by sdboyer on Friday June 10, 2016 @12:59PM (#11877792)
dep mgmt, at least, is poised to improve a lot in the coming months. we've got growing consensus around some metadata files, and i'm nearly done with my SAT solver (github.com/sdboyer/vsolver), which will be in glide, and maybe others, soon.