News for hackers. Stuff that matters.
Re:Gogs: a self-hosted Git Service written in Go by lukeholder on Saturday August 23, 2014 @09:44PM (#8217397)
What i love about Go from a user of some of the open source apps like Gogs, is that its so easy to get the application running.

A single binary with zero dependancies is so awesome. With a ruby on rails app I need to worry about ruby version, ruby implementation, gem versions, compilation of C based gems, etc etc. With Go i simply copy up the binary and run it.

Re:Gogs: a self-hosted Git Service written in Go by andmarios on Saturday August 23, 2014 @10:38PM (#8217497)
To be fair (and I am a Go fan) it is the same with Java and Scala. That's the main reason I switched from gitlab to gitbucket. Also with Java and Scala you can have all the resources inside your jar file. With Go you need a directory structure to store your templates, CSS, etc.

The second reason I switched was RAM consumption and here Go shines compared to Java, although I'll have to let Gogs run for a few days to see what happens.

I gave it a spin and it looks nice. I can see migrating from gitbucket to gogs, mainly for being able to run it on a small VPS. One thing that I didn't like was that the source version can only run from the source code directory. If I copy public and templates directories it should run from anywhere.

Re:Gogs: a self-hosted Git Service written in Go by kawera on Sunday August 24, 2014 @01:16AM (#8217772)
It is possible to pack static assets directly inside go binaries using statik[0] or rice[2]. I've been using then for small webapps and they both work very well.

[0] https://github.com/rakyll/statik

[1] https://github.com/GeertJohan/go.rice

Re:Gogs: a self-hosted Git Service written in Go by mholt on Saturday August 23, 2014 @11:19PM (#8217571)
(Slight nit-pic: To run a jar file, you need the JRE installed, whereas a Go binary contains the runtime.)

But you're right about static resources, they need to be in a directory structure. Fortunately, zip files make them easy to distribute.

Re:Gogs: a self-hosted Git Service written in Go by heavenlyhash on Sunday August 24, 2014 @02:32PM (#8219245)
In addition to the bundle-the-jre option (which is extremely workable), you can also feed java bytecode into robovm [1], which spins it through LLVM and produces native binaries for any platform you can go with LLVM. (Most of the documentation emphasizes IOS, but speaking from experience, yes, you absolutely can make mac desktop and linux native binaries as well.)

What robovm can do is a great example of the joys of toolchains with a well-defined stable intermediate representation. (Or one of the joys, I should say -- the flowering of languages on top of the jvm is another.)

Go's first-class AST work is excellent stuff, but I'd love it even more if they fleshed out a well-defined, stable, portable intermediate format like LLVM's IR and java's bytecode. Making a release of software and knowing it can run on architectures you haven't even thought of yet is liberating. Sure, architectures don't come along "very often" (but they're starting to happen more often!), and sure, you can use a full vm at the bottom layer (but it's slow! It can't do a fraction of the optimizations possible with an IR/bytecode), and sure, you can just pretend the source is an IR (but if some library you use chooses a different build tool chain than you, you're going to have a bad time -- don't laugh, golang is too young to have bifurcated much yet, but give it time, and look at what happened around the go-get mess already; or, consider what happens when someone just adds more tooling like source generation, which has also already happened)... Taking the long view, I think an intermediate representation has proven incredibly useful for the jvm as a platform, and I hope golang also grows to offer such a route sometime in the future.

[1] http://robovm.com/

Re:Gogs: a self-hosted Git Service written in Go by tempodox on Monday August 25, 2014 @07:33AM (#8221323)
There was a time when you just compiled the sources when you wanted to port a system to a new platform.
Re:Gogs: a self-hosted Git Service written in Go by heavenlyhash on Wednesday August 27, 2014 @12:28AM (#8230856)
That is the greatest possible defence of golang's current course and heading, and god knows I want as much of my world to build from source as possible, but

> sure, you can just pretend the source is an IR (but if some library you use chooses a different build tool chain than you, you're going to have a bad time

In the jvm world right now, for example, I won't touch maven with a ten foot poll. And yet I regularly use libraries produced by people who do. The portable, stable bytecode is a boundary between me and them where the buck stops. Anyone can use any build toolchain they like, and the interface for collaboration is well-defined.

Don't get me wrong, there are many things to love about golang, and there are many things to love about building from source. But I do think it's worth pausing to consider the flexibility that an intermediate representation can unlock.

Re:Gogs: a self-hosted Git Service written in Go by pcwalton on Sunday August 24, 2014 @03:04PM (#8219339)
LLVM IR is neither portable nor stable.
Re:Gogs: a self-hosted Git Service written in Go by mcpherrinm on Monday August 25, 2014 @03:22AM (#8220907)
But that isn't really relevant here: robovm is just a step in a build pipeline: Java -> java bytecode -> llvm IR -> native binary you distribute
Re:Gogs: a self-hosted Git Service written in Go by Alupis on Saturday August 23, 2014 @11:23PM (#8217582)
you can bundle the jre into your executable to remove all foreign deps
Re:Gogs: a self-hosted Git Service written in Go by adrusi on Sunday August 24, 2014 @01:29AM (#8217801)
Which is a security risk in that the jre can't be updated without an update to your product. Granted the same applies to go since it statically links to its runtime.
Re:Gogs: a self-hosted Git Service written in Go by hueving on Sunday August 24, 2014 @08:16AM (#8218362)
Not really a security risk if you think of it like an embedded library in your software. A JRE vulnerability is then just like a vulnerability in the app itself and the vendor needs to update it.
Re:Gogs: a self-hosted Git Service written in Go by _delirium on Sunday August 24, 2014 @01:40PM (#8219112)
The majority of JRE vulnerabilities are also not really relevant to a local app in the first place. They're sandbox vulnerabilities that let carefully crafted applets break out of the sandbox and execute arbitrary code. But regularly installed desktop/server software doesn't run in the applet sandbox, and is already assumed to be able to execute arbitrary code.
Re:Gogs: a self-hosted Git Service written in Go by condiment on Saturday August 23, 2014 @11:23PM (#8217581)
I've been experimenting with it as well and it looks like you have to copy several directories to get it to work out of the box - templates, public, custom, log, and data.

But I'm inclined to agree that this is one area Go projects have a hard time with. The portability of the executable is great, but if you're doing anything that requires static assets, configuration, or data outside the context of the runtime, you can easily end up with a tough-to-manage mess of configuration.

What we've done for our internal Go projects is wrap them up with an .rpm installer that handles the creation of directories throughout the system, drops init scripts where they're appropriate, and sets up necessary cron jobs and logrotate rules. I don't think this is the best approach, but at the same time I'm not familiar with any good alternatives.

Re:Gogs: a self-hosted Git Service written in Go by elithrar on Sunday August 24, 2014 @05:31AM (#8218108)
> But I'm inclined to agree that this is one area Go projects have a hard time with. The portability of the executable is great, but if you're doing anything that requires static assets, configuration, or data outside the context of the runtime, you can easily end up with a tough-to-manage mess of configuration.

This is mostly a sysadmin/devops issue, since all languages have the same issue.

I solve it through a CM tool (Ansible, in my case) that sets up my directories, rsyncs my templates and TOML config, and uploads my binary. Not having to worry the server running the same env as my dev environment removes a whole class of bugs/frustration.

Re:Gogs: a self-hosted Git Service written in Go by serverhorror on Monday August 25, 2014 @06:32AM (#8221210)
> This is mostly a sysadmin/devops issue, since all languages have the same issue.

You're kidding right? I thought that whole "devops" movement (call it what you like) was to get rid of the attitude of "other people's problem".

I live in environments that are (for the most part) regulated and demand by law that there is a strict separation between the people running and developing software, often open to interpretation is the 3rd role of people configuring the software.

The problem is that "I solve it..." just doesn't work. Either people solve it by working cooperatively together or it simply won't ever be deployed.

There's no such thing as a sysadmin problem, devops problem, developer problem. (No go downvote me for the tone of the next sentence) -- For me it's either get your ass moving, stand up, walk over to people and talk to them or get lost!

Re:Gogs: a self-hosted Git Service written in Go by elithrar on Monday September 01, 2014 @10:34AM (#8253017)
I missed this at the time, but I think you read past what I said: I used "devops" in the tools sense, not the people sense. A programming language (the syntax, the compiler, the linter) can't be everything to everyone, and therefore most don't attempt to shoehorn in things like static asset bundling, etc.

That's why frameworks exist, or packages like go-bindata or rakyll/statik (https://github.com/rakyll/statik), or CM tools that can wrap it all together.

Re:Gogs: a self-hosted Git Service written in Go by andmarios on Saturday August 23, 2014 @11:59PM (#8217634)
If the project permits it (size, complexity, what kind of bugs I expect to encounter) I like to have my static assets in a separate git repository.

System configuration I think is a problem independent of the programming language and anything you try will feel as hack. As long as it works, it is ok. :)

Re:Gogs: a self-hosted Git Service written in Go by philsnow on Sunday August 24, 2014 @12:18AM (#8217672)
> With Go you need a directory structure to store your templates, CSS, etc.

Is there anything stopping one from appending a zip archive to the go binary, and having the binary open the executable as a zip file to get at the static assets ?

Re:Gogs: a self-hosted Git Service written in Go by kyrra on Sunday August 24, 2014 @12:29AM (#8217693)
There are some 3rd party build tools to help with that. Below are a couple I found.

https://github.com/GeertJohan/go.rice

https://github.com/jteeuwen/go-bindata

EDIT: to add, the Go team is looking to add a generate command in go1.4 to help with this. See: https://groups.google.com/forum/m/#!topic/golang-dev/ZTD1qtp...

Re:Gogs: a self-hosted Git Service written in Go by mjibson on Sunday August 24, 2014 @12:29AM (#8217692)
go.rice (https://github.com/GeertJohan/go.rice) does things like this.
Re:Gogs: a self-hosted Git Service written in Go by DAddYE on Sunday August 24, 2014 @12:40AM (#8217709)
Thanks guys I was looking for something like that and I was aware of go-data, this with the next go generate should be very neat!
Re:Gogs: a self-hosted Git Service written in Go by pjmlp on Sunday August 24, 2014 @02:58AM (#8217924)
> A single binary with zero dependancies is so awesome.

You mean, like every single language out there with native code compilers?

A big generation gap must have happened, for static compilation to be touted as awesome feature.

Re:Gogs: a self-hosted Git Service written in Go by justincormack on Sunday August 24, 2014 @03:55AM (#8217981)
Glibc does not properly support static binaries, you need libc around even for static binaries mostly (eg dns resolution). Its annoying. You can make real static binaries with Musl libc though, or on the BSDs.
Re:Gogs: a self-hosted Git Service written in Go by pjmlp on Sunday August 24, 2014 @04:07AM (#8217997)
Yes, but it doesn't change the fact how strange it is to see people amazed by having a language compiler supporting static compilation.

Two decades ago it was the only option in most operating systems.

Re:Gogs: a self-hosted Git Service written in Go by justincormack on Sunday August 24, 2014 @04:46AM (#8218058)
Yes, I blame glibc for deprecating what should be perfectly normal...
Re:Gogs: a self-hosted Git Service written in Go by pjmlp on Sunday August 24, 2014 @05:04AM (#8218080)
May be. But it is an issue for the C developers that choose to link with it, need to sort out.

Not an excuse for the lack of understanding, that static compilation is a standard feature that any language compiler to native code usually provides.

Re:Gogs: a self-hosted Git Service written in Go by ithkuil on Sunday August 24, 2014 @08:25AM (#8218376)
Well currently this applies to Go as well: when linking to the "net" package it links to the system libc shared library for the same reason.

You can build your binary with the netgo build tag, but you must take into consideration that your binary might not behave correctly when running on systems with customised name resolution (NSS) settings, such as nss-ldap etc.

Re:Gogs: a self-hosted Git Service written in Go by uniclaude on Sunday August 24, 2014 @04:55AM (#8218072)
It's not about what those languages allow, it's about how developers use them. You very rarely see C software distributed this way.

Go developers only distribute their open source software in a way that you only have to take the source, compile it, and run it, without having to worry about dependencies.

Re:Gogs: a self-hosted Git Service written in Go by pjmlp on Sunday August 24, 2014 @06:47AM (#8218211)
> You very rarely see C software distributed this way.

I see it all the time in the last 30 years of my career.

Maybe GNU/Linux kids nowadays don't.

Besides C is not the only language with native compiler.

Re:Gogs: a self-hosted Git Service written in Go by elementai on Monday August 25, 2014 @11:45AM (#8222248)
Busybox, lvm and some other similar packages are frequently static in GNU/Linux for obvious reasons.
Re:Gogs: a self-hosted Git Service written in Go by shurcooL on Saturday August 23, 2014 @10:11PM (#8217440)
I agree, that is one of my most favorite aspects of Go.

    go get -u github.com/go/project
If you're writing native Go code, it doesn't matter if there's 0 or 1000+ dependencies, that one command will take care of everything. So there's no need to fear having extra dependencies, and you are welcome to reuse more code.

Go takes the common parts of writing software and abstracts them away. Take any two distinct Go projects, and the only difference will be:

- Import Path

- Actual Code

Everything else works exactly the same no matter what project it is (assuming it follows idiomatic Go package conventions). Docs via godoc.org, standard testing, go vet, http://go-lint.appspot.com/, http://gocover.io/

No need to figure out how to install that particular project/library, how to get docs, how to run tests, etc.

Re:Gogs: a self-hosted Git Service written in Go by swanson on Saturday August 23, 2014 @10:29PM (#8217477)
You mean like `bundle install` or `pip install -r requirements.txt` or `npm install package.json`? I mean, it's great that Go has something similar, but it's not like other languages are missing dependency managment...
Re:Gogs: a self-hosted Git Service written in Go by andmarios on Saturday August 23, 2014 @11:19PM (#8217569)
Mind you that these are compile time dependencies. Go produces static binaries that can run as standalone apps. So only the developer has to install them. Ruby, nodejs and python have runtime dependencies, thus the user has to install them too (or use the bundled ones if they exist).

Also their package managers have many issues you don't see with go (because go doesn't have a package manager, rather a source code dependency manager). For example python and ruby have many versions and not everything is available in all versions. What a user will do if he want to run a 2.7 and a 3.x python app? Sometimes they use the system compiler to produce binaries. I have seen it fail occasionally; for example charlock_holmes on a gentoo hardened system. Npm just floods your filesystem with npm_modules directories. I use a package manager so I won't have to install the same thing multiple times at multiple locations. Also it is very difficult to understand its directory structure. I searched the docs for hours in order to find where it stores the programs I install as a user (its path). Go is difficult to understand too in that aspect but it has all this info in one online document. You read it until you understand it.

Finally there are always the questions; should I use pip or easy_install? What does rvm, what gem and what bundle(r)? Why should users learn them? How may I update my current gems and remove old versions? Where is gruntjs and how do I run it?

Re:Gogs: a self-hosted Git Service written in Go by DAddYE on Sunday August 24, 2014 @12:36AM (#8217700)
Wait what? Ocaml, Haskel, Rust? Nix?

Probably many more. If `go get` was good enough as you intended to say why do we have `godep, godeps, gom, gondler, goop, vendorize, party, gpm` and many more? Which one should I pick? It's better do GOPATH="_vendor:$GOPATH" go install or use one of them? Also, should I install go with apt/brew/pac or use gvm?

I like go, I use it for fun and work, but really, is getting annoying seeing all this `gophers` totally out of reality.

Re:Gogs: a self-hosted Git Service written in Go by andmarios on Sunday August 24, 2014 @07:18AM (#8218276)
There are third party tools for every language. With go until now I haven't need any of them, so I don't know them. I read go's docs and I do fine with the official tool when I want to compile an app from source.

On the other hand, there isn't a way to not learn about gem, bundle, pip, easy_install, npm when you try to install apps written in their languages. Some times it is easy, as “gem install gollum” and some times it is difficult, as trying to install gitlab.

Haskell I really don't know because any haskell app I ever needed was in my distro's repositories. To be fair most Python packages are too.

Re:Gogs: a self-hosted Git Service written in Go by Jweb_Guru on Sunday August 24, 2014 @12:47AM (#8217719)
The reason that Go can't handle dynamic library dependencies, for the most part, isn't a strength--it's a weakness. I guarantee you that if it could call out to C efficiently, it would. Part of the price it pays for its goroutines is that its relocatable stacks do not play nicely with C calling conventions, meaning it can be extremely expensive to call out to a C routine. For that reason, Go essentially reimplements libc in its own standard library. All the other languages you named don't do that, because they don't have to.

Anyway, Go can't produce dynamically linked libraries yet even for Go applications. It really has no choice but to bundle them into one application.

Re:Gogs: a self-hosted Git Service written in Go by andmarios on Sunday August 24, 2014 @07:41AM (#8218314)
I agree that this limits Go's use scope. But when you accept that Go is most suited for web apps and some console tools, it is nice and works well for the users and the application packager.

The only package manager (pm) I like, is the pm of the distro I am using. If I am on Fedora, I like yum. When I am on Gentoo I like portage etc. The problem with most web apps is that they have dependencies that don't exist in your distro's pm. I'll take a static binary over a 3rd party pm any day. Let's not forget that one of the most important reasons for the existence of docker is to not have to set up an app in many different systems.

Re:Gogs: a self-hosted Git Service written in Go by frik on Sunday August 24, 2014 @02:07AM (#8217853)
Go's goroutines as I understand are the same or very similar to Lua's coroutines and WinAPI's Fibers. Right?

For the later two can calling C functions is cheap. I would assume if you run Go with its default one thread option, it's cheap too. (Switching threads or even processes means context switching is what is usually expensive).

Re:Gogs: a self-hosted Git Service written in Go by unassuming_rat on Sunday August 24, 2014 @02:53AM (#8217914)
No, Go routines are not the same as coroutines in other languages.

In order to provide efficient lightweight threads, Go allocates only a small amount of memory for each stack, growing it as function calls require it by appending another segment in a linked list. This makes thread creation very cheap.

The problem with segmented stacks is that they can cause "stack thrashing." Suppose a goroutine is right at the boundary between two stacks. It calls a function that exhausts the available segment, forcing the allocation of a new segment, which is then added to the linked list. The function runs, and then frees the segment upon return. If this boundary happens to fall within a tight loop, you'll see a severe drop in performance, to the point that it's completely unacceptable.

Recognizing this, Go relatively recently switched away from segmented stacks. It now uses a strategy called relocatable stacks, or contiguous stacks. See https://docs.google.com/document/d/1wAaf1rYoM4S4gtnPh0zOlGzW... for details. Similar concerns prompted Rust to move away from segmented stacks, but as relocatable stacks require a garbage collector that was not a viable way forward for Rust, so it just dropped the idea completely.

The problem with both approaches is that C has no knowledge or support of either segmented or relocatable stacks. This means that calling out to C in Go involves a completely different calling convention that involves switching to a larger stack, running there, and then switching back--which is very expensive, especially compared to languages like Lua where the C FFI is basically free. That's why Go reimplements glibc by performing its own system calls--to avoid the C FFI.

In conclusion, every language feature--no matter how enticing--has a cost.

Re:Gogs: a self-hosted Git Service written in Go by Veratyr on Sunday August 24, 2014 @02:30AM (#8217885)
Unlike Go, many projects (particularly web based) written in these languages have native dependencies (for databases etc.) and it's uncommon for installation to be as simple as you say.

Ruby's Nokogiri for example is very very common and I've never found it simple to install.

Python has psycopg, numpy and a variety of others that are often very complex to install.

Node at least differentiates itself in that most of the database drivers have pure JS fallback implementations, so they'll work without the relevant libraries installed, just not as fast.

Re:Gogs: a self-hosted Git Service written in Go by masklinn on Sunday August 24, 2014 @03:46AM (#8217974)
> Python has psycopg, numpy and a variety of others that are often very complex to install.

    (test)> pip -q install pillow psycopg2 numpy
    pip -q install pillow psycopg2 numpy  9.85s user 5.43s system 44% cpu 34.400 total
That was complex indeed.

(it wasn't)

Re:Gogs: a self-hosted Git Service written in Go by rtpg on Sunday August 24, 2014 @10:59AM (#8218702)
I've had issues with installing those libraries involving C-language dependencies that were failing compiling (this was mainly due to the recent Xcode tool update that makes llvm barf sometimes).

It's not going to get simpler than a single executable.

Re:Gogs: a self-hosted Git Service written in Go by DAddYE on Sunday August 24, 2014 @01:41AM (#8217817)
Without going so far: `gem install ...` which basically comes with any linux distro and in osx since 5 years.
Re:Gogs: a self-hosted Git Service written in Go by shurcooL on Sunday August 24, 2014 @04:45AM (#8218056)
I'm coming from/comparing to C++ mostly, which is the language that Go replaced for me.
Re:Gogs: a self-hosted Git Service written in Go by girvo on Saturday August 23, 2014 @10:13PM (#8217443)
Funnily, it's also why I sort of like Java web apps. Deploying a JAR or a WAR as a single file is pretty great.
Re:Gogs: a self-hosted Git Service written in Go by kinofcain on Saturday August 23, 2014 @10:55PM (#8217533)
Easier than some other languages, but you still need something to run what's in your jar. Being able to deploy executables eliminates a whole layer of dependencies, which ends up being really handy.
Re:Gogs: a self-hosted Git Service written in Go by girvo on Saturday August 23, 2014 @11:18PM (#8217567)
I definitely agree! Statically linked binaries are my favourite part of Go, and one of the reasons why I've started moving towards it at work for some projects. Really takes a lot of hassle out of it all.
Re:Gogs: a self-hosted Git Service written in Go by pjmlp on Sunday August 24, 2014 @02:56AM (#8217919)
You can compile that jar to native code with the likes of Excelsior JET, for example.
Re:Gogs: a self-hosted Git Service written in Go by jingo on Sunday August 24, 2014 @03:53PM (#8219480)
This is why I prefer static binaries and preferably "crunched" binaries.

Same goes for the use of "read-only" mounts and "RAM disks" as opposed to dependence on other, less reliable media.

I hope your comment continues to be upvoted; usually I see downvotes at any mention of static linking. Why, I can never be sure. If static binaries make some power user's life easier why should anyone else care? With reasonably-sized, well-written programs (or the use of a system like Go's), compilation can be quite fast. And easy enough for anyone to do if and when libraries change.

Re:Gogs: a self-hosted Git Service written in Go by joe2010xtmf on Saturday August 23, 2014 @10:06PM (#8217435)
Thanks for your words, we really appreciate!
Re:Gogs: a self-hosted Git Service written in Go by lukeholder on Saturday August 23, 2014 @10:23PM (#8217465)
Thanks for your work on Gogs! looks forward to seeing it grow. Would love to help out on the front end - let me know if you are needing any contributors.
Re:Gogs: a self-hosted Git Service written in Go by joe2010xtmf on Sunday August 24, 2014 @09:36PM (#8220217)
Hi there, how can I contact you? My e-mail is on the GitHub.
Re:Gogs: a self-hosted Git Service written in Go by vectorpush on Saturday August 23, 2014 @10:17PM (#8217453)
Indeed. Gitlab is a similar software that I absolutely love, but the setup process is quite lengthy (though straightforward) and updating to new versions can be a considerable effort. I'll definitely be giving this a try, especially since the design is of a similar quality to gitlab.
Re:Gogs: a self-hosted Git Service written in Go by geerlingguy on Saturday August 23, 2014 @10:36PM (#8217492)
The new Omnibus installer (which is basically a nice wrapper around Chef) makes the install process much simpler, FYI; it was released sometime late in the 6.x era.
Re:Gogs: a self-hosted Git Service written in Go by sytse on Sunday August 24, 2014 @05:47AM (#8218126)
GitLab B.V. CEO here, thanks for mentioning the Omnibus installer packages, they indeed make the installation much faster (2 minutes total) and updates are simple as well.
Re:Gogs: a self-hosted Git Service written in Go by general_failure on Saturday August 23, 2014 @10:15PM (#8217449)
Agreed. Gitlab is awesome but very painful to setup
Re:Gogs: a self-hosted Git Service written in Go by sytse on Sunday August 24, 2014 @04:24AM (#8218023)
GitLab B.V. CEO here, have you tried our packages? With them you can now install GitLab in two minutes.