News for hackers. Stuff that matters.
Re:Introduction to Go 1.1 by melvinmt on Tuesday March 26, 2013 @09:55AM (#5443245)
For those of us who are wondering if 1.1 is stable enough to upgrade: Google is already using Go 1.1 in production.

And the company is betting big time on Go, more than 50% of their codebase will be (re)written in Go in a couple of years.

Source: core dev on the golang team.

Re:Introduction to Go 1.1 by bradfitz on Tuesday March 26, 2013 @11:38AM (#5443997)
I am a core dev on the golang team.

Teams and engineers at Google can use whichever language they want. (trade-offs abound) We're trying to make Go be the most compelling language with the fewer trade-offs so they're willing to switch. It's getting increasingly popular.

It's possible that 50% of new code will be written in Go in a couple years, but I've never seen that written as a goal. (And we do have goals. And usage charts over time, for each language, inside and outside the company, etc.) There's no way we'll rewrite 50% of all code in a couple years... there's so much code.

Re:Introduction to Go 1.1 by melvinmt on Tuesday March 26, 2013 @12:59PM (#5444568)
Thanks for clarifying (can't edit my statement), the engineer I spoke to was wildly enthusiastic about Go so it may have been wishful thinking on his part. Even though it's clear now that the existing codebase won't be rewritten nearly as much, I still think it's cool that Google is considering to write 50% of all new projects in Go (if that part is still true). I'm hoping a lot of companies will follow.
Re:Introduction to Go 1.1 by BarkMore on Tuesday March 26, 2013 @10:52AM (#5443659)
I doubt that Google is rewriting 50% of their code in two years. Perhaps they said that 50% of new projects will be written in Go.
Re:Introduction to Go 1.1 by melling on Tuesday March 26, 2013 @10:49AM (#5443631)
Some support for Android would be great. Go would make a great mobile development language.
Re:Introduction to Go 1.1 by myko on Tuesday March 26, 2013 @11:03AM (#5443733)
I love Android, but Java is the pits. I would love the libraries to be re-written in Go.

How feasible is something like this, considering Dalvik is a major piece of the way Android behaves? Would the VM just go away?

I really hope this is a skunkworks project happening at Google right now. Or actually I'd prefer it have major support up the chain, but I'll take a skunkworks project.

Re:Introduction to Go 1.1 by pkulak on Tuesday March 26, 2013 @12:32PM (#5444378)
Dalvik is a major piece, but nothing is required to use it; there has been native support for a while now. Re-writing all the Android libraries in Go would be a heck of a project though, and then you'd have to maintain both forever. And I'm not entirely sure what the benefits would be, since I don't really know how Dalvik compares in runtime and memory use to Java 7. It would be pretty awesome though. I actually like Java, but developing mobile apps in Go would be a blast. Plus, Google could get entirely out of this Oracle situation if they transitioned entirely.
Re:Introduction to Go 1.1 by grey-area on Tuesday March 26, 2013 @01:14PM (#5444675)
They'd be better off just starting again with a new minimal stack and API, with the lessons learned from their first iteration of the API (and I'm sure there were lots of things they'd do differently if they had the chance), run it in parallel, and gradually deprecate the old API as the new one gets up to speed, just as Apple has done several times over the past couple of decades (AppKit, UIKit etc). Moving a big group of developers from one API to another can be done, it just takes time, and a little judicious use of pressure...

Android now has the installed base for developers to be forced to go along with whatever changes Google requires, and Dalvik could be kept around for quite a while as a backstop. I don't think the problems with Oracle are going away, so this might come to pass eventually, but I doubt it is on the roadmap right now.

Re:Introduction to Go 1.1 by scott_karana on Tuesday March 26, 2013 @12:54PM (#5444523)
Considering that Go doesn't support shared libraries, it could make for really inefficient use of device RAM. Android's already a little heavy in that regard.
Re:Introduction to Go 1.1 by sambeau on Tuesday March 26, 2013 @07:32PM (#5446743)
I was under the impression that even with shared libraries each running process gets its own copy of the library in its memory space.
Re:Introduction to Go 1.1 by vetinari on Wednesday March 27, 2013 @05:44AM (#5448527)
Yes, they do, but it is shared, copy-on-write (read-only for .text), meaning that although each running process has it mapped into its memory space, there is only one copy in physical RAM.
Re:Introduction to Go 1.1 by sambeau on Friday March 29, 2013 @05:19AM (#5459875)
For some reason I thought that was a fallacy. I must now go and re-read Tanenbaum.
Re:Introduction to Go 1.1 by coldtea on Tuesday March 26, 2013 @11:29AM (#5443920)
>And the company is betting big time on Go, more than 50% of their codebase will be (re)written in Go in a couple of years. Source: core dev on the golang team.

They have been known to oversell the language.

No way is the quote accurate.

Re:Introduction to Go 1.1 by BarkMore on Tuesday March 26, 2013 @11:34AM (#5443962)
> They have been known to oversell the language.

There's a lot of hype about Go, but I am not aware of cases where Google is overselling the language. Can you give examples?

Re:Introduction to Go 1.1 by coldtea on Tuesday March 26, 2013 @11:52AM (#5444091)
>There's a lot of hype about Go, but I am not aware of cases where Google is overselling the language. Can you give examples?

Not Google. Just the Golang guys (Pike and co).

1) They sold it as a "systems programming language" at the beginning, to replace C/C++. When that didn't play well with actual system programmers, they reverted it to mean mostly server and similar back-end programming.

2) They emphasize as often as they can the "compilation speed", while for one is not a great concern for most projects, and second, other languages are just as speedy on that front.

3) They made claims of Go being "very fast" based on its statically compiled nature, whereas in practice it's slower than Java/Scala et co.

4) They downplay the fact that their GC is basically crap.

5) They make frequent statements about Go being used all around Google, but I've read Google employees deny that and say that it's use is quite marginal. The 2 greatest success stories they had offered are a simple component for Google Downloads (not the one that handles the whole thing) and a load balancer for MySQL fro YouTube. Not the kind of adoption to write home about.

Re:Introduction to Go 1.1 by tptacek on Tuesday March 26, 2013 @12:01PM (#5444152)
1. It clearly is a systems programming language. Pike has written whole articles about how difficult it has been to convince C++ people ot switch.

2. It clearly does have an extremely fast compiler.

3. It clearly is very fast, and Pike never claimed it was the fastest.

4. The GC works and is an active area of development. They are open about this. What do you want them to say?

5. Could you cite them actually saying Go was used all over Google?

Re:Introduction to Go 1.1 by pron on Tuesday March 26, 2013 @01:53PM (#5444934)
1. Maybe, depending how you define "systems programming language". Go, unlike Rust, is not really low level. Its just as "systems level" as Java is, and many C++ shops have switched to Java. So Go is like Java in that respect. Not a C/C++ replacement in the post-Java era.

2. Sure. Much faster than C++. But, as Go is really a Java alternative rather than a C++ alternative, the compiler is not that much faster than Java's.

3. Yes, but not fast enough as a C/C++ replacement. It's not even as fast as Java.

4. The Go team has made some interesting, and risky, design decisions with the GC. They write: "Go must support what we call interior pointers to objects allocated in the heap... This design point affects which collection algorithms can be used, and may make them more difficult, but after careful thought we decided that it was necessary to allow interior pointers because of the benefits to the programmer and the ability to reduce pressure on the (perhaps harder to implement) collector. So far, our experience comparing similar Go and Java programs shows that use of interior pointers can have a significant effect on total arena size, latency, and collection times." It will be interesting to see how this decision pans out.

5. I think he pretty much just did. Or, to be precise, he quoted what is likely a false prediction.

It is true that both Go and Rust were first touted as a C/C++ replacement. Go clearly isn't while Rust clearly is. Go is a Java alternative minus the dynamic code-loading, and as such, hardly revolutionary. You get similar compilation speed, a slower runtime and none of the dynamic stuff. Go is sure a nice language, but Java shops are better off moving on to Clojure or Scala if they need a better language.

Re:Introduction to Go 1.1 by tptacek on Tuesday March 26, 2013 @02:09PM (#5445035)
The Go compiler is much faster than javac.

I don't accept your premise of "Java is the floor for acceptable performance in systems programming languages". Plenty --- perhaps most! --- new serverside code is written in languages much, much slower than Java. I'm not sure if I accept your premise that Golang is consistently slower than Java (if it is, the difference is marginal). On the other hand, Golang code starts instantly and has no runtime requirement.

I'm peripherally aware of some kind of rivalry between Rust and Go, but I'm not interested in discussing it; I use programming languages as tools to solve problems, not as tribes to join. When Rust is ready for prime time, I'll check it out and see if it solves any of my problems better than C, Golang, or Ruby, and if it does, I'll start using it. I have no plans to take seriously the idea of benchmarking the whole of Go against the whole of Rust.

Re:Introduction to Go 1.1 by pcwalton on Tuesday March 26, 2013 @02:26PM (#5445142)
There is not really a rivalry between Rust and Go; they had different goals and Go is a fantastic language for its goals. For example, Mozilla uses both Rust and Go for different things. I don't see any need for tribalism either.
Re:Introduction to Go 1.1 by pron on Tuesday March 26, 2013 @02:32PM (#5445166)
All I'm saying is, Go was touted as a C/C++ replacement while it clearly isn't; it's a Java alternative, and as such, more than justifies comparisons of the two. When you compare them, you see some advantages for Go and some for Java.

I don't think there's a rivalry between Rust and Go, and if there is -- there shouldn't be. They serve completely different purposes: Go is an alternative to Java, while Rust is an alternative to C/C++. It's not about speed, either. It's about level of control.

Re:Introduction to Go 1.1 by tptacek on Tuesday March 26, 2013 @02:40PM (#5445208)
This stuff about Golang "touted" seems to turn on picky definitions "touting" that include "stating the objectives of the language project". But more importantly, they are just not relevant to developers.

So, having watched Rob Pike talk about Golang many times, and read a lot of what he's written about the language, I object to the way you're distilling what he's saying about the project. But, more importantly, I object to the sideshow.

Re:Introduction to Go 1.1 by pron on Tuesday March 26, 2013 @02:57PM (#5445314)
I'm not being picky, this isn't a sideshow, and I think it's very relevant to developers. As most "systems" developers are quite familiar with C and with Java, and Go is the new kid on the block, it is important to point out that Go essentially solves the same problems Java does, and not those that you'd turn to C to solve. This is especially important to point out because Go's objectives initially placed the language in the C space, or so it seemed to many.
Re:Introduction to Go 1.1 by tptacek on Tuesday March 26, 2013 @02:59PM (#5445331)
Let's agree to disagree that this is an important point.
Re:Introduction to Go 1.1 by burntsushi on Tuesday March 26, 2013 @06:44PM (#5446541)
> it is important to point out that Go essentially solves the same problems Java does

One of the major selling points of Go is its concurrency features built into the language. Namely, first class channels and lightweight threads. Java has neither of those. In particular, very few languages support lightweight threads in an M:N scheduling style (Haskell, Erlang, Rust), which in and of themselves makes Go a very attractive tool for concurrent programming.

Re:Introduction to Go 1.1 by pron on Tuesday March 26, 2013 @07:43PM (#5446772)
Actually, these aren't lightweight threads because, AFAIK, they require cooperative multitasking. Java actually has fork-join, an extremely sophisticated scheduler, that allows much finer control over scheduling than Go, and also supports non-cooperating tasks. I believe it's being used as the basis for Akka's actors. Java also has quite a few queue implementations suitable for a different concurrency scenarios. So, in that respect, Go offers easier to use, though fewer, concurrency options, while Java offers finer control over concurrency.

But the point wasn't to debate the merits of Go over Java or vice versa, just to say that they are both intended to be used in the same domains.

Re:Introduction to Go 1.1 by burntsushi on Tuesday March 26, 2013 @08:20PM (#5446939)
Go's lightweight threads do not require cooperative multitasking in the traditional sense. In Go, lightweight threads are preempted by the runtime whenever an operation that can block is made (IO, channel send/receive, system calls, etc.). However, lightweight threads can hog the CPU, and in that sense, they must be cooperative. But this rarely shows up outside of pathological cases in practice.

More importantly, I wasn't talking about scheduling. I was talking about lightweight threads. They are also commonly known as: goroutines, green threads, fibers, user space threads, etc. This has a dramatic effect on the style of concurrent programming, as spawning a light weight thread is cheap relative to spawning an OS thread.

Joe Armstrong explains [1] things better than I can:

    Processes in Java or C# are pretty much like the objects in the flawed object 
    system which we described earlier - you can’t have many of them, and if you 
    have more than a few hundred processes the system will start misbehaving and 
    to do any real work with them you have to write your own scheduler and start 
    combining multiple threads of control into single processes. All of this 
    gives concurrent programming a bad name - and would probably make any sane 
    person think that concurrent programming was a difficult and should be avoided 
    whenever possible. The opposite is true.
And:

    In Erlang processes are *light weight*. This means that very little 
    computational effort is required to create or destroy a processes. Light-weight 
    processes in Erlang are one to two order of magnitude lighter than operating 
    system threads.
    
    Not only are Erlang processes light-weight, but also we can create many 
    hundreds of thousands of such processes without noticeably degrading the 
    performance of the system (unless of course they are all doing something at the 
    same time).
In Go, "goroutines" are analogous to "processes" in Erlang in the above excerpt.

My point here was that Go solves the concurrency problem much differently than Java does, to a point where Go is much better suited to write concurrent programs than Java is. Joe Armstrong's argument in his paper applies pretty well here.

I'm not really sure what is to be gained by debating over whether Go was "intended" to be in the same domain as Java. I've used Go to solve problems that I might have otherwise solved in a variety of other languages (C, Haskell, Python). Had I ever been a Java programmer, I'm sure that would have been in the list too. Experience in speaking with other Go programmers leads me to believe that I am not unique in this regard.

[1] - "Concurrency oriented programming in Erlang", 2003. https://guug.de/veranstaltungen/ffg2003/papers/ffg2003-armst...

Re:Introduction to Go 1.1 by pron on Tuesday March 26, 2013 @08:56PM (#5447085)
Take a look at Akka actors. The JVM offers similar mechanisms, only with finer control over scheduling. But Go is fine and does have some advantages over the JVM (faster startup time) as well as disadvantages (dynamic code loading, runtime instrumentation, monitoring). Use it in good health!
Re:Introduction to Go 1.1 by tptacek on Tuesday March 26, 2013 @11:11PM (#5447589)
Isn't the "heaviness" of Go's threads due to do with how the thread stacks are allocated, not so much with how the threads are scheduled?
Re:Introduction to Go 1.1 by pron on Wednesday March 27, 2013 @06:02AM (#5448560)
In that case, Java allows for a very similar thing with fork-join. If you want to see exactly how, in two weeks I'll release an open-source library that wraps Java's fork-join as Go-like lightweight threads, only more preemptive and with better scheduling.
Re:Introduction to Go 1.1 by enneff on Tuesday March 26, 2013 @12:11PM (#5444226)
You should read this http://talks.golang.org/2012/splash.article which addresses points 1 and 2.

As for 3 and 4, Go _is_ fast compared to a lot of languages, and has just gotten a lot faster. I think we're pretty realistic about the GC, in that we say it's simple and it works, and for most people it is more than adequate. We continue to spend a lot of time working on the GC.

One thing we do say, however, is that unlike Java, Go gives you more control over memory use so that you don't need to put so much pressure on the garbage collector.

5 - there's a bunch of Go usage at Google that we can't talk about. It is being used increasingly all over the place, but growth is pretty organic at this point. The public things we can talk about, we do. The Google Downloads thing is not a small component, by the way, but rather the entire download server. We hope to release part of that as an open source project soon, so you can get an idea as to what it actually does. One of the more impressive things about that particular service is that it serves massive traffic with Go's built in net/http package.

Re:Introduction to Go 1.1 by pjmlp on Tuesday March 26, 2013 @02:03PM (#5445000)
Regarding issue 1, even though I tend to bash a bit Go regarding the lack of certain features, I have to defend it here.

There are desktop operating systems being written in GC enabled system programming languages, namely Native Oberon and AOS. Both were and are used at Zurich Technical University (ETHZ) in operating systems research.

In the late 90's some developers even used them as their main OS.

You just need to have an escape mechanism to do the usual low level tricks, which is achieved by having a virtual package like unsafe, system or similar.

The problem with getting new languages adopted for systems programming is that they will only get adopted if there is a mainstream OS that makes use of them thus forcing the developers at large to use it as such.

Only now we see big companies replacing OS code that used to be written in C by C++. So how long it will take for any of the new contenders to defy C++ and in which OS?

As for the Oberon systems, you might find this interesting,

http://www.inf.ethz.ch/personal/wirth/books/ProjectOberon.pd...

http://www.ocp.inf.ethz.ch/wiki/Documentation/Oberon

http://sage.com.ua/en.shtml?e1l0

Re:Introduction to Go 1.1 by netcraft on Tuesday March 26, 2013 @10:41AM (#5443570)
im interested in the rewritten aspect - are these things that would have otherwise needed to be rewritten, or are you gaining benefits from the rewrite? Or is it an "eat your own dogfood" sort of deal? 50% of google's codebase must be significant, I can't imagine doing that "just because". And rewritten from what? C/C++ or python or something else?
Re:Introduction to Go 1.1 by melvinmt on Tuesday March 26, 2013 @11:02AM (#5443731)
I'm not sure why, I also thought it was a bold statement coming from a Google employee and we'll have to see if they actually are able (and willing) to do that.

But I can imagine, seeing how obsessive Google with performance usually is, that it mostly will be rewritten to save "millions of years" in optimizations. I believe Google is mostly using Java and Python.

Re:Introduction to Go 1.1 by enneff on Tuesday March 26, 2013 @12:13PM (#5444239)
> coming from a Google employee

Who? I am convinced you have misheard, as nobody in their right mind would make that claim. We have a ton of code, and thousands and thousands of programmers, so it's just not realistic. These kind of hyperbolic statements are harmful as they build distrust in what we're doing. I, a Go core dev, have no interest in being anything but straightforward on this topic.

Re:Introduction to Go 1.1 by melvinmt on Tuesday March 26, 2013 @12:52PM (#5444514)
I won't name the source but I never intended to bring "harm and distrust" to the Go community (I really like Go). I can't edit my OP but thanks for clarifying the statement.
Re:Introduction to Go 1.1 by BarkMore on Tuesday March 26, 2013 @11:31AM (#5443935)
> I believe Google is mostly using Java and Python.

Google uses C++ for most compute-intensive code.

Re:Introduction to Go 1.1 by bradfitz on Tuesday March 26, 2013 @11:39AM (#5444004)
We don't use much Python. Mostly C++ and Java.
Re:Introduction to Go 1.1 by SiVal on Tuesday March 26, 2013 @12:19PM (#5444285)
Is Google's use of Python growing or shrinking? And how does it compare to your use of Go?
Re:Introduction to Go 1.1 by swah on Tuesday March 26, 2013 @01:41PM (#5444863)
The other day I was surprised to see that Go didn't have sets. After searching Google Groups, it was clear that the rationale is that "sets are easy to implement with hashmaps".

I feel though that having a set, or a multiset, whatever... is exactly the kind of abstraction that makes it easy to think about your programs.

In the end I felt that my Go port of some Python felt much more error prone that the original code.

Re:Introduction to Go 1.1 by burntsushi on Tuesday March 26, 2013 @06:52PM (#5446584)
> I feel though that having a set, or a multiset, whatever... is exactly the kind of abstraction that makes it easy to think about your programs.

I agree. But many abstractions meet that criteria. That doesn't mean you include it in the language.

Also, it isn't just easy to implement a set with a map in Go, it's downright trivial:

    type Set map[string]int

    // Create set
    set := make(map[string]bool)

    // Add to set
    set["new element"] = true

    // Remove from set
    delete(set, "new element")

    // Membership test
    if set["new element"] {
        // "new element" is in the set
    } else {
        // "new element" is not in the set
    }
Of course, this lacks common set operations like intersection, union, difference, etc. But in my experience, sets are most useful in their ability to represent a unique collection of elements with fast membership testing.
Re:Introduction to Go 1.1 by swah on Tuesday March 26, 2013 @08:18PM (#5446933)
Thanks for answering. You're right, but in my case, I was adapting http://norvig.com/spell-correct.html, and the set operation (constructor, actually) was exactly what I needed.

Of course, it was simple to write a removeDuplicates fn that inserted elements from the array to a map, and back to an array. But that is the point of my parent comment... :)

Once, while writing something in Java, I found out that I needed a Multiset (didn't knew the term) and that Java had a implementation. As strange as it sounds, that made me very productive and confident my code was working... (Of course, soon you're writing getters and setters and hating Java again..)

Re:Introduction to Go 1.1 by burntsushi on Tuesday March 26, 2013 @08:38PM (#5447010)
Ah, I see. What if you represented a multiset as a map just like in my parent comment, but with integers instead of booleans?

    type Set map[string]int

    // Add to set
    set["element"]++

    // Remove from set
    s[el] = max(0, s[el]-1)

    // Membership
    s[el] > 0
Here's a working example: http://play.golang.org/p/zMvVF1yERc

N.B. This is a fiendish way of making good use of "zero" values in Go.

Re:Introduction to Go 1.1 by swah on Tuesday March 26, 2013 @09:54PM (#5447295)
It seems you're just counting ocurrences? The Multiset actually stores them IIRC.

So to me it seems the Set type should be map[string][]type_of_elem, if that is possible.

[edit]

I'm confusing Multiset with Multimap... sorry.

[note]

Actually Multimap/Mutliset is not shipped with Java, but from Google... So it's not that "included".

Re:Introduction to Go 1.1 by burntsushi on Tuesday March 26, 2013 @11:07PM (#5447570)
> It seems you're just counting ocurrences?

Yeah. Remember, a multi-set is just a bag. If the bag is `{a, b, a, a, b}`, then it can also be represented as a simple frequency vector: `{a: 3, b: 2}`. There's no need to actually store duplicate values.

> So to me it seems the Set type should be map[string][]type_of_elem, if that is possible.

OK, now I see. It depends on what your definition of equality is. If the entire value represents identity, then my solution works. But if only part of your value (say, some but not all fields in a struct) represents identity, then my approach won't work at all. My approach relies on Go's built in definition of equality, which isn't flexible and cannot be changed. For instance, if you're storing values with this type in your set:

    type MyValue struct {
        Id int
        Tag string
    }
And these two values are equivalent:

    MyValue{5, "abc"}
    MyValue{5, "xyz"}
Then you'd have to roll your own implementation. (Since in the eyes of Go, these two values are not equal. And that cannot be changed.)

There are elegant ways around this, but it is certainly more clumsy in Go than it would be in a language with some sort of ad hoc polymorphism (like overloading in Java or type classes in Haskell). I think your idea is pretty close:

    type MultiSet map[MyValueIdentity][]MyValue

    type MyValueIdentity int
Anyway, best of luck to you. Don't be afraid to drop by the #go-nuts channel on IRC. We're a friendly bunch :-)
Re:Introduction to Go 1.1 by GhotiFish on Tuesday March 26, 2013 @10:26AM (#5443470)
>And the company is betting big time on Go, more than 50% of their codebase will be (re)written in Go in a couple of years.

!!! wow !!!

Re:Introduction to Go 1.1 by mseepgood on Tuesday March 26, 2013 @11:32AM (#5443952)
Don't believe anything without a credible source citation. Not everything someone writes on the internet is true.
Re:Introduction to Go 1.1 by trailfox on Tuesday March 26, 2013 @12:18PM (#5444276)
Wild speculation hardly == "wow"