News for hackers. Stuff that matters.
Re:How We Went from 30 Servers to 2: Go by waterside81 on Tuesday March 12, 2013 @07:08PM (#5365625)
Go's definitely picking up momentum. I know that Mozilla is shifting much of its services infrastructure to Go.

We're porting over our Arabic sentiment engine, currently in Python/Cython to Go. If you're dealing with simple data structures, going from Python to Go is almost a line-for-line port, but the performance benefits are, of course, massive. Our benchmarks show a 40x speed improvement so far.

Lastly, for anyone thinking about taking the plunge, use Go tip (from their source code repo) - don't use their "stable" releases. They fix bugs so fast you'll always want to be current.

Re:How We Went from 30 Servers to 2: Go by DASD on Tuesday March 12, 2013 @07:11PM (#5365638)
Do you have a citation for Mozilla using Go? I would be a bit surprised given their(Mozilla) development of Rust.
Re:How We Went from 30 Servers to 2: Go by waterside81 on Tuesday March 12, 2013 @07:15PM (#5365658)
Sure, check out their github pages (here's a few):

* https://github.com/mozilla-services/heka-mozsvc-plugins

* https://github.com/mozilla-services/heka

My friend works for Mozilla - that's how I knew.

Re:How We Went from 30 Servers to 2: Go by DASD on Tuesday March 12, 2013 @07:34PM (#5365752)
Thanks waterside81. Can anyone explain the difference between Mozilla and Mozilla services? A cursory glance seems some of the Github repositories overlap.
Re:How We Went from 30 Servers to 2: Go by rfk on Tuesday March 12, 2013 @08:23PM (#5365961)
Mozilla Services is the team within Mozilla that builds/runs much of the backend infrastructure, e.g. the firefox sync servers, marketplace servers etc. The existence of separate "mozilla" and "mozilla-services" github projects is largely a historical accident, since different teams started moving to github organically at different times.

(Source: I work for Mozilla, on the Services team)

Re:How We Went from 30 Servers to 2: Go by mh- on Tuesday March 12, 2013 @10:39PM (#5366459)
Source: I work for Mozilla-Services
Re:How We Went from 30 Servers to 2: Go by azakai on Tuesday March 12, 2013 @07:16PM (#5365659)
Rust and Go are not similar or competing. They are both new, but aside from that the differences are huge. So if a project uses one, that doesn't mean the other was a possibility for that project too.
Re:How We Went from 30 Servers to 2: Go by jbooth on Tuesday March 12, 2013 @10:44PM (#5366480)
They're not similar at all, but both were designed to replace C++. That sort of means that they're competing for the use-case of "things I would write in C++ if it wasn't a god-awful nightmare to do so".

From there, though, you're right that they take very, very different approaches.

Re:How We Went from 30 Servers to 2: Go by azakai on Tuesday March 12, 2013 @11:34PM (#5366650)
> They're not similar at all, but both were designed to replace C++.

In practice, the main use for Go appears to be like this story - an alternative to RoR, Node, Python/Django, etc. - and not a C++ alternative. Go and Rust may both start from the idea of doing C++ or something similar "right", but end up in very different places.

Re:How We Went from 30 Servers to 2: Go by coldtea on Wednesday March 13, 2013 @05:31AM (#5367553)
>They're not similar at all, but both were designed to replace C++.

Well, Rust WAS designed to replace C++, both in the intention behind its design and the way it was done.

Go designers had this vague intention of "replacing C++", but the way they have designed the language they only really replace Java or some scripting language. Which they admit (the get mostly Python etc converts than C/C++ converts).

Re:How We Went from 30 Servers to 2: Go by steveklabnik on Tuesday March 12, 2013 @07:29PM (#5365727)
The Rust team agrees: https://github.com/mozilla/rust/wiki/Doc-language-FAQ (control-f for 'this Google language')
Re:How We Went from 30 Servers to 2: Go by DASD on Tuesday March 12, 2013 @07:21PM (#5365689)
Indeed they are not similar but wildly different tools are often used in a similar or competing use cases(eg. the blog's post of the company using Ruby and conversion to Go). I had asked because if you were to reverse the usage do you think Google is likely using much Rust since it is developing Go?

Just to clarify, I see a bright future for both languages.

Re:How We Went from 30 Servers to 2: Go by tmzt on Tuesday March 12, 2013 @09:35PM (#5366241)
It would be interesting to see Rust used in Android in some of the parts that are now C or C++.
Re:How We Went from 30 Servers to 2: Go by Symmetry on Wednesday March 13, 2013 @11:08AM (#5368957)
The way I would put it is that Go was designed to incorporate some of the simplicity and speed of C as well as the ease of use of Python, while Rust was designed to combine the speed and flexibility of C++ with the safety of Haskell.
Re:How We Went from 30 Servers to 2: Go by steveklabnik on Wednesday March 13, 2013 @05:28PM (#5371229)
... and the concurrency of Erlang. (and OCaml might be more apt than Haskell).
Re:How We Went from 30 Servers to 2: Go by stcredzero on Tuesday March 12, 2013 @07:44PM (#5365798)
> Rust and Go are not similar or competing.

Rust is more "kitchen sink" (done as elegantly and cleanly as possible) and Go is more minimalist.

Re:How We Went from 30 Servers to 2: Go by pcwalton on Tuesday March 12, 2013 @07:56PM (#5365845)
Being "kitchen sink" isn't a design goal of Rust. Rather, Rust is a lower-level, safer language. Go is higher-level.

Rust strives to be as minimalist as possible without sacrificing the goals of low-level control over memory and C++ performance (optional GC), memory safety, race-free concurrency, and type safety (no null pointers).

Re:How We Went from 30 Servers to 2: Go by duaneb on Tuesday March 12, 2013 @08:43PM (#5366041)
I don't think he meant kitchen sink, more that there are fundamental aspects from many parts of languages that are bound together well, which makes

> Rust strives to be as minimalist as possible without sacrificing the goals of low-level control over memory and C++ performance (optional GC), memory safety, race-free concurrency, and type safety (no null pointers).

somewhat funny.

Re:How We Went from 30 Servers to 2: Go by codygman on Wednesday March 13, 2013 @04:37AM (#5367431)
Why funny? (coming from someone who's thinking about giving Rust a try)
Re:How We Went from 30 Servers to 2: Go by duaneb on Thursday March 14, 2013 @12:37PM (#5376007)
He said "rust doesn't have the kitchen sink", and then listed the kitchen sink. Rust is awesome, you should use it.
Re:How We Went from 30 Servers to 2: Go by codygman on Thursday March 14, 2013 @01:14PM (#5376266)
Alright, I'll download the compiler/etc right now :D
Re:How We Went from 30 Servers to 2: Go by laut on Wednesday March 13, 2013 @12:42AM (#5366856)
Go is from 2009. Ruby is from 1995. Although it took about 10 years for Ruby to really start to be popular in the English speaking world.
Re:How We Went from 30 Servers to 2: Go by gnuvince on Wednesday March 13, 2013 @12:49AM (#5366868)
Rust, not Ruby.
Re:How We Went from 30 Servers to 2: Go by cpprototypes on Tuesday March 12, 2013 @07:53PM (#5365833)
Just curious, what were the tradeoffs for profiling the Python code and rewriting slow parts in C vs total rewrite in Go? For high level languages, the usual argument has been to rewrite just the slow parts in C or some other low level language.
Re:How We Went from 30 Servers to 2: Go by waterside81 on Tuesday March 12, 2013 @08:43PM (#5366046)
And that's what we did during our first pass: identify the slow parts and port them to C. This gave us about 30% better performance. Without getting too much into the nitty gritty of our specific case, this wasn't enough.

We needed some massive speed improvements, I'm talking in the order of 100X faster. The nature of our algorithms was such that they could be done in parallel (i.e map/reduce) - an ideal candidate for Go's goroutines. We actually tried to make it parallelizable at first in Python, using gevent (and even just multiprocessing) and the results were not great.

One other aspect that really guided us towards Go was memory usage. Python was just sucking up so much memory whereas our Go implementation thus far is so much thinner.

Re:How We Went from 30 Servers to 2: Go by danieldk on Wednesday March 13, 2013 @05:53AM (#5367604)
One other aspect that really guided us towards Go was memory usage. Python was just sucking up so much memory whereas our Go implementation thus far is so much thinner.

It's funny that you mention that, because in computationally intensive work memory management is an issue with Go. E.g. I wrote a maximum entropy parameter estimator in Go, which was terribly slow until I circumvented the garbage collector by preallocating a huge block of memory and doing my own management. In C malloc() and free()-ing had practically no overhead. After putting the Go garbage collector out of the game, the Go version was approximately within 2x of the C version.

I am interested how Go gave you one or two orders of magnitude speedup, while rewriting hot spots in C didn't...

Re:How We Went from 30 Servers to 2: Go by Ziomislaw on Wednesday March 13, 2013 @06:16AM (#5367658)
> I am interested how Go gave you one or two orders of magnitude speedup, while rewriting hot spots in C didn't...

paralellization

Re:How We Went from 30 Servers to 2: Go by danieldk on Wednesday March 13, 2013 @09:28AM (#5368194)
One can easily do that in C with OpenMP.
Re:How We Went from 30 Servers to 2: Go by Symmetry on Wednesday March 13, 2013 @11:14AM (#5369011)
If you can get away with only parallelizing the C code that'll work, but if you're replacing individual Python functions calls it won't so much. You could use a lot of Python processes, but that might be awkward depending on how much they need to coordinate and might lead to too much memory usage.
Re:How We Went from 30 Servers to 2: Go by codygman on Tuesday March 12, 2013 @08:37PM (#5366015)
One thing you don't get in the C/Python combo is total static typing. The other big win of Go is elimination of a lot of runtime errors.

So it comes down to which will be more of a win for you and your product. Go has a big advantage here IMO though because of static typing AND performance increase.

Re:How We Went from 30 Servers to 2: Go by pjmlp on Wednesday March 13, 2013 @03:31AM (#5367288)
Like any other language with static typing and available compiler implementations.
Re:How We Went from 30 Servers to 2: Go by codygman on Wednesday March 13, 2013 @04:42AM (#5367445)
Is that relevant though? The parent is asking about a comparison between python+c and Go for the OP's software, so doesn't it make sense to answer to that?

Of course, you are correct. Rewriting in any other language with static typing or an available compiler implementation would also provide the benefits of static typing and detection of runtime errors.

Re:How We Went from 30 Servers to 2: Go by pjmlp on Wednesday March 13, 2013 @05:52AM (#5367603)
Because those benefits are not some special feature of Go.

On the other hand I think youth generation of programmers lack exposition to programming languages, like we used to have in the old days, hence they make quite limited comparisons.

Re:How We Went from 30 Servers to 2: Go by beatpanda on Wednesday March 13, 2013 @01:22AM (#5366964)
Hi, I'm in the market* for an Arabic sentiment engine for some academic research, can you contact me? My email is in my profile.

*Note that "in the market" doesn't mean I have a lot of money to spend :D

Re:How We Went from 30 Servers to 2: Go by adamnemecek on Wednesday March 13, 2013 @02:31AM (#5367151)
What is an "Arabic sentiment engine" :-D? Like textual analysis as to whether a piece of Arabic text is positive or negative?