News for hackers. Stuff that matters.
Re:We switched to Node.js: the good and the not so good by DoubleCluster on Monday January 14, 2013 @06:39AM (#5054237)
> It terms of performance, we also have seen a significant (about 25%) bump in terms of feeds processed by second per server.

They rewrote the entire codebase and obtained for just a 25% gain? It doesn't sound like they are very happy to be coding in javascript now either.

Everyone: please don't rewrite your code, it's almost never worth it. The one exception is rewriting a core part of an algorithm in C for speed (the last 10x speedup).

Re:We switched to Node.js: the good and the not so good by julien on Monday January 14, 2013 @06:56AM (#5054286)
Well, the reason we rewrote was not to explode all benchmarks. The reason we rewrote was to be able to ease the maintenance of our code =)
Re:We switched to Node.js: the good and the not so good by DoubleCluster on Monday January 14, 2013 @07:16AM (#5054328)
That's fine. Did the code become easier to maintain? It's not clear from the article. The "The good" section is relatively weak and the "The not-so-good" has some painful points about memory management and api instability.

It sounds a bit like you guys rewrote it because node.js is hot and you then stuck with the rewrite because of sunk costs. Or am I imagining things?

Thanks for sharing your experiences with us. I'm just asking those questions because I want to learn more about them.

Re:We switched to Node.js: the good and the not so good by julien on Monday January 14, 2013 @07:21AM (#5054339)
Well, we assumed everyone knew about the good... so yes, we're happy with the rewrite and will probably never go back. Also, a 25% saving in servers for us translate in several thousands of dollars saved monthly. Not negligible :)
Re:We switched to Node.js: the good and the not so good by rartichoke on Monday January 14, 2013 @09:17AM (#5054695)
Yeah but in your article you said you're not even sure if this is because of node. You made some architectural changes in your code base and hinted that might have been a reason for the speed boost too.

The article might as well be written as "we refactored our code and got a nice speed boost".

Re:We switched to Node.js: the good and the not so good by octo_t on Monday January 14, 2013 @07:23AM (#5054345)
How much development time did this cost though, when you could have been adding new features and improving the user experience?
Re:We switched to Node.js: the good and the not so good by julien on Monday January 14, 2013 @07:31AM (#5054367)
Hard to assess exactly, but we were not adding feature and improving the user experience mostly because maintaining the previous code base and evolving it was so costly.
Re:We switched to Node.js: the good and the not so good by mukundmr on Monday January 14, 2013 @07:48AM (#5054406)
from a long term maintainability perspective, javascript can be notorious. you could always follow a disciplined approach for development to help with maintainability, but that applies to all programming languages. i don't see anything about javascript that makes it more maintainable especially for large projects.
Re:We switched to Node.js: the good and the not so good by julien on Monday January 14, 2013 @08:01AM (#5054443)
It's not the language, it's the people, the community.
Re:We switched to Node.js: the good and the not so good by necrodome on Monday January 14, 2013 @08:16AM (#5054480)
what's different about node community from ruby community? are you sure that's not confirmation bias?
Re:We switched to Node.js: the good and the not so good by julien on Monday January 14, 2013 @08:58AM (#5054637)
Well, again, most of the dependencies we used in Ruby did not see any update in the 3 years we've been using them. We also reported several bugs in those libraries/dependencies which were never fixed. This led us overtime to use our own branch of all the significant dependencies we had (including the MySQL gem for EM, the redis gem... and several other key ones).

Most of node modules are still in active development. As I've stated in the blog post, that's a pro and con, but we estimated that the pro was greater than the con :)

Re:We switched to Node.js: the good and the not so good by LargeWu on Monday January 14, 2013 @09:21AM (#5054705)
What leads you to believe all of these node modules that you now have dependencies on will still be in active development in 3 years?
Re:We switched to Node.js: the good and the not so good by mgkimsal on Monday January 14, 2013 @01:23PM (#5055809)
They might not be, but that doesn't matter as much as being able to report bugs and have them fixed now, while people are still actually developing/fixing things.
Re:We switched to Node.js: the good and the not so good by embwbam on Monday January 14, 2013 @01:06PM (#5055735)
TypeScript makes it way more maintainable.
Re:We switched to Node.js: the good and the not so good by amikazmi on Monday January 14, 2013 @10:58AM (#5055129)
What was the reason you choose to rewrite for ease the maintenance in node?

Couldn't you rewrite for ease the maintenance in ruby?

Re:We switched to Node.js: the good and the not so good by Peaker on Monday January 14, 2013 @01:26PM (#5055822)
My experience is the exact opposite.

Trying to make do with horrible code for long periods of time, suffering through bugs in every change... Until we finally rewrite it. I've never regretted a rewrite, and it has always ended up significantly better than the original (possibly because the horror threshold for rewriting is high, so that's not saying much).

Re:We switched to Node.js: the good and the not so good by DoubleCluster on Tuesday January 15, 2013 @04:51AM (#5059475)
In my experience it's quite doable (with a good IDE) to refactor the working but messy code into something maintainable. It might take the same amount of time as a rewrite but you'll still have all the features and corner cases covered. Usually refactoring will be much faster though.
Re:We switched to Node.js: the good and the not so good by Peaker on Tuesday January 15, 2013 @08:35AM (#5059979)
I disagree. Incremental changes are not always capable of reaching point B from point A.

A large, extremely messy code-base can make even trivial changes unsafe, let alone large changes.

The effort required not only to figure out what point B is, but also how to safely reach it from point A is immense. Much much harder than a rewrite.

However, it is a very good idea to meticulously read the bad old code and write down a list of things it handles -- to make sure none of it is missed in the rewrite.

Re:We switched to Node.js: the good and the not so good by viraptor on Monday January 14, 2013 @07:45AM (#5054402)
I'd look at it from another side. They took a working project which they knew well and transferred it completely to a different platform. The first "complete" version of the rewrite had 25% more throughput.

In that case it's a pretty good achievement, since the rewrite will have a lot more low-hanging fruit regarding performance improvements, than a polished, existing product. I'm used to see rewrites which are at least a little bit worse than previous versions due to all the work done to squeeze everything out of version N-1.

Re:We switched to Node.js: the good and the not so good by omarkj_ on Monday January 14, 2013 @06:55AM (#5054284)
There are other reasons for rewriting like getting to a more easily maintainable code or when a previous solution has simply outlived it's usefulness.

Also, C is not the only language worth rewriting in.

Re:We switched to Node.js: the good and the not so good by voidlogic on Monday January 14, 2013 @10:36AM (#5055014)
"Also, C is not the only language worth rewriting in" Agreed!

I would really like to see someone write a node.js and Go back-end for the same front-end and do a head to head comparison.

As someone who spent a fair amount of time rewriting node.js prototypes in Go, I'm probably biased. I feel like javascript is a much less maintainable language. Perhaps is was the original node.js implementations (I don't think it was), but the Go versions were always faster, used less memory and IMHO were more readable.

Re:We switched to Node.js: the good and the not so good by iends on Monday January 14, 2013 @12:16PM (#5055485)
Any particular lessons learned rewriting from node to Go?

I've got a small but growing node/socket.io app I figured I'd have to one day rewrite in go if I really wanted it to scale.

Re:We switched to Node.js: the good and the not so good by voidlogic on Monday January 14, 2013 @01:27PM (#5055828)
Nothing too unexpected, off the top of my head, I've noticed:

* Use Go tip; You can grab a snapshot review all the open issues for that snapshot (most are enhancements).

* Like any re-factor doing it sooner as opposed to later is less work :)

* Do some "from scratch" Go projects before doing re-factor projects to get your legs under you (if they are not already there)

* Write Go in Go, not C/Python/Java in Go. This is harder than you think when you get started, but, if you ask for help and people tell you you are fighting the system, carefully consider their advice.

* A lot of the Go community likes to use single letter variable names in contexts like receivers, struct state (just look at the stdlib), buck the system, don't do that, use short camel case names. The next guy / you in six months will be glad you did.

* If you have a Java/C++ background you might often write a single threaded version of a daemon and later multithread it later, this is generally an unnecessary step in Go.

* The Go versions really are not much larger (LOC)

* There is lots of useful Go code on github (don't be afraid to try them)

* If you are doing front-endy kind of stuff supplement "net/http" with Gorilla where needed whenever you can rather than rolling your own. ttp://www.gorillatoolkit.org/

* "go tool prof" is a great tool, know how to use it and its top20 / web commands. Even if you don't feel the pain, use it and you will learn what things you do are expensive and it will keep trouble from sneaking up on you.

* If you are using a SQL based store, use a driver that implements the interfaces in "database/sql" rather than providing its own interface. This will make your life very simple if you need to migrate between mySQL <-> Postgre, etc

* LiteIDE is a nice lean cross platform Go IDE that includes syntax highlighing, autocomplete (with gocode) and debugging support. The only think I had to do was write my own syntax highlighting theme, based on Solarized, because I thought the included ones were gross.

Re:We switched to Node.js: the good and the not so good by nivertech on Monday January 14, 2013 @08:42AM (#5054568)
I think the reason they only got 25% gain is because they already used Async IO with Ruby eventmachine.

Another reason might be, that their workload is IO-bounded, so faster VM like V8 doesn't help.

Re:We switched to Node.js: the good and the not so good by jahewson on Monday January 14, 2013 @09:22AM (#5054709)
If the initial version was IO-bound wouldn't they get a 0% gain from re-writing?
Re:We switched to Node.js: the good and the not so good by diroussel on Monday January 14, 2013 @09:37AM (#5054770)
If my program reads a file one byte at a time, and it's IO-bound, can I improve performance by re-writing?
Re:We switched to Node.js: the good and the not so good by jahewson on Thursday January 17, 2013 @12:39PM (#5073839)
Then it's not really IO-bound is it? It's stupidity-bound. Once you re-write it properly then it'll be IO-bound.
Re:We switched to Node.js: the good and the not so good by jonpacker on Monday January 14, 2013 @06:57AM (#5054290)
They did not say anywhere that the 25% speed gain was the reason they did it...
Re:We switched to Node.js: the good and the not so good by dnu on Monday January 14, 2013 @07:02AM (#5054307)
It depends on how large the code base is. Rewriting a 10k LOC project isn't a big deal, compared to a 10 mln one. And a rewrite doesn't have to target performance only, but ease of maintenance too.
Re:We switched to Node.js: the good and the not so good by PommeDeTerre on Monday January 14, 2013 @07:42AM (#5054388)
If ease of long-term maintenance is a goal, then JavaScript is the wrong approach.
Re:We switched to Node.js: the good and the not so good by nkohari on Monday January 14, 2013 @08:25AM (#5054501)
Well that's just, like, your opinion man.
Re:We switched to Node.js: the good and the not so good by gbog on Monday January 14, 2013 @08:47AM (#5054590)
You have better ways to ask for explanations. Not the OP but I can give some reasons for Javascript and Node.js to not be the best option for a rewrite if the goal is longterm maintainability:

- js / node.js is very young, it might fade out of fashion, and in 10 years it is not impossible that good coders in js will be hard to find.

- Readability and simplicity are prominent in this context, and js has a syntax that is less than optimal in this regard.

- Maintenance tooling may be lacking.

If I were to choose a language for a project that I new will be big and will need care in 10 or 20 years, I'd hesitate, and maybe choose Java (which I hate) or Python (which I hate less). If in a risk-things mood, I'd give Go a try.

Re:We switched to Node.js: the good and the not so good by nkohari on Monday January 14, 2013 @11:05AM (#5055153)
Technology and business requirements move too fast to worry about writing software that will be around 10 years from now. Choosing the best technology for your particular use case, that will also allow your software to evolve over time, is much more important than trying to predict the future.

Also, I've gotta say, you completely undermined all 3 points of your argument by saying you'd choose Go.

Re:We switched to Node.js: the good and the not so good by gbog on Tuesday January 15, 2013 @02:25AM (#5059148)
> Technology and business requirements move too fast to worry about writing software that will be around 10 years from now

This is a common belief but I do not think it applies very well in most cases. If you write the last pic sharing thing, maybe you can dismiss worries, but if you build the next Google, entreprise or scientific software, or even something like Github, you should hope your baby to be well and alive in 10 years, and choose your tech stack accordingly. I guess.

Re:We switched to Node.js: the good and the not so good by nkohari on Tuesday January 15, 2013 @12:33PM (#5061467)
Google in 2002 was nothing like Google in 2012. The only thing the same is the name. The entire software industry has changed at least twice in the last decade or so. If more than 10% of the original code that powered Google is still in production, I would be absolutely shocked.

Remember how Twitter was originally written in Rails, and then it collapsed under its own weight when they tried to scale it? Their business requirements changed, so they rewrote the parts that needed to be rewritten. On the flip side, if they had originally set out to try to build the system that now powers Twitter, not only would they likely have built the wrong thing entirely, they never would have launched.

Software services are evolutionary. You have to always be willing to burn pieces of it down and rewrite them as conditions change around you.

Re:We switched to Node.js: the good and the not so good by gbog on Wednesday January 16, 2013 @05:16AM (#5066184)
> Software services are evolutionary

Sure, but changing the tech stack is much harder than rewriting some parts of it. And some setups are easier to adapt than others. Thus choosing the best available option when starting a new project is important, and the likeliness of the technology to be alive and well in ten years should be taken into consideration, among other parameters (your own experience, the current availability of good developers).

Moreover, the "trendiness" of a technology should be counted as a negative factor, because it would tend to be overestimated and cast shadows on potentially better (but less sexy) alternatives.

Re:We switched to Node.js: the good and the not so good by codygman on Tuesday January 15, 2013 @01:22AM (#5059009)
I don't think he undermined the second, but would like to know your opinion. Here is mine:

"- Readability and simplicity are prominent in this context, and js has a syntax that is less than optimal in this regard."

Go code is almost always very simple and readable. What makes you think him choosing Go undermines the simplicity and readability arguments?

Re:We switched to Node.js: the good and the not so good by nkohari on Tuesday January 15, 2013 @12:35PM (#5061488)
Simplicity and readability are subjective criteria. In my opinion, Go has many merits, but readability is not one of them.
Re:We switched to Node.js: the good and the not so good by codygman on Monday January 21, 2013 @06:14PM (#5094352)
Just out of curiousity, what are your top 5 most readable programming languages that you've used?
Re:We switched to Node.js: the good and the not so good by nkohari on Monday January 28, 2013 @02:53PM (#5130277)
Of the languages that I possess more than just a passing understanding, I'd have to say: CoffeeScript, Ruby, Python, JavaScript, and C#.
Re:We switched to Node.js: the good and the not so good by OriginalSyn on Monday January 14, 2013 @09:07AM (#5054657)
I don't see how finding JavaScript programmers is going to be a problem in 10 years... even if they started now it would take, at least, that long to fully deprecate the language out of the browser. NodeJS may go away but I think betting on JavaScript going away are some pretty long odds.
Re:We switched to Node.js: the good and the not so good by niggler on Monday January 14, 2013 @09:26AM (#5054728)
Node.js may be young but 98% of the code can be reused (and if you are careful, you probably wrote the fallbacks already). V8 can be built separately.

When you understand the zen of javascript and try to write in a consistent manner, JS is more readable than C. I equate these concerns with arguments about how lisp or scheme is unreadable.

The ecosystem for tooling is expanding, albeit slowly.

However, most of your code probably could be implemented in a way that can be run in browser (e.g. XLS parser: http://niggler.github.com/js-xls/) which is where I see the real value in node. Aligning the languages means fewer moving parts and potential points of failure (as opposed to having to worry about quirks in implementations of many languages and worrying about features supported in one context but not the other)

Re:We switched to Node.js: the good and the not so good by blacktulip on Monday January 14, 2013 @06:58AM (#5054292)
I kind of agree that 25% gain doen't justify codebase rewrite, let along language/framework switching. Perhaps they are doing this with future traffic peak in mind. Premature optimisation, I know. Probably they've already cleared up other important items on their todo list.

What interested me most is the community part. I've always thought ruby/rails community is awesome. Maybe I should start learning some JavaScript now.