Re:Scaling node.js to 100k concurrent connections
by babuskov on Thursday August 09, 2012 @10:56PM
(#4364631)
Interesting. What's the system's uptime? Close to 12h or rather not? If the latter, this still means 1-2 restarts per day.
Regarding crashes, do you know of any special things to look out for? I do crash dumps and log uncaught exceptions, but sometimes node simply dies without any trace in the log files.
Re:Scaling node.js to 100k concurrent connections
by benologist on Friday August 10, 2012 @12:04AM
(#4364829)
The highest I've seen on a dyno (this is all on Heroku) is over a day, web #13 and #14 are on 22 hours now. I think they're actually eventually running out of memory or being retired and replaced by Heroku rather than crashing but I'm not sure, it's not being caught in the exception catching.
Most of the crashes come down to stupid things, it's so easy to make a mistake when you don't have a compiler watching your back. External dependencies can hurt if they're laggy or unavailable. Unterminated requests are a really easy accident as well.
At this point I just use exception catching and dump the results into Redis unless I'm specifically hunting down a bug and want the crash to occur:
It took a while to iron out most cases that can crash, right now I have:
web.1: up for 12h
web.2: up for 12h
web.3: up for 12h
web.4: up for 12h
web.5: up for 12h
web.6: up for 4h
web.7: up for 1h
web.8: up for 12h
web.9: up for 12h
web.10: up for 12h
web.11: up for 32m
web.12: up for 12h
web.13: up for 7h
web.14: up for 7h