News for hackers. Stuff that matters.
Re:Building A Node.js Server That Won't Melt by jeffasinger on Tuesday January 15, 2013 @01:15PM (#5061743)
Why you would do this, instead of putting your node server behind a more battle tested proxy, like Varnish or HAProxy, and limiting the number of simultaneous connections.
Re:Building A Node.js Server That Won't Melt by amatix on Tuesday January 15, 2013 @01:57PM (#5062093)
Do both: each layer and service in an application should be able to manage its own load. Helps prevent cascading failures too.

Read up on Netflix's architecture, where they've talked about this a lot. And the book "Release It!: Design and Deploy Production-Ready Software" (Nygard), though Java-oriented in its examples, covers the concepts well.

Re:Building A Node.js Server That Won't Melt by lloydhilaiel on Tuesday January 15, 2013 @01:22PM (#5061804)
Our problem is the the number of simultaneous connections we can support isn't static, it varies depending on the type of traffic bursts (i.e. New vs. Returning users).

But agree that a higher level proxy is useful - because this way your overloaded application doesn't even need to deal with traffic (we hope to use toobusy to have applications instruct the routing layer to temporarily block / reroute traffic in times of load).

Re:Building A Node.js Server That Won't Melt by jeffasinger on Tuesday January 15, 2013 @01:51PM (#5062050)
The reason that I asked that is I feel that most applications end up needing something at that layer eventually anyways, but not knowing exactly what level of traffic results in too much load is a good reason to do this.
Re:Building A Node.js Server That Won't Melt by davidw on Tuesday January 15, 2013 @04:22PM (#5063089)
It seems to me that letting the computer calculate when it's in trouble is preferable to have a human guessing.
Re:Building A Node.js Server That Won't Melt by gfloyd on Tuesday January 15, 2013 @02:18PM (#5062270)
I think this would simplify implementing a load-balancing proxy in some ways. If a server responds with a 503 generated by node-toobusy, the load balancer will know immediately to reroute the request to another server, rather than having to wait for a timeout or some other threshold.
Re:Building A Node.js Server That Won't Melt by toong on Tuesday January 15, 2013 @05:36PM (#5063545)
So this load-balancer is balancing between A&B. Now A gives back a few 503's, you are going to send everything over to B ?

[edit] I do get your point, I was just saying: don't start implementing a naive load-balancer :-)

Re:Building A Node.js Server That Won't Melt by benologist on Tuesday January 15, 2013 @01:27PM (#5061847)
You'd do it because it's

  npm install toobusy 
then somewhere in your project

  require("toobusy");
If you only need one server that's going to be up to a lot faster than putting anything in front of it.
Re:Building A Node.js Server That Won't Melt by byoung2 on Tuesday January 15, 2013 @06:47PM (#5063974)
If you only need one server that's going to be up to a lot faster than putting anything in front of it.

A single server setup might be simpler, but it won't be faster. Varnish serving a cached page from memory is going to be faster than 5 asynchronous calls that take 5ms of CPU time (and filesystem I/O in the case of the database and template given in the example). With varnish, even with a 1 second TTL (and 1 second grace), your first request will take the 5ms hit, but the next 199 for that second will be served from memory.

Now with Varnish serving 199 out of 200 requests from memory, if your backend is still toobusy, by all means serve a 503, and Varnish can cache that too.

Re:Building A Node.js Server That Won't Melt by AndyKelley on Tuesday January 15, 2013 @08:23PM (#5064482)
You have a problem, and so you try to solve it with caching. Now you have two problems.

I think that's how that quote goes.

Re:Building A Node.js Server That Won't Melt by byoung2 on Tuesday January 15, 2013 @10:45PM (#5065110)
I think of caching for apps like clothes for people...while you could survive naked, it's more comfortable with clothes, and you're protected from heat, cold, etc. Of course there is the problem of being over/under dressed, but you have to wear something.
Re:Building A Node.js Server That Won't Melt by benologist on Wednesday January 16, 2013 @04:05AM (#5066038)
That's super unnecessary for a single server, you can literally just use a variable outside of your request handling as a cache.

  var cache;

  module.exports = function(request, response) {
    if(cache) {
      return response.end(cache); 
    }

    // get my data from wherever
    cache = the_data;

    return response.end(cache);
  }
Now 199 out of 200 requests are from memory, there are zero extra moving parts, and you're using a cool part of the language instead of a 3rd party tool you have to select and configure.
Re:Building A Node.js Server That Won't Melt by byoung2 on Wednesday January 16, 2013 @05:38AM (#5066247)
How do you selectively serve the cached response to some users based on cookies, header, etc., and expire it after a set amount of time? How would you gather stats about how many cache hits vs misses you have? How do you serve the cached response when your server is pinned? These are problems 3rd party tools have solved.
Re:Building A Node.js Server That Won't Melt by benologist on Wednesday January 16, 2013 @07:11AM (#5066451)
You can expire things with setInterval and a counter. You serve a cached response if your server has a cached response, or if you are using it as a fallback you would combine it with something like toobusy maybe so you have a functional (if not fresh) 'under load' page.

Where it gets really fun with NodeJS is you can do all of your data work outside of the requests so you can pull all of your content out at the start and refresh it on an interval independently of the users, which can eliminate some or all of their trips to the database if you're lucky and it fits in ram and is viable etc.

The less moving parts that need to cooperate to serve your site the better - most things don't warrant a deep stack of technology to serve HTML and run CRUD operations.

Re:Building A Node.js Server That Won't Melt by jongleberry on Tuesday January 15, 2013 @09:03PM (#5064667)
and what if your whole site is dynamic? varnish doesn't do anything.
Re:Building A Node.js Server That Won't Melt by byoung2 on Tuesday January 15, 2013 @10:40PM (#5065095)
Very few sites are so dynamic that something can't be cached for at least a second. Even a heavily dynamic site like HN with people commenting all the time could at least cache for logged out users.
Re:Building A Node.js Server That Won't Melt by lazyjones on Tuesday January 15, 2013 @01:29PM (#5061860)
I'd prefer that solution too, especially since such proxies can mitigate some types of DoS attacks (e.g. known patterns in the "Referer" header, or from known IP address ranges) and load will generally be much lower if some of the popular content is cacheable.
Re:Building A Node.js Server That Won't Melt by Cushman on Tuesday January 15, 2013 @01:19PM (#5061780)
Because that is a lot of work, and this is a drop-in solution for any node application?