News for hackers. Stuff that matters.
Re:We 30x'd our Node parallelism by eshyong on Friday December 13, 2019 @11:06AM (#21782757)
I mean, running multiple node runtimes (aka multiprocessing) actually sounds like a reasonable compromise for parallelism. That's the standard solution for dynamic languages without great multithreading support. If you needed great multithreading support then Node probably wasn't the right choice for you in the first place, but for most applications, it's probably fine.

However, running multiple containers for parallelism sounds a little bit crazy. In the worst case, each container may be running on its own server, but even assuming multiple containers per host, I'm guessing they were running an insignificant number of instances, which is probably why they were able to save $300k in server costs.

Re:We 30x'd our Node parallelism by phoe-krk on Friday December 13, 2019 @11:12AM (#21782816)
Yet this is the case mentioned in the article.

> We were running 4,000 Node containers (or "workers") for our bank integration service.

Re:We 30x'd our Node parallelism by eshyong on Friday December 13, 2019 @11:23AM (#21782932)
Yes, I agree. I'm arguing against your characterization of node as a poor runtime. For many line-of-business applications node is a fine choice. However, plaid has to integrate with many banks which only expose web pages, not APIs, so I'm guessing that they have to do a non-trivial amount of CPU work to scrape and process HTML responses. For this, it may not be such a good choice.

All I'm saying is that the choice of language is (usually) not the issue. Poor architecture design causes a lot more problems than whether you choose python/java/ruby/node for your webapp.

Re:We 30x'd our Node parallelism by neillyons on Saturday December 14, 2019 @04:15AM (#21788591)
Yeah I’m confused why they didn’t start with running multiple processors per container. There must be a reason?