News for hackers. Stuff that matters.
Re:Help Scale NPM by judofyr on Tuesday November 26, 2013 @11:42AM (#6802387)
Wouldn't it be better to make NPM more distributed so that anyone could set up a mirror and help out?

EDIT: Not saying it would be easy; I'm just wondering if you've considered this direction.

Re:Help Scale NPM by indexzero on Tuesday November 26, 2013 @12:11PM (#6802600)
Anyone can make a mirror. That's the glory of CouchDB. Just kick off replication and BOOM you've got the npm registry. There are community mirrors in Europe (http://npmjs.eu) and Australia.

If you want to run or use a community mirror that's totally great!

npm config set registry http://my.awesome.community.mirror

Re:Help Scale NPM by addandsubtract on Tuesday November 26, 2013 @05:21PM (#6804700)
This is great and all, but when Heroku relies on the main npm registry to be up (when it's not) you're screwed.
Re:Help Scale NPM by twalling on Wednesday November 27, 2013 @09:13AM (#6808087)
Heroku customers can always clone the nodejs buildpack and simply point the npm registry at another mirror.
Re:Help Scale NPM by nfriedly on Wednesday November 27, 2013 @09:28AM (#6808153)
I've started to always check in my node_modules directory. Heroku automatically runs `npm rebuild` in that case, so native modules always work. I've found it to be both faster and more reliable.
Re:Help Scale NPM by applecore on Tuesday November 26, 2013 @12:19PM (#6802654)
I'm surprised they're using CouchDB and not MongoDB.
Re:Help Scale NPM by nathan7 on Tuesday November 26, 2013 @01:15PM (#6803031)
npm is pretty much the prime use-case for CouchDB. REST API out of the box, replication is a core feature and not just a scaling feature (multi-master, MVCC) and the validation/access control is pretty much made for it. The npm registry is implemented as a CouchApp for a reason.
Re:Help Scale NPM by remon on Tuesday November 26, 2013 @03:23PM (#6803932)
I agree that from a functionality perspective this is largely true, at least on paper. That said, if running costs become a significant bottleneck CouchDB becomes a less obvious choice. Sometimes there's a reluctance to migrate from one technology to another as requirements change over time but this seems one of those occasions where exactly that step is required. Sometimes it's good to take a step back, look at your current requirements in terms of cost and performance and determine what technology best suits your needs. I would question the know-how and objectivity of anyone that would land on CouchDB/node.js in this instance.
Re:Help Scale NPM by nathan7 on Tuesday November 26, 2013 @04:20PM (#6804296)
There's no node involved in the registry itself. This is purely CouchDB.
Re:Help Scale NPM by nawitus on Tuesday November 26, 2013 @01:11PM (#6802995)
CouchDB has better replication support (e.g. multi-master, MVCC).
Re:Help Scale NPM by cleverjake on Tuesday November 26, 2013 @12:55PM (#6802892)
a lot of the central node community started on couch
Re:Help Scale NPM by twerquie on Tuesday November 26, 2013 @12:52PM (#6802858)
Why?
Re:Help Scale NPM by agentultra on Tuesday November 26, 2013 @11:57AM (#6802487)
It's hard but the payoff is worth it. Just look at CPAN or any linux pacakge distribution system. The package hosting is all mirrored.
Re:Help Scale NPM by quarterto on Tuesday November 26, 2013 @12:07PM (#6802572)
Re:Help Scale NPM by dmourati on Tuesday November 26, 2013 @01:20PM (#6803065)
This was what I said when the repository went down for over eight hours a few weeks back. Part one is scaling the server, part two is making the client aware of the mirrors and gracefully handling temporary unavailability. From the looks of it, this all seems above the heads of node's current leadership which frankly sours me on relying on node at all.
Re:Help Scale NPM by malandrew on Tuesday November 26, 2013 @03:55PM (#6804141)
Make it into a docker app please. I would love to be able to set up a bunch of these commonly used registries as a distributed app I can run on my machine with very little overhead. Have it set up to pull in bulk changes as a cron job or via torrents maybe?

Having my own personal NPM and own personal RubyGems would be awesome.

Re:Help Scale NPM by regularfry on Tuesday November 26, 2013 @06:41PM (#6805118)
Your own personal rubygems is downright trivial, given a constrained set of gems. Upload .gems to a directory, build an index, serve static files. I'm guessing there's something about npm packages which makes that impossible.
Re:Help Scale NPM by iloveponies on Wednesday November 27, 2013 @08:29AM (#6807894)
Yes, npm itself is the only reason why.
Re:Help Scale NPM by candyluver13 on Wednesday November 27, 2013 @09:05AM (#6808058)
https://github.com/jmgunn87/mynpm <- docker app is jmgunn87/mynpm
Re:Help Scale NPM by jenius on Tuesday November 26, 2013 @11:57AM (#6802493)
Would also be interested to hear someone from nodejitsu's response to this.