News for hackers. Stuff that matters.
Re:Scaling Django to 8 Billion Page Views by jacques_chester on Wednesday September 25, 2013 @04:27AM (#6443308)
The obvious question is: did you try other RDBMSes?
Re:Scaling Django to 8 Billion Page Views by nrser on Wednesday September 25, 2013 @06:28AM (#6443655)
no, that's sort of why i was asking for details. I'd be interested to know if postgres or some other open source solution could net us the 10-100x performance he's referencing. I've heard great things about postgres, but I've never used it under heavy load.

I'm under the impression that some proprietary databases on expensive hardware can achieve those levels, and some friends that do oracle installations claim that the total cost-per-query is competitive with open source / commodity hardware setups, but I've never really considered using them at my companies, and it seems like a lot of start-ups are in the same boat.

Re:Scaling Django to 8 Billion Page Views by jacques_chester on Wednesday September 25, 2013 @10:17AM (#6444699)
Yeah, the proprietary databases can be very expensive. I was told at an IBM internship day that DB2 was doing billions of transactions / day for UPS in the mid 1990s ... on clustered[1] mainframes. Not something most startups can easily get their hands on.

Basically the technological bifurcation seems to exist near the limits of a high-value corporate credit card. Which naturally changes over time.

The biggest shift in data economics in the last 5 years has been the emergence of SSDs. For startups the 2nd biggest shift has been the surged in performance of PostgreSQL; they've made huge strides in scaling it up over the last few years.

[1] http://en.wikipedia.org/wiki/IBM_Parallel_Sysplex

Re:Scaling Django to 8 Billion Page Views by mattrobenolt on Wednesday September 25, 2013 @06:33AM (#6443670)
FWIW, PostgreSQL is our primary data store. It works well. ;)
Re:Scaling Django to 8 Billion Page Views by nrser on Wednesday September 25, 2013 @10:20AM (#6444716)
like i said, i've heard good things.

what kind of QPS have you guys seen a box go up to? anywhere near the 'tens to hundreds of thousands' that [bhauer](https://news.ycombinator.com/user?id=bhauer) threw out there?

45K req/sec is big, btw. props, yo.