Western civilization runs on the mainframe

Re:Western civilization runs on the mainframe by rbranson on Monday August 02, 2010 @12:20PM (#1568041)

Web companies operate in a different world entirely. If Twitter goes down, it's sad, but nobody really cares. They also don't have very much money, so running lean on development time and hardware is critical to survival. If Twitter had the IT budgets like banks and insurance companies, things would be a little different. Of course scaling isn't that hard when you have billion dollar budgets. Per byte of content processed, Twitter probably operates at orders of magnitude more efficiency than banks or insurance companies. Each byte of data HAS to cost almost nothing, or the business can't exist. Nobody would pay for social networking, so it has to be dirt ass cheap.

[ Parent | on HN ]

Re:Western civilization runs on the mainframe by gaius on Monday August 02, 2010 @12:42PM (#1568111)

The fascinating thing is that, including VC, Twitter really did - does - have a comparable IT budget. They could have just licensed Tibco Rendezvous, which is how banks and exchanges distribute thousands of price updates/sec, to the level of this desk at this bank is allowed to see that, but that desk isn't, and the other desk over there gets it 20 seconds later. Instead they went of and tilted at windmills, all the while telling themselves that they were breaking new ground.

[ Parent | on HN ]

Re:Western civilization runs on the mainframe by rbranson on Monday August 02, 2010 @01:37PM (#1568329)

I don't think the economics quite work out that way. Most enterprise scaling products are extremely expensive, have long sales cycles, and require lots of expertise to install and maintain them. They affect your application architecture design because of the per-server or per-socket licensing. You're also pretty much going to have to use that once it's purchased and put in place. If it no longer fits your business need, you've just wasted a ton of money. In fast-moving businesses this is absurd. For slow-moving industrial-age Fortune 100s, it makes more sense.

It also really sucks when you run into a problem and can't open the damn thing up. Case in point: I was working with some code that was instrumented with HP OpenView OVPA to collect performance analytics. When the new version of OVPA came out, we were planning a huge rollout to bring all 4k+ servers up to date. Problem is, the new major release had moved to a pthreads library and changed a bunch of the threading code, causing a 2 second lockup on start due to some sloppy code. For the vast majority of server software, this is almost undetectable, but these were CGIs that were invoked millions of times per hour. It took me about a day to locate the problem with truss (Solaris' strace). I had to spend a week in 5am meetings with HP's offshore development team to convince them that it was their bug. Of course, it took another week to get a patch.

[ Parent | on HN ]

Re:Western civilization runs on the mainframe by jacquesm on Monday August 02, 2010 @04:03PM (#1568853)

That's very aptly put.

Unless you need many thousands of servers the majority of funded web application companies could very well afford the best of what the OTP world has to offer them. But since that isn't 'sexy tech' they want to re-invent those wheels.

Twitter has since switched to Erlang, I think they've realised that customer satisfaction beats new technology any day.

[ Parent | on HN ]

Re:Western civilization runs on the mainframe by kristianp on Tuesday August 03, 2010 @08:24AM (#1570871)

Twitter actually uses Scala, I believe.

[ Parent | on HN ]

Re:Western civilization runs on the mainframe by jacquesm on Tuesday August 03, 2010 @08:48AM (#1570931)

Aye, you're right:

http://www.theregister.co.uk/2009/04/01/twitter_on_scala/

Memory error, I should have my dimms checked ;)

[ Parent | on HN ]