News for hackers. Stuff that matters.
Re:Goodbye Academia by silverlake on Monday March 03, 2014 @04:18PM (#7336552)
You can write a superficial clone of any web app. But can you handle 8000 tweets/sec in real-time? The innovation is in the back-end.
Re:Goodbye Academia by pyrrhotech on Monday March 03, 2014 @04:22PM (#7336571)
It's been done notoriously badly at Twitter, no offense if you are an employee. Sure it's gotten better since they switched to the JVM, but still scaling to that size has been done by hundreds of companies now. Once there are textbooks out describing the techniques, you are simply implementing known methodologies with perhaps a few customizations and tweaks. I don't really see much innovation there TBH.
Re:Goodbye Academia by silverlake on Monday March 03, 2014 @04:42PM (#7336703)
I'm neither an employee nor a twitter user. But I've built phone networks and appreciate the absurd complexity of very large systems. If you believe you can build twitter, you're either a brilliant programmer or ignorant of the real challenges. Look at Twitter's code on github and tell me that's just a simple tweak on a known methodology.
Re:Goodbye Academia by AllTheThings on Monday March 03, 2014 @04:59PM (#7336817)
Twitter, Facebook, and LinkedIn (along with other big tech companies) all produce a lot of academic research that documents the architectures and protocols that they use, along with their performance. These companies innovate, and the material from their research often makes it into these "textbooks ... describing the techniques".
Re:Goodbye Academia by cnlwsu on Tuesday March 04, 2014 @11:50AM (#7341358)
Lets not trivialize a truly difficult problem just cause mongodb has some huge numbers on their marketing literature. Its naive to believe that scaling to the level of twitter is easy.

Sometimes you have to be innovative just to implement simple ideas in those text books or to make it actually efficient. Its easy to use type 1 UUIDs as ids but if its not practical to store 128bit ids for every value or you need to pack it into 64bits or need better ordering guarantees. you build something like snowflake.