Re:Twitter Shifting More Code to JVM, Citing Performance and...
by raganwald on Monday July 04, 2011 @12:59PM
(#2727044)
It is incredibly easy to design for growth when looking in a rear view mirror. For example, it is easy NOW to see that Twitter's problem was scaling, and that they should have planned to grow their scalability while evolving their user experience.
However, at the time it might have gone the other way: Perhaps their user base and volume might have increased at a slower place, leaving plenty of time to evolve their scalability, while their user experience might have required relentless change.
In which case, they should have planned for their user experience to "scale" rather than their infrastructure.
Re:Twitter Shifting More Code to JVM, Citing Performance and...
by eropple on Monday July 04, 2011 @03:35PM
(#2727583)
True. That said, designing for scalability isn't that hard, and does not take that much more time than just bunging out whatever comes first to mind.
Re:Twitter Shifting More Code to JVM, Citing Performance and...
by jshen on Tuesday July 05, 2011 @01:43AM
(#2728781)
You're assuming that you know ahead of time what the bottlenecks are, and more importantly, which trade offs to make, because scaling is almost always about trade offs.
A scalable solution is usually a more rigid solution as well, and that can be a problem for a startup that may need to try a few ideas before they hit on what works.
Re:Twitter Shifting More Code to JVM, Citing Performance and...
by dasil003 on Monday July 04, 2011 @06:56PM
(#2728117)
This is armchair architect thinking. Sure, coding to the interface makes scaling easier, it also makes maintenance, testing and everything else easier: it's just a good idea. But the devil is in the details. When you have to scale like Twitter you are getting into the territory where you have to do a lot of legwork just to figure out how to scale a particular interface. Performance and consistency issues start to gnaw away at the ideal interface you carefully designed back when you were 20k users and 10 reqs/s.
The key is, when designing your growth architecture (and I don't consider it overengineering--a few simple decisions allow you to scale pretty well to a fairly large load), is to code to interface rather than code to implementation. If things evolve/change over time, you've already got half the work done for you because things don't spontaneously break on change.