News for hackers. Stuff that matters.
Re:Google's ‘Gopher Team’ by ancarda on Monday July 29, 2013 @09:01AM (#6120747)
I really don't understand why dl.google.com isn't just running Nginx or some web server. It's just serving files. Why does it need software written in house?
Re:Google's ‘Gopher Team’ by skriticos2 on Monday July 29, 2013 @09:48AM (#6121068)
Nginx was built with 'single server' architecture in mind while Google generally operates on a 'distributed everything' architecture. This involves stuff like distributed file systems/data stores. My guess is it's not compatible with the single file system interfaces of Nginx. Also note that dl.google.com serves stuff on a massively bigger scale than the majority of the rest of the internet. On that scale I'd guess they face some rather uncommon challenges and bottlenecks that can only be addressed by custom software.
Re:Google's ‘Gopher Team’ by x1024 on Monday July 29, 2013 @03:21PM (#6123257)
True, but a static-file server should be trivially scalable. That is, scalable by placing it behind a load balancer(or using any number of alternative methods that don't acutally touch the server itself). Even if the server implements Access Control, it should still be trivially scalable.

Also, nginx doesn't have such...interesting ideas as the custom server: http://talks.golang.org/2013/oscon-dl.slide#19

I mean, if by "rather uncommon challenges" you're referring to the "what are threads?" design philosophy of the original... then yeah, it does require "custom software".

Re:Google's ‘Gopher Team’ by ihsw on Monday July 29, 2013 @10:25AM (#6121260)
dl.google.com isn't just a place to dump files to the public web, it's also got complicated ACL schemes that define restrictions on data access. dl.google.com is used internally.
Re:Google's ‘Gopher Team’ by GeneralMayhem on Monday July 29, 2013 @09:38AM (#6120985)
Google is the only company on the planet with anything approaching its scale requirements. It's entirely possible that even nginx breaks down at their level of speed/concurrency/distribution.
Re:Google's ‘Gopher Team’ by x1024 on Monday July 29, 2013 @03:27PM (#6123293)
True, but it isn't the only company serving apt packages, and it isn't even the biggest one. Considering that their apt server was as slow as it was(http://talks.golang.org/2013/oscon-dl.slide#9) - they obviously did something wrong that everyone else was doing right.

Perhaps the problem was that dl.google.com was being used for too many things at once, but this doesn't excuse building custom software that offers worse performance than free off-the-shelf products that literally everyone else has been using for years.