News for hackers. Stuff that matters.
Re:Docker not ready for primetime by cortesoft on Sunday August 28, 2016 @04:33PM (#12378391)
I know you probably aren't trying to address all cases, but just because you can't re-build your prod infrastructure in minutes or hours doesn't mean you aren't doing devops right.

Many larger companies can't do this; my company has 70+ datacenters with tens of thousands of servers. We can't re-build our prod infra in minutes or hours. We are still doing devops right :D

Like I said, I know you aren't talking about my situation when you made your statement... I just get frustrated when people act like there are hard and fast rules for everyone.

Re:Docker not ready for primetime by StreamBright on Sunday August 28, 2016 @04:51PM (#12378468)
Well I am not talking about a 1M node outage. That you cannot fix with anything. I am talking about a maximum datacenter wide outage, that actually happens pretty often. Amazon has game days, Netflix has chaos monkeys for the same reason. Make sure that you can rebuild parts of your infra pretty quick.
Re:Docker not ready for primetime by toomuchtodo on Sunday August 28, 2016 @06:13PM (#12378793)
As long as your VMs are prebaked and your cloud provider supports ASG-esq primatives (i.e. "I need X instances running at a time, instantiate with such and such meta data") anyone can rebuild their prod infra quickly. You don't need Docker or containers to do that.
Re:Docker not ready for primetime by cortesoft on Sunday August 28, 2016 @07:25PM (#12379046)
Some of us work for those cloud providers :)
Re:Docker not ready for primetime by toomuchtodo on Sunday August 28, 2016 @08:12PM (#12379213)
I have no doubt ;) it's why I always strive for accuracy here!
Re:Docker not ready for primetime by cortesoft on Sunday August 28, 2016 @06:01PM (#12378750)
Oh, of course. Our datacenters often go offline (both planned and unplanned), and we are always ready to handle that. We are pretty much constantly re-provisioning servers... with so many physical machines, hard drive and other hardware failures are a daily occurrence.
Re:Docker not ready for primetime by ckdarby on Sunday August 28, 2016 @06:41PM (#12378884)
I'm going to have to call bullshit, Where do you work or what company do you own?

I work at MindGeek, depending on the time of the year it would be fair to say we rank within the top 25 bandwidth users in the world. We are not even close to that amount of servers and we deal with some of the largest traffic in the world. What company is running in 70+ datacenters!? World's largest VPN provider? Security company providing all the data to the NSA?

Maybe it is just my broad assumptions but I would hope that the major big 10 that come to mind such as Google, Amazon, Microsoft, etc would be able to rebuild their production regions in hours.

Re:Docker not ready for primetime by Sanddancer on Sunday August 28, 2016 @06:52PM (#12378923)
Not the OP, but first thing that comes to mind as to why you'd need a lot of datacenters are CDNs like Cloudflare or Akamai. Stuff like that, you need lots of servers, lots of storage, and low latency. You'd also need a good amount of configurations because things like video streaming would require different server settings than, say, protecting a site that's being DDOSed.
Re:Docker not ready for primetime by cortesoft on Sunday August 28, 2016 @07:24PM (#12379038)
Ding ding ding :D

I don't work for one of those two, but I do work for a very large CDN.

Re:Docker not ready for primetime by ceejayoz on Sunday August 28, 2016 @10:04PM (#12379536)
> I work at MindGeek, depending on the time of the year it would be fair to say we rank within the top 25 bandwidth users in the world.

"I'd have thought I've had heard of them..."

One Wiki search later... yup. I've heard of them.

Re:Docker not ready for primetime by Steeeve on Monday August 29, 2016 @04:00AM (#12380642)
Ha! I went to their website... Hmm... never heard of them. Went to Wikipedia... Oh, now I know who they are! Yeah. Lot's of bandwidth.
Re:Docker not ready for primetime by fnord123 on Monday August 29, 2016 @04:04AM (#12380650)
>What company is running in 70+ datacenters!?

People running edge networks and therefore need servers local to everywhere in the world to keep latencies down. Maybe it's not as much of an issue for MindGeek (the parent company for a lot of video streaming sites). I would guess you guys need a lot of throughput but latency isn't so much of a problem. Or you simply don't need to serve some parts of the world where it might be illegal to distribute some types of content.

FWIW, Cloudflare has 86 data centers: https://www.cloudflare.com/network-map/

Re:Docker not ready for primetime by cortesoft on Monday August 29, 2016 @12:23PM (#12382972)
Or they are customers of CDNs.
Re:Docker not ready for primetime by user5994461 on Monday August 29, 2016 @04:35PM (#12385084)
[Note: MindGeek = the pornhub network]

Short version: That's a video streaming websites which is rather simple, yet bandwidth intensive.

Outsourcing the caching and video delivery means MindGeek can do with little servers and a few locations.

Nonetheless the CDN you're outsourcing to does need a lot of servers at many edge locations.

Actually, if we think in terms of "top bandwith users in the world". It's possible that your company is far from being in the list. It's likely dominated by content delivery / ISP / and other providers, most of which are unknown to the public.

Re:Docker not ready for primetime by ckdarby on Friday September 02, 2016 @09:16PM (#12417238)
Would you say Netflix & Youtube are rather simple? Handling +100 million users is never rather simple...
Re:Docker not ready for primetime by user5994461 on Saturday September 03, 2016 @07:02AM (#12418723)
Youtube had the challenge of being one of the first streaming services, and it's operating at an unprecedented scale. I am actually wondering, how many orders of magnitude youtube has more traffic than pornhub?

I am in distributed systems and try to work exclusively on hard problems. So when I say "simple", that is biased on the high end of the spectrum.

If you go to pornhub.com and looks at "popular keyword", you'll only find thousand or ten thousands videos. In a way, there is not that much content on pornhub.

All major websites have challenge. Pornhub is a single purpose website and a lot of the challenge is in video delivery, which can be outsourced to a CDN nowadays.

"simple" is maybe too strong a word. I am trying to convey the idea that it has limited scope and [some of] the problems it's facing are understood by now and have [decent] solutions.

That's not to say it's easy ;)

Re:Docker not ready for primetime by cortesoft on Sunday August 28, 2016 @07:23PM (#12379029)
I work for a large CDN.