News for hackers. Stuff that matters.
Re:AWS announces forks of Elasticsearch and Kibana by t0mas88 on Thursday January 21, 2021 @06:13PM (#25865940)
A lot can be said about AWS and open source, but it is clear they have created some "secret sauce" on the networking, storage and virtualization side of things which is their core business. Unlike Elastic they never promised open source and never used it as marketing. So it completely fair for them to keep those things closed as they provide a big part of the competitive advantage to their cloud.

Those components underpin things like Aurora (which does tricks with storage and replication that MySQL can't) and this warm/hot storage. So there is probably no practical way to open source those elasticsearch changes without opening up their storage system as well and even then it wouldn't run outside of AWS.

Re:AWS announces forks of Elasticsearch and Kibana by catmanjan on Thursday January 21, 2021 @06:48PM (#25866322)
The way I see it is the secret sauce is just black boxing it so you don't have to/can't worry about those things...
Re:AWS announces forks of Elasticsearch and Kibana by t0mas88 on Friday January 22, 2021 @03:45AM (#25869701)
There is a slideshare about their networking approach for VPC somewhere it's quite clever. But I can't find anything for EBS and what features it might have that make Aurora replicate the way it does.
Re:AWS announces forks of Elasticsearch and Kibana by robhu on Friday January 22, 2021 @06:07AM (#25870597)
I'd be interested in seeing that slideshare if you have it to hand?
Re:AWS announces forks of Elasticsearch and Kibana by t0mas88 on Friday January 22, 2021 @09:06AM (#25871745)
Edit: I found this video https://www.youtube.com/watch?v=St3SE4LWhKo it uses the slides I mean but looks like there are a few versions from different events. Google for "another day, another billion packets"

I remember what it looks like, tried to Google it but can't find it. The gist of it was that they didn't create any tunnels or VPN like infrastructure to create VPC, because it wouldn't scale to large numbers of nodes. Especially because your servers can be on completely different hosts all of their internal network.