News for hackers. Stuff that matters.
Re:Open Distro for Elasticsearch by sailfast on Monday March 11, 2019 @01:47PM (#19361040)
As an active developer and maintainer of a cluster, I strongly disapprove of this move by AWS.

By forking on an earlier instance of the codebase for ES and Kibana they're not only creating an open source version of the code but also attempting a fork of the community - plugin developers, user groups, etc. It's extremely frustrating. EDIT: If folks actively building plugins have felt pain here because of decisions by Elastic, then my view on this point changes considerably, obviously.

I find it very hard to believe there was no compromise to be had here between the current Elastic roadmap for their product and their previous work.

How is this good for users of the current tools? How does this help create better capabilities on a common platform? Will this encourage users to build businesses on elasticsearch open source elements?

I'm not sure where the community ends up after this, so I'm not sure I can support by using AWS' elasticsearch tools.

Re:Open Distro for Elasticsearch by sidi on Monday March 11, 2019 @02:08PM (#19361234)
As someone who is involved in the ecosystem and maintains some open-source projects based on ElasticSearch, I am similarly curious about the move to not build upon existing projects (there could be a good reason, but haven't heard it yet).

Why wasn't ElastAlert used / built upon for alerting? https://github.com/yelp/elastalert

Or why wasn't ElasticSearch SQL used / built upon for SQL? https://github.com/NLPchina/elasticsearch-sql

Or why wasn't SearchGuard used / built upon for Security? https://github.com/floragunncom/search-guard

All of them have Apache 2.0 license.

Re:Open Distro for Elasticsearch by edraferi on Monday March 11, 2019 @02:24PM (#19361412)
> why wasn't ElasticSearch SQL used / built upon for SQL?

According to the release blogpost [0], "SQL Support [...]is an improved version of the elasticsearch-sql plugin."

Sounds like they are building on it, as you suggested.

[0] https://aws.amazon.com/blogs/aws/new-open-distro-for-elastic....

Re:Open Distro for Elasticsearch by stoke on Monday March 11, 2019 @02:39PM (#19361561)
> Or why wasn't SearchGuard used / built upon for Security? https://github.com/floragunncom/search-guard

Looks like it was. https://github.com/opendistro-for-elasticsearch/security-ssl...

floragunn GmbH Copyright 2015-2018 floragunn GmbH

Re:Open Distro for Elasticsearch by ssgao on Monday March 11, 2019 @02:11PM (#19361267)
For SearchGuard, only the basic auth is Apache 2.0. Anything practical you need to pay to use it.
Re:Open Distro for Elasticsearch by termez442 on Tuesday March 12, 2019 @04:54AM (#19366387)
From a look at the source code, Amazon has Apached 2.0 the entire Floragunn Searchguard codebase (including the commercial part)
Re:Open Distro for Elasticsearch by sagichmal on Monday March 11, 2019 @02:17PM (#19361327)
Why isn't AWS paying to use it, then? It's not like they don't have the money?
Re:Open Distro for Elasticsearch by tragicpapercut on Monday March 11, 2019 @02:38PM (#19361553)
It's not just about AWS, other people need to use this as well. Why are security features a paid for addition? Why not make core features a paid for addition instead? I've always hated Elastic (the company) for this decision, please stop trying to give away free things that are not secure, and then charge for security.
Re:Open Distro for Elasticsearch by _msw_ on Monday March 11, 2019 @02:40PM (#19361581)
I'd recommend browsing the code to find out more about the security module:

https://github.com/opendistro-for-elasticsearch/security/blo...

Re:Open Distro for Elasticsearch by ssgao on Monday March 11, 2019 @02:36PM (#19361526)
there's no "they", there's only us. AWS paying to use it means we are paying to use it. Between that and a truly Apache 2.0 project, I think it's a no-brainer for everyone.
Re:Open Distro for Elasticsearch by sagichmal on Monday March 11, 2019 @02:43PM (#19361614)
Why shouldn't you pay to use it, then?

More generally: why should AWS get to leverage its near-monopolistic position in the SaaS market to perform essentially hostile takeovers of the software projects it packages and runs? Simply because they have the resources to do it?

Re:Open Distro for Elasticsearch by e1ven on Monday March 11, 2019 @04:57PM (#19362753)
I'm very glad that they've done this.

The code was open, and the company decided to close it for the new versions, so it forks. Forks are common when people don't like the corporate direction of an OSS project - It's a strength, not a weakness. Openoffice/Libreoffice, Hudson/Jenkins etc.

Imagine if Linus said that the next version of Linux would be paid-only.

Within 15 minutes there'd be a new librelinux repo that people could contribute to instead.

Someone was going to make a new fork anyway - This is Amazon putting their money where their mouth is to fund and support that new fork.

Re:Open Distro for Elasticsearch by dhd415 on Monday March 11, 2019 @05:27PM (#19362998)
To be clear, no Elasticsearch code or features that were OSS were closed. As new Elasticsearch features are developed, some of them are now released as OSS and others are released with a commercial license.
Re:Open Distro for Elasticsearch by anth_anm on Monday March 11, 2019 @11:13PM (#19365067)
Okay, are you glad that Amazon now owns Elasticsearch?
Re:Open Distro for Elasticsearch by therealx on Monday March 11, 2019 @08:16PM (#19364197)
If ES didn't do the weird license change thing, we wouldn't be here. Either Amazon wouldn't have used it, or they'd be paying. Including non-com features in the default install was not a good way to play things.

Would you feel the same if AWS built a drop in/api compatible system from scratch?

Re:Open Distro for Elasticsearch by unethical_ban on Monday March 11, 2019 @02:44PM (#19361619)
Beyond the thoughtful ideas below, an official blog post would be cool for that. Specifically, Elastalert and ODfE Alerting are completely different.
Re:Open Distro for Elasticsearch by MeteorMarc on Monday March 11, 2019 @02:36PM (#19361528)
From the pom.xml of Open Distro for ElasticSearch: the fork is from elasticsearch-6.5.4, which is the latest stable release. Only alternative would have been v6.6.1. EDIT: also, nothing is there to believe they would not bump to higher versions in later versions.
Re:Open Distro for Elasticsearch by praseodym on Monday March 11, 2019 @03:22PM (#19361954)
6.6.1 is the latest stable release. 6.5.4 was released 19 december 2018 [1], so they’re not too far behind, though.

[1] https://www.elastic.co/blog/elastic-stack-6-5-4-released

Re:Open Distro for Elasticsearch by sailfast on Monday March 11, 2019 @04:25PM (#19362465)
Thanks for this clarification!
Re:Open Distro for Elasticsearch by SeaCow on Monday March 11, 2019 @02:40PM (#19361583)
From the related AWS blog (so at least a few grains of salt):

>>This means that the majority of new Elasticsearch users are now, in fact, running proprietary software. We have discussed our concerns with Elastic, the maintainers of Elasticsearch, including offering to dedicate significant resources to help support a community-driven, non-intermingled version of Elasticsearch. They have made it clear that they intend to continue on their current path.

... maybe I'm misreading your comment, but it sounds like AWS tried to find a compromise, including contributing to upstream, and were shut down (and all their content about future contributing seems to say "we'll send it upstream, but it depends on whether Elastic accepts it," so seems to corroborate). It honestly seems like you get more certainty that everything you're using from AWS is open and will remain open, and Elastic is going to keep creating uncertainty / drive to proprietary source-available.

Re:Open Distro for Elasticsearch by sailfast on Monday March 11, 2019 @04:32PM (#19362536)
There is certainty around the open-source bits and the license - this is true. That said elastic the company has also seems to have brought the product a long way after becoming a company. Having a dedicated staff of engineers and developers behind a thing is helpful for that, and they should be able to make some money as part of this if we want to see future open source products advance similarly. (There is a difference too between consulting / services and money “at scale” - AWS is removing an at-scale component here)

That said, I also totally understand disagreements with Elastic’s decisions on charging for security, etc and why that causes concern. As a user I wish I wasn’t staring a fork in the face and I’m hopeful things get back to equilibrium here soon as a single community.