jasontedor

Let’s say we do that, use UTC everywhere. You live in London. It’s 3 PM. For you, that’s mid-afternoon. It’s your brother’s birthday. You want to phone him and wish him a happy birthday. He lives in Sydney. In a world without timezones, it’s 3 PM for him too. So you give him a call and start singing loudly when he answers. He’s mad at you because actually it’s the middle of the night and he’s sleeping.

Or think about traveling. At home, you set your alarm for 6 AM. You really like waking up at this time even when you travel. You want time to hit the gym, and eat a nice breakfast. Now, in a world without timezones, you travel to SF. When your flight lands at 5 PM, what phase in the day is it? Local “morning”? Local “late evening”? Are you going to have a hard time catching a taxi? What time do people eat breakfast there? What time should you set your alarm for?

So you need a translation layer from your location to another location to know what phase in the day it is for people in the other location. Is it their “morning”? Their “afternoon”? Their “business hours”? When do they sleep?

That translation layer exists. That’s timezones.

Abolishing timezones doesn’t make coordination problems easier. In fact, it makes them harder. The time on the clock might be the same around the world, but when people do things (wake up, eat breakfast, conduct business, grab a cocktail) would vary around the world. (Yes, it’s true there’s regional variance to these things today, but for the most part you can rely on morning having a rough relationship with when people wake up and sunrise, for example.)

I think James Gleick had a much better write up about this: http://www.maths.mic.ul.ie/posullivan/A%20Bug%20and%20a%20Cr...

Jason from Elastic here.

I want to clarify one aspect here. Elasticsearch and Kibana will be dual licensed under the Elastic License or the SSPL license, not only SSPL. The distributions that we provide will be licensed under the Elastic License, which does not have the copyleft requirements that are sometimes concerning to legal teams.

That is, if you download our distribution and run it in your infrastructure, you are subject to the Elastic License. The same license that most (90%+) of our users are already running under today, when they download our default distribution from elastic.co. This is why we say the vast majority of our users are not impacted by this change.

Note: we are considering making changes to the Elastic License to simplify it. Please see more on our [blog][0].

Disclaimer: I am on the Elasticsearch team and work for Elastic; I welcome any and all feedback.

[0]: https://www.elastic.co/blog/license-change-clarification

Jason from Elastic here.

I want to clarify this. Building a SaaS product with Elasticsearch or Kibana on the backend is okay and not prohibited, as long the service is not “managed” Elasticsearch nor Kibana. For example, a music service that uses Elasticsearch on the backend to provide music catalog search is okay. A service that offers site search powered by Elasticsearch is okay. A log search service powered by Elasticsearch is okay (again, provided it doesn’t expose Elasticsearch functionality). We welcome applications built with Elasticsearch or Kibana in the backend, even those that would be seen as competitors, as long as they are not managed Elasticsearch nor Kibana services. The Elastic License allows this, and you can do it for free. See this [item][0] in our FAQ. If you have any doubt, reach out to [email protected] and we would be happy to clarify.

Disclaimer: I am on the Elasticsearch team and work for Elastic; I welcome any and all feedback.

[0]: https://www.elastic.co/pricing/faq/licensing#i-build-a-saas-...

Jason from Elastic here.

>To give the story a bit more context, I always felt that Elastic built on Lucene without giving much back.

I would love to understand why you have this perspective? Investing in and contributing to Lucene is a core part of the Elasticsearch team. We contribute more to Lucene than any other organization, some of our employees are amongst the top committers, and several of our employees are members of the Lucene PMC. We give back to and [celebrate][0] Lucene openly. We love Lucene.

Disclaimer: I am an engineer on the Elasticsearch team; I welcome any and all feedback.

[0]: https://www.elastic.co/celebrating-lucene

I think that the full context of the sentence you quoted will help:

> In this case, if you base your number of threads off grepping lscpu you take another dependency (on the util-linux package), which isn’t needed. You also get the wrong answer, as you do by grepping /proc/cpuinfo. So, what this will end up doing is just increase the number of context switches, possibly also adding a performance degradation.

The idea is that a common way to size an application that exploits concurrency is to base the number of application threads on the hardware threads available. This portion of the article is saying that if you do so on the basis of `lscpu` or grepping `/proc/cpuinfo` you could mis-size relative to the actual number of cores available to schedule the threads of your application on. Doing so will lead to excessive context switching for your application, and that can hurt.

Jason from Elastic here.

A shard can handle concurrent writes from clients, there are multiple write threads scaling to the number of logical processors exposed to Elasticsearch, and we have invested in making Lucene accommodate this concurrency[0][1][2].

[0]: http://blog.mikemccandless.com/2011/05/265-indexing-speedup-... [1]: https://issues.apache.org/jira/browse/LUCENE-3023 [2]: http://blog.mikemccandless.com/2017/07/lucene-gets-concurren...

Disclaimer: I am an engineer on the Elasticsearch team; I welcome any and all feedback.

Jason from Elastic here.

It is indeed good reading, but as you say, old. Since then, we have invested tremendously in the [data replication][0] and [cluster coordination][1] subsystems, to the point that we have closed the issues that Kyle had opened. We have fixed all known issues related to divergence and lost updates of documents, and now have [formal models of our core algorithms][2]. The remaining issue along these lines is that of [dirty reads][3], which we [document][4] and have plans to address (sorry, no timeline, it's a matter of prioritization). Please do check out our [resiliency status page][5] if you're interested in the following these topics more closely.

Thanks for all of the feedback in this entire thread.

[0]: https://github.com/elastic/elasticsearch/issues/10708 [1]: https://github.com/elastic/elasticsearch/issues/32006 [2]: https://github.com/elastic/elasticsearch-formal-models [3]: https://github.com/elastic/elasticsearch/issues/52400 [4]: https://www.elastic.co/guide/en/elasticsearch/reference/7.6/... [5]: https://www.elastic.co/guide/en/elasticsearch/resiliency/cur...

Disclaimer: I am an engineer on the Elasticsearch team; I welcome any and all feedback.

I think this is addressed by the use of k-anonymity which is described in the section of the article titled “Enabling Anonymous Searches with k-Anonymity”.

Jason from Elastic here. We are actively developing cross datacenter replication (internally we are calling it "cross cluster replication" so you will likely see it referred to this in the future but of course this is subject to change).

I can not give a timeframe, but it is one of the top features on the ES roadmap. :)

Source: I am an engineer working on this feature.

Elastic | Software Engineer (Elasticsearch core/infrastructure team) | Distributed | https://elastic.co/

Elastic is the company behind the Elastic Stack (fka the ELK Stack). The Elasticsearch core/infrastructure team is looking for an exceptionally talented backend engineer. We are a fully-distributed and diverse team (we are open to any age, gender, ethnicity, sexuality, etc.). The formal job posting is on our careers page[1]. Briefly, we are looking for a tenacious engineer with deep knowledge of Java, the JVM, and loves working on the core foundations of a server like the backend components of Elasticsearch. You can apply directly via[1] or send me a résumé via email at my first name at elastic.co (disclaimer: I am the area lead of the team you would be hired into; if you email me I will reply to you, although I will be frank with you). I look forward to hearing from you.

[1]: https://www.elastic.co/about/careers/engineering/jobs/914024

I'm sorry that you're frustrated. Maybe the following will help?

There is such a flag for your use case[1]. Start the node in the Docker container with

  discovery.type: single-node

This allows the node to bind HTTP and transport protocols to non-localhost interfaces and not enforce the bootstrap checks[2][3]. The node can not form a cluster with another node, but that's okay for development use cases. It's design was intended to address the Docker development use case.

I hope that helps!

Disclaimer: I work at Elastic; I welcome any and all feedback

[1]: https://www.elastic.co/guide/en/elasticsearch/reference/5.5/...

[2]: https://github.com/elastic/elasticsearch/pull/23595

[3]: https://github.com/elastic/elasticsearch/pull/23598

Thank you for noticing this, it is not considered experimental anymore. I integrated a change[1] to update the docs; it should percolate through to the site within thirty minutes.

[1]: https://github.com/elastic/elasticsearch/pull/23621

Jason from Elastic here. It wasn't the same release as there were critical changes[1] between the RC and the GA, and the RC release will receive no support.

[1] https://www.elastic.co/downloads/past-releases/elasticsearch...