MeiliSearch: Zero-config alternative to Elasticsearch, made in Rust

Re:MeiliSearch: Zero-config alternative to Elasticsearch, ma... by heipei on Wednesday March 25, 2020 @04:25PM (#22688199)

I know the project doesn't claim it, but the title somewhat implies this: I honestly don't understand people claiming ElasticSearch is hard to operate, especially not at small scales. If anything, ElasticSearch for me has been one of the easiest pieces of infrastructure to operate, for me pretty much "zero-config". Let me elaborate: You can run ElasticSearch via Docker command-line, if you want a cluster you just supply IPs of the other nodes. Then you start indexing documents with simple HTTP calls. You can add or remove nodes at any time and don't have to do anything but to start another ElasticSearch instance. If you run out of space or performance just start another node. Everything needed for management, indexing, search is available through HTTP APIs, no tools needed.

Clustered ElasticSearch has been rock-solid for me and I've used it in anger many times. The level of maintenance needed is close to zero, both initially and long-term. Compare that with the abysmal experience of setting up a sharded MongoDB cluster for example...

Please enlighten me how ElasticSearch is "a lot of work to operate" (heard that one multiple times), and what you're comparing it to.

[ Parent | on HN ]

Re:MeiliSearch: Zero-config alternative to Elasticsearch, ma... by jniedrauer on Wednesday March 25, 2020 @06:50PM (#22689206)

I've been bitten by elasticsearch twice in my career, and I've seen others bitten by it as well. Once you put it in production, you can't just run it from docker on your workstation. You have to set up a cluster with enough capacity for whatever load you're going to throw at it, gracefully handle failures, updates, scaling up as load increases, etc.

There are so many switches and dials to tune, and unless you really learn it in depth, you won't know which ones you need. It's difficult to even determine what hardware requirements you have. And it's a hard sell to tell your business guys "I think elasticsearch will work better if we give it more... CPU? Memory? Disk speed? I'm not really sure." and can't provide any concrete metrics to back that up.

Another place where footguns abound is upgrading from one version to another, especially if you've got plugins installed. There are tricks that you have to learn the hard way.

At this point, I think long and hard before reaching for a solution like elasticsearch. If I've got a DBA whose entire job it is to master the tech and wield it expertly, that's one thing. But if I'm part of an early stage startup, I just can't justify the lost time and potential for catastrophe.

[ Parent | on HN ]

Re:MeiliSearch: Zero-config alternative to Elasticsearch, ma... by natefox on Wednesday March 25, 2020 @11:45PM (#22690645)

> Once you put it in production, you can't just run it from docker on your workstation.

But that's true for any data store. This isn't any different. Nor is an RDBMS. They all need HA/replication. And that is rarely trivial.

Honestly, I think this is why managed/hosted solutions (AWS RDS for example) are so popular - they remove a large part of the complexity for you.

[ Parent | on HN ]

Re:MeiliSearch: Zero-config alternative to Elasticsearch, ma... by hashhar on Thursday March 26, 2020 @02:32AM (#22691120)

Not all data stores. You can go quite far with an out of the box Redis instance or even PostgreSQL. No fiddling needed unless you are in triple digit QPS ranges.

[ Parent | on HN ]

Re:MeiliSearch: Zero-config alternative to Elasticsearch, ma... by cormacrelf on Wednesday March 25, 2020 @11:36PM (#22690612)

I once wasted a whole day trying to get two instances up on GKE. Permissions problems, about ten configs for the JVM alone, many more for ElasticSearch. You would fix one, restart, wait ten minutes, browse 50 pages of logs, google for half an hour, add a config, and goto 1. Never got it going in the end.

[ Parent | on HN ]

Re:MeiliSearch: Zero-config alternative to Elasticsearch, ma... by amelius on Thursday March 26, 2020 @11:19AM (#22694377)

My experience is roughly the same, unfortunately.

Personally I don't understand why there are so few search libraries/systems to choose from, given that "search" is one of the fundamental pillars of CS.

[ Parent | on HN ]

Re:MeiliSearch: Zero-config alternative to Elasticsearch, ma... by JeanMarcS on Thursday March 26, 2020 @10:02AM (#22693618)

Just been bitten by the plugin issue after an apt upgrade.

First time it happened for me and I was pretty angry at it

[ Parent | on HN ]

Re:MeiliSearch: Zero-config alternative to Elasticsearch, ma... by Kerollmops on Wednesday March 25, 2020 @04:31PM (#22688247)

MeiliSearch is "zero-config" compared to ElasticSearch in terms of setup to make it work for end-user instant and relevant search engine. Our engine follows the Algolia engine in terms of typo-tolerance, relevancy, and speed.

Here is a little comparison to enlighten your questions: https://docs.meilisearch.com/resources/comparison_to_alterna....

[ Parent | on HN ]

Re:MeiliSearch: Zero-config alternative to Elasticsearch, ma... by heipei on Wednesday March 25, 2020 @04:35PM (#22688282)

Thanks, hadn't seen that, that makes a lot more sense. I agree that ElasticSearch is definitely not "zero-config" when it comes to building certain bespoke applications on it that go beyond simple filtering or query-relevance document search.

[ Parent | on HN ]

Re:MeiliSearch: Zero-config alternative to Elasticsearch, ma... by ksec on Wednesday March 25, 2020 @06:56PM (#22689240)

May be adding Vespa [1] to comparison?

[1] https://vespa.ai

[ Parent | on HN ]

Re:MeiliSearch: Zero-config alternative to Elasticsearch, ma... by dijit on Wednesday March 25, 2020 @06:16PM (#22688992)

Elasticsearch is easier than mongo in some ways and harder in others.

I run a few 10TiB ES clusters (which, is not much to be fair) but infrequently find that I have to reindex or reshard the cluster because I can’t just add another node. There’s something to be said for understanding the index rotation too, and access patterns.

It’s easy to make an ES cluster, it’s difficult to maintain one, it’s nearly impossible to debug one.

- if you consider that “it’s slow” is what you have to debug.

[ Parent | on HN ]

Re:MeiliSearch: Zero-config alternative to Elasticsearch, ma... by kstrauser on Wednesday March 25, 2020 @07:22PM (#22689376)

That's approximately how large our clusters are. Fortunately, ours are read-only, so our admin story is:

- Hey, a node died! - Run Terraform to stand up a whole new cluster and restore it from a snapshot. - Update the app to point at the new cluster. - Run Terraform to delete the old cluster.

I'm pretty happy with this arrangement.

[ Parent | on HN ]

Re:MeiliSearch: Zero-config alternative to Elasticsearch, ma... by jniedrauer on Wednesday March 25, 2020 @06:51PM (#22689212)

> if you consider that “it’s slow” is what you have to debug.

This is exactly it. This is a problem you encounter with every database engine, but in most of them you can quickly find the bottleneck and fix it. With elasticsearch... it's a frustrating and expensive game of trial and error.

[ Parent | on HN ]

Re:MeiliSearch: Zero-config alternative to Elasticsearch, ma... by antpls on Friday March 27, 2020 @08:11AM (#22702207)

> I run a few 10TiB ES clusters

For information, what does "10TiB" refer to in this context?

Is it the size of what ES takes in RAM, or the size of ES' index, or is it the total size of the corpus that ES must index? Or corpus size + index ?

[ Parent | on HN ]

Re:MeiliSearch: Zero-config alternative to Elasticsearch, ma... by heipei on Thursday March 26, 2020 @08:00AM (#22692632)

Would be interested to hear why you can't add another node in some cases.

[ Parent | on HN ]

Re:MeiliSearch: Zero-config alternative to Elasticsearch, ma... by dijit on Thursday March 26, 2020 @06:16PM (#22698533)

If you have 5 shards on an index, adding a 6th node isn’t going to improve performance.

(Contrived example for the sake of illustrating the point)

[ Parent | on HN ]

Re:MeiliSearch: Zero-config alternative to Elasticsearch, ma... by fareesh on Thursday March 26, 2020 @07:08AM (#22692365)

We built a "Yelp for Colleges" product several years ago. The product needed a unified search where students could search for either a course or a college or a question from the forums with typeahead / autocomplete to get them to where they wanted to go quickly, with support for misspellings.

In all there were about 50k documents, and we mostly cared about the title field. Elasticsearch would randomly bloat up to occupy a huge amount of RAM. Restarting it would make it work for a few days. It would also occasionally crash.

We got rid of it and went with some levenshtein distance based database query

I'd love to use it again sometime but the experience was not good, and Googling for information brought up all kinds of very complex use-cases shared by others

[ Parent | on HN ]

Re:MeiliSearch: Zero-config alternative to Elasticsearch, ma... by Kerollmops on Thursday March 26, 2020 @07:40AM (#22692512)

Go on and try MeiliSearch, 50k documents are easily handled by the engine and with not much RAM usage.

It will take you something like 10 minutes to start and populate MeiliSearch, you will be able to test it just by going to the server HTTP url in no time!

[ Parent | on HN ]

Re:MeiliSearch: Zero-config alternative to Elasticsearch, ma... by specialist on Thursday March 26, 2020 @10:41AM (#22693990)

Neat.

I implemented the student facing course catalog web interface for a single org. One of the funnests (most fun) parts was the heuristics in the query parser. Like patterns for recognizing course numbers and boosting those exact match results. Really helps you appreciate all the fit & finish that goes into proper search engines.

This was the olden days, when we just used Lucene directly.

[ Parent | on HN ]

Re:MeiliSearch: Zero-config alternative to Elasticsearch, ma... by atombender on Thursday March 26, 2020 @12:16AM (#22690753)

Elasticsearch is very memory-intensive, and it's difficult to know exactly how much memory it will actually use, so you just have to throw a lot of RAM at it to avoid OOMing, then monitor it carefully, and hope your query concurrency won't accidentally blow the limits. Understanding why Elasticsearch is caving unpredictably under load is difficult, and GC pausing can be a significant performance sink.

[ Parent | on HN ]

Re:MeiliSearch: Zero-config alternative to Elasticsearch, ma... by rodgerd on Wednesday March 25, 2020 @05:46PM (#22688791)

> I honestly don't understand people claiming ElasticSearch is hard to operate, especially not at small scales.

The problem is that ES is deceptively simple to operate. As millions of people who have found things like their medical records shared with the world can attest.

[ Parent | on HN ]

Re:MeiliSearch: Zero-config alternative to Elasticsearch, ma... by winrid on Wednesday March 25, 2020 @04:34PM (#22688266)

I've had issues scaling writes to it. You can get around it, but maybe this would be better in a high write environment.

[ Parent | on HN ]