qdequelen

Hey PSeitz, Meilisearch CEO here. Sorry to hear that you failed to index a low volume of data. When did you last try Meilisearch? We have made significant improvements in the indexing speed. We have a customer with hundreds of gigabytes of raw data on our cloud, and it scales amazingly well. https://x.com/Kerollmops/status/1772575242885484864

> Are LLM tools better or worse than e.g. meilisearch or elasticsearch for searching with snippets over a set of document resources?

Absolutely worse, LLM are not made for it at all.

Thanks for the highlight!

Do you handle high throughput volumes? I would need this for testing to host a database service at scale.

Did you only get the schema.json?

Hey, I'm the CEO of Meilisearch. If your issue is performance, I would love to give you a try with Meilisearch. You'll be able to create an "as you type" experience with our engine that responds in less than 50ms!

Thanks @CaroFG, I'm sure that if anyone has questions, the engineering team would love to answer!

Hello, I'm the Meilisearch CEO. I think you're also correct, Jabo.

I just want to clarify. Meilisearch's pricing doesn't start at 1.2K/month, but at 0/month. We have a usage-based pricing that is basically 0.25/1000 documents and searches. And, funny thing, we are thinking about splitting the searches and documents, too, but we wanted to have more data to be sure to select the right unit price for each. :)

Make the web faster with Rust.

Hey muhammadusman, I'm the Meilisearch's CEO. We have a complete comparison table. Note that it represents our point of view. https://docs.meilisearch.com/learn/what_is_meilisearch/compa...

Both Meilisearch and Typesense are really different regarding resource consumption and performance. I would say that where Typesense would have a better indexing performance (Meilisearch has recently improved indexation speed), Meilisearch will guarantee a much faster search performance while keeping impressive relevancy. Regarding the consumption, as Typesense is entirely on RAM and Meilisearch is using memory mapping, Meilisearch would take more disk space but less RAM.

To answer your question precisely, we handle all the space-separated languages and have specific tokenizers for Chinese, Japanese, Korean, Thai, and Hebrew. We plan to add more languages in the future.

Thanks for your feedback!

You should try it again since we intensively improved the indexation performances. Most of our actual users no longer have performance problems, even on hundreds of millions of documents.

Yes, it's planned!

Yes, indeed, it's crucial to have an SSD. With it, loading will be instant (a few ms).

You should definitely try Meilisearch again. We have optimized a lot of the consumption and indexation performance. Even with all the improvements, we think it's essential to continue focusing on it during 2023.

And indeed, Meilisearch uses memory-mapping, which means that everything is on disk, and it will try to take as much memory as possible. For your information, we successfully ran a 115M documents dataset on a 1Gb RAM machine.

If you remove the URLs from indexation, it'll generally save a ton of place and will be much, much faster to index. We are thinking about not indexing URLs by default; you can help us by explaining your use case here -> https://github.com/meilisearch/product/discussions/553

Just a detail, if you're making a `du -sh` on your computer, the size on the disk will stay unchanged because we are doing soft deletion ;). Don't worry. It will be physically deleted after a while if you need it in the future.

If you kept the default configuration of Meilisearch, the maximum size of the HTTP payload is 100Mb (for security). You change it here -> https://docs.meilisearch.com/learn/configuration/instance_op...

addDocumentsInBatches() is just an helper to send your big json array into multiple parts, not absolutely sure you'll need it. (Code -> https://github.com/meilisearch/meilisearch-js/blob/807a6d827...)

Hello, I'm the Meilisearch CEO, your issues could be because you sent your data without configuring your index, and it's what I read from your comment. Just change your setting to not index URLs.

Check out the doc: https://docs.meilisearch.com/reference/api/settings.html#upd... With the payload: `["title", "descriptionHTML"]`

It will change everything!

Can I get your use case? What is the data you have to index? If it's logs, indeed, Meilisearch will not be the best fit. I would recommend https://quickwit.io. If it's structured or unstructured data, even if 10TB seems huge for our use cases, I would love to put you in contact with my engineering team. We would find a way to manage it.

Hello, I would love to know what is your needs/limitation that makes you ask for cluster/sharding. Meilisearch is cloud-native, so we don't need clusters of machines to provide a highly available solution. By default, our service is highly available and vertically scalable. We don't have geo-replication today, but it will come later in 2023. I don't know your need for scalability, but we can handle almost every use case. You may only be in an Enterprise plan.

We support almost all languages. The only thing is that in some languages, without help and intervention from the community, we stop at the precision we understand from a "French/English" person. Today we handle perfectly Latin-based languages and all languages that are space separated. We have also worked with the community to improve Chinese, Japanese, Thai, and Korean, which is under review.

Cool! Meilisearch's CEO here. We have spent so much time improving indexation speed. I'm so glad the experience is better for you now! Even more improvements will be released in the future.

Good to know. If you find some Russian language support improvement we can do, don't hesitate to create an issue on our tokeniser https://github.com/meilisearch/charabia.

Moreover, it's Hacktoberfest. If you want to help us improve the language support, it would be awesome!

Hello, I'm Quentin, Meilisearch's CEO. Meilisearch Cloud is encrypted at rest. We're using, at this time, a k8s with replicated and encrypted persistent volumes. We just didn't talk about it yet.

Thanks! Happy to get you as a user :)

Hello everyone, I'm Quentin, the CEO of Meilisearch. We are very proud to announce today our incredible fundraising. Over the last three years, we have succeeded in establishing ourselves as a serious competitor in the search industry. In just 3 years, we have managed to win over the hearts of developers by enabling them to create fast, relevant, and easy-to-implement search experiences in their applications.

But this was only the beginning. Thanks to this new investment, we will continue to invest in open source and the community and start the next big phase of our company. Offering the same quality of research to companies on a completely different scale.

We have a lot of ideas to make Meilisearch even more scalable and easier to integrate into the devs stack. Geo-replication, data sync with primary databases, search bar customizable in one line of javascript, and many more.

Stay tuned for news. I will be happy to answer all your questions here.

Indeed for the moment, I would not advise going to more than a few million documents, especially if you want to make regular updates. With the new update that we will release in March, I think we will go up to a few tens of millions of documents with much higher update frequencies. However, we keep in mind that Meilisearch should do much, much more. This will be a work in progress for us during the year, as we plan in the future to allow Meilisearch to be no longer a search binary with limited resources but as a globally distributed solution that will have no limits to scale. But it's still too early to talk about our long-term plans on how we will improve user-facing search. #serverless