ES grew out of Lucene, which provided an inverted index of all the text in a document, with a bunch of NLP related features bolted onto that. While ostensibly designed to be developer friendly, ES had and still has a horribly hacked together API with bugs and mis-documented misfeatures all over the place. In my experience it's anything but developer friendly. But if all you need is a text index on a document store with a static schema, it does its job reasonably well.

Mongo started as a very developer-friendly data store with lots of overstated claims to being a database. While earlier versions of Mongo were wildly dangerous to use as a business critical database, it has since matured and is now quite good at being a developer-friendly document-centric database. In my experience Mongo truly is developer-friendly as long as you don't try to use it as a full-blown transactional database with lots of complex data shapes and indexes.

I would not trust ES with anything but text search on a document store, and I would not trust Mongo with anything resembling multi-document transactions. With that said, they are both good at specific, different things.

MySQL and Postgres have their own baggage that makes them pretty terrible in some aspects. IMHO a JSON-over-HTTP API should really be table stakes for a database to be considered developer-friendly nowadays. (But please don't butcher HTTP like ES did and then claim to have that.)

> IMHO a JSON-over-HTTP API

No no no no, again no.

We don't need yet another shitty query language bolted onto one of the most error prone and annoying to type serialization formats while transmitting data on top of a by default stateless protocol that makes no sense for a database.

I'm sick of it.

SQL. The same queries will work in 95% of the case on any SQL db. There is a driver in almost every language that is robust. There are implementations great for every use case; embedded, scalable, transactional. That's friendly.

There is nothing wrong with SQL, it's freaking awesome.

Sure, keep the SQL. Just make it a field in a JSON POST payload, and send the results back as JSON.

The "drivers in almost every language" suck. They all suck. I've never seen a SQL driver and wire protocol that was not awful in some way. The statefulness is part of what makes them awful. We have better ways to keep track of state now.

> I've never seen a SQL driver and wire protocol that was not awful in some way.

Have you seen the PostgreSQL wire protocol[1]? I recently built a logical replication client driver for a project and found the protocol to be excellent. After looking at the documentation, I'm no longer limited to languages that have drivers for Pg, because I know how easy it'd be for me to just write one.

Just because some SQL drivers and wire protocols are awful (looking at you, Oracle[2]) shouldn't mean one should go running to the hills, let alone to JSON.

----

1: https://www.postgresql.org/docs/current/protocol.html

2: https://noss.github.io/2009/04/28/reverse-engineering-oracle...

An HTTP/JSON protocol doesn't have to replace the standard one. But having such a standard protocol makes sense in the age of web apps, particularly when NoSQL offerings that are perceived by the market as competitors (leaving aside whether they really are - perception matters more here) do that already.

> An HTTP/JSON protocol doesn't have to replace the standard one.

So, two protocols? Two standard protocols is rarely better than one.

> having such a standard protocol makes sense in the age of web apps

Only if the existing standard protocol cannot work with the "web", and we have plenty of history proving otherwise. Replacing the existing standard with the loose JSON would be, strictly, a downgrade; and unnecessary, because we already do interoperate JSON and SQL. See: PostgREST and the many REST & GraphQL frontends on PostgreSQL.

> particularly when NoSQL offerings that are perceived by the market as competitors (leaving aside whether they really are - perception matters more here) do that already

This is really more of going into a pig's pen and wrestling with them. A database should do the job of a database. Competing for perception in a market that cannot make sane decisions for itself is how we get MongoDB.

When was your experience with Mongo? I'm wondering because I use Mongo now with transactions (supported since version 4) and I like to know about problems I might face in the future.

Elasticsearch is a distributed index, not a reliable document store. You can expect availability problems and data loss. This is fine, because you can rebuild the index from your source of truth.

In my experience with ES, definitely don't treat it as a source of truth; allow it to be rebuilt.

True.

I run a startup that uses both of these.

Mongo is a heck of a lot easier to configure and develop with, and works great as a general database with a rapidly changing schema.

Elasticsearch is great at solving specific problems like searching for items in a specific way, but it's got quite a learning curve and is pretty painful to host and configure compared to Mongo where you can have a prod instance going in seconds.

Is it a good idea to use a database instead of a search engine?

Yes. Elasticsearch is not nearly as easy to use and is not really designed to be a transactional database.

Apples and Oranges. Some use-cases may overlap, but not all. For instance, why would you use MongoDB for centralized logging, as opposed to the ELK stack?