There are much better options than LIKE and regexes in SQL for text search. To mention but two:
SELECT 'a fat cat sat on a mat and ate a fat rat'::tsvector @@ 'cat & rat'::tsquery;
SELECT * FROM articles
WHERE MATCH (title,body)
AGAINST ('database' IN NATURAL LANGUAGE MODE);
All main engines support full text search in one way or another. TSQL, Oracle, DB2... even SQLite.
What more, these engines all support inverted tree indexes in one form or another - i.e. the same type of index Lucene uses - to make this efficient. This is completely different from the typical LIKE or regex query against a column with a BTree index.
Also:
> Squeezing the query language into some sql-like shape will provide marginal gain at best.
... is disingenuous at best. In both examples I raised (and in the ones you'll find if you query the others) the engines have functionality to accept user input pretty much as is - meaning as your typical mom would type it in a search field.
If you need extra criteria from there and the dialect's full text syntactic features, you can use regular SQL conditions (and joins, and aggregates, etc.) on the subset of records found. The most discriminating criteria/index will be the one related to the search query in most cases. And when not, lucky you - your query planner did not hammer everything with the same sledgehammer, thus demonstrating why you should be using SQL.
What I see at your zombodb link is just a way to make Postgres proxy requests to ES - complete with ES syntax and/or Query DSL. The only difference is that instead of connecting to ES directly you connect to Postgres and then add "SELECT" thing in front of your ES Query DSL. If it makes something easier, fine, but it's not "Postgres full text over ES".
Think of it as browser prefixes. Sooner or later all engines agree on a common syntax, with each one needing to support some legacy code.
And in contrast with what happens when you use some third party app, you can further filter matched rows, join them with other tables to your heart's contents, aggregate them, apply window functions, and what have you.
Take a look at the most recent versions of the SQL standard, say SQL:2016, and you see it handles document type data (JSON) pretty well too. The syntax isn't what we're used to, but it works pretty well.
I know that SQL can do more when you separate its conceptual foundation from relational databases. It's an expression language, an interface language to structured data.
I used to work on a platform that could translate SQL into NoSQL databases and SQL databases alike, it involves translating the expression into something that fits the native data model. For something similar look at Apache Drill, Dremio, Hive or PrestoDb and how they hide different types of databases behind interfaces that make them compatible with SQL.
I can admit, shoehorning SQL as it is onto a non-relational source (ex. Graph databases or search) isn't totally clean. But I believe it can evolve further as a standard to address that.
It requires more work in middleware to interface between SQL expressions and heterogeneous technology. And getting creative with the language constructs themselves. The power of data projection, data shaping and filtering alone, not to mention aggregation, etc. is undeniable in my opinion.
Disclaimer: I used to work for a company that built data federation software which supported SQL in front of heterogeneous sources. So what I am saying I know to be factual in most cases (although difficult!).
Developers are also unable to become involved or voice their concerns in any meaningful way without spending an even larger amount of money to participate in their committees. Oh, and AFAIK all development happens behind closed doors, so good luck figuring out why certain design decisions were made.
I mean, I understand there's a use case where it can be useful, such as SQL query builders or APIs that support only SQL, but in general case when you don't need one interface to rule them all, what's the use of shoehorning everything into SQL?
But for other domains, the query languages arose out of need, not because someone felt like it. SQL is hot garbage for for non-trivial logic. I can't even imagine what splunk would look like if you had to query it with SQL.
If anything, SPL (or rather the piped operators approach) would make a lot more sense as the groundwork for a universal query language that works across many domains.
Lucene for example doesn't use SQL because its really solving a different problem - text search. Its a language dedicated to what could only be expressed using something like `LIKE` and regexes in SQL.
Same with Splunk, it addresses a different domain and solves different problems that cant be easily expressed in SQL.
Same with MongoDB.. its not a relational db. Just because there is a mapping from some SQL queries to some Mongo queries, does not mean they are identical databases.
The query language directly impacts the AST generated, which is necessarily tied very closely to the exact capabilities/internals of a system. This post just feels like its based on a cursory glance of what the systems do.
If all Lucene had was regexes, it would not perform much better than a SQL database throwing regexes at strings. Its precisely because the query language is finer-grained that it can be better optimized for that usecase.