I am using it as the backbone for my hobby project that auto-generated federated GraphQL API's:
https://github.com/GavinRay97/GraphQLCalcite
The experience has been incredibly positive and the community has been incredibly helpful & supportive. It's one of the coolest technical projects I've ever seen and has sparked my interest in query engines and relational databases.
I posted some JMH benchmarks of an app that parses a GraphQL query, converts it to Calcite relational expressions, and then executes it against an in-memory DB and it ran on the orders of milliseconds:
https://lists.apache.org/thread/hofjx628864t0kt4kk8vo4tjfrxb...
Something very similar to Calcite but much lesser-known is the "Teiid" project:
https://github.com/teiid/teiid
Highly recommend checking the code out. It's got a brilliant query optimizer/planner tailored to cross-datasource queries, a cache system, and a translator architecture that can convert a generic SQL dialect to dozens of SQL/NoSQL flavors.
Also integrates other data sources like REST API's, S3, flat files etc as queryable data sources.
For example, I have a former student who's been working on an adapter to Apache Arrow. Beyond having a fast execution engine, this should also eventually enable Calcite to work pretty seamlessly with other systems using Arrow without serialization overhead.
He's one of the only people I know that's a committer/PMC of both Arrow and Calcite, super nice and brilliant guy.
Maybe he'd have good feedback or ideas around this?
month | authors | commits | files | churn
---------+---------+---------+-------+-------
2022-02 | 21 | 62 | 84 | 4387
2022-01 | 26 | 109 | 243 | 32429
2021-12 | 33 | 99 | 198 | 10461
2021-11 | 19 | 49 | 77 | 6960
2021-10 | 26 | 64 | 371 | 13626
2021-09 | 18 | 41 | 68 | 2258
2021-08 | 11 | 17 | 25 | 1924
2021-07 | 17 | 30 | 51 | 2704
2021-06 | 14 | 31 | 28 | 1708
2021-05 | 9 | 17 | 35 | 1606
2021-04 | 11 | 46 | 99 | 4224
2021-03 | 16 | 36 | 143 | 84711) https://oss.gitsense.com/insights/github?q=pull-age%3A%3C%3D...
select
commit_ym AS month,
count(distinct(author_email)) as authors,
count(distinct(commit_id)) as commits,
count(distinct(path_id)) as files,
sum(total) as churn
from
z1_commits_422 as commits,
z1_changes_422 as changes,
z1_code_churn_422 as churn
where
commits.id=changes.commit_id and
changes.code_churn_id=churn.id and
lang='java'
group by commit_ym
order by commit_ym desc
limit 12
By having most of Git's history in SQL, I can slice, dice and cross-reference code history, which is how my product works.The primary advantage of Calcite for connecting multiple data sources are 1) easily joining data sources that use completely different APIs (assuming there is a Calcite adapter available) and 2) supporting more complex queries than the original data source supports without having to write code (other than SQL) to do the processing.
And the project being founded by someone who has written multiple RDBMS.
Calcite is a wonder.
Generally all that is required to connect to multiple data sources from CSV to Elasticsearch is just writing a JSON configuration file. Then can get SQL access via JDBC with the able to join all those sources together.
Unfortunately, I would not be too surprised if the query execution Calcite was found to be less performance-optimized than Hydra. There is ongoing work for improvement there. That said, there are users of Calcite at Google, Uber, Spotify, and others who have made great use of various parts of the framework.
[0] https://calcite.apache.org/