News for hackers. Stuff that matters.
Re:Monoliths Are the Future by willvarfar on Thursday January 30, 2020 @03:10PM (#22194982)
https://prestosql.io/

It can access all those different databases.

You can also make your own connectors that make your services appear as tables, which you can query with SQL in the normal way.

So if the new accounts micro-service doesn't have a database, or the team won't let your analysts access the database behind it, you can always go in through the front-door e.g. the rest/graphql/grpc/thrift/buzzword api it exposes, and treat it as just another table!

Presto is great even for monoliths ;) Rah rah presto.

Re:Monoliths Are the Future by joefkelley on Thursday January 30, 2020 @03:27PM (#22195168)
It's been about 4 years since I've been in this world, but I remember there being several products all doing a very similar thing: Presto, Hive, SparkSQL, Impala, perhaps some more I'm forgetting. Is the situation still the same? Or has Presto "won out" in any sense?
Re:Monoliths Are the Future by dmitrig01 on Thursday January 30, 2020 @03:59PM (#22195511)
Hive and Impala are databases.

Presto and SparkSQL are SQL interfaces to many different datasources, including Hive and Impala, but also any SQL database such as Postgres/Redis/etc, and many other types of databases, such as Cassandra and Redis; the SQL tools can query all these different types of databases with a unified SQL interface, and even do joins across them.

The difference between Presto and SparkSQL is that Presto is run on a multi-tenant cluster with automatic resource allocation. SparkSQL jobs tend to have to be allocated with a specific resource allocation ahead of time. This makes Presto is (in my experience) a little more user-friendly. On the other hand, SparkSQL has better support for writing data to different datasources, whereas Presto pretty much only supports collecting results from a client or writing data into Hive.

Re:Monoliths Are the Future by joefkelley on Thursday January 30, 2020 @06:45PM (#22197282)
I think some of this might be misinformed.

I know Hive can definitely query other datasources like traditional SQL databases, redis, cassandra, hbase, elasticsearch, etc, etc. I thought Impala had some bit of support for this as well, though I'm less familiar with it.

And SparkSQL can be run on a multi-tenant cluster with automatic resource allocation - Mesos, YARN, or Kubernetes.

Re:Monoliths Are the Future by freewilly1040 on Thursday January 30, 2020 @03:15PM (#22195036)
Do issues arise when trying to match types across different persistence layers?
Re:Monoliths Are the Future by willvarfar on Thursday January 30, 2020 @03:17PM (#22195061)
In practice, no, haven't hit any.
Re:Monoliths Are the Future by hello_moto on Thursday January 30, 2020 @04:40PM (#22196014)
Presto can't deal with ElasticSearch. It can query basic data but it's not optimized to translate SQL to native ES query (SQL ES query is for paying customer).