News for hackers. Stuff that matters.

A comment on Modern Data Practice and the SQL Tradition. Parent comment

Re:Modern Data Practice and the SQL Tradition by alexhutcheson on Friday November 08, 2019 @10:35AM (#21483119)
I wish SQL had syntax more similar to dplyr[1] for complicated queries. I dislike having to do a nested sub-query every time I re-group the data - the query ends up being a deeply indented mess that needs to be read from the inside-out. Using "WITH" clauses helps a lot, but it's still not as easy to read as the more imperative dplyr style.

[1] https://dplyr.tidyverse.org/

[ Parent | on HN ]
Re:Modern Data Practice and the SQL Tradition by gigatexal on Friday November 08, 2019 @10:39AM (#21483165)
Be wary of WITH statements as they are optimization barriers as some engines can't optimize them -- that's changing but just FYI. I believe the latest Postgres release is handling them properly now -- I could be wrong.
[ Parent | on HN ]
Re:Modern Data Practice and the SQL Tradition by alexhutcheson on Friday November 08, 2019 @10:43AM (#21483222)
Good note. Subqueries are sometimes optimization barriers as well - it depends on query engine. If you care about performance you need to look at the query plan.
[ Parent | on HN ]
Re:Modern Data Practice and the SQL Tradition by gigatexal on Friday November 08, 2019 @11:03AM (#21483483)
"If you care about performance you need to look at the query plan." Always I would say, once your data gets above toy size.
[ Parent | on HN ]
Re:Modern Data Practice and the SQL Tradition by mizay7 on Friday November 08, 2019 @01:02PM (#21484924)
we use azure data explorer for a lot of data storage. It is a terribly boring name but it highly performant at a large range of data sizes, stores rectangular (table) data with clear schematization, and uses the piping type of syntax of tidyverse. I love it. I really wish they would rebrand and market it more.
[ Parent | on HN ]