News for hackers. Stuff that matters.
Re:A gentle-ish introduction to worst-case optimal joins by CHY872 on Thursday June 02, 2022 @07:49AM (#31593716)
Increasingly databases can get do-overs on bad plans. A micro-level example might be dataflow splitting particularly expensive tasks so as to better parallelize them: https://cloud.google.com/blog/products/gcp/no-shard-left-beh...

Spark will re-run the query planner after doing the map stage of a map-reduce operation so as to leverage the information collected: https://databricks.com/blog/2020/05/29/adaptive-query-execut...

Both examples represent changing parameters about a lower level plan than simply join operators, but the industry is certainly moving in this direction.

Re:A gentle-ish introduction to worst-case optimal joins by sitkack on Thursday June 02, 2022 @10:16PM (#31603449)
Most systems that make expensive scans should be updating or creating a count-min-sketch histogram of the data to allow schedulers and query planners to do a better job.

https://en.wikipedia.org/wiki/Count%E2%80%93min_sketch

https://duckduckgo.com/?t=ffab&q=count-min+sketch+site%3Avld...

https://www.semanticscholar.org/search?q=count-min%20sketch%...