News for hackers. Stuff that matters.
Re:Cost-Based Optimizer in Apache Spark 2.2 by makmanalp on Thursday August 31, 2017 @11:54AM (#15141157)
What's cool about these statistics-based approaches is that you mostly don't even need fully up-to-date statistics, just overall decent stats, unless you have an insane amount of churn. Meaning - you can get query speedup without insertion overhead: you choose to take that overhead any time you want using ANALYZE.

Very neat stuff from the databricks team!

Re:Cost-Based Optimizer in Apache Spark 2.2 by trhway on Thursday August 31, 2017 @04:36PM (#15143700)
For non latency sensitive queries you can also run dynamic sampling.