Re:Why query planning for streaming systems is hard
by SpicyLemonZest on Sunday August 15, 2021 @06:25PM
(#28193104)
Robust operators in particular seem like an important area of research to me. I've never had time to sit down and think about it in detail, but I can't count the number of escalations I've had that would have been instantly solved if only we had a gracefully degrading broadcast join.
[ Parent
| on HN ]
Re:Why query planning for streaming systems is hard
by timgarmstrong on Monday August 16, 2021 @01:32AM
(#28195215)
The theme does come up in some papers - Google's F1 papers for example.
A lot of it doing the engineering work to make known techniques work - you can have a broadcast join that spills to disk (using a hybrid hash join or similar) and then layer on other techniques to make the spilling more incremental and reduce the penalties from spilling (e.g. bloom filters). It's just an order-of-magnitude increase in complexity to go from a simple broadcast join to a robust one.