News for hackers. Stuff that matters.

A comment on Chronon, Airbnb's ML Feature Platform, Is Now Open Source. Parent comment

Re:Chronon, Airbnb's ML Feature Platform, Is Now Open Source by hobs on Tuesday April 09, 2024 @04:53PM (#39984124)
https://en.wikipedia.org/wiki/Sixth_normal_form Basically we've had time travel (via triggers or built in temporal tables or just writing the data) for a long time, its just expensive to have it all for an OLTP database.

We've also had slowly changing dimensions to solve this type of problem for a decent amount of time for the labels that sit on top of everything, though really these are just fact tables with a similar historical approach.

[ Parent | on HN ]
Re:Chronon, Airbnb's ML Feature Platform, Is Now Open Source by ezvz on Tuesday April 09, 2024 @05:14PM (#39984329)
6NF works well for some temporal data, but I haven't seen it work well for windowed aggregations because the start/end time format of saving values doesn't handle events "falling out of the window" too well. At least the examples I've seen have values change due to explicit mutation events.
[ Parent | on HN ]
Re:Chronon, Airbnb's ML Feature Platform, Is Now Open Source by hobs on Tuesday April 09, 2024 @07:32PM (#39985426)
Agree, you don't really want to pre-aggregate your temporal data, or it will effectively only aggregate at each row-time boundary and the value is lower than just keeping the individual calculations.
[ Parent | on HN ]