Re:Self-Adjusting Computation
by assface on Sunday May 31, 2015 @06:28PM
(#9635849)
There is a significant storage overhead due to all of the data that is collected about the computation (the "dynamic dependency graph").
The storage overhead is massive. It's a non-starter with this approach. Our experiments with self-adjusting computation were in the range of 30-100x for simple algorithms. That means if you have a 1TB data set, you need 30TB just to store the intermediate results.
A relational database with materialized views or the special purpose systems that you cite (DBToaster, LINVIEW) are better approaches.
Re:Self-Adjusting Computation
by umutacar on Monday June 01, 2015 @12:01PM
(#9639663)
storage overhead of self-adjusting computation depends on the granularity at which dependencies are tracked, which is in the control of the programmer. my post above provides more information.
The storage overhead is massive. It's a non-starter with this approach. Our experiments with self-adjusting computation were in the range of 30-100x for simple algorithms. That means if you have a 1TB data set, you need 30TB just to store the intermediate results.
A relational database with materialized views or the special purpose systems that you cite (DBToaster, LINVIEW) are better approaches.