News for hackers. Stuff that matters.
Re:Differential Dataflow for the Masses by snthpy on Saturday November 09, 2024 @03:13AM (#42093202)
Love this project!

Since "Differential Dataflow" has a specific meaning though and this isn't it, I wonder if a different title wouldn't be better?

"DBSP for the masses" presumes someone already knows what DBSP is so that doesn't work.

What about "Incremental Dataflow for the masses" or "Incremental Dataflow in Python"?

Re:Differential Dataflow for the Masses by rebanevapustus on Saturday November 09, 2024 @04:01AM (#42093369)
Incremental Dataflow for the Masses sounds much better :)
Re:Differential Dataflow for the Masses by snthpy on Saturday November 09, 2024 @03:18AM (#42093213)
Also, I never fully grokked Differential Dataflow but I get DBSP so I would love an explanation of how Differential Dataflow differs from DBSP.
Re:Differential Dataflow for the Masses by rebanevapustus on Saturday November 09, 2024 @04:01AM (#42093366)
https://github.com/brurucy/pydbsp/issues/6 it's been answered! (to the best of my knowledge)
Re:Differential Dataflow for the Masses by keithalewis on Wednesday November 13, 2024 @04:15PM (#42130071)
This seems to make the mistake of identifying a vector with a stream.
Re:Differential Dataflow for the Masses by rebanevapustus on Wednesday November 13, 2024 @06:30PM (#42131391)
In this implementation streams are not vectors nor lists. They are ordered hash maps.
Re:Differential Dataflow for the Masses by keithalewis on Friday November 15, 2024 @10:57AM (#42148047)
Just pointing out what the theoretical documentation says. A stream should have operations for empty, current, and next. You don't want implementation details leaking into the abstract model.
Re:Differential Dataflow for the Masses by erichocean on Thursday November 14, 2024 @12:14PM (#42138424)
Electric Clojure v3 also implements differential dataflow. [0]

[0] https://www.youtube.com/watch?v=mP9e_FDjHLU

Re:Differential Dataflow for the Masses by rebanevapustus on Thursday November 14, 2024 @05:04PM (#42141710)
Electric Clojure v3 cannot be "Differential Dataflow" because it *seems* to be restricted to DAGs only. Differential Dataflow is highly expressive, it supports iteration.
Re:Differential Dataflow for the Masses by dustingetz on Thursday November 14, 2024 @05:42PM (#42141985)
(electric founder here) we are lowercase differential dataflow my friend. Fwiw our model does support cycles and iteration, we simply didn’t bother to formally reify cycles yet, because we have a functional effect system under the hood and using that to implement cycles by userland side effect does the right thing, is perfectly efficient, and has very little downside.
Re:Differential Dataflow for the Masses by rebanevapustus on Friday November 15, 2024 @10:17AM (#42147649)
Thanks for clarifying it!

I checked some of your videos, and I couldn't quite grasp how much of (capital) DDflow do you borrow.

What does "our model does support cycles and iteration" mean? Do these also have the incremental semantics that guarantee bounds over the size of updates?