Re:Data Parallel, Task Parallel, and Agent Actor Architectures
by jgraettinger1 on Tuesday July 18, 2023 @10:39PM
(#36781509)
Estuary Flow derivations [1], I suspect (I work here)? You write a program which has messages pushed to it, which maintains and updates an internal state, and which publishes output messages.
You can build cyclic data-flows [2] - not just DAGs - which seems a key ingredient.
You can also write contract-based tests that verify the end-to-end behaviors of your "agents" [3].
Today, you're limited to SQLite or TypeScript. Looking forward, we aim to open it up to any program packaged in a docker container.
You can build cyclic data-flows [2] - not just DAGs - which seems a key ingredient.
You can also write contract-based tests that verify the end-to-end behaviors of your "agents" [3].
Today, you're limited to SQLite or TypeScript. Looking forward, we aim to open it up to any program packaged in a docker container.
[1] https://docs.estuary.dev/concepts/derivations/
[2] https://docs.estuary.dev/concepts/derivations/#approving-tra...
[3] https://docs.estuary.dev/concepts/tests/