Re:Show HN: A distributed workflow engine written in Go
by importsaas on Monday October 02, 2023 @07:20AM
(#37736744)
Potential frenemy here :)
First up, congratulations on the launch, and project. The timing and need for such a project is certainly apt.
Sharing my feedback based on spending 30 minutes as per the guide.
- a config with an example could be introduced earlier. Currently it is deeper in the docs.
- docker compose example could be introduced earlier. Currently it's just in the repo.
- would be great if the destination of the job could also be dynamic. Currently you have a nice yaml way to run SQL/go/bash. I think a job is truly potent when it can pull data from multiple places, but also push that to multiple places as well.
- it's great that it is thinking distributed from the get go.
- having a queue support was smart. having S3 has a source/destination would also be interesting
- how would you do safely do multi-tenant? Eg a SaaS decides to use tork, having 10 customers. I want to run a similiar job for each, and in a way that data doesn't get mixed up.
- hit some errors using an old docker engine. Something I haven't run into for many other projects that can run a docker compose version:3
ERR node ... failed health check error="Error response from daemon: client version 1.42 is too new. Maximum supported API version is 1.41"
WRN error getting CPU usage error="not implemented yet"
DBG received heartbeat cpu-percent=0 heartbeat-time=1696246437 hostname=temp.local node-id=... status=DOWN
- Am wondering if this could someday be a good alternative to swarm/portainer to run long living containers. That with the workflow may help in devops
- how do you schedule jobs. Wasn't clear after seeing examples in 30m
- I notice your git repo deletes branches every few days. So early in a project maybe it can be avoided?
- web ui is promising. Reminds me of inngest territory. Didn't have time to see what the other two tabs. Maybe add those screenshots.
- it's not clear what you mean by extensible. Because of being able to run any image, bash?
- docs could have a section for CRUD like GET requests to see running jobs. Filter by job status, filter by date.
First up, congratulations on the launch, and project. The timing and need for such a project is certainly apt.
Sharing my feedback based on spending 30 minutes as per the guide.
- a config with an example could be introduced earlier. Currently it is deeper in the docs.
- docker compose example could be introduced earlier. Currently it's just in the repo.
- would be great if the destination of the job could also be dynamic. Currently you have a nice yaml way to run SQL/go/bash. I think a job is truly potent when it can pull data from multiple places, but also push that to multiple places as well.
- it's great that it is thinking distributed from the get go.
- having a queue support was smart. having S3 has a source/destination would also be interesting
- how would you do safely do multi-tenant? Eg a SaaS decides to use tork, having 10 customers. I want to run a similiar job for each, and in a way that data doesn't get mixed up.
- hit some errors using an old docker engine. Something I haven't run into for many other projects that can run a docker compose version:3
- Am wondering if this could someday be a good alternative to swarm/portainer to run long living containers. That with the workflow may help in devops- how do you schedule jobs. Wasn't clear after seeing examples in 30m
- I notice your git repo deletes branches every few days. So early in a project maybe it can be avoided?
- web ui is promising. Reminds me of inngest territory. Didn't have time to see what the other two tabs. Maybe add those screenshots.
- it's not clear what you mean by extensible. Because of being able to run any image, bash?
- docs could have a section for CRUD like GET requests to see running jobs. Filter by job status, filter by date.
Godspeed!