Re:Show HN: Orchest – Data Science Pipelines
by pplonski86 on Thursday August 13, 2020 @04:44AM
(#24141216)
Scheduling periodic tasks can be useful for creating ETLs or to create alarms (every 1h hour check data for new values if condition is meet then send the email alert)
Re:Show HN: Orchest – Data Science Pipelines
by yannickperrenet on Thursday August 13, 2020 @05:39AM
(#24141518)
Agreed! ;)
Currently, tasks can only be scheduled to run at a set time. Could you elaborate a little on why you would want tasks to run periodically? We have some ideas on why this might be helpful, but would love to hear your take on this. (Periodic task scheduling is absolutely something we can add.)
The front-end of the application actually makes calls to our API for multiple things, among which execution of tasks (our internal endpoints can be found here https://github.com/orchest/orchest/tree/master/orchest/orche...). Exposing a (REST) API to the user to interact with the pipeline and start executions is on our roadmap.
I hope this answers your questions.