News for hackers. Stuff that matters.

yannickperrenet

A while ago I wrote a simple script (including cli) to convert your exported bookmarks to markdown [1]. Hope it helps.

[1] https://github.com/yannickperrenet/bookmarkdown

Re:Show HN: Orchest – Data Science Pipelines on Thursday August 13, 2020 @05:39AM (#24141518)
Agreed! ;)
Re:Show HN: Orchest – Data Science Pipelines on Wednesday August 12, 2020 @10:26PM (#24139361)
Grid Studio is indeed the project Rick worked on before starting to work on Orchest. Great to hear that you are familiar with it.

Currently, tasks can only be scheduled to run at a set time. Could you elaborate a little on why you would want tasks to run periodically? We have some ideas on why this might be helpful, but would love to hear your take on this. (Periodic task scheduling is absolutely something we can add.)

The front-end of the application actually makes calls to our API for multiple things, among which execution of tasks (our internal endpoints can be found here https://github.com/orchest/orchest/tree/master/orchest/orche...). Exposing a (REST) API to the user to interact with the pipeline and start executions is on our roadmap.

I hope this answers your questions.

Re:Show HN: Orchest – Data Science Pipelines on Wednesday August 12, 2020 @10:11PM (#24139285)
Thanks for your input. We are planning to also support an API and/or CLI (besides the GUI) to set up the pipelines through scripting. Depending on what we hear from the community we will definitely add this sooner rather than later.

You make a great point for the use of monitoring the progress of a run. During pipeline runs the visual pipeline interface (also in view only mode during or after running pipeline runs as part of an experiment) can be helpful as a visual indication of where the pipeline failed, how long pipeline steps took to execute and whether certain pipeline steps are still running.

Lastly, for team collaboration we think the visual interface is indeed helpful to explain (especially to the less technical people) the structure of the code, what exactly the pipeline sets out to solve and how the pipeline solves it. In addition, the possibility to add intermediary pipeline steps that serve as dashboards can be helpful for collaboration.