Re:Show HN: Cuneiform – A Functional Workflow Language
by chrisseaton on Friday February 05, 2016 @08:03AM
(#11041079)
> Programs written in a general purpose programming language (like Python) are [not] automatically parallelizable
Of course they are - your super-scalar, out-of-order processor does it for you at the very least.
And at a higher level, the same speculative techniques that can be used to automatically parallelise a C program can also be used for Python, Ruby, etc. It's just that nobody has implemented that yet.
Re:Show HN: Cuneiform – A Functional Workflow Language
by joergen7 on Friday February 05, 2016 @10:34AM
(#11042046)
Perhaps I was a bit imprecise by saying that Python programs are not automatically parallelizable.
What I was trying to say was that Python programs are not automatically parallelized by the standard Python interpreter.
Thanks for pointing it out.
Programs written in a general purpose programming language (like Python) are neither automatically parallelizable nor do they run on more than one machine. Of course, most general purpose languages offer facilities for parallelization and/or distribution. Python has the multiprocessing library. In R there is the parallelMap library. But responsibility over how parallelization is used in such a program still resides with the programmer.
This is different in Cuneiform. Everything is automatically parallelized here. Most task applications are relatively large computational pieces of work that run for seconds or minutes (in contrast to, e.g., an addition). In this scenario it makes sense to parallelize wherever possible.
Cuneiform allows you to drive any API.
Another thing that is difficult in most general purpose programming languages is the incorporation of libraries not written in that particular language. E.g., it is non-trivial to use a Python library in Java. Even calling command line applications requires a small wrapper (os.system() in Python). In Cuneiform this is the normal mode of operation. Writing a wrapper around a Bash script is just as hard as defining a function.