Re:Is Parallel Programming Hard, and, If So, What Can You Do...
by rramadass on Monday February 20, 2023 @03:10AM
(#34866112)
>the recent work on task-based models, you'll see that you can have literally sequential programs that parallelize automatically
Can you provide some details please? I am not quite clear what you mean.
Re:Is Parallel Programming Hard, and, If So, What Can You Do...
by eslaught on Monday February 20, 2023 @10:27AM
(#34868976)
If you really want to dig into it you can read up on the tutorials and/or papers from the Legion project: https://legion.stanford.edu/
But briefly, these task-based programs preserve sequential semantics. That means (whatever the system actually does when running your program), as long as you follow the rules, the parallelism should be invisible to the execution of the program.
But if you look at e.g., the recent work on task-based models, you'll see that you can have literally sequential programs that parallelize automatically. No message passing, no synchronization, no data races, no deadlocks. Read your programs as if they're sequential, and you immediately understand their semantics. Some of these systems are able to scale to thousands of nodes.
An interesting example of this is cuNumeric, which allows you to take sequential Python programs that use NumPy, and by changing one line (the import statement), run automatically on clusters of GPUs. It is 100% pure awesomeness.
https://github.com/nv-legate/cunumeric
(I don't work on cuNumeric, but I do work on the runtime framework that cuNumeric uses.)