Re:To reinvent the processor
by jcranmer on Saturday May 11, 2019 @01:44PM
(#19887370)
> I think an under researched area is the application of deep learning to compilers. What could be done there for parallelism?
I don't think it's under-researched, because we already know that it's only ever going to do a lousy job. The problem of autoparallelization is mostly stymied by difficulty in computing the legality of transformations, and secondarily by the problem of tuning parameters [1]. We've realized that it's easier to just get the users to tell us about these things, sidestepping the problem for the most part.
> We are still kind of stuck in programming models developed back when core frequencies were scaling geometrically.
The programming model for parallelism is dataflow. We already know that: the tricky thing of parallelism is communication, so annotating all of the edges of communication makes the scheduling problem much, much easier.
[1] Theoretically, machine learning is good at tuning parameters. In practice, you get good results with the dumbest algorithms already, and the available headroom for performance is generally swamped by the fact that small changes such as function size can dramatically alter the performance characteristics of code elsewhere in the application.
I don't think it's under-researched, because we already know that it's only ever going to do a lousy job. The problem of autoparallelization is mostly stymied by difficulty in computing the legality of transformations, and secondarily by the problem of tuning parameters [1]. We've realized that it's easier to just get the users to tell us about these things, sidestepping the problem for the most part.
> We are still kind of stuck in programming models developed back when core frequencies were scaling geometrically.
The programming model for parallelism is dataflow. We already know that: the tricky thing of parallelism is communication, so annotating all of the edges of communication makes the scheduling problem much, much easier.
[1] Theoretically, machine learning is good at tuning parameters. In practice, you get good results with the dumbest algorithms already, and the available headroom for performance is generally swamped by the fact that small changes such as function size can dramatically alter the performance characteristics of code elsewhere in the application.