Re:A 1000-processor chip powered by an AA battery (2016)
by vidarh on Tuesday February 16, 2021 @06:02AM
(#26152824)
A lot more things can be parallelised than we do today.
Consider this: We mostly parallelize compilation today at the unit of a single compilation unit.
We do this for two reasons: It's simple, for languages where separate compilation is possible, and it's good enough when you have a small-ish (smaller than the number of compilation units) number of fast cores.
But a lot of compilation can be reduced into far more fine-grained parallel steps.
E.g. parsing can be turned into a chain of array operations along the line of finding possible start/ends of tokens, finding possible starts and ends of quoted segments, finding possible starting points for matches against various rules based on the possible token starts etc. You can further build syntax trees as arrays of vectors. If you have many enough cores it may pay off to subdivide the work to extreme levels like that.
But it's hard to reason about. We're not used to it, and most of us are no good at conceptualising it. Some are.
For a real-life example of a compiler that actually runs on GPU's and aims for massive parallelisation (it compiles a subset of Dyalog APL), check out "co-dfn" by Aaron Hsu. It's very terse and complex to read, and frankly I've only spent time looking at it very superficially, but it's an amazing piece of work. Now, if it was easier for mere mortals to actually write code like that, and we had a more compelling reason (many, but slower cores), we'd probably see a lot more parallelisation. In the short run it'd be extremely painful, but in the long run it might well benefit us.
Consider this: We mostly parallelize compilation today at the unit of a single compilation unit.
We do this for two reasons: It's simple, for languages where separate compilation is possible, and it's good enough when you have a small-ish (smaller than the number of compilation units) number of fast cores.
But a lot of compilation can be reduced into far more fine-grained parallel steps.
E.g. parsing can be turned into a chain of array operations along the line of finding possible start/ends of tokens, finding possible starts and ends of quoted segments, finding possible starting points for matches against various rules based on the possible token starts etc. You can further build syntax trees as arrays of vectors. If you have many enough cores it may pay off to subdivide the work to extreme levels like that.
But it's hard to reason about. We're not used to it, and most of us are no good at conceptualising it. Some are.
For a real-life example of a compiler that actually runs on GPU's and aims for massive parallelisation (it compiles a subset of Dyalog APL), check out "co-dfn" by Aaron Hsu. It's very terse and complex to read, and frankly I've only spent time looking at it very superficially, but it's an amazing piece of work. Now, if it was easier for mere mortals to actually write code like that, and we had a more compelling reason (many, but slower cores), we'd probably see a lot more parallelisation. In the short run it'd be extremely painful, but in the long run it might well benefit us.