News for hackers. Stuff that matters.
Re:DLVM: A modern compiler framework for neural network DSLs by grandmczeb on Thursday February 22, 2018 @02:51AM (#16435973)
Lots of modern models have very late binding variables which are hard to precompile for (sentence length in MNT, for example). That means you're going to need to do some form of specialization at runtime, so a JIT makes sense.
Re:DLVM: A modern compiler framework for neural network DSLs by deepnotderp on Thursday February 22, 2018 @10:18AM (#16438096)
Just treat it as an infinite loop , there's no need to JIT in an optimized version that late.
Re:DLVM: A modern compiler framework for neural network DSLs by grandmczeb on Thursday February 22, 2018 @03:11PM (#16440950)
One of the core operations of the transformer network[1] is a (LxL) x (LxE) matrix multiply (where L is the sentence length and E is the network width). Can you be more specific about how you would get good performance without specializing on L?

[1] https://arxiv.org/abs/1706.03762

Re:DLVM: A modern compiler framework for neural network DSLs by deepnotderp on Thursday February 22, 2018 @03:59PM (#16441337)
You use the loop based GEMM kernel and inject the loop counters as the input size.
Re:DLVM: A modern compiler framework for neural network DSLs by grandmczeb on Thursday February 22, 2018 @05:02PM (#16441876)
L can be as small as 1 and bigger than 512. For small L it makes sense to do different optimizations than large L. A loop based GEMM doesn’t help with that.