News for hackers. Stuff that matters.

A comment on Decorator JITs: Python as a DSL. Parent comment

Re:Decorator JITs: Python as a DSL by Scene_Cast2 on Monday February 03, 2025 @10:34AM (#42919195)
If this is the way forward, I'd love for a better developer experience.

I'm currently wrangling with pytorch.compile (flex attention doesn't like bias terms - see issues 145869 and 144511 if curious). As much as I love speed and optimization, JIT (at least the pytorch flavor) currently has weird return types that break VS Code's intellisense, weird stack traces, limitations around printing, and random issues like the bias term, and limitations such as not supporting sparsity.

Speaking of pytorch JIT workflows - what's a nice way of having a flag to turn off compilation?

[ Parent | on HN ]
Re:Decorator JITs: Python as a DSL by dleeftink on Monday February 03, 2025 @11:12AM (#42919681)
> random issues like the bias term

I'd like to know more about this!

[ Parent | on HN ]
Re:Decorator JITs: Python as a DSL by Scene_Cast2 on Monday February 03, 2025 @02:45PM (#42922016)
Basically this style of code - https://github.com/pytorch-labs/attention-gym/pull/84/files - has issues like this - https://github.com/pytorch/pytorch/pull/137452 https://github.com/pytorch/pytorch/issues/144511 https://github.com/pytorch/pytorch/issues/145869

For some higher level context, see https://pytorch.org/blog/flexattention/

[ Parent | on HN ]
Re:Decorator JITs: Python as a DSL by singhrac on Monday February 03, 2025 @03:20PM (#42922477)
You can decorate a function with @torch.compiler.disable() to disable a specific function (and anything further down the call stack).
[ Parent | on HN ]