Hidet: A Deep Learning Compiler for Efficient Model Serving

Re:Hidet: A Deep Learning Compiler for Efficient Model Serving by brucethemoose2 on Thursday April 27, 2023 @11:58PM (#35737331)

Hmm, does not work for Stable Diffusion yet :(

But inductor/triton didn't work in the 2.0 nightlies either, and now it works fine for SD.

[ Parent | on HN ]

Re:Hidet: A Deep Learning Compiler for Efficient Model Serving by mlazos on Friday April 28, 2023 @01:20AM (#35737696)

This bug was in the front end (dynamo) so I’m not surprised it happens with hidet too. I fixed this for the 2.0.1 release.

[ Parent | on HN ]

Re:Hidet: A Deep Learning Compiler for Efficient Model Serving by brucethemoose2 on Friday April 28, 2023 @01:39AM (#35737817)

I am getting backend errors right now. For instance, with the unet:

> torch._dynamo.exc.BackendCompilerFailed: backend='hidet_backend' raised: NotImplementedError: hidet: Tensor.to(..., device=...) is not supported for symbolic tensors...

Early torch 2.0 was full of backend unimplemented/unsupported errors too, but I can't even remember when dynamo issues popped up (or even when everything started working) since I stayed on torch nightly.

[ Parent | on HN ]

Re:Hidet: A Deep Learning Compiler for Efficient Model Serving by mlazos on Friday April 28, 2023 @01:56AM (#35737921)

Ah ok this wasn’t what I thought then. I haven’t taken a look at Hidet yet so I won’t be much help here :( I thought you were talking about 2.0.0

[ Parent | on HN ]

Re:Hidet: A Deep Learning Compiler for Efficient Model Serving by yyding on Friday April 28, 2023 @02:21AM (#35738075)

Hi, do you mind submitting an issue to the hidet github repo so we can have a look? Thank you!

[ Parent | on HN ]

Re:Hidet: A Deep Learning Compiler for Efficient Model Serving by akbarnur on Friday April 28, 2023 @09:36AM (#35741115)

Hey @bructhemoose2 can you file an issue, we will try to fix it ASAP: https://github.com/hidet-org/hidet/issues

[ Parent | on HN ]