Show HN: Running LLMs in one line of Python without Docker

Re:Show HN: Running LLMs in one line of Python without Docker by jiayq84 on Wednesday October 04, 2023 @05:35PM (#37772144)

Thanks - we definitely agree that llama.cpp is great. Big fan of their optimizations. We are more or less orthogonal to the engines though - in the sense that we serve as the infra/platform to run and manage those implementations easily. For example, we support running a wider range of models - for example sdxl is one single line too:

lep photon run -n sdxl -m hf:stabilityai/stable-diffusion-xl-base-1.0 --local

It's really about how to productize a wide range of models as easy as possible.

[ Parent | on HN ]

Re:Show HN: Running LLMs in one line of Python without Docker by brucethemoose2 on Wednesday October 04, 2023 @05:45PM (#37772253)

SDXL is indeed a monster to install and setup. The UIs are even worse.

IDK if the GPL license is compatible with your business, but I wonder if you could package Fooocus or Fooocus-MRE into a window? Its a hairy monster to install and run, but I've never gotten such consistently amazing results from a single prompt box + style dropdown box (including native HF diffusers and other diffusers-based frontends). The automatic augmentations to the SDXL pipine are amazing:

https://github.com/MoonRide303/Fooocus-MRE

[ Parent | on HN ]

Re:Show HN: Running LLMs in one line of Python without Docker by jiayq84 on Wednesday October 04, 2023 @05:50PM (#37772295)

Oh wow yeah, that is a beast. Let me give it a shot.

[ Parent | on HN ]