News for hackers. Stuff that matters.

A comment on Show HN: Running LLMs in one line of Python without Docker.

Re:Show HN: Running LLMs in one line of Python without Docker by rw2 on Wednesday October 04, 2023 @04:27PM (#37771225)
Is not having to build a dockers image worth $100 a month? I do find server setup to be a pain but I think if I will use a model for a year, I can take the time(3-4 hours) to set it up. Only with constant switching of models would I use a service like this.

I never set up bigger models like LLAMA on servers. Other hacker news people can chime in.

[ Parent | on HN ]
Re:Show HN: Running LLMs in one line of Python without Docker by jiayq84 on Wednesday October 04, 2023 @04:39PM (#37771397)
It's not only about "building a docker" but also maintaining multiple models, multiple environments and a lot of users. Imagine there is a group of engineers each needing to deploy their own models: one needs tensorflow 1.x, one needs tensorflow 2.x, one needs pytorch and one needs a very strange combination of dependencies. Trust me, things get complex very easily:

https://github.com/leptonai/examples/blob/main/advanced/whis...

I definitely agree that for a fixed use case, building a docker once and for all is probably the simplest and best approach. However, it quickly gets more complex and out of hand.

Also the basic plan is free for independent developers - isn't it worth it? :)

[ Parent | on HN ]