Llama.cpp: Full CUDA GPU Acceleration

Re:Llama.cpp: Full CUDA GPU Acceleration by barbariangrunge on Tuesday June 13, 2023 @09:56AM (#36310279)

I’m newish to python, I’ve only used it for machine learning projects and some web scraping. Could somebody elaborate on venv? I just started using it but now everyone in this thread is saying how much they hate it. Is there an alternative?

[ Parent | on HN ]

Re:Llama.cpp: Full CUDA GPU Acceleration by BryantD on Tuesday June 13, 2023 @10:53AM (#36311107)

venv is fine. Remember that this is a self-selected sample of people. You’re going to bump into the flaws and gotchas but it’s a perfectly usable tool.

[ Parent | on HN ]

Re:Llama.cpp: Full CUDA GPU Acceleration by throwawaymaths on Tuesday June 13, 2023 @12:49PM (#36313025)

Uhm so I was professionally setting up ML distros and ML containers for cloud deployments. Venv is not fine, especially if you've seen how other langs do it.

[ Parent | on HN ]

Re:Llama.cpp: Full CUDA GPU Acceleration by collinmanderson on Tuesday June 13, 2023 @09:54PM (#36320387)

Could you say more what you don’t like about venv compared to how other languages do it?

[ Parent | on HN ]

Re:Llama.cpp: Full CUDA GPU Acceleration by throwawaymaths on Tuesday June 13, 2023 @10:24PM (#36320637)

Just store your deps in project directory instead of using hidden fucking magic.

[ Parent | on HN ]

Re:Llama.cpp: Full CUDA GPU Acceleration by tekknik on Wednesday June 14, 2023 @08:23AM (#36324788)

The hidden magic is adjusting env vars used by python, LD, etc. Adjusting paths only seems like magic until you understand what it’s doing.

I’ve done with with plenty of languages including C/C++

[ Parent | on HN ]

Re:Llama.cpp: Full CUDA GPU Acceleration by throwawaymaths on Wednesday June 14, 2023 @11:06AM (#36327097)

Yes I know how it works but try explaining that to someone learning flask at a boot camp.

[ Parent | on HN ]

Re:Llama.cpp: Full CUDA GPU Acceleration by tekknik on Saturday June 17, 2023 @09:30PM (#36376475)

I mean I wouldn’t expect someone at a boot camp to understand this nor was this a topic of conversation, so /shrug?

[ Parent | on HN ]

Re:Llama.cpp: Full CUDA GPU Acceleration by collinmanderson on Friday June 16, 2023 @01:37PM (#36360653)

The good news is they're working towards encouraging a standard .venv in project directory.

https://peps.python.org/pep-0704/

[ Parent | on HN ]

Re:Llama.cpp: Full CUDA GPU Acceleration by orra on Tuesday June 13, 2023 @01:08PM (#36313316)

IMHO venvs are fine as an implementation detail, a building block for a slicker tool.

The annoyance with venvs is you have to create and activate them. In contrast for cargo (or stack or dotnet or yarn or pipenv or poetry), you just run the build tool in the project directory.

Another limitation of venv is it doesn't solve the problem of pinning a versions of Python, so you need another tool.

[ Parent | on HN ]