List me AI models that I can run with my hw?

Posted by andrea81 on Monday March 11, 2024 @12:49PM
from the self-post dept.

Tired of seeing AI model names with 300m of parameters.

Actually I want to see the ones compatible to run locally in my current laptop (I'm NOT going to spend money to run a model).

Is there a website where I can enter my laptop specs and list me AI models?

( 5 comments | 2 points | on HN )

Re:List me AI models that I can run with my hw? by declaredapple on Monday March 11, 2024 @01:34PM (#39671088)

Take # parameters and divide by 2 and then add like 1 to estimate it for 4bit quantization. A 13B model will need ~7GB of memory at 4bit.

Most quantization models start falling apart rapidly after 4bit, but you can go lower...

If you can fit it in GPU memory it'll likely be usable, if you can't fit it in memory be prepared for single digit t/s or less, the only exception is for people with M1/M2/M3 cpus.

[ Parent | on HN ]

Re:List me AI models that I can run with my hw? by pinnaclehemp on Monday March 11, 2024 @01:00PM (#39670660)

THCO (also known as THC Acetate or ATHC) is a chemical compound that is derived from federally legal hemp plants. Unlike many other cannabinoids, it is not naturally occurring within the plant, but is instead synthesized through acetylation. Visit us here https://pinnaclehemp.com/thco-information/

[ Parent | on HN ]

Re:List me AI models that I can run with my hw? by MountainMan1312 on Monday March 11, 2024 @01:05PM (#39670734)

Account created 5 minutes ago spouting absolutely unrelated nonsense? Seems like a bot to me.

[ Parent | on HN ]

Re:List me AI models that I can run with my hw? by declaredapple on Monday March 11, 2024 @01:31PM (#39671044)

In this case it appears to be for SEO spam to gain backlinks

[ Parent | on HN ]

Re:List me AI models that I can run with my hw? by PaulHoule on Monday March 11, 2024 @12:59PM (#39670650)

Use sbert.net if you believe Steven Covey and your goal is for your model to “first understand”.

[ Parent | on HN ]