News for hackers. Stuff that matters.
Re:GPT4All releases one click install for Mac,win64,Linux wi... by brucethemoose2 on Tuesday April 18, 2023 @09:18AM (#35613593)
https://huggingface.co/TheBloke/vicuna-AlekseyKorshuk-7B-GPT...

Try this model, which uses excludes some of the "ethics" filtering.

Also, IDK what you mean by a "small" GPU, but I am getting ~3.5 tokens/sec on a laptop 2060, and I think I can squeeze more out of it. And there are methods for splitting up the load in small VRAM pools.

There are no comments on this comment yet.