News for hackers. Stuff that matters.

A comment on Stability AI Launches the First of Its StableLM Suite of Language Models.

Re:Stability AI Launches the First of Its StableLM Suite of ... by Filligree on Wednesday April 19, 2023 @12:49PM (#35630408)
The 7B model just barely fits in a 3090, but I've yet to get useful output; it usually gets stuck in a repetition loop.

Spent some time playing with sampling parameters, to no effect. Has anyone else had a go?

[ Parent | on HN ]
Re:Stability AI Launches the First of Its StableLM Suite of ... by nlightcho on Wednesday April 19, 2023 @01:21PM (#35630784)
Make sure you are using the instruction tuned model. The base model will be difficult to prompt.

It works in 8-bit with about 12GB of VRAM usage. Here's sample code:

https://gist.github.com/AlexanderDzhoganov/a1d1ebdb018e2e573...

[ Parent | on HN ]
Re:Stability AI Launches the First of Its StableLM Suite of ... by cheald on Wednesday April 19, 2023 @06:40PM (#35634225)
If you're getting stuck in a sampling loop, try raising the temperature (0.6+) and/or increasing the repetition penalty (1.1-1.2 often work well).
[ Parent | on HN ]