News for hackers. Stuff that matters.
Re:70B Llama 2 at 35tokens/second on 4090 by Palmik on Wednesday September 13, 2023 @12:01PM (#37498499)
Fine tuned gpt-3.5 is 8x the cost of regular gpt-3.5 for inference, other thread where this is discussed: https://news.ycombinator.com/item?id=37494335

This may be acceptable for some use cases, bot not for others where even regular gpt-3.5 may not be economical.

There are no comments on this comment yet.