News for hackers. Stuff that matters.
Re:Show HN: Fine-tune your own Llama 2 to replace GPT-3.5/4 by bfirsh on Tuesday September 12, 2023 @05:13PM (#37488474)
Founder of Replicate here. It's early indeed.

OpenAI aren't doing anything magic. We're optimizing Llama inference at the moment and it looks like we'll be able to roughly match GPT 3.5's price for Llama 2 70B.

Our current models are far from efficient, but I'm surprised 7B turned out 100x more expensive. By our math, it's about ~10x more expensive right now for the base model. For fine-tuned models, we're roughly the same price. Rony – will message you on Discord to dig in more. :)

Re:Show HN: Fine-tune your own Llama 2 to replace GPT-3.5/4 by yixu34 on Wednesday September 13, 2023 @02:05AM (#37492776)
We're working on LLM Engine (https://llm-engine.scale.com) at Scale, which is our open source, self-hostable framework for open source LLM inference and fine-tuning. We have similar findings to Replicate: Llama 2 70B can be comparable to GPT 3.5 price, etc. Would be great to discuss this further!
Re:Show HN: Fine-tune your own Llama 2 to replace GPT-3.5/4 by Dowwie on Wednesday September 13, 2023 @01:14PM (#37499077)
How heavy of a lift is it to optimize inference?