News for hackers. Stuff that matters.
Re:Our Humble Attempt at “How Much Data Is Needed to Fine-Tune by swyx on Monday September 25, 2023 @12:40AM (#37639730)
> ...our experiment showed that fine-tuned GPT-3.5 models were significantly faster than [base GPT-3.5] by 3.6 to 3.76 times.

does this pass the sniff test? under what situations could a finetuned single tenant model be faster than a batchable multitenant base model?

Re:Our Humble Attempt at “How Much Data Is Needed to Fine-Tune by gnahzby on Monday September 25, 2023 @12:59AM (#37639810)
This result really surprised us too, but it seems like we were not alone in observing this: https://medium.com/@gabrielgrinberg/openai-api-fine-tuned-gp...

That being said, both accounts that we experimented on were personal accounts ( with a very limited budget) so maybe there are some special behaviors (e.g. some rate limit that only exists for base models)

Re:Our Humble Attempt at “How Much Data Is Needed to Fine-Tune by eqian99 on Monday September 25, 2023 @01:59AM (#37640091)
Yeah we were definitely surprised! This was a small scale experiment; we uploaded our notebook for latency tests here if anyone is interested taking a look at the code / trying it their own finetuned models: https://github.com/eqian99/finetuning-experiment
Re:Our Humble Attempt at “How Much Data Is Needed to Fine-Tune by dnnssl2 on Monday September 25, 2023 @02:23AM (#37640226)
Under the same conditions where enterprise versions of the API have significantly less latency and better reliability than personal. OpenAI can change anything about the underlying infrastructure.