News for hackers. Stuff that matters.

A comment on OpenAI: Improvements to the fine-tuning API and expanding our cus. Parent comment

Re:OpenAI: Improvements to the fine-tuning API and expanding... by throwaway74432 on Thursday April 04, 2024 @02:40PM (#39934246)
Number of tokens is still a useful metric, as their endpoints have Tokens Per Minute quotas. Decreasing number of tokens used means increasing throughput, up until the Request Per Minute quota.
[ Parent | on HN ]
Re:OpenAI: Improvements to the fine-tuning API and expanding... by afro88 on Thursday April 04, 2024 @02:43PM (#39934281)
But the sentence was "Indeed was able to improve cost..."
[ Parent | on HN ]
Re:OpenAI: Improvements to the fine-tuning API and expanding... by mechagodzilla on Thursday April 04, 2024 @02:46PM (#39934313)
From OpenAI’s perspective the cost improved!
[ Parent | on HN ]
Re:OpenAI: Improvements to the fine-tuning API and expanding... by throwaway74432 on Thursday April 04, 2024 @02:49PM (#39934338)
"...and latency" Higher throughput = lower latency*

*Under certain conditions

[ Parent | on HN ]