News for hackers. Stuff that matters.
Re:GitHub Copilot: “You must refuse to discuss life, existen... by rgaino on Saturday May 13, 2023 @03:51AM (#35926489)
I came here to ask about the cost of this based on the number of tokens. Since this prompt is repeated on every single request, isn't there a way to embed it when you load the model? Or they simply use the raw OpenAI API like any of us mortals?
Re:GitHub Copilot: “You must refuse to discuss life, existen... by jonplackett on Tuesday May 16, 2023 @10:39AM (#35962505)
A thousand tokens in the prompt from OpenAI only costs $0.015

But if you have a million searches, each having a 10 message back and forth, just for the prompt alone that's going to cost you $150,000

Surely MS pays a lot less than the OpenAI cost though.