News for hackers. Stuff that matters.
Re:Copilot Internals by melony on Sunday December 18, 2022 @02:02PM (#34041444)
Why does “cushman-ml” suggest a 12B model instead of the 175B model?
Re:Copilot Internals by peaslock on Sunday December 18, 2022 @02:29PM (#34041831)
Just guessing here: OpenAI named their models alphabetically in order of the model size:

https://blog.eleuther.ai/gpt3-model-sizes/

Davinci performed similarly to a 175B GPT-3 model and the next smallest model is Curie which corresponds to a 6.7B GPT model.

Now, there have been two GPT models fine tuned on Code, namely davinci-codex and cushman-codex. Since cushman != curie, it is presumably based on the GPT-model in between, namely the one with 13B. The initial Codex model as known to have 12B parameters, so it may still be that model:

https://github.com/will-thompson-k/tldr-transformers/blob/ma...

See also A.2 here: https://arxiv.org/pdf/2204.00498.pdf#page=6

Re:Copilot Internals by alextheparrot on Sunday December 18, 2022 @03:01PM (#34042302)
Code is the base model in more recent iterations [0]

[0] https://beta.openai.com/docs/model-index-for-researchers

Re:Copilot Internals by varunkmohan on Sunday December 18, 2022 @05:45PM (#34044802)
Most likely latency and cost reasons. A model that's 10x as big requires 10x the hardware to serve at the same latency. Since most generations are not too long, a smaller finetuned model should work well enough.