BigCode Project Releases StarCoder: A 15B Code LLM

Re:BigCode Project Releases StarCoder: A 15B Code LLM by sauwan on Thursday May 04, 2023 @03:14PM (#35820663)

I'm assuming higher is better? How does this compare to GPT-3.5/4?

[ Parent | on HN ]

Re:BigCode Project Releases StarCoder: A 15B Code LLM by qwerty3344 on Thursday May 04, 2023 @03:25PM (#35820822)

[ Parent | on HN ]

Re:BigCode Project Releases StarCoder: A 15B Code LLM by qwerty3344 on Thursday May 04, 2023 @03:25PM (#35820821)

I think it's still significantly behind GPT 3.5/4, both of which can get 67% on HumanEval, and 88% with Reflexion

[ Parent | on HN ]

Re:BigCode Project Releases StarCoder: A 15B Code LLM by enum on Thursday May 04, 2023 @04:53PM (#35821907)

Keep in mind that StarCoder(Base) is just a pretrained LM. The extra stuff that makes 3.5/4 like RLHF gets built on this.

[ Parent | on HN ]

Re:BigCode Project Releases StarCoder: A 15B Code LLM by manojlds on Thursday May 04, 2023 @05:14PM (#35822170)

Aren't GPT-3 etc base LM and ChatGPT the instruction tuned? Or am I wrong?

[ Parent | on HN ]

Re:BigCode Project Releases StarCoder: A 15B Code LLM by dpf on Friday May 05, 2023 @06:52AM (#35827595)

code-davinci-002 is a base LM, and the other 3.5 models (text-davinci-{002,003}, gpt-3.5-turbo, and ChatGPT) use instruction tuning and/or RLHF. Source: https://platform.openai.com/docs/model-index-for-researchers

[ Parent | on HN ]