Testing ability of GPT4 and open-source LLMs to detect C++ bugs

Re:Testing ability of GPT4 and open-source LLMs to detect C+... by chenzhekl on Sunday April 16, 2023 @02:06AM (#35587681)

It is amazing to see how GPT is so much better than other LLMs. Should this be attributed to OpenAI's better training data?

[ Parent | on HN ]

Re:Testing ability of GPT4 and open-source LLMs to detect C+... by brucethemoose2 on Sunday April 16, 2023 @02:42AM (#35587820)

Its possible that LLaMA 65B would be stronger with some instruction tuning, but it looks like its only been done on smaller LLaMa models.

[ Parent | on HN ]

Re:Testing ability of GPT4 and open-source LLMs to detect C+... by nullsense on Sunday April 16, 2023 @06:00AM (#35588648)

I really want to see an instruction tuned LLaMA 65B. Hope it will happen.

[ Parent | on HN ]

Re:Testing ability of GPT4 and open-source LLMs to detect C+... by fhd2 on Sunday April 16, 2023 @02:11AM (#35587698)

Presumably. Vicuna was fine-tuned on ChatGPT responses, but from a quick glance through the examples in the blog post, there are none asking it to find a bug: https://vicuna.lmsys.org/

I've found Vicuna 13B to be quite capable at _generating_ code. But not as good as GPT-4. Model size probably also plays a large role: 13B parameters runs nicely on normal-ish home user hardware, GPT-4 not quite.

[ Parent | on HN ]

Re:Testing ability of GPT4 and open-source LLMs to detect C+... by davidjones332 on Sunday April 16, 2023 @04:05AM (#35588141)

government black project

[ Parent | on HN ]