GitHub Copilot Leaked Prompt

Re:GitHub Copilot Leaked Prompt by spacetime_cmplx on Friday May 12, 2023 @10:35PM (#35924924)

Why is everyone so confused about this? Isn't verifying the easy part? You put it into the GPT-3.5/4 API as a system prompt and see it answers like the actual chatbot. If it does, you've either extracted the actual prompt (congrats!) or something else that works just as well (congrats!). If it doesn't, it's a hallucination. If you're worried about temperature setting throwing you off, keep trying new questions until you find one that the original chatbot gives the same answer consistently.

It's like a trapdoor function.

Am I missing something?

[ Parent | on HN ]

Re:GitHub Copilot Leaked Prompt by perryizgr8 on Friday May 12, 2023 @10:49PM (#35925013)

It may not be the exact same model as GPT. They may have tweaked some parameters and almost definitely trained it on additional content relevant to the task of helping with coding. So you probably can't get the same output with just the same prompt.

[ Parent | on HN ]

Re:GitHub Copilot Leaked Prompt by spacetime_cmplx on Friday May 12, 2023 @10:54PM (#35925035)

Sure, in which case the real prompt is as useless as a hallucinated one, so what's the difference?

[ Parent | on HN ]

Re:GitHub Copilot Leaked Prompt by qup on Saturday May 13, 2023 @05:12AM (#35926812)

I guess that now verifying it isn't the easy part, as you boldly claimed the comment before?

[ Parent | on HN ]

Re:GitHub Copilot Leaked Prompt by yeck on Saturday May 13, 2023 @08:26AM (#35927702)

I don't think the purpose of getting the prompt leaked was to then use the prompt but just to expose the limitations of this approach to steering an LLM.

[ Parent | on HN ]