News for hackers. Stuff that matters.
Re:Advent of Code 2023's new AI/LLM Policy by foooorsyth on Monday October 16, 2023 @12:26PM (#37902215)
>Seems like they need to adapt the level of difficulty

"They" is one guy - Eric Wastl. He has a very specific style and hey may not want to change the entire style of the challenge just because some dorks want to cheat. I hope that he doesn't change it -- AoC is very approachable, especially inside the first ~15 or so days. It's a charming set of puzzles that people look forward to. I don't want it to become an elite competitive programming competition just to stay ahead of LLMs. I want to be able to partake with 1-2 hours a night after my kids go to bed.

With that said, I don't think LLMs will be helpful with some of the tougher problems. It's a hard thing to test, because I assume CoPilot would do well with previous year challenges due to the massive amount of public code available on the same problems. I don't think it'd do well on designing the IntCode VM or the modular arithmetic problem on first time seeing them without examples on GitHub. But perhaps Eric has been feeding some LLMs the unseen problems for this year and the LLMs are cracking them? I'm skeptical but perhaps they've crossed that rubicon.

Re:Advent of Code 2023's new AI/LLM Policy by yunwal on Monday October 16, 2023 @12:43PM (#37902527)
> With that said, I don't think LLMs will be helpful with some of the tougher problems. It's a hard thing to test, because I assume CoPilot would do well with previous year challenges due to the massive amount of public code available on the same problems.

I'm not 100% sure on this, but I think copilot isn't trained on anything past January 2022 (the gpt-3.5 and 4 cutoff dates), so I think it still shouldn't do well with AoC questions from 2022

Re:Advent of Code 2023's new AI/LLM Policy by avarun on Monday October 16, 2023 @01:00PM (#37902801)
You mean September 2021, right?
Re:Advent of Code 2023's new AI/LLM Policy by yunwal on Monday October 16, 2023 @02:07PM (#37903750)
I'm basing that off this community post and a few other forums that noted that GPT knows about events after September 2021. Point stands either way.

https://community.openai.com/t/what-is-the-actual-cutoff-dat...

Re:Advent of Code 2023's new AI/LLM Policy by danielbln on Tuesday October 17, 2023 @05:15AM (#37912391)
It seems the cut-off has shifted from September 21 to January 22 recently.