That's not to say that you can't still do a foot race between A and B. It's just now you have to constrain it, it has become something different.
Designing such a robot would be an achievement in its own right, but not really in the spirit of the competition.
If you just rented some time on someone else's robot, its just plain cheating.
I would love to see teams compete, in a separate venue/ranking, based on their own model's ability to write code with minimal prompting.
It would be kinda cool to see what can be done in different 'classes' of model. How far can you get with a local model running on an 8gb card, or just using an M2 mac. How far can you get with a custom tuned chatgpt vs a programming specialized model. Etc...
Yes, an AIdvent of Code would be interesting...
Last year, AI was only able to solve the first 3 or 4 problems, so it's really not like using AI takes all human ingenuity out of the equation. What it does is enable you to autocomplete sub-problems that are already solved problems so you can focus on the good stuff. I'd personally prefer if there were 2 leaderboards - one with AI, and one without. Motorcycle races are fun too :)
There's two views: (1) programming as a craft and an art, a human creation, and (2) programming as a hurdle to overcome to accomplish more important things. People with both views might wish to practice.
In the case of AoC, the only objective incentive I can think about is impressing a recruiter. Maybe it's enough if you're motivated enough.
I play chess and some players end up thinking that cheating allows them to reach their real level and that it's just bad luck or conspiracy that prevent them to get to that level without resorting to cheating. Talk about cognitive dissonance.
Weirdly enough, basically this happens all the time at races [0]. Not a motorcycle, but a runner in my local competitive scene used a bicycle to fake her strava data for a HM in 2017 [1].
I don't get it either.
[0] https://www.wired.com/story/marathon-investigation-cheaters-...
[1] https://washingtonpost.com/news/morning-mix/wp/2017/02/23/ho...
It's also likely that a lot of AI users will be skilled developers too. They already know they can solve the problems, they also think they deserve the leaderboard rankings, so using AI is ok.
This reminds me of the Project Euler position on people posting their solutions publicly. The assumption is others would seek them out, get credit they don’t deserve and not learn as much.
I understand trying to prevent skewing the results, but personally it doesn’t affect me since I’m not doing it to compete in the website, I’m doing it for self edification.
I believe this actually helps me compete in the thing that really matters, getting a job, bringing high quality products to market and generally solving real problems people have.
Are LLMs starting to solve some of that? Sure. So then I want to compete for the jobs that still can’t be solved by them and involve more complicated and interesting solutions.
If people want to handicap their problem solving abilities or compete for jobs supervising boilerplate generation, all the better for me. I’ll take a hopefully more rewarding career over making the leaderboard on AoC and PE. Though I do feel sorry for their maintainers.
> Is it fun to hit <tab> over and over?
It’s probably a mixture of entertainment (I’ve seen countless apps where it looks like people just tap the screen as fast as possible without even really watching what’s going on) and imagining they’re learning something by watching answers appear (although IMO the lack of engagement leads to lack of retention; sort of like Socrates’ position on reading vs discussion).
Almost. But the crucial difference is — imagine an alternative universe, where humanity didn't have horses or any other animals or devices, even rickshas or wheels, to help moving around the world, and suddenly the first primitive cars were being invented just a couple of years ago.
Wouldn't you feel enthusiastic to use those new devices and test them everywhere?
Bringing a motorcycle to a 5K is obviously cheating. The motorcycle is a factor of 10 faster than its closest competitor. That is not where LLM/AI is currently at. The code generated by AI is not 10 times better than human code. It is just human code cleverly regurgitated.
Comparing AI/LLM to the motorcycle is giving it too much credit. AI is more like a 500CC motorcycle vs a 300CC motorcycle.
Sure they are pretty close, and the human might win once in a while, but the AI has just enough of an advantage that is guaranteed to burn the human out and win 95+% of the time.
I use copilot a lot and use chatgpt a lot to help with coding problems and doing things like writing short bash scripts for me, so I feel like I have a good grasp of what it's capable of doing, and the amount of work you'd have to do to break the problem into small enough chunks that it can understand and solve it is similar, if not more, than the amount of work you'd have to do to just solve it yourself. At least in its current state.
I think it defeats the purpose of the AoC to use AI, but I can imagine for some people it's a different game to see how they can use a screw (gaming term) to circumvent the system.
On the other hand, there may be some value to learning how to apply AI tools as a solver. If test cases are available for a given problem space, and AI can be applied to build code to meet those "requirements", then it is probably a skill we should be learning.
While I don't believe people are going to play fair here, it would be nice if AoC submissions had an [AI-assist] checkbox. Then there could be two leaderboards, and the AI folks could still have some good competition with each other.
Ultimately, the AoC leaderboards are unfair as they favor people in timezones that fit the release of each "day". So being on the leaderboard isn't really something that many people can do, even if they are brilliant problem solvers.
I understand the appeal of using the best and fastest means when there are real-world stakes — taking a motorcycle is better than running for many transport scenarios. But the whole point of this is to have fun! Is it fun to hit <tab> over and over?