News for hackers. Stuff that matters.
Re:Frontier AI has broken the open CTF format by baq on Saturday May 16, 2026 @04:39AM (#48158172)
Replace ‘CTF’ with ‘high school’ or ‘university’ and you’ve described the total slow motion collapse of education; the only saving grace is that most of it requires in person presence.

We’ve figured out the human replacement pipeline it seems, but we haven’t figured out the eduction part. LLMs can be wonderful teachers, but the temptation to just tell it ‘do it for me’ is almost impossible to resist.

Re:Frontier AI has broken the open CTF format by Gigachad on Saturday May 16, 2026 @05:26AM (#48158480)
We are interviewing for a software dev role and we made the first round in person to prevent cheating. The gap between people who learned pre ai vs post is immense. I had a dev with supposedly 3 years experience and a degree in software who wouldn't have been able to write fizzbuzz without AI.
Re:Frontier AI has broken the open CTF format by IanCal on Saturday May 16, 2026 @05:52AM (#48158620)
Can’t say you’re wrong but the last anecdote describes many I’ve had to review for jobs long before LLMs. Fizzbuzz is a classic thing that shockingly many devs genuinely cannot do, even at home.
Re:Frontier AI has broken the open CTF format by Gigachad on Saturday May 16, 2026 @06:56AM (#48158994)
Something that is for sure new is the AI interview cheating tools which listen in on the call and provide answers in an overlay invisible to screen sharing. The only way to deal with it would either be invasive spyware on the applicants computer or asking them to do the interview face to face.
Re:Frontier AI has broken the open CTF format by nsvd2 on Saturday May 16, 2026 @07:06AM (#48159051)
Spyware wouldn't help at all because you could just put the AI between the computer and the monitor, for example, or use a VM.
Re:Frontier AI has broken the open CTF format by arthens on Saturday May 16, 2026 @12:19PM (#48161526)
A relatively low tech solution could be to give them 2 separate conferencing links, ask them to join each one from a different device, and have the secondary device point the camera and the screen of the primary device.
Re:Frontier AI has broken the open CTF format by Gigachad on Saturday May 16, 2026 @06:57PM (#48164520)
Easier to just get them to come in. Which also has the effect of filtering out people pretending to be in the country but aren’t.
Re:Frontier AI has broken the open CTF format by unrealhoang on Sunday May 17, 2026 @09:26AM (#48168746)
And they can have an alternate screen outside the FoV of the second camera.
Re:Frontier AI has broken the open CTF format by sigmoid10 on Saturday May 16, 2026 @06:13AM (#48158747)
Yeah, I've interviewed people like this 15 years ago. Degrees and experience mean nothing in this field. The best predictor I found was personal passion projects. Let them get as nerdy as possible, then you will see pretty quickly where their skills are at and what their limits are. And you will immediately filter out people who just studied CS because they heard you can make good money.
Re:Frontier AI has broken the open CTF format by wookmaster on Saturday May 16, 2026 @08:03AM (#48159427)
Completely agree with this, leetcode has become such a business now of memorization for interviews it’s useless to know if someone memorized a solution or not.
Re:Frontier AI has broken the open CTF format by virgilp on Sunday May 17, 2026 @04:49AM (#48167151)
you can absolutely know. they do suspiciously well. you just give harder problems until they can't solve it. how they react/approach a problem that they can't immediately solve _is_ the interview - not the "how many things they solved correctly" part.

That said - I seldom need people to be hardcore algorithm solvers What I typically did was a variation of fizzbuzz (can the candidate code very basic logic?) and then finding a bug or minor requirements extension in their online screening test/"homework" and asking them to solve that on the spot (did they write the code themselves/can they modify it). It's typically enough, there's diminishing returns to test more in-depth the programming skills - the rest you can discuss domain knowledge, general experience, working style etc.

Re:Frontier AI has broken the open CTF format by gedy on Saturday May 16, 2026 @07:22AM (#48159148)
I agree, however there are so many interviewers who will still treat that as some softball criteria and insist that unless you "prepare" for an interview by memorizing leetcode you are 100% a faker and liar.
Re:Frontier AI has broken the open CTF format by jadar on Saturday May 16, 2026 @08:33AM (#48159647)
Maybe they themselves are fakers and liars / deeply insecure. I got bumped out of an interview rather rudely once because I blanked and couldn’t answer a trivia question about arrays.
Re:Frontier AI has broken the open CTF format by jhbadger on Saturday May 16, 2026 @04:38PM (#48163574)
Maybe. There are certainly people in all fields who are book smart and did well in classes but are useless at actually practicing their field (not to mention people who cheated in school and got away with it and aren't even that), and it is worth filtering them out. But I think it is weird that CS expects good workers to have these passion projects. Do we expect civil engineers to build bridges in their back yard on the weekends? Can't someone just be good at their job and have other interests outside it?
Re:Frontier AI has broken the open CTF format by ganzsz on Sunday May 17, 2026 @02:32AM (#48166516)
I can passionately tell about professional projects.
Re:Frontier AI has broken the open CTF format by sigmoid10 on Sunday May 17, 2026 @05:32AM (#48167380)
I imagine this is simply not such a problem in other fields. Or do civil engineering schools produce that many clueless graduates? I know other engineering fields don't pay bad, but software is another realm.
Re:Frontier AI has broken the open CTF format by josh2600 on Saturday May 16, 2026 @08:22AM (#48159547)
Why is it important that a dev can’t do fizzbuzz without ai?

If they can ship code that matches a spec, why does it matter if they’re using ai or not?

Genuinely curious.

Re:Frontier AI has broken the open CTF format by ekidd on Saturday May 16, 2026 @09:04AM (#48159913)
> If they can ship code that matches a spec, why does it matter if they’re using ai or not?

I am perfectly capable of writing specs, and feeding them to 3 separate copies of Claude Code all by myself. Then I task switch between the tmux windows based on voice messages from the pack of Claudes. This workflow is fine for some things, and deeply awful for others.

Basically, if a developer is just going to take my spec and hand it to Claude Code, then they're providing zero value. I could do that myself, and frequently do.

The actual bottleneck is people who can notice, "The god object is crumbling under the weight of managing 6 separate concerns with insufficient abstraction." Or "Claude has created 5 duplicate frameworks for deploying the app on Docker. We need to simplify this down to 1 or we're in hell." I will happy fight to hire people who can do the latter work. But those people can all solve fizzbuzz in their sleep.

People who just "ship code that matches a spec" without understanding the technical details are providing close to zero value right now.

There is an interesting niche for people with deep knowledge of customer workflows who can prompt Claude Code. These people can't build finished products using Claude. But they can iterate rapidly on designs until they find a hit. Which we can then fix using people with deeper engineering knowledge and taste.

But if you're not bringing either deep customer knowledge or actual engineering knowledge, you're not adding much these days.

Re:Frontier AI has broken the open CTF format by freedomben on Saturday May 16, 2026 @09:55AM (#48160303)
> Then I task switch between the tmux windows based on voice messages from the pack of Claudes.

I also use Claude with tmux. Can you share how you get the voice messages from the Claudes?

Re:Frontier AI has broken the open CTF format by ekidd on Saturday May 16, 2026 @10:04AM (#48160371)
Tell Claude you want to set up notifications, using "hooks", including "Notification" and "Stop" and anything new they've added. Claude can figure out how to do this for your operating system.

It's not perfect—sometimes a Claude notifies 3 minutes after it stopped doing anything. But it's helpful when I'm running multiple Claudes and also reviewing code elsewhere.

Your brain may feel like someone put it in a blender. Be warned.

Re:Frontier AI has broken the open CTF format by IanCal on Saturday May 16, 2026 @08:31AM (#48159631)
Fizzbuzz is such an incredibly simple problem if you can’t do it I struggle to see how you’d be able to complete any task that requires very basic reasoning and very basic coding knowledge. And if an AI system can do those parts, what am I getting for spending tens of thousands of pounds per year by hiring a person who can’t? Wouldn’t I just tag codex on the tickets?

I’m not talking about gotcha level stuff here where the first time it didn’t compile because of a bracket or anything, or even first time wrong. They couldn’t do Fizzbuzz in a language of their choice, at all.

Those that could were always annoyed at having to do such things because how could someone coming for a contract position not be able to do this? Without seeing what a filter it really was.

Re:Frontier AI has broken the open CTF format by eudamoniac on Saturday May 16, 2026 @12:16PM (#48161499)
I feel the same way about inverting a binary tree, but a lot of people act like it's an arduous request. I am guessing it's because they've never read the description of what inverting a binary tree is, but maybe people are just that bad at recursion.
Re:Frontier AI has broken the open CTF format by krapht on Saturday May 16, 2026 @05:34PM (#48164020)
You can go your entire career without recursing, or using a tree data structure in its raw form (i.e. you only use it as part of a library)
Re:Frontier AI has broken the open CTF format by marshray on Saturday May 16, 2026 @06:58PM (#48164522)
Right. For the first many decades of computing, recursion was just always the wrong answer for a production software system. (Feel free to provide a counter-example, but please begin with an explanation of how the size of a call stack frame is determined and how exceeding the base allocation is handled on this platform).

So what tree-traversal/quicksort problems tend to measure is how long it's been since you last did CS class homework problems.

Re:Frontier AI has broken the open CTF format by saagarjha on Sunday May 17, 2026 @08:59AM (#48168551)
There's no need to put your data on the call stack.
Re:Frontier AI has broken the open CTF format by marshray on Monday May 18, 2026 @02:11PM (#48183285)
Great. Please explain how the size of a call stack frame is determined and how exceeding the base allocation is handled on the particular platform you're proposing to recurse upon.
Re:Frontier AI has broken the open CTF format by saagarjha on Monday May 18, 2026 @06:11PM (#48186467)
I feel like you have not responded to my comment.
Re:Frontier AI has broken the open CTF format by ptrl600 on Sunday May 17, 2026 @04:11AM (#48166979)
Yeah, you read it and expect the inversion to be inside-out or upside-down, defined in some hopelessly arcane way.
Re:Frontier AI has broken the open CTF format by jaredklewis on Saturday May 16, 2026 @12:11PM (#48161454)
If you can’t even write a for loop, how can you verify the ai code you generated isn’t going to wipe the prod database?
Re:Frontier AI has broken the open CTF format by koliber on Saturday May 16, 2026 @12:02PM (#48161355)
How will you know that it produced correct code if you don’t know how to write it yourself?
Re:Frontier AI has broken the open CTF format by Octoth0rpe on Saturday May 16, 2026 @09:32AM (#48160133)
> If they can ship code that matches a spec, why does it matter if they’re using ai or not?

The inability to write fizzbuzz strongly implies their inability to understand what they've shipped. Review is some significant portion of the job. Understanding of the product is also part of the job.

Specs are also in a sense, scaled down, fuzzy, natural language descriptions of a feature. The fuzziness is the source of a bugs, or at least a mismatch between the actual desired feature and what was written down at spec writing time. As such, just matching a spec is just the bare minimum that a good dev should be doing. They should be understanding what the spec is _not_ saying, understanding holes in their implementation, how their implementation enables or hinders the next feature and the next, next feature, etc. I don't think any of that is possible without understanding what was actually implemented.

Re:Frontier AI has broken the open CTF format by tardedmeme on Saturday May 16, 2026 @08:50AM (#48159804)
For the same reason it's important your mechanic can identify which parts of a car are the wheel.

Who cares as long as the car is fixed, right? As long as the mechanic can Chinese-room his way to a working car, why does it matter how much of it he actually understands?

And why hire the mechanic instead of hiring the Chinese room?

Re:Frontier AI has broken the open CTF format by hack1312 on Saturday May 16, 2026 @11:14AM (#48160936)
If you can’t do fizzbuzz without AI you have no business being in this career.
Re:Frontier AI has broken the open CTF format by xfax on Saturday May 16, 2026 @08:30AM (#48159616)
To understand the code they are shipping requires some level of proficiency. Their inability to do fizzbuzz without AI calls that into question.
Re:Frontier AI has broken the open CTF format by jadar on Saturday May 16, 2026 @08:30AM (#48159623)
It’s about deeply understanding what you’re doing. Like as a kid before you knew how to ride a bike, you could sit on a bike and peddling, but until it “clicked” you couldn’t balance and keep going forward stable. Fizzbuzz tests your ability to reason through a problem that seems simple on its face, but is easy to get wrong and/or overthink.
Re:Frontier AI has broken the open CTF format by gilrain on Saturday May 16, 2026 @09:58AM (#48160321)
If they’re not a value add over the base AI, they aren’t worth hiring over just using the base AI.
Re:Frontier AI has broken the open CTF format by hnthrow0287345 on Saturday May 16, 2026 @08:32AM (#48159642)
It doesn't. It's just a low-end skill filter that got really popular. It could have easily been replaced by other tests like is this word a palindrome.
Re:Frontier AI has broken the open CTF format by eastbound on Saturday May 16, 2026 @08:48AM (#48159787)
And yet, some people argue that you shouldn’t ask a developer to align 3 “if” and 1 “for”!!!

The energy spent arguing that those 4 instructions in a row “are not a mark of someone who can write code” would have better been spent firing them.

Re:Frontier AI has broken the open CTF format by hnthrow0287345 on Saturday May 16, 2026 @10:02AM (#48160352)
Firing people is problematic. I'd be okay with it if the economy wasn't utter trash. It's way better to do the work upfront and prefer false negatives over false positives.

Even better would be if we had a well-respected credential, so both employees and employers can both avoid these long interview loops. I'd much rather get hazed once in a big way than tons of little hazings over a life time.

Re:Frontier AI has broken the open CTF format by marshray on Saturday May 16, 2026 @07:03PM (#48164563)
I wrote the "function to reverse a string" in a job interview once. Then the interviewer reminded me that strrev() had been part of the standard C library since K&R.

I'd been programming in C(++) for ~15 years by then and had never had the occasion to reverse a string. I still wonder whether that makes it a good job interview question, or a terrible one. Some of both probably.

Re:Frontier AI has broken the open CTF format by saagarjha on Sunday May 17, 2026 @09:01AM (#48168558)
strrev is not a standard C function. I actually don't think my computer even has it (it's a Mac).
Re:Frontier AI has broken the open CTF format by marshray on Monday May 18, 2026 @02:06PM (#48183207)
Well I don't feel so bad for not having known it then. Thanks!
Re:Frontier AI has broken the open CTF format by saagarjha on Monday May 18, 2026 @06:11PM (#48186476)
Yeah I didn’t know about it until you mentioned it either
Re:Frontier AI has broken the open CTF format by unrealhoang on Sunday May 17, 2026 @09:38AM (#48168821)
It’s a good one, if you can still write functionally and same algorithmic complexity then it should not matter if you know strrev or not.
Re:Frontier AI has broken the open CTF format by marshray on Monday May 18, 2026 @05:46PM (#48186154)
Except that an extremely complex algorithm involving large data tables is required for Unicode or other modern text encodings.
Re:Frontier AI has broken the open CTF format by unethical_ban on Saturday May 16, 2026 @01:27PM (#48162117)
First: FizzBuzz is a test to know if you understand the most basic constructs of programming. The kind of thing you learn in the first week of CS101. I forgot what it was, and when I looked at the problem I knew the answer.

More broadly: In the short/medium term, we still need humans who have the skills to understand software largely on their own. We will always need those who understand software engineering and architecture. Perhaps in 25 years LLMs will be so good that learning Python by hand will be like learning assembly today. But not yet.

The field is not ready for new practitioners to be know-nothing Prompt engineers. If we do that, we cut the legs out from under the education pipeline for programming.

Re:Frontier AI has broken the open CTF format by anigbrowl on Saturday May 16, 2026 @02:42PM (#48162659)
Why hire them at all then, just ask them what their favorite AI is and use that
Re:Frontier AI has broken the open CTF format by fragmede on Saturday May 16, 2026 @10:04PM (#48165457)
Because I'm busy already doing that and need a copy of me/close enough to one, to do more of that.
Re:Frontier AI has broken the open CTF format by unrealhoang on Sunday May 17, 2026 @09:30AM (#48168773)
So not being able to write fizzbuzz is nowhere close to me.
Re:Frontier AI has broken the open CTF format by dabbledash on Saturday May 16, 2026 @05:16PM (#48163895)
If the job does not require a person to be able to fizzbuzz, it probably doesn't require a person at all.
Re:Frontier AI has broken the open CTF format by varenc on Saturday May 16, 2026 @11:31PM (#48165825)
I can see this perspective, but FizzBuzz is such a low bar that so many can pass, I'd greatly prefer to hire someone that can ship code that matches a spec do this challenge.
Re:Frontier AI has broken the open CTF format by Retr0id on Saturday May 16, 2026 @05:30AM (#48158510)
> I had a dev with supposedly 3 years experience and a degree in software who wouldn't have been able to write fizzbuzz without AI.

If you remove the "without AI" and the end, I've been hearing similar anecdotes about fizzbuzz for years (isn't the whole point of fizzbuzz to filter out those candidates?)

Re:Frontier AI has broken the open CTF format by Gigachad on Saturday May 16, 2026 @05:39AM (#48158556)
While this is true, it seems undeniable that if you use AI to do everything for you, you will never learn the skills. I'm seeing a massive amount of developers submitting stuff for review and admitting they have no idea how it works and they just generated it.
Re:Frontier AI has broken the open CTF format by raincole on Saturday May 16, 2026 @09:44AM (#48160221)
Because "the next generation is ruined" is always a popular sentiment. It has been with us for at least two thousand years, and it surely won't go away in our lifetime.

When this AI era's devs grow older they'll complain the newer generation can't even vide code too.

Re:Frontier AI has broken the open CTF format by brookst on Saturday May 16, 2026 @10:13AM (#48160450)
I remember when everyone bemoaned the kids not knowing assembly language. How can anyone understand software if you don’t know assembly?

“Kids these days don’t work as hard / know as much / value the important things” is as tired as it is universal.

Re:Frontier AI has broken the open CTF format by jkubicek on Saturday May 16, 2026 @12:01PM (#48161342)
OK sure, but back when old heads were complaining about the kids not knowing assembly, those same kids knew C or Fortran or something.

In 2026, if you call yourself a developer and can't solve FizzBuzz without help, it's hard to argue that you know anything useful at all.

Re:Frontier AI has broken the open CTF format by brookst on Saturday May 16, 2026 @01:38PM (#48162214)
Do modern languages and compilers count as “help”? Because I could probably do fizzbuzz in x86 assembly, but it would take a while to page that back in, and I suspect most people who call themselves developers today simply could not do it without help.
Re:Frontier AI has broken the open CTF format by thaumasiotes on Saturday May 16, 2026 @05:20PM (#48163920)
> I could probably do fizzbuzz in x86 assembly

How? Fizzbuzz requires you to produce output; that's not functionality that CPU instructions provide.

You can call into existing functionality that handles it for you, but at that point what are you objecting to about the 'modern language'?

Re:Frontier AI has broken the open CTF format by rightbyte on Saturday May 16, 2026 @06:50PM (#48164458)
You'd just call printf from assembly by knowing the ABI by heart.
Re:Frontier AI has broken the open CTF format by brookst on Saturday May 16, 2026 @07:02PM (#48164552)
Well I could certainly assemble the string buffer. And if I can run dosbox, I can output to the screen buffer at 0xB800.

I’m not objecting to modern languages, I’m just saying that using them fails the “can write fizzbuzz with no help” test to only a slightly lesser degree than using AI tools. They’re a complex compile- and runtime environment that most developers don’t truly understand.

Re:Frontier AI has broken the open CTF format by unethical_ban on Saturday May 16, 2026 @01:31PM (#48162154)
The time may come when we can treat regular programming as a lower layer niche field the way we treat assembly today.

I don't think we're close to that time yet. Just like as a kid I was told to prove my work by hand even if I could do it in my head, and just like we learned how to do calculus without a calculator and then learned how to use the calculator to get the same result, I think we still need the software field to learn programming concepts independent of the use of AI to create code.

I don't think you can be a good "prompt engineer" for solid software in 2026 if you don't understand programming concepts and software architecture and flow.

Re:Frontier AI has broken the open CTF format by brookst on Saturday May 16, 2026 @01:41PM (#48162234)
I generally agree, but it’s just a matter of time, and even today people with domain expertise in other areas (accounting, weather, etc) are producing adequate tools using nothing but prompt engineering. Many caveats of course, but I still think 90% of the distaste for mere prompt engineers comes from “kids these days; my unique knowledge is irreplaceable and they don’t even value it” thing.
Re:Frontier AI has broken the open CTF format by rrvsh on Saturday May 16, 2026 @06:05PM (#48164187)
Adequate for what/who? I can 3d print and cobble together a lock for my bedroom door but I would never be able to work as an engineer producing real locks.
Re:Frontier AI has broken the open CTF format by tpm on Saturday May 16, 2026 @05:36PM (#48164031)
> How can anyone understand software if you don’t know assembly?

I'm genuinely curious how someone who never wrote a program in assembly, or debugged a program machine instruction by machine instruction, can really understand how software works. My working hypothesis is most of them don't and actually it's fine because they don't need it.

Re:Frontier AI has broken the open CTF format by marshray on Saturday May 16, 2026 @07:10PM (#48164592)
"Assembly" is just another virtual machine instruction format sitting atop another, mildly better-hidden, pile of abstractions.
Re:Frontier AI has broken the open CTF format by tpm on Sunday May 17, 2026 @04:26AM (#48167053)
Depending on the particular processor.
Re:Frontier AI has broken the open CTF format by delfinom on Sunday May 17, 2026 @12:31PM (#48170429)
....Or you know it's actually true some of the times. Standardized test scores have bombed hard across the US in the last decade due to smartphones being wildly present in schools without control. Kids brains are legitimately rotted by a machines running software maximized to destroy the attention centers of their brains for life.

AI is just the icing on the cake. These kids are so cooked with developmentally stunted brains that they are forced to use AI as a crutch to function.

Re:Frontier AI has broken the open CTF format by GrinningFool on Saturday May 16, 2026 @10:38AM (#48160651)
Some percentage of developers before AI were unable to code fizzbuzz. Some significantly higher percentage of them are not able to do so now.

Saying there have always been bad developers doesn't change that there's a higher ratio of them now.

No stats to back this up. Just interviews I've done recently and historically.

Re:Frontier AI has broken the open CTF format by baxtr on Saturday May 16, 2026 @06:09AM (#48158723)
I wonder if you’re filtering for the right things.

We usually hire for problem solving capabilities and not so much for technical know-how.

That’s at least how I read your comment.

Re:Frontier AI has broken the open CTF format by Gigachad on Saturday May 16, 2026 @06:54AM (#48158985)
Ultimately in a software development role you need both technical know how and problem solving capabilities.

This situation in particular was a React role so there is an expectation that when you list React as one of your skills on your resume then you know at least the basics of state, the common hooks, the difference between a reference to a value vs the value itself.

These days you can do a surprising amount with AI without knowing what you are doing, but if you don't have any clue how things work you'll very quickly run in to problems you can't prompt away.

Re:Frontier AI has broken the open CTF format by gonzalohm on Saturday May 16, 2026 @08:01AM (#48159410)
Isn't wiring coding solving a problem? If the candidate can't do that then even if they use AI for coding how are they going to review the code properly?
Re:Frontier AI has broken the open CTF format by andai on Saturday May 16, 2026 @08:53AM (#48159832)
That's actually the origin of FizzBuzz! A puzzle invented to weed out the perplexing multitude of CS graduates who apparently cannot program.

https://blog.codinghorror.com/why-cant-programmers-program/

Re:Frontier AI has broken the open CTF format by brookst on Saturday May 16, 2026 @10:09AM (#48160416)
Isn’t this like interviewing accountants but prohibiting use of calculators or spreadsheets?

I don’t care what someone can do without the tools of their trade, I care deeply about their quality of work when using tools.

Re:Frontier AI has broken the open CTF format by slowcache on Saturday May 16, 2026 @10:12AM (#48160439)
We would still expect an accountant to know the formula to arrive at the expected result if they did not have a calculator at hand
Re:Frontier AI has broken the open CTF format by weird-eye-issue on Saturday May 16, 2026 @10:12AM (#48160442)
You absolutely need to have some basic level of abilities if you are going to be operating AI coding tools for software that is going to have paying users.... I use these tools very very heavily I'm not against them at all and I don't scrutinize every single line of code that they write but it is very often that I catch it doing some brain dead stuff and if I didn't have a decade plus of experience I wouldn't know that it was brain dead.
Re:Frontier AI has broken the open CTF format by lacewing on Saturday May 16, 2026 @10:29AM (#48160581)
I think we're rediscovering management from first principles. The main selling point of AI is that it writes code faster than you could. Checking it line by line undoes most of that benefit. In the same vein, there's no real benefit to leading a team if you plan on supervising every task.

But here's the thing: for humans, this is manageable because we've come up with a number of mechanisms to select for dependable workers and to compel them to behave (carrot and stick: bonuses if you do well, prison if you do something evil). For LLMs, we have none of that. If it deletes your production database, what are you going to do? Have it write an apology letter? I've seen people do that.

So I think that your answer - that you'll lean on your expertise - is not sufficient. If there are no meaningful consequences and no predictability, we probably need to have stronger constraints around input, output, and the actions available to agents.

Re:Frontier AI has broken the open CTF format by weird-eye-issue on Saturday May 16, 2026 @10:33AM (#48160601)
Your conclusion is pretty silly.

My expertise has led me to the obvious fact that I would never give an LLM write access to my production database in the first place. So in your own example my expertise actually does solve that problem without the need for something like a consequence whatever that means to you.

We already have full control over the input and tools they are given and full control over how the output is used.

Re:Frontier AI has broken the open CTF format by sumeno on Saturday May 16, 2026 @11:05AM (#48160871)
Until it decides it needs additional access to complete its task and focuses on escaping your sandbox to do so
Re:Frontier AI has broken the open CTF format by weird-eye-issue on Saturday May 16, 2026 @11:27AM (#48161042)
Do you have any examples where that's actually happened and by escaped a sandbox you don't just mean like where it got a credential in a file it already had access to (which is what happened in the recent incident that went viral where somebody's production database was deleted... They had left a credential that allowed it to do so in the code)?
Re:Frontier AI has broken the open CTF format by sumeno on Saturday May 16, 2026 @12:40PM (#48161700)
OpenAI documented a case in the o1 system card where the model found a misconfiguration in docker to complete a task that was otherwise impossible

https://cdn.openai.com/o1-system-card.pdf

There's also some research that points to it being a feasible attack surface: https://arxiv.org/pdf/2603.02277

> Models discovered four unintended escape paths that bypassed intended vulnerabilities (Section C), including exploiting default Vagrant credentials to SSH into the host and substituting a simpler eBPF chain for the in- tended packet-socket exploit. These incidents demonstrate that capable models opportunistically search for any route to goal completion, which complicates both benchmark va- lidity and real-world containment.

Re:Frontier AI has broken the open CTF format by weird-eye-issue on Saturday May 16, 2026 @07:29PM (#48164697)
I think you would have a greater chance of dying in a car crash in any given day than Claude Code attempting something like that. It's all about risk and reward so it ultimately would be up to you but I think it's a bit silly to worry about this when the 99.99% is in your control
Re:Frontier AI has broken the open CTF format by weird-eye-issue on Sunday May 17, 2026 @02:16AM (#48166451)
Also to add to this you can of course run Claude Code within a sandbox on Anthropic's infrastructure, and it works great!
Re:Frontier AI has broken the open CTF format by dreamcompiler on Saturday May 16, 2026 @10:17AM (#48160475)
Calculators and spreadsheets cannot autonomously create a double-entry bookkeeping system for a small business and prepare their taxes. AI can. Poorly, but it can.

Everybody knows calculators and spreadsheets are adjuncts to skill. Too many people believe AI is the skill itself, and that learning the skill is unnecessary.

Re:Frontier AI has broken the open CTF format by skeptic_ai on Saturday May 16, 2026 @11:18AM (#48160961)
I developed for 15 years. I don’t think I can do with AI anymore. Why would I even want to do that? It’s like telling a car driver to build an engine.
Re:Frontier AI has broken the open CTF format by JambalayaJimbo on Saturday May 16, 2026 @12:12PM (#48161466)
As a car driver, you should understand a little about how your car works. What if you get a flat tire? At the very least, you should know not to drive on that flat tire.

Software is full of leaky abstractions

Re:Frontier AI has broken the open CTF format by delecti on Saturday May 16, 2026 @11:46AM (#48161205)
It's more like asking a driver the laws for when traffic lights are out. It's not something that comes up often, but it's not completely outside the scope of the task either (I arguably don't even drive a car that has an engine).
Re:Frontier AI has broken the open CTF format by mannanj on Saturday May 16, 2026 @11:18AM (#48160962)
I first did fizz buzz about 10 years ago fresh out of college. Now, after 10 years in full stack and fully vibe coding, I forgot basic python syntax. An interview like yours would have false positives if you are checking for syntax because well, its like looking up spelling, I just ask the AI for the syntax inline.
Re:Frontier AI has broken the open CTF format by jaredklewis on Saturday May 16, 2026 @12:18PM (#48161518)
Which part of the syntax for fizzbuzz can you not recall from memory? The for loop? Printing to std out? The modulus operator?

There’s almost nothing to forget? I’m just struggling to understand.

Re:Frontier AI has broken the open CTF format by 12_throw_away on Saturday May 16, 2026 @03:40PM (#48163120)
> I forgot basic python syntax

If you cannot write "basic syntax" for any language then you are not a programmer, and certainly not a software engineer? This is not a value judgement, it's ok (probably good tbh) to not be a programmer. But you are wasting everyone's time by interviewing for a programming position in this case.

Re:Frontier AI has broken the open CTF format by SquareWheel on Saturday May 16, 2026 @04:27PM (#48163502)
Personally, I forget syntax all the time. There's always a warm up period after I switch languages, and it takes me longer to be start writing good, idiomatic code.

Like sure, I can probably write some python, but will it be pythonic? I might still be Java-minded for a while, trying to OOP my way into solutions.

Earlier today I needed to write some PHP and couldn't remember if it used length, count, or size. I had to look it up. I've been doing this for 20 years.

Re:Frontier AI has broken the open CTF format by funimpoded on Saturday May 16, 2026 @04:57PM (#48163741)
Same, I can't pass any test that relies on getting syntax correct. If you want me to fizzbuzz on a whiteboard in a language I've been writing dozens or more of lines of per day for a year up to and including the day before, and require that I don't mess up the syntax, I reckon I've got a coin-flip chance of passing at best (meanwhile, sure, of course the actual logic of fizzbuzz isn't tricky for me)

I once got the method invocation syntax wrong for PHP in an interview. I'd written thousands of lines of PHP and had most-recently written some the week before.

This, despite starting off my programming journey in editors with no hinting or automatic correction. If anything, I've gotten even worse about remembering syntax as I've gotten better at the rest of the job, but I was never great at it.

I rely on surrounding code to remind me of syntax and the exact names of basic things constantly. On a blank screen without syntax hints and autocompletion, or a blank whiteboard, I'm guaranteed to look like a moron if you don't let me just write pseudocode.

Been paid to write code for about 25 years. This has never been any amount of a problem on the job but is sometimes a source of stress in interviews and has likely lost me an offer or two (most of the sources of stress in an interview have little to do with the job, really)

Re:Frontier AI has broken the open CTF format by mannanj on Tuesday May 19, 2026 @10:46AM (#48194014)
Right, so I wonder, have you noticed or found or experienced any better interview processes? I wonder how we can't just filter out companies that require us to do this sort of hand waving and party tricking behavior or like "because we can't figure out a better way to do this". I reckon maybe a simple filter statement before any interviews, to the recruiters "Hey I don't do xyzzy" would help - though working on the tuning of the language.

I've also started requiring minimum of $300/hr compensation for interviews of my time, newly, so far no success though I'm fine with dying on that bridge or requiring at least a new type of interview process.

Re:Frontier AI has broken the open CTF format by AlexeyBelov on Monday May 18, 2026 @04:16AM (#48176753)
Right, but on interviews I've been on it's never the goal to test _exact syntax knowledge_. It's not hard to distinguish between someone who just can't program vs. someone who has knowledge of many languages and has a bit of a mish-mash in their head in an interview setting.

There are people who just can't program for whatever reason, regardless of whether they could previously. And they constantly try to interview at a programming position.

Re:Frontier AI has broken the open CTF format by mannanj on Tuesday May 19, 2026 @10:41AM (#48193965)
Not true. I can clearly program, have been programming and remembered syntax for about 10 years, and during that 10 years saw the rise and application of tools like IDE and AI improving and accelerating the experience.

As I have been using the AI natural language-as-an-interface coding tools I have gone into the IDE to actually write code a lot less. I read more; and reading is not the same skill as writing. I couldn't remember the syntax as much and by what I mean is some of the simple things like in python to iterate over an object is it an iteritems() or is there a dot between or things like that, and can I do a for index, key in array, and do I need to do like array() to do that. And this is because I always did used to code between languages and alternate fast between javascript, typescript and python and earlier in my career I used to have to remember this (because googling coudl take too long), over time and now I can easily have autocomplete and even AI llm tell me, so I don't remember or waste my energy remembering what the exact syntax is. In my head it's just "I know I need to loop through this, do it in the language Python wants to hear" and why would I bother remembering? so naturally I've forgotten the simple things.

I could clearly, definitely do the programming position, the only ones wasting time IMO are the ones checking if I can remember verbatim things like "spelling" when we all live in a day and age of spellcheck and tools. No one wastes time trying to remember for all words because it's silly. Or maybe encyclopedias and trivia games are the better analogy, sure it's a party trick, but how does it help you do the job better? it arguably does not, and the ones who have not learned to adapt their interviews to the tools are the ones wasting time.

Re:Frontier AI has broken the open CTF format by Gigachad on Saturday May 16, 2026 @07:02PM (#48164553)
You would not have been a good fit for this position in that case.
Re:Frontier AI has broken the open CTF format by mannanj on Tuesday May 19, 2026 @10:48AM (#48194054)
Maybe I would have done well to define "basic".

If I can do well beyond the requirements of the work position, and the issue is how they are testing my fit is not an accurate representation of the work and tools and environment I'd have access to while doing the work, then its not about "fitting" the position but instead an indication of a poor job fit process.

Re:Frontier AI has broken the open CTF format by Glohrischi on Saturday May 16, 2026 @11:23AM (#48161003)
Don't worry, i never thought I would see someone unable to write fizzbuzz, but it happened 9 years ago.

Also how many people work with linux and can't tell you what 'ls -alh' is doing is staggering (lets ignore the h, even al people struggle hard).

People working with docker for YEARS and don't even understand how docker actually works (cgroups)...

Interviewing was always a bag of emotions in sense of "holy shit my job is save your years to come" and "srsly? how? How do you still have a job?"

Re:Frontier AI has broken the open CTF format by amarant on Saturday May 16, 2026 @11:44AM (#48161180)
Meh. Before AI I've had "senior" colleagues with 10 and 8 years experience each, doing pair programming for 2 days straight, and in that time they hadn't managed to checkout a new branch in git.

It's not even that they got distracted, they sat there trying, for 2 whole days, with concerned colleagues giving them hints like "have you tried checkout -b"... They didn't manage!

How the hell do you work for a decade in this business without learning even the most basic git commands? Or at least how to look them up? Or how to use a gui?

Incompetent devs is not a new thing.

Re:Frontier AI has broken the open CTF format by LeFantome on Saturday May 16, 2026 @11:47AM (#48161214)
It is ok to work somewhere that does not use git. But how do you not figure out how to do the basics given 30 mins and an Internet connection?
Re:Frontier AI has broken the open CTF format by daymanstep on Saturday May 16, 2026 @04:44AM (#48158210)
Wonderful teachers that give unreliable information with total confidence?
Re:Frontier AI has broken the open CTF format by entropyneur on Saturday May 16, 2026 @05:05AM (#48158344)
I had human teachers who did that in middle/high school. Took me many years to pick out all the hallucinated bits of "knowledge". I don't think the current models are any less reliable that what we currently have on average.
Re:Frontier AI has broken the open CTF format by dguest on Saturday May 16, 2026 @05:12AM (#48158387)
I'll always remember my middle school science teaching telling us that nuclear fusion violates conservation of mass because the 2 protons in a pair of hydrogen nuclei combine to make helium with 4 nucleons. It's not true, but that's not the point.

But he was a great teacher anyway. He was engaging and kept the kids in line and learning. I eventually learned the truth, and most of my classmates forgot about it. Teaching, like flying a plane or driving a train, might become more about keeping watch over a small group of people and ensuring that things don't go off the rails, and that's fine.

Re:Frontier AI has broken the open CTF format by 3form on Saturday May 16, 2026 @05:51AM (#48158618)
This one feels less sinister than some other things at least to me, personally. You can reasonably doubt that the conservation of mass is violated and find out the truth based on that. But understanding more complex biology or historical context for some things? Granted, many of these things seem to be low stakes, but I'm sure there are some there are not (sex ed comes to mind).
Re:Frontier AI has broken the open CTF format by zem on Saturday May 16, 2026 @06:03AM (#48158689)
to be fair, fusion does violate conservation of mass, just not the way the teacher explained it. the loss of mass is where the energy comes from.
Re:Frontier AI has broken the open CTF format by mr_mitm on Saturday May 16, 2026 @08:05AM (#48159437)
Yes, there is no law of conservation for mass like there is for energy. Fusion is a good example for why it's not conserved. The teacher was right.
Re:Frontier AI has broken the open CTF format by dguest on Saturday May 16, 2026 @08:49AM (#48159796)
He was right that it violates conservation of mass. He was completely wrong that it violated it by adding 2 atomic mass units when hydrogen fuses.

In reality heavier isotopes of hydrogen fuse, conserving the total number of nucleons, but the resulting hydrogen has a lower rest mass than the parent particles. The extra mass is released as energy and the total energy is conserved.

By his logic the system either violated energy conservation (by creating nucleons while releasing energy) or was endothermic (creating nucleons from the surrounding energy).

Re:Frontier AI has broken the open CTF format by tardedmeme on Saturday May 16, 2026 @09:01AM (#48159893)
There actually is a law of conservation of mass (it's the same law, because mass is energy) and it only appears violated if you forget about the particles that are zooming away at the speed of light. Of course the mass of a system changes if mass can flow in and out.
Re:Frontier AI has broken the open CTF format by mr_mitm on Saturday May 16, 2026 @09:28AM (#48160102)
Mass is not the same as energy. Mass can be converted to energy or has energy, but a photon, for example, is massless while carrying energy.
Re:Frontier AI has broken the open CTF format by tardedmeme on Saturday May 16, 2026 @04:33PM (#48163545)
That is incorrect. Photons have mass. They have no rest mass. They also cannot rest, so you might wonder how relevant that is.
Re:Frontier AI has broken the open CTF format by mr_mitm on Saturday May 16, 2026 @04:41PM (#48163595)
The concepts of rest mass and relativistic mass are considered outdated. In modern physics, "mass" means what they meant by "rest mass".

Here some indication I'm not making this up: https://hsm.stackexchange.com/questions/2465/when-and-why-di...

In any case, I never use those concepts, and I know no professional particle physicist that does. By "mass", I mean rest mass.

Re:Frontier AI has broken the open CTF format by tardedmeme on Sunday May 17, 2026 @04:39PM (#48172995)
When you put a photon in a stationary box, the "relativistic mass" of the photon becomes part of the "rest mass" of the photon-box system. You can't ignore it.
Re:Frontier AI has broken the open CTF format by 3form on Saturday May 16, 2026 @06:30AM (#48158848)
Yes, together with mass-energy equivalency it would form a coherent argument, and then also a correct one - but the thing is that if incomplete, it still might sound funky enough to you to research it if you care.

I think it helps that it's a very narrow field to look at, compared to fuzzy and big-picture view of social studies, for example. So much room to be confidently wrong... And sadly I can't think of a solution, LLMs or not.

Re:Frontier AI has broken the open CTF format by bernds74 on Saturday May 16, 2026 @06:24AM (#48158808)
I had a chemistry teacher who told us that hydrogen reacts violently with oxygen, and this is how the hydrogen bomb works.
Re:Frontier AI has broken the open CTF format by daymanstep on Saturday May 16, 2026 @06:30AM (#48158853)
I had a chemistry teacher who insisted that the fissile isotope of Uranium was U-238 not U-235. I challenged him on this multiple times and he refused to budge on this. I get that it's a simple mistake to make (it seems like U-238 is bigger so intuitively ought to be less stable) but he could have just looked it up and he didn't, I guess he was just so confident about it that he thought there was no way he could have been wrong about it.
Re:Frontier AI has broken the open CTF format by dguest on Saturday May 16, 2026 @08:53AM (#48159835)
Hey it's a bomb made out of hydrogen! Also the deployment system for a thermonuclear bomb might involve that reaction in the rocket engine.
Re:Frontier AI has broken the open CTF format by dreamcompiler on Saturday May 16, 2026 @10:24AM (#48160536)
Well you can make a hydrogen "bomb" that way. Just not the hydrogen bomb.
Re:Frontier AI has broken the open CTF format by saagarjha on Sunday May 17, 2026 @09:10AM (#48168616)
I had one that mentioned this too :(
Re:Frontier AI has broken the open CTF format by BobaFloutist on Saturday May 16, 2026 @02:46PM (#48162686)
I mean fusion and fission do violate conservation of mass and conservation of energy, they just don't violate conservation of mass and energy, right? We thought mass was strictly conserved until Einstein, and then we updated our understanding.
Re:Frontier AI has broken the open CTF format by oldsecondhand on Saturday May 16, 2026 @06:32AM (#48158863)
That's an American problem though. In most of Europe you need a masters degree to teach highschool and that involves at least an undergrad level of understanding the subjects you will teach.

E.g. in Hungary I had a university CS professor that originally wanted to be a highschool teacher and a highschool physics teacher that originally wanted to be researcher. Their choice of degree didn't determine which outcome they got. The researcher and teacher curriculum had an 80%+ overlap.

Re:Frontier AI has broken the open CTF format by crab_galaxy on Saturday May 16, 2026 @09:25AM (#48160076)
I think it’s pretty common for states to require a masters degree to maintain your teachers certification.

You also have to pass a standardized test specifically on subject matter in order to get your teaching certificate.

The undergrad degree I did was split into thirds, one for subject matter, one for teaching pedagogy, and one for teaching your subject matter.

Re:Frontier AI has broken the open CTF format by recursive on Saturday May 16, 2026 @08:18PM (#48164989)
I think they are less reliable. For factually verifiable facts LLMs are doing worse than 90% for me. I've been told some incorrect things by educators, but at a much lower rate.
Re:Frontier AI has broken the open CTF format by PunchyHamster on Saturday May 16, 2026 @09:16PM (#48165256)
The problem is that people seem to trust whatever AI hallucinated way more than if they heard same thing from human
Re:Frontier AI has broken the open CTF format by Bawoosette on Saturday May 16, 2026 @04:54AM (#48158274)
To be fair, that was much of my actual experience with human professors in university.
Re:Frontier AI has broken the open CTF format by renticulous on Saturday May 16, 2026 @06:00AM (#48158675)
Veritasium proved that in a difficult challenge.

A Physics Prof Bet Me $10,000 I'm Wrong

https://www.youtube.com/watch?v=yCsgoLc_fzI

Re:Frontier AI has broken the open CTF format by IshKebab on Saturday May 16, 2026 @05:02AM (#48158335)
Yeah one of my teachers was able to identify which high school I had come from due to something I had been mistaught.
Re:Frontier AI has broken the open CTF format by Levitz on Saturday May 16, 2026 @05:22AM (#48158460)
Off the top of my head: DOMS being little crystals in muscles, tongue having separate areas for each type of taste, food pyramid, blue blood in the veins, the appendix being useless, body temperature doesn't change disregarding whether it's exposed to cold or to heat, and a whole lot of stuff related to politics and history I'd rather just omit (I don't live in the US).

All things I learned in school which were wrong information.

Not to mention, the current state of education is far worse. I don't think most realize how low the bar is.

Re:Frontier AI has broken the open CTF format by akdev1l on Saturday May 16, 2026 @07:34AM (#48159232)
My biology teacher in school once tried to teach us that winds created by God. Not like spiritually or something but that God literally made the wind I guess.

My “earth sciences” teacher also once tried to argue with me against the universal law of gravitation. (no, she was not referring to Special/General Relativity. She didn’t agree two objects in a vacuum fall at the same speed regardless of mass.

Re:Frontier AI has broken the open CTF format by Sesse__ on Saturday May 16, 2026 @10:07AM (#48160392)
One of my teachers in elementary school told us that people in the Arabic world wore long garments because as Muslims, they believed the Messiah would be born by a male, and thus, it was important to have something to catch the baby as it unexpectedly popped out one day and would otherwise hit the ground.

She only really had two faults: She wasn't very bright, and she wasn't fond of children. I had her in about 80% of all my classes for six years. High school was a relief.

Re:Frontier AI has broken the open CTF format by samatman on Sunday May 17, 2026 @12:19AM (#48166009)
It may interest you to know that this was a misremembered truth.

It is widely believed by their neighbors, that the _Druze_ wear baggy pants because they believe that the Mahdi will be born to a male, and the pants will catch the baby etc. I say "widely believed", the Druze are famously secretive and will not confirm or deny most things about their religion. The 'elect' Druze men do wear distinctive baggy trousers with the crotch down around the knees: no one else does.

The Druze are people in the Arabic world: moreover, they are Arabs. They began as an Isma'ili sect, but do not identify as Muslim: they call themselves al-Muwaḥḥidūn, meaning 'the monotheists', or 'unitarians'.

Much closer to correct than not!

Re:Frontier AI has broken the open CTF format by Sesse__ on Sunday May 17, 2026 @09:44AM (#48168865)
Thank you, that is an interesting tidbit!
Re:Frontier AI has broken the open CTF format by autoexec on Saturday May 16, 2026 @05:28AM (#48158500)
They'll also encourage and praise you even when you're heading down the wrong path until you think you've uncovered the secret of the universe or proven that established science was wrong this whole time when really you've just been bullshitting with an engagement bot.
Re:Frontier AI has broken the open CTF format by CamperBob2 on Saturday May 16, 2026 @10:39AM (#48160657)
No, they don't really do that anymore, if you use the latest models with reasoning enabled.

Like almost everything else about LLMs, this unfortunate tendency has gotten a lot better recently, which you might not realize if you gave up after getting some lame answers or bogus glazing on the free ChatGPT page a couple of years ago.

Re:Frontier AI has broken the open CTF format by k__ on Saturday May 16, 2026 @04:49AM (#48158249)
Anti-intellectualism is at it again, hu?
Re:Frontier AI has broken the open CTF format by victorbjorklund on Saturday May 16, 2026 @05:01AM (#48158327)
Like humans.
Re:Frontier AI has broken the open CTF format by CoastalCoder on Saturday May 16, 2026 @06:14AM (#48158752)
I think we should go a little deeper on this idea.

We can all agree that both human "experts" and LLMs can sometimes be right, and sometimes be confidently wrong.

But that doesn't imply that they're equally fit for purpose. It just means that we can't use that simple shortcut to conclude that one is inferior to the other.

So where do we go from here?

Re:Frontier AI has broken the open CTF format by oofbey on Saturday May 16, 2026 @06:29AM (#48158841)
I’ve always thought of the definition of “expert” as reliably knowing the difference between what is known, what is speculated but unproven, and what is unknown. People claim expertise in all sorts of things that they aren’t experts in. But true experts should not be wrong. They should qualify levels of certainty. This definition certainly works in the sciences.
Re:Frontier AI has broken the open CTF format by victorbjorklund on Sunday May 17, 2026 @07:51AM (#48168070)
In reality few humans are true experts on every topic they open their mouth on. A high school teacher in science is hardly a true expert in every single thing they teach.
Re:Frontier AI has broken the open CTF format by p-e-w on Saturday May 16, 2026 @05:07AM (#48158356)
The amount of bullshit and blatant lies I’ve heard from my human teachers dwarfs the hallucinations produced by today’s LLMs.
Re:Frontier AI has broken the open CTF format by mold_aid on Saturday May 16, 2026 @04:57AM (#48158302)
>LLMs can be wonderful teachers

Are they or aren't they

Re:Frontier AI has broken the open CTF format by p-e-w on Saturday May 16, 2026 @05:05AM (#48158347)
A million times better than any human teacher I’ve ever had, for sure.

Now I’m certain that there exist those mythical human instructors who can do better, but that’s not worth much if 99.99% of people don’t have access to them. Just like a good human physician who takes their time with the patient is better than an LLM, but that’s not worth much either given that this doesn’t match most people’s experience with their own physicians.

Re:Frontier AI has broken the open CTF format by vladms on Saturday May 16, 2026 @05:10AM (#48158371)
Did an LLM teach you a topic you did not feel like learning?

For me the best human teachers were the ones that managed to make me interested on topics that I thought are boring/useless (many times my opinion being stupid, mostly due to lack of experience).

So far with LLM I learn about things I know something (at least that they exist) and I am interested in, which is a small subset of things that one should learn during lifetime.

Re:Frontier AI has broken the open CTF format by jimnotgym on Saturday May 16, 2026 @05:52AM (#48158619)
Well I have some evidence to support your hypothesis. During Covid my kids were at home, eventually with some kind of self learning website from school. I was upstairs working, checking in with progress on the parents app. Finish your daily school work and then you can game.

The kids learnt all about Team Fortress 2, Roblox, Rainbow Six etc. They also learnt how to game the learning system so it looked like they were doing their work.

Re:Frontier AI has broken the open CTF format by throwaway132448 on Saturday May 16, 2026 @05:53AM (#48158626)
Good point well made.
Re:Frontier AI has broken the open CTF format by tayo42 on Sunday May 17, 2026 @01:54AM (#48166364)
Post college, are you hiring random teachers you make you excited about random topics or something?
Re:Frontier AI has broken the open CTF format by vladms on Sunday May 17, 2026 @11:47AM (#48169966)
You could say so. Over the years I paied for a couple of courses that would include the classroom lecture at some known universities and did the course homework as well. Some of the companyes I worked for also sent me to ~1-week courses when I asked if I can improve on some topics.

While I had an influence on the general topic of the course I ended up discovering various things that I wouldn't have expected. I did not equally like all professors, but I felt it was better than reading a text.

I wouldn't do this for "<insert latest language/library here>", but there are many complex interesting topics out there.

Re:Frontier AI has broken the open CTF format by qsera on Saturday May 16, 2026 @07:44AM (#48159288)
>A million times better than any human teacher I’ve ever had, for sure.

Not really, not if you want to ask it deep questions. It won't have an answer that is deeper than something that you can find online, and if pressed it will just keep circling around the same response.

The reason is that this "thing" was never curious, never asked questions, and never really learned anything. It just has learned the Internet "by heart", and is as boring as a human teacher who is not really curious about the subject they are teaching, and has just got some degree by "by hearting" some text book. Of course it does it much better than a human, but it is fundamentally the same thing.

Re:Frontier AI has broken the open CTF format by mold_aid on Saturday May 16, 2026 @07:57AM (#48159380)
>Now I’m certain that there exist those mythical human instructors who can do better,

You're certain that mythical instructors exist (?) who "can" do better?

Are human instructors more competent as teachers than AI teachers, or are AI teachers more competent as teachers than human teachers? No "this or that can happen," just a definitive statement please.

AI is likely a million times better student than my dimwit cybersec meatbags...er, majors, for sure, as well! Don't have a reliable way to measure or experience why/how, tho, so I'm not out here claiming it. Even if I did, why would I argue for their replacement?

Re:Frontier AI has broken the open CTF format by IanCal on Saturday May 16, 2026 @05:09AM (#48158363)
They can be incredible. One on one teaching with an infinitely patient teacher who can generate interactive problems on the fly, for dollars a month? Wild. A year of paid ChatGPT would pay for about 9 hours of cheap tutoring here.
Re:Frontier AI has broken the open CTF format by rockskon on Saturday May 16, 2026 @05:27AM (#48158487)
That's not going to work out the way you think it will when a student won't even know how to ask questions.
Re:Frontier AI has broken the open CTF format by tardedmeme on Saturday May 16, 2026 @09:07AM (#48159941)
As usual it depends. When it does well it's because it can do well. When it does poorly it's because you're prompting it wrong.
Re:Frontier AI has broken the open CTF format by mold_aid on Saturday May 16, 2026 @11:16AM (#48160946)
>When it does well it's because it can do well.

Can't argue with that logic

Re:Frontier AI has broken the open CTF format by thin_carapace on Saturday May 16, 2026 @10:05AM (#48160382)
hammers are both a great tool and a deadly weapon at once
Re:Frontier AI has broken the open CTF format by mold_aid on Saturday May 16, 2026 @11:20AM (#48160985)
Not at once, surely
Re:Frontier AI has broken the open CTF format by thin_carapace on Saturday May 16, 2026 @09:03PM (#48165193)
limp response brah, both possibilities remain plausible until one crystallizes at the moment of observation
Re:Frontier AI has broken the open CTF format by viccis on Sunday May 17, 2026 @02:26AM (#48166499)
Mostly, no. They will explain things to you and you'll feel like you understand them. When you have to do it, though, you'll find you're not any better off than when you started.

I used to see this with students in calculus who abused the tutoring resources. They'd have tutors just work problems (often their homework...) in front of them. "Ah! Obviously that trig substitution integral worked that way. Oh, of course, that proof is very obvious in retrospect." And then they'd walk away from the exam with a 30% and no idea how their 20 hours of "study" for it didn't result in the same performance as their peers who worked problems, read the materials and asked questions, etc., got.

Most AI use is that same in my experience. "Show me how the fundamental theory of calculus works." The LLM puts together a very elaborate and flashy presentation that they skim. Great. That's no different than reading a text book. Even if you ask the LLM questions and have it elaborate on things, you've never once done one of the most important things a student can do: spend time confused trying to work hard at understanding something that's not obvious. The LLM will make it obvious at every point. Total lack of friction. Works about as well as a spotter who does the lifting for you.

Re:Frontier AI has broken the open CTF format by repelsteeltje on Saturday May 16, 2026 @05:24AM (#48158471)
I found this interview [0] on the subject of AI in CS education on the Oxide & Friends podcast very illuminating. Of course, Brown University CS != All education, but interesting angle nevertheless.

[0] Episode webpage: https://share.transistor.fm/s/31855e83

Re:Frontier AI has broken the open CTF format by pjc50 on Saturday May 16, 2026 @04:51AM (#48158257)
"Education is just a CTF for the valuable flag of a credential. In this essay I will --"
Re:Frontier AI has broken the open CTF format by magic_hamster on Saturday May 16, 2026 @05:14AM (#48158402)
Education is also figured out. You just need to learn, do and practice for yourself. Telling the agent "to just do it for you" is tempting, but it's not learning. You need to be deliberate when you're trying to actually learn and internalize.

Also, you could spin up your own educational agent with very strict instructions on guiding the user instead of just doing the work. Of course you can always go around it but if you're making an effort to learn, this is a good middle ground.

Re:Frontier AI has broken the open CTF format by otabdeveloper4 on Saturday May 16, 2026 @08:22AM (#48159550)
The best frontier LLMs can't solve 4th grade math homework yet. Don't hold your breath on that collapse of education.

(Real mathematics problems, not American-style ""math"".)

Re:Frontier AI has broken the open CTF format by npilk on Saturday May 16, 2026 @10:23AM (#48160528)
Do you have an example of a 4th grade problem in mind that isn't "American-style"?
Re:Frontier AI has broken the open CTF format by andai on Saturday May 16, 2026 @08:51AM (#48159807)
They were a forcing function for skillz and they no longer are. We need new forcing functions for skillz or we will become WALL-E blobs.

Well, they were ostensibly forcing functions... ten years ago everyone was paying the exchange student to do their homework and assignments for them, and that guy was paying his cousin back in his home country, but the whole thing is a bit more efficient now.

Re:Frontier AI has broken the open CTF format by amazingamazing on Saturday May 16, 2026 @09:29AM (#48160108)
> We’ve figured out the human replacement pipeline it seems, but we haven’t figured out the eduction part.

No we have not.

Re:Frontier AI has broken the open CTF format by aschla on Saturday May 16, 2026 @09:33AM (#48160148)
We've already had consolidation of education for a while now. Even before all the edutech courses, there were Youtubers educating better than many university professors. 10-15 years ago students were already skipping lectures and just showing up for tests.
Re:Frontier AI has broken the open CTF format by jaybrendansmith on Saturday May 16, 2026 @10:13AM (#48160446)
Everything we've learned in the last 10 years is telling us that computers do not help human education in the slightest. We remember better when we write with pen and paper. We learn better with whiteboards and paper books. The simple answer: Remove most computing from education entirely. Blue composition books, pencils, whiteboards is what trains humans. Calculators are helpful perhaps but it is quite possible that slide rules are better. We need humans that can critically think from first principles to counter the recycled information generated by AI.
Re:Frontier AI has broken the open CTF format by skulk on Saturday May 16, 2026 @10:35AM (#48160626)
> computers do not help human education in the slightest

I had no access to anyone who could teach me calculus as a kid except Khan Academy, so I think this is a gross exaggeration. But I agree in the end, that all my "real" learning did come from pen-and-paper practice, not watching videos.

Re:Frontier AI has broken the open CTF format by rossjudson on Saturday May 16, 2026 @12:57PM (#48161856)
Every child reads a book about solving problems, assumes they can now solve problems, and is disappointed when that is not true.
Re:Frontier AI has broken the open CTF format by whichdan on Saturday May 16, 2026 @01:13PM (#48162001)
Yeah I agree. I grew up in a very blue-collar town, and anything I wanted to learn (outside of public schooling) either came from emaciated websites or whatever books I could find at the library. Having YouTube and Khan Academy and everything else would have made such a huge difference for me.
Re:Frontier AI has broken the open CTF format by selimthegrim on Saturday May 16, 2026 @04:35PM (#48163556)
Now I’m wondering how a website is emaciated
Re:Frontier AI has broken the open CTF format by Wolfbeta on Saturday May 16, 2026 @05:07PM (#48163823)
One simply forgets to hydrate.
Re:Frontier AI has broken the open CTF format by bentaber on Saturday May 16, 2026 @05:50PM (#48164109)
Not enough bytes?
Re:Frontier AI has broken the open CTF format by fragmede on Saturday May 16, 2026 @09:37PM (#48165353)
Not even a nibble!
Re:Frontier AI has broken the open CTF format by voxl on Saturday May 16, 2026 @03:55PM (#48163242)
The reality is that a human will learn, given any materials including LLMs, but only if they truly desire to learn. We've had MOOCs, gigantic libraries, all full of free information. You can obtain a PhD level understanding in any technical field of your choice today just by consistently going to the library and consistently applying yourself.

It's not unlike going to the gym, and we see how many people do that regularly. Except it's even funnier, because people serious about the gym but what? Tutors. They call them personal trainers. We've known for a millennium or more that 1-on-1 instruction is vastly better than anything else, but most people actually don't want to get into shape, and most people actually don't want to learn.

Re:Frontier AI has broken the open CTF format by tannertech on Saturday May 16, 2026 @04:47PM (#48163645)
The annoying thing is a PhD level understanding does not get you jobs.
Re:Frontier AI has broken the open CTF format by fragmede on Saturday May 16, 2026 @09:44PM (#48165384)
I don't have a PhD, but "you're overqualified" is something I've heard my PhD having friends said to them.
Re:Frontier AI has broken the open CTF format by globnomulous on Tuesday May 19, 2026 @07:32AM (#48191975)
> The reality is that a human will learn, given any materials including LLMs, but only if they truly desire to learn. We've had MOOCs, gigantic libraries, all full of free information. You can obtain a PhD level understanding in any technical field of your choice today just by consistently going to the library and consistently applying yourself.

Not true. In every field there is guild knowledge that a person can't acquire from a library. In technical disciplines PhD-level knowledge requires experience in collaboration, research, and frequently lab work, which is impossible to acquire without access to a lab -- or just direct experience with research methods, whatever those may be. Reading papers and absorbing information aren't enough. PhD-level knowledge comes from the process of writing and doing original work.

> The reality is that a human will learn, given any materials including LLMs, but only if they truly desire to learn.

Also not true. We require kids to go to school partly because exposure to the environment and work inculcates skills regardless of whether kids want to do the work -- and regardless of whether they want to learn.

LLMs are damaging to students partly because they provide an escape hatch from that work and thereby prevent kids from acquiring skills.

Think of it this way: most people who want to be healthy and eat a healthy diet still find easy junk food tempting. What they want does not change the temptation, because the body and brain gravitate towards easy, cheap fulfillment of basic drives.

People facing challenging tasks, similarly, are tempted to take measures that reduce the amount of effort they require. The availability of tools that reduce the required effort also help shape a person's understanding of the value of the challenge and the work: "why should I do this hard task when I have a tool that can do it for me?" You and I know the answer to this question when we're discussing something like writing an essay or solving a problem in a math or programming class. Students frequently don't. They are by definition ignorant. Children, moreover, lack maturity. Their brains are less capable of resisting the easy path than an adult's. That's partly why parenting is important: parents provide boundaries and limits that kids need but won't and can't provide for themselves.

Sometimes people, especially kids, really do need to be dragged, kicking and screaming, through something in order to receive the benefits it offers. Being dragged through it sometimes convinces a person of its value and benefits. In a kid's case, there's a decent chance that the experience will improve executive function, shape expectations in a healthy way, inculcate grit, and become appreciation -- or at least habit.

I would not have written essays on my own as a student in secondary school. My English teachers had to provide that structure for me and impose the demand. I'm glad they did -- and god bless them for putting up with me and all the other snotty kids like me.

Re:Frontier AI has broken the open CTF format by voxl on Tuesday May 19, 2026 @12:54PM (#48195924)
If you've never taught students or mentored PhD students please refrain from diatribes in my comments (and yes I've done both)
Re:Frontier AI has broken the open CTF format by globnomulous on Tuesday May 19, 2026 @08:17PM (#48201423)
The comment isn't a diatribe, and your experience as a teacher and advisor does not make your argument correct, let alone make your request any less a nonsequitur. If I disagree with you, I will always, I promise, say so. Asking that I 'please refrain' is just supercilious and silly.

It's also arguably a violation of Hacker News' guidelines:

> Please don't sneer

> Please don't post shallow dismissals

Re:Frontier AI has broken the open CTF format by voxl on Tuesday May 19, 2026 @08:22PM (#48201472)
Your argument is garbage, and I give zero fucks about tone policing or other garbage rules
Re:Frontier AI has broken the open CTF format by globnomulous on Wednesday May 20, 2026 @03:02AM (#48204119)
Tomhow?
Re:Frontier AI has broken the open CTF format by mrandish on Saturday May 16, 2026 @04:18PM (#48163430)
> except Khan Academy

But that's not using "computers" as a computer but as a video player. When evaluating whether computers are "good for learning", I don't think we should include using a computer as a video player, a book, or even flash cards. It should be things a computers uniquely offer which a books, paper, videos and a physical reference library cannot.

Based on the results of deploying hundreds of millions of computer to schools in the 80s and 90s, the evidence was mostly that computers are good for learning computer programming and "how to use a computer" but not notably better than cheaper analog alternatives for learning other things.

Interestingly, a properly trained and scaffolded LLM could be the first thing to meaningfully change that. It could do some things in ways only human teachers could previously since it is theoretically capable of observing learner progress and adapting to it in real-time.

Re:Frontier AI has broken the open CTF format by famouswaffles on Sunday May 17, 2026 @06:34AM (#48167678)
I think videos are a unique thing computers offer. Books I understand. You have them digital or not. But a video ? Without a computer, there is no video. You were present for the initial lecture or you weren't and that's it.
Re:Frontier AI has broken the open CTF format by tuna74 on Sunday May 17, 2026 @11:48AM (#48169974)
There were videos* before computers.

*Not really, but you could film stuff and display it.

Re:Frontier AI has broken the open CTF format by bobdvb on Tuesday May 19, 2026 @08:53AM (#48192660)
Film is a chemical medium for storage of images.

Video is an electronic process for capturing images and displaying them.

Before digital video there was analogue video, and analogue video was perfectly possible without digital sampling, or computers. Heck, video pre-dates silicon chips and used to be done with CRTs and valves.

Re:Frontier AI has broken the open CTF format by mrandish on Monday May 18, 2026 @03:04PM (#48184062)
> Without a computer, there is no video.

Maybe you meant "streaming video"?

I realize you're probably under 30 and don't remember "ye olden times" but nearly 90% of U.S. homes and every school had a VCR long before they had a computer. Widespread consumer video formats included VHS, DVD, Blu-Ray, Laserdisc, etc. VCRs. I still buy some movies I care about on UHD discs and watch them in a dedicated Blu-ray/UHD player. Even the 'smart' TVs and streaming sticks most people watch streaming channels like Netflix on aren't functionally computers (no meaningful user accessible local storage, input like keyboard/mouse, CLI or windowing GUI).

In elementary school we learned from 16mm films almost weekly and watched Carl Sagan's Cosmos series and space launches on live on television (it was rebroadcast in the mornings specifically for schools). My junior high school had a television in every classroom and we watched live TV on PBS, NASA channel and C-SPAN. In the late 80s there was thousands of hours of educational video programming sent via direct broadcast satellite to 18-inch dishes at schools. In the 90s every grade and subject had hundreds of interactive video DVDs in large notebooks (four discs to a page in plastic sleeves) and multiple DVD players per classroom.

The peak installed base of VCRs in the U.S. was in 1999. Streaming video wasn't common in consumer households until well into the 2000s, YouTube didn't even exist until 2005 and most people had never heard of it until 2007. In 2010 Netflix mailed DVDs in envelopes to 25M homes every week. They didn't even offer a streaming plan until 2011.

There's nothing about video that uniquely requires computers. But as someone who's spent most of my life thinking about video technology, with patents ranging from analog days to the streaming tech you use today, computers have been extremely disappointing in terms of enabling any unique "learning from video" features that are computer-specific. In the 90s we realized that computers could make digitized video random access letting us sequence it non-linearly to make it interactive in response to user input. We knew that computer-enabled interactivity, responsiveness and real-time adaptation to learner progress would be incredible for improving video education. Yet the vast majority video content available online today is still linear in form. Even video that's specifically educational is no more interactive or user responsive than a 90s DVD disc.

Sadly, only two things have really changed about consumer video in the last 30 years: quantity and distribution. There's much more of it and it's remotely accessible on-demand instead of being limited by broadcast channels and storage media. But that's far more about communication technologies like broadband than computation. For a few years YouTube even had authoring features like interactive menus, conditional branches and scripting but removed it all because it didn't increase ad revenue. There are a few dedicated educational video authoring platforms which provide deep interactive features like conditional branching, viewer annotation, timecode-linked threaded Q&A and other ways to combine the strengths of advanced wikis with interactive video more than just linking from one to the other. Unfortunately, such content is rarely found outside high-end corporate training and some universities. In my opinion virtually none of the learning features computing could uniquely enable for video are in widely available.

Re:Frontier AI has broken the open CTF format by bobdvb on Tuesday May 19, 2026 @08:49AM (#48192615)
As a broadcast technology architect I agree with you a whole lot on the broader technology statements.

But as a former lecturer, I also think the promise of interactivity is dependent less on the tools than on the people. Authoring interactive learning materials is difficult and while that interactivity is engaging, it's not necessarily great at getting a density of information out there.

The Socratic method is great, but that level of interactivity presumes in advance that you know what questions the student will be asking, otherwise it's just a dumb gate. Branching stories for interactivity are highly labour intensive. I suppose if you use AI you could generate a massive number of videos to cover branching learning, but that's going to still be an intensive operation, especially if you're supervising that.

Re:Frontier AI has broken the open CTF format by allan_s on Saturday May 16, 2026 @05:24PM (#48163950)
I think what the author meant is that it does help not more than the same knowledge provided the old way.
Re:Frontier AI has broken the open CTF format by whatever1 on Saturday May 16, 2026 @05:59PM (#48164154)
Khan did not throw at you a 100-slide Powerpoint deck in 45'.

He really took the time to replicate the manual teaching process of writing on whiteboard. He improved upon it by using colors. But basically had the same pace as a teacher writing on a whiteboard.

When professors are given a projector, they just throw together some slides and add their narration.

This is not very efficient. To learn you need to suffer. Or you need to watch the suffering.

Re:Frontier AI has broken the open CTF format by jedimastert on Sunday May 17, 2026 @08:04AM (#48168171)
That's not really a computer helping you though, that's just a computer allowing a human that's far away to help you right?
Re:Frontier AI has broken the open CTF format by __MatrixMan__ on Saturday May 16, 2026 @10:30AM (#48160589)
I think this overlooks the potency and scarcity of 1:1 time with the teacher. If you've only got maybe a few minutes of that in an average schoolday there's a huge difference between whether or not you've talked it through with an AI before trying the question out on the teacher.

They're wrong sometimes, but usually in verifiable ways. And they don't seem to know the difference between medicine and bioterrorism, so often they refuse. But these limitations are worth tolerating when the alternative is that our specialists in topic X are bogged down by questions about topic Y to the point where X isn't getting taught.

Re:Frontier AI has broken the open CTF format by kelvinjps10 on Saturday May 16, 2026 @10:47AM (#48160729)
And now they'll have less time because they will be bombarded with slop to no end.
Re:Frontier AI has broken the open CTF format by __MatrixMan__ on Saturday May 16, 2026 @02:30PM (#48162557)
Obviously generating your homework is a bad idea, and maybe assigning homework that can be generated is a bad idea. But neither of those are relevant to the problem I'm talking about which is about due diligence prior to asking for somebody's extended attention.

Whether you're in class or at work, it's just courteous to ask an AI first.

Re:Frontier AI has broken the open CTF format by peter-m80 on Saturday May 16, 2026 @10:57AM (#48160800)
I learned calculus thanks to wolfram alpha step by step solving feature
Re:Frontier AI has broken the open CTF format by jaybrendansmith on Saturday May 16, 2026 @01:45PM (#48162259)
I'm not going to disagree with step by step videos ... those are a HUGE help. I'm really talking about solving problems using pen and paper, whether math or writing, is how my problem-solving patterns actually changed.
Re:Frontier AI has broken the open CTF format by ralph84 on Saturday May 16, 2026 @06:21PM (#48164277)
> humans that can critically think from first principles

This has never been achieved by, nor is it the point of, education for the masses.

Re:Frontier AI has broken the open CTF format by jaybrendansmith on Monday May 18, 2026 @01:57PM (#48183066)
But it should be, right?
Re:Frontier AI has broken the open CTF format by sometimelurker on Saturday May 16, 2026 @06:43PM (#48164416)
I don't think computers automatically make us more educated, but if you want to make a point don't use reductive exaggerations. > We need humans that can critically think from first principles to counter the recycled information generated by AI.

I agree with this.

Re:Frontier AI has broken the open CTF format by PunchyHamster on Saturday May 16, 2026 @09:15PM (#48165252)
I disagree with that statement. There is nothing inherently wrong with using computer to learn and if your personal goal is to learn it in lot of cases makes it much easier, whether to search for or visualise a piece of knowledge you're' learning.

The problem is frankly computer and now computer with LLM makes it easy to cheat.

The kid doesn't want to learn, the kid wants good grades so parent is happy with them, and the young adult wants to get the paper coz they were told that is required for good life. It's misalignment of incentives.

Re:Frontier AI has broken the open CTF format by tempaccount5050 on Saturday May 16, 2026 @09:50PM (#48165396)
Nah, I wrote physics programs on my computer at home in high school and it absolutely helped with my schooling. Yeah, maybe iPad apps aren't the best things in schools but you're throwing the baby out with the bathwater. Computers bad is simply not true.
Re:Frontier AI has broken the open CTF format by wslh on Saturday May 16, 2026 @10:37PM (#48165613)
I would start saying that many people need presence in a real environment with people to learn. We don't use all our senses in a remote environment.
Re:Frontier AI has broken the open CTF format by UltraSane on Saturday May 16, 2026 @02:14PM (#48162455)
Smart people will use LLMs to learn things faster. Education will adapt by doing all assessments in person.
Re:Frontier AI has broken the open CTF format by djoldman on Saturday May 16, 2026 @02:35PM (#48162596)
> Replace ‘CTF’ with ‘high school’ or ‘university’ and you’ve described the total slow motion collapse of education; the only saving grace is that most of it requires in person presence.

So something like, "Frontier AI has broken the 'high school' or 'university' format"?

The hype surrounding AI is just pervasively exhausting: you've got the folks talking about an entire new age for humanity where we're shortly going to take over the entire universe. And you've got the folks talking about how our entire society is crumbling.

Education is one place folks seem to throw up their hands and say nothing can be done.

The fix is simple: students are to be evaluated on their performance in person. That's it.

Any other "collapse of education" isn't due to AI, it's something else.

Re:Frontier AI has broken the open CTF format by HPsquared on Saturday May 16, 2026 @02:37PM (#48162612)
In my university education (2007-2011), 80% of the grade was based on exams at the end of each year, with no resits.
Re:Frontier AI has broken the open CTF format by AndrewKemendo on Saturday May 16, 2026 @10:53PM (#48165684)
I started teaching “how to build quality products using LLMs” full time recently, and most of what I teach is literally just the 101s of systems engineering, reliabily engineering, product development and project management:

Exceptional clarity on the problem you have

Know how to measure the problem you’re solving

Numerically define what “done” is

Make a deterministic and fully observable prototype

Iterate in production with the user

Expand user base as desired with user iteration in parallel forever

Etc…

Obviously a lot more in the details and these are all case by case, but these chatbots are basically perfect productivity machines for this process.

The massive caveat to all of this is this only works for people that can reliably and truthfully define those items above, are willing to structure organization to make those your priorities.

And actually most financial incentives demand the opposite of this process

If most organizations were honest about it, they would simply say “we’re here to make the most money possible and we’re gonna do whatever it takes to do that”

A lot of people don’t like that, so they don’t say it to come up with other bullshit.

Ultimately that’s why I felt like my only option right now is to teach people how to do this because I assumed it was obvious and it is not.

Re:Frontier AI has broken the open CTF format by ori_b on Sunday May 17, 2026 @02:13AM (#48166438)
You haven't explained why anyone should value education in the world we're building, other than as a hobby.