I used o3 to find a remote zeroday in the Linux SMB implementation

Re:I used o3 to find a remote zeroday in the Linux SMB imple... by Aurornis on Saturday May 24, 2025 @10:03PM (#44085002)

I’ve developed a few take-home interview problems over the years that were designed to be short, easy for an experienced developer, but challenging for anyone who didn’t know the language. All were extracted from real problems we solved on the job, reduced into something minimal.

Every time a new frontier LLM is released (excluding LLMs that use input as training data) I run the interview questions through it. I’ve been surprised that my rate of working responses remains consistently around 1:10 for the first pass, and often takes upwards of 10 rounds of poking to get it to find its own mistakes.

So this level of signal to noise ratio makes sense for even more obscure topics.

[ Parent | on HN ]

Re:I used o3 to find a remote zeroday in the Linux SMB imple... by ngneer on Sunday May 25, 2025 @08:36AM (#44087377)

I do the same, but entry level problems that require healthy analysis. New frontier LLMs do not manage to do so well at all.

[ Parent | on HN ]

Re:I used o3 to find a remote zeroday in the Linux SMB imple... by Aachen on Sunday May 25, 2025 @10:48AM (#44088228)

> challenging for anyone who didn’t know the language.

Interviewees don't get to pick the language?

If you're hiring based on proficiency in a particular tech stack, I'm curious why. Are there that many candidates that you can be this selective? Is the language so dissimilar that the uninitiated would need a long time to get up to speed? Does the job involve working on the language itself and so a specifically deep understanding is required?

[ Parent | on HN ]

Re:I used o3 to find a remote zeroday in the Linux SMB imple... by limflick on Sunday May 25, 2025 @06:53PM (#44092005)

Aren't most interviews like this? Most dev openings I see posted mention the specific language who's expertise they're looking for and the number of years of experience needed working with said language as well.

It can be annoying, but manageable. I've never coded in Java for example, but knowing C#, C++ and Python I imagine it wouldn't be too hard to pick up.

[ Parent | on HN ]

Re:I used o3 to find a remote zeroday in the Linux SMB imple... by Aachen on Monday May 26, 2025 @06:14PM (#44102161)

Huh, okay. That's not how we run interviews but I guess it's at least a thing, even if not common around here that I've seen yet (I'm not super current on interview practices though)

Regarding the job ads, yes they'd describe the ideal candidate but I haven't the experience that the perfect candidate ever actually shows up. Like you say, knowing J, T and Z, the company is confident enough that you'll be able to quickly pick up dotting the Is and crossing the 7s

[ Parent | on HN ]

Re:I used o3 to find a remote zeroday in the Linux SMB imple... by ponector on Monday May 26, 2025 @01:09PM (#44099334)

That is the market nowadays. Employers seek not only deep knowledge in particular language, but also particular libraries. If you cannot answer interview questions about implementation of some features - you are out.

[ Parent | on HN ]

Re:I used o3 to find a remote zeroday in the Linux SMB imple... by tuetuopay on Monday May 26, 2025 @10:37PM (#44103486)

> Interviewees don't get to pick the language?

For leetcode interviews, sure. Other than that, at least familiarity with the language is paramount, or with the same class of language.

[ Parent | on HN ]