nonamee

Not from my experience. Missing 10% were caused by medium effort and that agent was not able to do play tests. So it is just testing and issuing additional prompts. Nothing hard. No thinking required.

It would be nice benchmark for models. Something like generate web browser from web standards in rust, c, and a language of your choice. An automatic judge would score the results by amount of found vulnerabilities and functionality. We could also see if generating code by LLM in rust makes it safer or if language doesn't matter anymore. And you know that it will work eventually, because AI labs need models capable to write new fully working models themselves to reach AGI or whatever you call it.

The narrative has a point. The LLMs are getting very good at this task. I got 95% of functionality by using LLM to port our Qt desktop app to thin web client and web server. Astra medium, two prompts, 2 hours of LLM processing. Each of two LLM plans had about ten steps. The missing step for porting of these apps is to test against the original (ex. Ms office) app in the plan.

Very interesting article and great job! But does it also provide enough information so that someone else can replicate your results or is that a secret sauce? The article mentions some prompts, tweaks, skills, etc.

interesting article about doesitplay community and their observations, but it seems like a lost battle since majority of gamers simply do not care

No, the opposite, the only app which is save is the app you never ship (stays in a cloud). No binary or source code available to search for exploits by ai.