Re:Measuring AI Ability to Complete Long Tasks: Opus 4.5 has...
by ruszki on Sunday December 21, 2025 @04:01AM
(#46343331)
In the case of OP, they cannot even test it, because they have no clue how it works. They cannot test whether the goal was achieved or not.
The other day I generated an MCP server for AST of Java. I had no clue how that works. I couldn’t test it because I had no idea how that looks like. Btw, AI even lied in tests, because it literally mocked out everything from live code. So everything was green, and literally nothing was tested, and it was untestable manually by me.
The other day I generated an MCP server for AST of Java. I had no clue how that works. I couldn’t test it because I had no idea how that looks like. Btw, AI even lied in tests, because it literally mocked out everything from live code. So everything was green, and literally nothing was tested, and it was untestable manually by me.