News for hackers. Stuff that matters.
Re:Microsoft has a problem: nobody wants to buy or use its s... by sk7 on Monday December 08, 2025 @02:40PM (#46196643)
Tests are called "evals" (evaluations) in the AI product development world. Basically you let humans review LLM output or feed it to another LLM with instructions how to evaluate it.

https://www.lennysnewsletter.com/p/beyond-vibe-checks-a-pms-...

Re:Microsoft has a problem: nobody wants to buy or use its s... by azemetre on Monday December 08, 2025 @03:20PM (#46197113)
Interesting, never really thought of it outside of this comment chain but I'm guessing approaches like this hurt the typical automated testing devs would do but seeing how this is MSFT (who already stopped having dedicated testing roles for a good while now, rip SDET roles) I can only imagine the quality culture is even worse for "AI" teams.
Re:Microsoft has a problem: nobody wants to buy or use its s... by ethbr1 on Monday December 08, 2025 @06:09PM (#46198956)
Yes. Because why would there ever be a problem with a devqaops team objectively assessing their own work's effectiveness?