News for hackers. Stuff that matters.
Re:Meta's new LLM-based test generator is a sneak peek to th... by xboxnolifes on Friday February 23, 2024 @08:20PM (#39488236)
> I find it interesting that generally the first instinct seems to be to use LLMs for writing test code rather than the implementation. Maybe I've done too much TDD, but to me the tests describe how the system is supposed to behave. This is very much what I want the human to define and the code should fit within the guardrails set by the tests.

I feel the same way about how test code is viewed even outside of AI. A lot of the time the test code is treated as a lower priority code given to more junior engineers, which seems like the opposite of what you would want.

Re:Meta's new LLM-based test generator is a sneak peek to th... by zeroonetwothree on Friday February 23, 2024 @08:31PM (#39488306)
When I do code review I always review the tests first. If they look through and reasonable then I can be a lot less careful reviewing the rest.
Re:Meta's new LLM-based test generator is a sneak peek to th... by postalrat on Saturday February 24, 2024 @12:25AM (#39489348)
Tests never cover everything so exactly what are you looking for?
Re:Meta's new LLM-based test generator is a sneak peek to th... by Muromec on Saturday February 24, 2024 @05:14AM (#39490510)
For example, just check the list of units, see that there is one for a happy flow, but none for error flow. Check that a number of tests correlates well with apparent cyclomatic complexity of the code. Check that tests are actually defining and testing relevant behavior and not just "I called method A, dus the method B was called".
Re:Meta's new LLM-based test generator is a sneak peek to th... by pydry on Saturday February 24, 2024 @03:52AM (#39490193)
This is how I feel too. For me, tests usually end up being a concrete specification which I can execute.

Getting LLMs to write tests is like getting LLMs to write my spec.