Re:Automated Unit Test Improvement Using Large Language Mode...
by nonethewiser on Saturday February 17, 2024 @08:23AM
(#39409193)
> IME, the only "test coverage % rule" that I've ever seen work was "must not decrease the overall percentage of tested code".
That’s a stupid rule and not measuring what you think it is.
It fails if you just delete tested code.
It fails if you remove some code thats tested and add some tested code, but not enough.
These are all extremely common for any refactor. Here is a simplistic example but imagine the same principles applied to a large set of changes. Dozens of files, thousands of lines. You cant just manually account for that.
50%:
100 lines of code. 50 covered, 50 uncovered.
remove some function and its 44% from 40 covered, 50 uncovered. Failed.
Or remove some function and replace it with something better thats not as long. 44 covered, 50 uncovered. 47% coverage. Failed.
That’s a stupid rule and not measuring what you think it is.
It fails if you just delete tested code.
It fails if you remove some code thats tested and add some tested code, but not enough.
These are all extremely common for any refactor. Here is a simplistic example but imagine the same principles applied to a large set of changes. Dozens of files, thousands of lines. You cant just manually account for that.
50%: 100 lines of code. 50 covered, 50 uncovered.
remove some function and its 44% from 40 covered, 50 uncovered. Failed.
Or remove some function and replace it with something better thats not as long. 44 covered, 50 uncovered. 47% coverage. Failed.
A stack overflow post about this.
https://softwareengineering.stackexchange.com/questions/4007...
This inevitably leads to worthless tests to increase coverage and avoiding optionL refactors.