I'm sure your (manual) commits keep together the changes to the code and the matching changes to the test that keep the test green—but your Ctrl+S'es probably don't. In this strange workflow, the tests are required to be green every time you press Ctrl+S; and any time you press Ctrl+S and the tests aren't green, everything you did since the last time the tests were green disappears from the worktree.
One of the many points of automated testing is that it's automated. I don't have to mentally run all of my tests in the back of my head to tell whether or not my code is correct. With this setup, I do. I become scared to even hit Ctrl+S in case I made a typo somewhere. Or perhaps even worse, what if I made a typo in my test and just committed a half-finished implementation? I've taken automated testing, one of the most brilliant time-saving inventions in all of programming, and turned it into a chore.
And as a programmer, that's supposed to encourage me to do more TDD?
I understand that the article is specifically about addressing those kinds of doubts, but... I dunno. I can't imagine ever trying this for any program bigger than a couple thousand lines of code.
If the goal is to have tiny changes, why not just count up the changelog and revert it if anyone ever tries to commit more than 25 lines of code? It seems like that would be just as effective, and potentially less annoying.
In some ways, I think the feature branch->PR model of today is a step backwards from traditional central server single-trunk development of years gone by (CVS/Perforce/SVN). Branches and PRs work better for F/OSS development, and for monster refactoring, but I kinda long for the days of a normal team just committing to master/trunk several times a day.
I have never treated the individual commits inside a PR as anything other than "I'd like to backup my workstation to the cloud now", but if people are treating them as something meaningful, I think that's a mistake.
The idea behind rebasing is not to make it seem like "real history", but to have all changes in a commit to be related. They're not a reflection of how you built the code, but how you can understand it. They're documentation, and as such, you spend effort on them.
People like using clever features to make the history inaccurate, which seems like a bad idea to me. (It's "cleaner", but wrong.)
Are there workflows that avoid this problem but still enforce every commit being green?
But if there's too much divergence between unit test and integration test pass/fail rates, then unit test coverage needs to be improved; obviously, easier said then done when you have old pre-TDD that's tightly coupled and/or code that's interacting with the system and incurring side effects that are hard/expensive to test.
Replying just to highlight that it's possible, even common, to have a scenario in which:
> "passing tests" doesn't mean anything.
//implementation
var CONSTANTS = {
RED : "red"
}:
//test
test(function () {
assert.strictEqual(CONSTANTS.RED, "red");
});
I spent a lot of time arguing (unsuccessfully) that these kinds of tests accomplished literally nothing. I heard everything from, "well, this lets someone know that the constant shouldn't change" to "well, writing it twice will encourage programmers to be more careful."The worst was that people started doing it with translation strings. So we'd send a config file to the doc team, they'd rephrase something to be clearer, or remove a misspelling, and suddenly tests are failing and the translation string can't be committed until the next day. We had a release build delayed over this once.
Meanwhile, because the coverage report is already showing the code path that displays the string as green, nobody has written tests to figure out if the dialog containing the string actually shows up on screen.
Sometimes I feel like the entire concept of code coverage was just a mistake ;)
The outstanding issue with Java code coverage tools is switch statements, because there are unhandled paths generated by the compiler, but there is an easy solution: don't use switch statements. Enums are basically always a better option anyway.
I use test coverage as part of my workflow. Run my tests with coverage on and make sure my new code is fully covered. Kinda like a sanity check to make sure I'm not missing anything. Or if I'm stepping into old code and I want to see how well it's tested, I'll run the full suite with coverage on and see how bad it is.
The second management starts asking for x% code coverage is when it starts to break down, imo.
Few examples:
Say we have a parser and we discover that it chokes on certain input. I'll start a new branch and first commit will be "Failing test case for XXX" to demonstrate the bug. Then I go about fixing it (or sending it off to a colleague who is better suited to do the work). Once it's fixed, it goes into PR and it gets merged into master only if CI likes it. No one is inconvenienced by the fact that that particular test fails in one of the commits...
In this case, there are couple of benefits:
1. I get to see the tests go red. Happened to me a few times where I do the test and fix in the same breath cause the fix seems simple, but then turns out the fix did nothing as the test wasn't triggering the bug. I find that I'm less likely to do a commit that says "add failing test" than I am to dive right into fixing the code prior to running the tests.
2. It's easy for anyone to go back in time to see that test indeed got red, that is to verify that provided input indeed demonstrates a bug etc. If fix and failing test case were in the same commit, then one would have to carefully checkout just some files and not others.
3. If bug is being handed off to someone, it's pretty obvious where to start if there is a single failing test when they begin looking into it.
In a similar vein, another usecase for a commit that breaks the build is as a means of communication when "architecting" new features and delegating work. We are all developers, so code sometimes comes more easily than prose, and creating a new test case that fails is more expressive and less ambiguous than paragraphs of text in the ticket.
Say we get a ticket that says "We are having problems with trolls", and we want to make it a bit harder for them to register (contrived example and silly solution). I can create a new branch, make the following commit and assign to a junior developer:
+ assert_false(
+ UserRepository->create( email => '[email protected]' ),
+ 'Do not allow throwaway email addresses to register'
+ );
This way, whoever gets to work on the feature has a great starting point - ticket explains the business problem, and the failing test puts them right where they need to start looking. It can be a time saver for everyone involved vs me updating the ticket and saying:
"Hi DevJunior, I think the way to solve this is to hook into the registration process. The create method in UserRepository is probably the best place to implement this - it should return failure if someone tries to register with @mailinator.com"Lastly, it's great to have a failing test to come back to when needing to leave a certain branch for a while - be it end of the day, or switching focus to more urgent matters. I find I remember quicker where I left off if test suite tells me, than if everything is green and I have to hunt what was the next thing that was needed to complete the feature... Basically, using VCS and the testsuite as a distributed TODO list :)
I have never worked anywhere where I would check in a change without the tests passing. If you are making a huge refactoring, you do it a step at a time, with each incremental change still having passing tests.
When I was at Google I believe that changes were automatically rolled back if they broke enough tests globally. I don't recall anyone ever complaining except that the rollback was slower than they would have preferred. You simply can't check in a library change that breaks half the company, after all.
If people are writing adequate tests and consider commits to be "change lists" instead of just a random thing they want to back up from their local machine... this is a no-brainer. But it does require the culture of having your code reviewer check that they think there's enough code coverage. Otherwise "passing tests" doesn't mean anything.
(Google's tools happily showed lines of code that were covered by tests in the review UI, so you have objective and subjective data as a code reviewer to look at before approving.)