https://testing.googleblog.com/2013/08/testing-on-toilet-tes...
https://medium.com/expedia-group-tech/test-your-tests-f4e361...
There are different reasons to have tests.
Unit tests often break even during small refactorings but are great to quickly cover of lot of edge cases of some complex piece of code. They also quickly point to what exactly is wrong.
End to end tests on the other hand should be almost immune to refactoring problems, but they are very slow and often it is tricky to find where the bug is or even if there is a bug at all in the application - in those tests bugs are very often in test code.
Tests for those are unit tests, and fit the GP's description. And those are the best units to test, often you don't want to unit test anything else.
Of course, YMMV. For example, on some languages you must test if the data has the correct structure and the functions have the correct interface, what is best done by unit tests, and those tests will break often. Some problems are so complex that you must test intermediate steps, otherwise you will never get it right; those tests also break often, but when you need them, you need them.
Unit tests should be cheap to write and cheap to reimplement. Otherwise GP’s right and the tests end up shackling you to an implementation, instead of defining the requirements programmatically.
If you make a change and a unit test breaks, do your best to make sure it’s not an actual defect. If it isn’t and fixing the test looks like more work than you want, then delete the test. Maybe make a new one to cover a similar case if that makes sense.
“Less tests” is a really easy state to get to. Just highlight and delete. If I think having more tests is good and you think that it’s bad because you don’t want the tests there if they break, that’s not a problem! We can get back to the way you want things in less than a minute.
Someone writing a test is not a sworn oath to preserve it forever. It’s not even like a public method. No one is using it.
The sentiment of the test may still apply, but the implementation details mean you're testing, for instance, for empty set using a different strategy. Block delete and start over.
Or the feature you're adding is meant to allow us to deal with empty set: let our customers put things in their shopping cart even though we don't have any shipping addresses for them. In that case we 'broke' that requirement, and just delete the test and move on.
But getting cheap tests and high coverage is a bit of an art, often learned via the school of hard knocks. If you let too many requirements leak up into integration tests, you may find you have many, many tests to fix when a "never" or "always" changes to "occasionally", "mostly". Some people get brave and delete these tests, and discover Chesterton's Fence later when someone yells at them for deleting regression tests, because the test described 2 scenarios it was asserting, but it surreptitiously asserted 2 more, one of which the New Guy just broke but got a green build.
With a unit test it's tenable to make a test that looks like it checks one rule while actually checking only one rule.
Isn’t that last step super weird? I don’t see how not writing tests would have prevented the regression. At most vague hand wavey “people would be more careful” which I don’t buy. The problem was breaking a dependency. Tests are not themselves dependencies, they are an automated stand-in for them.
An example I have is in a pay computation engine. I have a public function that computes how much someone ought to get paid for a given shift based on who was working the shift, what they were doing and a bunch of other criteria, what day they were working on, what time of day, etc.
Within that I have a private function that works a with a narrower scope that just does the pro-rata calculation for time-of-day and day-of-week. That function itself has a fairly high-level interface with a highly complex implementation. IMO it makes sense to test that function, even though that function is private and is only used by the one public function.
My point being whether it's private or public isn't really the point. The key thing is whether the functionality is self-contained and decoupled from the rest of the code with a stable interface.
The library itself is public and could have other users, eg. you when invoking manually from cli when testing things out.
Test your library code with unit tests - this is easier because it can be done in isolation feom external systems.
Itegration testing of apps is difficult. Often it is sufficient to test by use - have a dev environment which is usable or test in prod if it's not a critically important app.
This is often sufficient, but I'm sure testing gurus have a recipe to apply unit tests to the app itself as well. I personally prefer not to have 10:1 test to code ratio, so I keep things practical.
This hides the private implementation details, but keeps them accessible for when it's needed for internal tests or workarounds. It's easy to find code which relies on the private namespace, and one can't accidentally use it.
These tests that cross from black to white box testing could be construed as functional tests, in which case my solution and yours aren’t so different.
Also important to remember that it’s a mistake to treat library and application code as the same thing. They are not the same, and from my observations a lot of arguments about The Right Way come down to trying to treat them as if they are.
They have the same pattern at two different levels. What’s the essence of public? It’s that there’s an encouragement for people to use it and you don’t know who is using it. Private is the opposite. Its usage is discouraged or not allowed beyond a certain boundary, so you know everyone using it in an endorsed way.
In a library, your public methods are public and your private methods are private. In a service, all of your code is private! Critically, it meets the definition that you can see all of the usage.
So, why do we test public methods in services? (I’m sure someone would say not to). We just want to be more confident they work as expected now and will continue to do so in the future (I think people forget that’s a [the, even] big reason for tests). Sometimes testing something “more private” gives you faster, more accurate tests.
Maybe you can see it coming that I’ll argue by analogy it can also make sense to test private methods in libraries. They’re not part of your external API. But, just like public methods in a service are an internal API, private methods are part of an internal API for the class that is used by your future self and coauthors.
Indeed. In a good code, one could say it's libraries all the way down.
For example, I've refactored a lot of business code such that it can be easily re-used in different parts of our application. Ta-da, I effectively have created a piece of library code, and I'll want to write tests for it to make sure it doesn't break any consumers, current or future.
Regardless, as you noted, my point is to not prevent access but rather make it obvious if you're using implementation details.
Framework doesn't care if half your unit tests are functional tests, if you slide E2E test code into the before*() call in an integration test. If your white box and black box tests are snuggled up to each other in the same test suite. If your tests only pass if they run sequentially, in this particular order.
It's no wonder we can't get the terminology straight. You can be years into writing tests before anyone or anything calls you out.
Absolutely not
Your private methods contain logic no?
Every coverage is worth
In fact I don't see any "controversy" here. Not testing private methods seems like a lousy excuse
exactly because they change less with implementation/api changes and are more "logic focused"
Sure, do test your API methods, but that might not always be easy or practical
If you want coverage then private method testing is giving you fake or inferior coverage. Those hard to reach paths are only truly tested via the public interface (i.e. called by the function that will call them in practice, not your contrived test function).