In this same vein, I propose another question:
How similar does the test look to the production code?
These kinds of super-granular, brittle tests have a way of looking very similar to the code they are testing (e.g. super strict mocking of every dependency almost looks exactly the same as the code under test!). That style of testing is basically writing the code twice, and verifying that the second copy does indeed equal the first. I'd rather just write the damn code twice, and save myself from having to figure out all the minute differences between testing libraries that framework jockies bikeshed over.I think this comes from "unit tests" where they have assertions such as
assertThisOtherSpecificMethodNameWasCalledWithTheseSpecificallyNamedParameters
If more methods were written to not call out to disk or some external service (e.g., a database or API) and we had a main method that handled the external communication, then the unit tests would be much less fragile. This would also eliminate the need for mocking since the methods that are tested have no side effects.For testing dependencies, an environment where a test instance of the database, API, or other external service is needed. Then the code could be tested against an actual implementation rather than a mock of it.
I am not really following. You can write a seperate function that handles database query, then call this function in your code under test, or one of your argument in the code under test is the databse object handler (this allows you to abstract from a specific db handler). But if you don’t mock out the handler with a “dummy handler”, you will be making real databse call. That is not UT.
You will be doing functional testing.
So you can only abstract so much to get your code testable.
You can, but that still requires that you mock that call in order to test the method that calls your database query method. But, if you make it such that your method takes the result of the database query as a parameter, then you can test your method without having to use a mock. In other words, you can change this (which requires mocking get_db_result to test):
def a_method_to_test(param1, param2):
db_result = get_db_result(param1)
# do something with db_result
to this: def a_method_to_test(param1, param2, db_result)
# do something with db_result
Now you can test a_method_to_test without having to mock out the call to get_db_result by just passing in whatever you want as the db_result parameter.Basically, doing this will separate code that manipulates data from code that either writes or reads data. You can unit test methods that manipulate data, but you will need to do functional/integration testing of code that writes or reads data from other sources.
If you think this sounds like mocking, you’re right (it is like mocking, it isn’t mocking). It has the bonus of making it easier to inspect the logic & expose it to the user ("hey I’m about to do these things, does that sound ok to you?").
Well, not really. Because now I have the DB stuff all contained in a separate function (or handler which can be proxy to the correct DB engine/type of DB), so if with careful design I should be okay.
def create_account(user_manager, user_details):
user_manager.create_account(user_details)
Here user_manager can abstracts away the exact DB type/engine like Django.Of course this is ideal, but that was the motive. I do agree on mocking == testing implementation in general, and functional testing is almost always the way to go... since mocking returns dummy data / fixture, might as well return from a real database. The downside is speed :/ (there are various of tricks like setUpClass, run every test or group of tests in docker containers in parallel, but takes much longer than UTs). Ugh trade-offs.
I'm still not sure how to get the good outcome every time. Maybe it's just well designed interfaces ️
Secondly it decreases your confidence in the refactor. In a classic refactor, nothing should break. This is because a good set of tests are a way of documenting and verifying the design/behaviour of the system.
Finally, often in a codebase with micro tests, there will be an expectation that you replace one set of micro tests with another.
That "feedback" is the pain you feel when you have to maintain one form of tight coupling (ordinary bad code) combined with another (micro-level unit tests).
Higher level tests won't cause you that 'guiding pain' because they aren't as tightly coupled to your implementation.
IMO you don't need to write tens of thousands of lines of unit test code to spot that you're building tight coupling in. It's something you can spot simply by knowing what to look for.
Moreover, if you make a design mistake or introduce tech debt lower down the stack, having a panoply of low level unit tests means that refactoring will break those tests even if you haven't introduced any bugs. IME that causes a dangerous cementing effect on bad code.
I think if you've got a module which is self contained, reusable and tightly scoped, it's worth surrounding with a lower level test. But, if you don't, you're doing more harm than good by surrounding a module with tests.
I don't think you do. I think a testing pyramid is a sign of an unhealthy approach and an unhealthy codebase.
I think what you need is a testhenge. For those who aren't familiar with henges, here's a typical (although poorly maintained) one:
http://www.english-heritage.org.uk/remote/www.english-herita...
Rather than being massive at the bottom and then getting thinner as it goes up, there is a continuous band around the top, covering everything, supported by uprights that dive deeper down, just where needed.
That is, you need a layer of high-level tests for everything (for a web application, some mix of browser tests or controller-level integration tests), along with component-level integration tests and isolated unit tests for the bits of the system that are particularly risky (complex, error-prone, critical, new, old, etc).