Unless you're doing TDD or chatGPT is too busy, there's almost no excuse now to not write some unit tests ;)
Usually I get good tests from ChatGPT when I approach it as an iterative process, requesting multiple improvements to the generated test based on what it gives to me. Note that it doesn't replace the skillsets you need to know to write good test coverage.
For example, you can ask to generate integration tests instead of unit tests in case it need context. Providing details on how the testing code should be generated really helps. Also, asking to refactoring the code in preparation to make it testeable, for example, or requesting to convert some functions to actual pure functions, or requesting to refactor a piece of code it generated to a separate function. Then you ask to generate tests for normal and also for boundary conditions. The more specific you get, the chance of getting a good and extensive tests from it is much higher.
Having the tests and code generated by ChatGPT really helps to catch the subtle bugs it usually generates on the generated code (fixing it manually), usually I get test coverage that proof the robustness I needed for production code.
This approach still needs manual fine tuning of the generated code, I think ChatGPT still struggle to get the context right but in general, when it makes sense to use it, I'm more productive writing tests in this way than manually.
In my day job, I write Ruby, and it didn't impress me much when I used it with Rspec. I'd says it was saving me maybe a dozen or so keystrokes in total to write a new spec.
My job is to first consider a dozen ways a new feature might be used; then consider how it could be misused; then write a backend portion and UI for it, testing it for usability and potential for error along the way, using everything I've already thought could go wrong; then monitor it once it's deployed to see if anything actually does go wrong. After actually testing it through everything I can think of by hand, function by function, line by line as I write it, it seems totally stupid to write code to test my own code.
Not to sound like a shit, but that's what end users are for ;)
Not everyone is as diligent and not everyone will have thought through/remembered all those edge cases when they happen to end up maintaining/adding to your code.
Personally I find that tests are a great way to ensure all that hard work you did thinking about those edge cases isn’t wasted. That and manually testing stuff is annoying after the first time.
Tests are useful when you change code that is unrelated, but still somehow connected. You would never notice the issue as you don't manually try the unrelated code, but the test would catch it.
If you had unit/integration tests for all potential edgecases you came up with, you don't have to manually test that over and over.
Well, I get it. You're right in principle. The truth is, though, I have had to rewrite software from scratch in new languages every 10 years, and these sorts of bugs only appear a few times per decade. Once a piece of central business logic works, it usually works forever; a change to that central logic will of course require changes and tests across the whole system, from user inputs to annual reports, but that can't be helped. I suspect you'd then have to rewrite the unit tests, too.
It's probably true that when I die, most of my software will slowly wind down and eventually be abandoned. But also that's probably true even if I were to write a lot of tests.
This is a textbook example of something that property based testing can catch. Yes, the tests are smarter than the person that wrote the code. At least, smarter in a different way or maybe it’s that the people who wrote the property based testing framework are super smart.
Look up QuickCheck for the OG or property based testing + your language.
I don’t exactly understand your bug, but the way it would work is something like you’d make some generators that create night stays and rates and put those together into a bill. Then you test the property that the number of line items on the bill is the number of nights + number of discounts (or whatever is right). It will search to break this property and if it breaks it, find a minimal case. Seriously check it out, it’s perfect for this sort of thing.
Edge cases could be due to accidental complexity. In this case, the solution is to remove accidental complexity, not to freeze it forever in tests.
Edge cases could be due to essential complexity. In this case, if people are making changes without remembering them, it means they are changing code they don't understand. Sure, tests are making it easier. I don't want it to be easier.
> tests are a great way to ensure all that hard work you did thinking about those edge cases isn’t wasted
It is also a great way to ensure that your best talent is wasted on building guard rails.
Think about it: your best engineers are spending time making worst engineers more productive. And they are not even doing it through mentoring, so your worst engineers will remain where they are.
About how long would you say that process takes you? Do you repeat it every time a minor change needs to happen to your code? If not, how do you verify the impact of your changes?
Usually though, in practice, this looks like tracing/logging several times in the middle of each function to make sure I'm getting it right. Where unit tests are supposedly most handy is when unexpected types are passed to functions, leading to unexpected errors. The majority of proofing against that can be done by strict typing and good code documentation.
Ultimately the best test is when you ask yourself "wait, what if someone sent this" and you try it to see if you can break your own code. That's just a spidey sense in the back of your head. If you didn't have the specific doubt to begin with, I don't know how you could write a unit test to disprove it anyway.
I am sorry if I am being a bit mean - I just have had to take over maintenance of things like this and have found it a nightmare from hell so far.
My main concern, of course, is not leaving a mess for myself. That should be every coder's priority. Then taking over someone else's projects wouldn't be so hard.
Nevertheless, I highly recommend having automated tests and running them in the presence of memory profilers etc like valgrind if you're using a compiled memory unsafe language like C or C++.
I would look for another job if I had to work with someone that refused to write them to be honest. It's not that I love tests, but I have had to fix other people's bugs.
I can't speak for others but that's precisely how and why I write tests. It goes like "Wait, what if someone sent this" -> check what happens -> write a test to automate that check -> optionally, fix the code to handle this specific case.
Then, over time you accumulate tests that check all these weird edge cases for you and it doesn't take much to run them over and over again every time you change the code.
> Unit testing will not save you from SQL injection or XSS attacks, so study those and bulletproof your work against them first before you worry about mathematical proof that your database call never results in an error under some odd condition.
I agree with the first part but not the conclusion. My point is that with both manual and automated testing, you still needs to think. It's just that automated tests let you build an executable knowledge base of all the edge cases, errors, security issues etc. you've thought about in the past, and run them with every code change. Hence the value.
I think a lot of people miss the (in my opinion) best use case for UnitTests. It's in-code self-documentation. Leaving a little nugget of your understanding of the functionality behind for other developers in the future. Even if a method is horribly named or the business logic is so complex it takes an hour to understand exactly what is happening, a simple group of (input -> method -> expected output) unit test can get you working with the method fairly fast.
Actually, if I am extending a giant god function that is un-tested. I find it exceedingly helpful to add some tests for the specific use cases I am adding, just to be sure I have understood the code path relevant to my change correctly.
The weird part is not that you don't write unit tests, plenty of developers don't. It's that in 25+ years, you never even had the stray thought "hm, maybe all this manual work that I keep repeating might be worth automating?" and then tested it out to see whether it was worth doing or not
1. It can't be tested.
2. I don't know how to test it.
3. The tests don't give enough value given the effort required to write them.
The less experience you have with tests, the worse the cost-benefit analysis comes out. This is why people disagree over unit tests. Good tests give a huge productivity gain, but the opposite is true of bad tests. Unfortunately, there's a vicious cycle:
You write bad tests
-> You don't value tests
-> You don't spend time learning to improve tests
-> You continue to write bad testsAnd what happens in most companies? We hired a junior engineer and don't have work for them... let them write tests!
Then you make a change, that change impacts 10 of those. You now need to go back and re-test, manually, those 10 thing and check they haven't broke.
That's mainly why I use unit tests, as regression testing.
Also as documentation, if I'm unsure around what a piece of code is actually doing, it's helpful to see tests that hit and find something like.
When_FooIsInThisState_ThenIExpectThisThingToHappen()
Then see the setup and expected output.
It's also good for incrementally building stuff. e.g. I have 3 scenarios I want to get working, I write a test for the first and get it passing. Then I move onto scenario 2, this requires changing the code slightly for scenario 1, but I don't have to re-test scenario 1 as I've got a unit test that is running each time I change the code that tells me it still passes, and so scenario 1 is still working. Then when I do scenario 3, I know 1 and 2 are still working without manually going back and testing them...
They can really save a lot of manual effort.
Unit tests provide no guarantee that those 10 things haven't broke. They can break in a way that's not under test. Or they can be working individually, but breaking as a whole.
You still have to test those 10 things manually before merging.
Sure, sometimes unit tests will show you regression early on. Early feedback can be useful. But it will never give you confidence, it doesn't eliminate manual testing.
The important part is that you're making a change and you don't have confidence that it is safe. This is a signal of bad design. In most cases the right solution is to fix the design, not throw tests at the problem. There are exceptions of course, but even in those cases tests are not something to celebrate or be proud of.
It's like getting obese and then celebrating statin therapy. I mean it's great, but not as great as not being obese in the first place.