News for hackers. Stuff that matters.
Re:What if writing tests was a joyful experience? by hamdouni on Thursday January 12, 2023 @04:47AM (#34351285)
I don't get it. In their example of fibo(15), they already wrote the fibo function but how do they know it is correct ? Ok it is nice to have an autofill for expected value but not from the same 'thing' that I want to test...
Re:What if writing tests was a joyful experience? by drothlis on Thursday January 12, 2023 @04:52AM (#34351327)
I'm not the OP but I think Fibonacci is a contrived example. In practice these "characterisation tests" are great for adding tests to an existing codebase. The "snapshot" just records the current behaviour; it doesn't judge whether it's correct or not, but if you refactor it will catch changes to the behaviour, and then it's up to a human to judge if the changes were desirable or not.
Re:What if writing tests was a joyful experience? by quickthrower2 on Thursday January 12, 2023 @04:56AM (#34351370)
Fibo would be excellent for property testing.

I.e. For all x, Fib(x+2)=Fib(x)+Fib(x+1)

A property based test produces values of x and tests this property (or invariant).

If it fails the framework usually then tries to find the simplest failing case.

Re:What if writing tests was a joyful experience? by thom on Thursday January 12, 2023 @06:23AM (#34351984)
I am always slightly uncomfortable about property based tests that just replicate the logic of the thing they're testing. I don't really know of an alternative in this case, and obviously there's rarely any harm in having regression tests. But it feels like a code smell somehow.
Re:What if writing tests was a joyful experience? by drothlis on Thursday January 12, 2023 @06:35AM (#34352078)
I suppose they're ideal for cases where there's a simple, obviously correct but slow implementation that you can use to test a more complex but faster implementation.
Re:What if writing tests was a joyful experience? by topaz0 on Thursday January 12, 2023 @12:52PM (#34357651)
You say it replicates the logic, but I think it's more useful to say that it describes the meaning: fib is the function on natural numbers that has that property plus the base cases fib(0) = 1 and fib(1) = 1. That's what fib means.

That said, there are plenty of cases where your reaction is justified, where the test is implemented the same way as the function it's testing when the correctness property could be specified more simply.

Re:What if writing tests was a joyful experience? by quickthrower2 on Thursday January 12, 2023 @06:39PM (#34361888)
You can chuck some regular tests in at the same time. They attack different classes of bugs.

Property based tests fuzz test values you may not have dreamed of. Given a string as the required type it could generate some exotic stuff.

Re:What if writing tests was a joyful experience? by funcDropShadow on Saturday January 14, 2023 @05:58PM (#34384969)
Your uneasiness is understandable, but often it is far simpler to check the correctness of a result than to find it in the first place. Then property-based testing really shines.

Even outside that special case it is always possible to generate just some input data and check for the absence of runtime errors or exceptions. That approach can be improved by adding assertions, pre- and postconditions to your code under test.

Additionally, there a different ways to come up with good properties, see: https://www.youtube.com/watch?v=zvRAyq5wj38

Re:What if writing tests was a joyful experience? by karma_fountain on Thursday January 12, 2023 @08:12AM (#34352958)
Fib(x)=0 does satisfy this property though.
Re:What if writing tests was a joyful experience? by karmakaze on Thursday January 12, 2023 @03:01PM (#34359475)
A full definition would include Fib(0) = 0, Fib(1) = 1 or if you prefer 1, 1.
Re:What if writing tests was a joyful experience? by quickthrower2 on Thursday January 12, 2023 @06:37PM (#34361858)
You need to manually check the first 2 values in a seperate test.
Re:What if writing tests was a joyful experience? by drothlis on Thursday January 12, 2023 @04:54AM (#34351342)
P.S. They're also great for writing tests for new code, for many of the reasons described in the OP.
Re:What if writing tests was a joyful experience? by brap on Thursday January 12, 2023 @06:08AM (#34351885)
Sometimes verifying a result is correct is easier than coming up with the result yourself.
Re:What if writing tests was a joyful experience? by onion2k on Thursday January 12, 2023 @06:44AM (#34352131)
I think what they've done is implemented a way of doing snapshot testing by putting the snapshot data in the test itself. In snapshot testing you just cache the result of a code block and use that cached data to check the code produces the same output as it did when you knew it worked. Putting the snapshot inline in the test makes it a lot easier to read.
Re:What if writing tests was a joyful experience? by wmanley on Thursday January 12, 2023 @06:46AM (#34352146)
IMO The real value in tests is making sure you don’t break things later, rather than for getting your current code working.
Re:What if writing tests was a joyful experience? by mingusrude on Thursday January 12, 2023 @07:00AM (#34352251)
Maintaining tests that only test that things haven't changed can be a nightmare if you actually want things to change (like when you're making a change to the logic of a system). They can also be a real pain to maintain since if they fail you almost never know why they failed, just that they failed. This gives you (or the future maintainer) very little help in figuring out what's going on.
Re:What if writing tests was a joyful experience? by wmanley on Thursday January 12, 2023 @10:17AM (#34354732)
Guidelines for tests that reduce maintenance burden rather than add to it:

* Test the externally visible useful properties of your system, rather than implementation details. Related: don’t bother with mocks unless there’s no other way

* Keep your tests DRY: if there’s some invariant you want to check have it appear once in your test code

* Don’t over-test: you can delete tests and checks that fully overlap with other tests

* If you need to over-test make it easy to regenerate the expected results from your implementation (characterisation tests) as espoused by the article. * the amount of testing effort should be proportional to how important the functionality is. Reduce the effort for low impact features

Re:What if writing tests was a joyful experience? by ricardobeat on Thursday January 12, 2023 @07:02AM (#34352271)
That’s a big assumption. In TDD you write the tests first, to help you achieve the results expected, this doesn’t fulfill that need. It is basically snapshot testing applied to any kind of function.
Re:What if writing tests was a joyful experience? by wmanley on Thursday January 12, 2023 @10:04AM (#34354498)
I am of the opinion that the true value of TDD is that tests get written at all. I suspect that in practice the tests tend to get written at roughly the same time as the code, rather than strictly before.
Re:What if writing tests was a joyful experience? by masswerk on Thursday January 12, 2023 @08:38AM (#34353280)
IMHO, this is really more a documentation of the original behavior and maybe a test for consistency of later implementations against the original one. But it's not about validity. I'd call it a "traditionality test".
Re:What if writing tests was a joyful experience? by scruple on Thursday January 12, 2023 @10:27AM (#34354893)
Not fib, but... I've written tests like this semi-recently for an algorithm that I had to devise. I went to Excel to generate the test data, based on a data pull from our production database. So, real production data, using a sequence of Excel spreadsheet functions, which also has multiple eyes on it (including legal, since this was regulatory in nature), fed as input to the algorithm to verify the output matched up.

I'm still on the fence about that but I'm not sure how else to test something that is essentially a glorified algebra formula.