> 3. If I had a nasty bug, I would set a breakpoint in the debugger (pdb) and duplicate all of the state around me as best I could and put it into a functional test that WOULD get rerun as part of the automatic testing cycle. I would love a tool that could extract program state and put it into a test for me.
Amen. I've wanted this so many times in Java. A really hacky way to do this is to serialize your objects while debugging and deserialize them in your test. The problem is if the classes change your test won't work because you can't deserialize anymore. Encapsulation and data hiding really get in the way here.
Bingo! I did serialize data structures and put them in my tests. In my case it was Python so everything was serialized into a readable textual format. Maybe use snappy compressed json? Being purely functional and having a fairly flat data structures really helped, I didn't have to serialize very far down. Another side effect of being flat is the garbage collector has less work to do. Reallocate early and often!
----
Edit to subdue harshness... I wrote a huge ETL tool in Python using all pure functions, list comprehensions, only named_tuple, no classes, no mutation, lots and lots of garbage, but it worked and worked well. I took a two pronged approach to testing
1. Proved each pure function in separate file, this was for me, this stuff never got run again. It was like a nursery for pure transformations
2. Wrote high level integration tests, 1 or 2 per module (about 20 total)
3. If I had a nasty bug, I would set a breakpoint in the debugger (pdb) and duplicate all of the state around me as best I could and put it into a functional test that WOULD get rerun as part of the automatic testing cycle. I would love a tool that could extract program state and put it into a test for me.
4. The majority of testing was to run the ETL tool on subsets of the input and validate the output, I had a handful of these of increasing complexity. The output validation was automatic.
Testing a 4-10 line functions is waste once it is constructed. Higher level, whole module tests should cover an individual breakage of a smaller pure function. Tests are baggage (sometimes useful), just as static types are baggage (also sometimes useful). We need baggage, gotta wear clothes, read and eat.