There's a class of integration tests dubbed consistency tests, which tracks the output that an interface provides give a specific input in production, and the test consists of playing back a sample of said production traffic to verify if the service remains consistent before promoting/dialing it up in production.
This class of integration tests is also used in UI tests, and Selenium provides explicit support for them.
https://elementalselenium.com/tips/62-web-consistency-testin...
Are you using object-oriented programming or a more functional style? Without classes, unit tests become a lot more readable and manageable.
https://docs.microsoft.com/en-us/visualstudio/test/generate-...
Also known as its legacy / research project name, "Pex". The underlying algorithm technique is called "dynamic symbolic execution".
There was also similar tool for Java called "Agitar Agitator".
Our Show HN: https://news.ycombinator.com/item?id=31286126 Our demo: https://www.youtube.com/watch?v=aD4CV7UL0RM
Would love to get your feedback.
I use Deno and its standard library for testing with Puppeteer. I've found unit tests overrated and scenarios underrated (ie testing pyramid).
After execution, I can scroll through a tree of called functions and convert it into permanent "implementation documentation" (saved in a document I can annotate eg. with data flow, and edited when the code itself changes), helping me understand both the structure and dynamic control flow of code I'm learning or relearning after months of absence. I currently do this manually in Google Doc bullets, in order to make sense of legacy code whose control flow is fragmented into many function calls I must piece together mentally (as described in http://number-none.com/blow/john_carmack_on_inlined_code.htm...), and as a starting point for restructuring types and function graphs.
It's an interesting idea for sure!
Suture is geared towards refactoring, so it doesn't do it for every function at once, and instead you have to specify methods manually.
Yet, this would be amazing for legacy projects with no unit tests: you could record the code in production, and generate unit tests from there, and add them to the non-tested project.
It would bring you a safety net allowing you to better work on that legacy project IMO.
This actually reminds me a an attempt at an internal tool I saw pass by whose goal was to save all IO for a given request (aka db call, log calls, network request, file read) for a given backend, and allow you to replay those locally for testing purposes independently from the language used by the backend. I think this didn’t lead anywhere, because it was harder than expected, some things couldn’t be recorded easily (like RNG use), and it also meant finding a way to have all IO libs use the precomputed values.
Plus, the very act of engineering in a dependency injection for the mock if not already provided, enforces a testable interface.
I love and hate unit testing. It's a fantastic idea that I've never seen successfully implemented in practice, mostly because the tests invariably fall by the wayside. At the very least, it's a burden to support them.
I'd love a tool that would observe my code during runtime - capturing all inputs and outputs from every function, as well as the value of each variable at each point in the execution flow. Because this would probably incur a performance hit, I'd want to have the option to toggle it on/off.
After execution, I can scroll through a list of called functions and decide which ones I want to save. Once I choose to save, the tool generates a unit test for the selected function. Since the tool is capturing I/O of all functions, it could retroactively mock each call within the function.
This would only be useful for deterministic functions, of course, and it would still be up to devs to document why the expected results are what they are. But my goodness it would be a time saver - one click unit tests. Yes please.