Have automated feature tests that test most of the "happy path" run-throughs of features that your users do on the front end, and then unit tests that test the minutiae.
My favorite project that I ever worked on was one that was set up this way. We had literally hundreds of feature tests, and thousands of unit tests. When running in a single thread on a local machine, it would take two hours, but when running on Circle CI with parallelization, it would run the entire suite in 6 minutes.
This enabled us to release features to production with a high amount of confidence, any time we wanted.
It was not uncommon for us to release bug fixes to production, while still on the phone with the complaining customer. We earned a lot of customer loyalty points any time we pulled that off. And the best part was that because we had the massive feature test suite running as part of of CI/CD, we were able to do that with the same amount of confidence that we would have compared to having a massive QA team with 50 people testing our complete app before every deployment. It was awesome.
We just released a fairly large system that took this testing approach - despite a large number of tests hitting a database, file system and a blob storage emulator, they still completed in only 5 minutes or so on our CI machine (or 1 minute locally on very beefy laptops).
Why? What about your application changed so that it is slower when testing compared to real world use? If anything it should be dramatically faster because people don't provide microsecond accurate automated responses. If the application naturally executes very quickly I would imagine it would take far long to set up the test scenario than to execute against it.
In my own applications if they take more than two seconds to deliver a response (even for 5mb input) then at the very least I have a critical performance defect. There are not many administrative tasks I can complete from start to finish in that frame of time.
I need to make 8 HTTP requests to properly setup my test from a clean slate, including setting up the deductions, previous earnings, company location and employee home address. If each of those requests takes 50ms, and then I need to make a request to actually execute my test, and verify everything went well, that could easily be 500ms.
And that's just what needs to happen to run one end to end test of the payroll calculation feature. It's absolutely valuable to have a few such tests, but I'm not going to run tests for all the weird one off scenarios (like what happens if someone lives in NJ, and works in Yonkers, and the company has a location in NJ as well?) because they'd take minutes to run. I could run that entire test as a unit test in 2ms for the payroll calculation. That lets me run all of the tests I need very quickly.
I don't know about jeffasinger's company, but in optimal circumstances, a test instance of our app running against a in memory h2 DB takes 3 minutes to start. The app internally already performs heavy lifting in parallel, so it's also not clear that running multiple instances of the app will make it that much faster...
You can build a desktop version of the software with hardware mocked out, which is very useful, but then you're doing something much more akin to an integration test.
An entire unit test suite should complete in well under 2 seconds.
Real world use hits an expensive optimized production cluster. Tests run on my laptop or a cheap testing machine.
Real users don't use every feature every time they perform an action.
A test case at the feature level will likely require several requests if its starting from clean state.
You say request so I guess you are talking about a network service request. When I first got into A/B testing many years ago we found we could use it to simulate traffic through the entirety of the Travelocity web site from homepage to checkout and back. They never used the A/B test system for test automation, but they could have. Back then the checkout pages were rarely tested with valid feature tests because they were at the end of the path. There were plenty of unit tests isolated to small bits of functionality at those pages. We discovered, thanks to session recording with things like Tea Leaf and Clicktale, just a bit later that those unit tests were worthless. The checkout pages failed all the time and cost the company tremendously in lost revenue. Some of those failures would show up in our A/B test experiments though.
My lesson learned from this is that testing things in isolation isn't reliable and provides false confidence. You tend to get back the answer you wanted such that is amounts to a self-fulling prophecy opposed to a criticism.
Rather, I'm arguing against feature tests to the exclusion of unit tests. Both are required in a decently tested systems.
Feature tests provide the assurance that the application works as expected and prevent broken changes going to production, as you've pointed out.
However, they're not a practical replacement for the instant feedback that unit tests provide. Our feature tests currently take 1.5->2.5 hours to run depending on CI system load.
That's... not great. On a given workday I get to try 3-4 builds? I don't know about how others work, but personally I like to get some feedback every like 10-25 lines of code. Unit tests provide that. They take 2 minutes to run. If I had to wait 1.5 hours every 25 lines of code, I would sure be much less productive.
Lets say I somehow convinced management to pay for 20x as many compute resources for CI so that I could get those results in 5-10 minutes rather than 1.5->2.5 hours. And 5-10 is the best case, as that's pretty close to the overhead for our CI system to just clone the code and get the secrets to deploy to a test environment and actually deploy to said test environment.
That's still my flow broken every time I need a test run, because I'm not going to sit and stare at build for 10 minutes, I'm going to answer that email from my product manager or review a team member's PR. If that build comes back with a failure, then I'm context switching back to actually fix it.
The point is that this is a funnel. The unit tests are for instant feedback and to catch obvious mistakes, like "Hey, that value might not be defined, you still need to calculate value B, hope you didn't remove/break the fallback calculation". This ensures that less mistakes incur the full 2.5 hour penalty of a CI build that ultimately fails a test. The feature tests are there for safety. The unit tests are there for productivity. The feature tests are "Does this feature work so we can deploy it". In theory you have at most 1-2 runs of these per feature as unit tests _should_ catch issues before they get there. They're simply too expensive in terms of time to be your first layer of defense, rather they're your final layer.
For example, I worked on payroll software at some point. After finding a bug, we'd want to ensure that could never happen again, and would want to add a test for it going forward. So for example, I may have needed to write a test around someone who worked in one city in Ohio, lived in another, previous income above some amount, and with certain tax advantaged benefits. Setting up this test via feature testing is certainly possible, but it's likely the test itself will take a significant amount of wall time to execute. It's way faster to just test the payroll calculation code, which means you can run the tests more often, and with less developer inconvenience.