barries
Feature tests often need to span large parts of an application, so there is a often significant amount of overhead (both code and test-time) in repeating identical-except-for-one-value.
Feature tests often can't test the corner cases of internal code. For example, one hallmark of quality software is that it degrades gracefully in the presence of unexpected inputs. So, while the UI might prevent out-of-range values, programmers often choose to also check value ranges at, for example, the top of a stored procedure. This means that you can't test that code with an app-level feature test because a correct UI won't let you enter values to trigger the stored proc's failure case.
Another big issue is the combinatorial explosion. If you have a processing pipeline, like filters in a sound or image processing app or validation and authorization checks in a line-of-business app, the number of configuration and data values that need to be tested for each stage needs multiply together if you only feature test. Unit testing allows you to make sure each of the stages works "well enough"[1], then you can use far fewer integration and feature tests to make sure that the stages cooperate properly and that system requirements are met (two overlapping, but different, concerns).
[1] However the engineer, team or industry defines "well enough".
[Edit: split up the wall 'o text]
Even if languages designed to abet formal proofs, like SPARK, were commonly used, proving anything but the most trivial properties requires a completely different mindset, significant additional training and experience. This puts formal proofs outside the abilities of the average engineer (including myself).
Finally, useful properties, like termination (lack of infinite loops/recursion), let alone termination within a time bound, or correctness with respect to non-trivial requirements is extremely difficult even for those with extensive experience in the area.
In addition to the difficulty of creating proofs, it is amazingly difficult to state non-trivial requirements formally and verify that the stated requirements are the right ones: often the source of the requirements (customers/users, supervisors, marketing) have neither motivation nor interest to state them formally or review formally stated requirements.
Even if there were redress for cases when "the system" makes a mistake, you can never get your quality of life back.