News for hackers. Stuff that matters.

barries

Re:How Does TDD Affect Design? on Sunday May 18, 2014 @11:49AM (#7763355)
There are several issues with relying solely on feature tests.

Feature tests often need to span large parts of an application, so there is a often significant amount of overhead (both code and test-time) in repeating identical-except-for-one-value.

Feature tests often can't test the corner cases of internal code. For example, one hallmark of quality software is that it degrades gracefully in the presence of unexpected inputs. So, while the UI might prevent out-of-range values, programmers often choose to also check value ranges at, for example, the top of a stored procedure. This means that you can't test that code with an app-level feature test because a correct UI won't let you enter values to trigger the stored proc's failure case.

Another big issue is the combinatorial explosion. If you have a processing pipeline, like filters in a sound or image processing app or validation and authorization checks in a line-of-business app, the number of configuration and data values that need to be tested for each stage needs multiply together if you only feature test. Unit testing allows you to make sure each of the stages works "well enough"[1], then you can use far fewer integration and feature tests to make sure that the stages cooperate properly and that system requirements are met (two overlapping, but different, concerns).

[1] However the engineer, team or industry defines "well enough".

[Edit: split up the wall 'o text]

Re:Quark: A secure Web Browser with a Formally Verified Kernel on Sunday September 08, 2013 @12:16PM (#6349185)
See also Frama-C (http://frama-c.com/), which can use multiple automated and interactive theorem provers. It's really a proof obligation management IDE that you can use with multiple provers / proof assistants.
Re:Quark: A secure Web Browser with a Formally Verified Kernel on Sunday September 08, 2013 @12:12PM (#6349171)
It's not feasible for large programs in any generally used language, especially imperative ones. This is because most languages have undefined constructs and constructs that are so flexible that proving useful things about them is hard.

Even if languages designed to abet formal proofs, like SPARK, were commonly used, proving anything but the most trivial properties requires a completely different mindset, significant additional training and experience. This puts formal proofs outside the abilities of the average engineer (including myself).

Finally, useful properties, like termination (lack of infinite loops/recursion), let alone termination within a time bound, or correctness with respect to non-trivial requirements is extremely difficult even for those with extensive experience in the area.

In addition to the difficulty of creating proofs, it is amazingly difficult to state non-trivial requirements formally and verify that the stated requirements are the right ones: often the source of the requirements (customers/users, supervisors, marketing) have neither motivation nor interest to state them formally or review formally stated requirements.

Re:You are committing a crime right now (2012) on Sunday September 08, 2013 @11:59AM (#6349128)
Can't upvote enough because this is the single most troubling aspect of our (US) justice system today: as soon as you are charged, your quality of life is ruined unless you are both wealthy and lucky enough to avoid pretrial incarceration. Even if you are both wealthy and lucky, the stress, effort and money required only to be found "not guilty" or get charges dismissed is a huge burden.

Even if there were redress for cases when "the system" makes a mistake, you can never get your quality of life back.