News for hackers. Stuff that matters.
Re:The tragedy of 100% code coverage (2016) by cbanek on Monday May 08, 2017 @11:53PM (#14297622)
> As an aside, while I'm rambling, all the examples in the article appeared to represent unnecessary abstraction, which is the opposite problem.

One other thing about code coverage, unit testing, and other testing fads is I think they actively affect the architecture, and usually in the overthinking it way.

Instead of having one tight bit of procedural code (which may have some state, or some dependency calls), people split it up into multiple classes, and then test each bit. This allows them to use the class architecture to mock things, but really has just multiplied out the amount of code. And in the end, you're running tests over the golden path probably even less. It's even possible to have 100% of the code covered, and not run the golden path, because you're always mocking out at least one bit.

Re:The tragedy of 100% code coverage (2016) by KallDrexx on Tuesday May 09, 2017 @08:49AM (#14299524)
I think there's an argument to be made that if the desire for testing is a main driver for your architecture then your tests are too granular, and you aren't testing any internal integration points. In my experience that means that your tests are so tied to your current architecture that you can't even refactor without having to rewrite tests.
Re:The tragedy of 100% code coverage (2016) by cowardlydragon on Tuesday May 09, 2017 @11:41AM (#14300949)
mocking / interaction / expect breaks encapsulation to perform "testing".

Thus it is often a test of the implementation's assumptions when first written, and even worse, when the code is maintained/edited, the test is merely changed to get it to pass, because unit tests with mocks are usually:

1) fragile to implementation 2) opaque as to intent

Whereas input/output integration points are more reliable, transparent, and less fragile to implementation changes if the interface is maintained.

However, if you must do mock-level interaction testing, Spock has made it almost palatable in Javaland.

This is one area where functional fans get to make the imperative folks eat their lunch.

Re:The tragedy of 100% code coverage (2016) by YZF on Tuesday May 09, 2017 @01:27PM (#14302057)
Exactly. I've seen tests where some code calls:

  printf("hello world");
And the test is:

  mock_printf(string) {
    if string != "hello world" then fail;
  }
Which is basically just duplicating your code as tests.
Re:The tragedy of 100% code coverage (2016) by tracker1 on Tuesday May 09, 2017 @11:46PM (#14305788)
I think that depends on the language/environment... for example, it's usually MUCH easier to get high coverage, and to do just enough mocking testing against Node-style modules in JS than logic in a typical C# or Java project.

Usually because interfaces need to be clearly/well defined and overridden... In JS there are tools (rewire, proxyquire, etc) that can be used with tests in order to easily inject/replace dependencies without having to write code to support DI nearly as much. In fact, I'd say that there's usually no reason not to be very close to 100% coverage in JS projects.

Re:The tragedy of 100% code coverage (2016) by sbov on Tuesday May 09, 2017 @12:11AM (#14297681)
It's strange to me that we continue to make languages that force us to make certain architectural choices to help facilitate testing.
Re:The tragedy of 100% code coverage (2016) by pjmlp on Tuesday May 09, 2017 @05:11AM (#14298621)
We have it since Eiffel and Common Lisp, but not all mainstream languages were keen on adopting design by contract.

On .NET it is a plain library, which requires the VS Ultimate editions to be useful and on Java it was mostly third party libraries.

C# design team is considering adding proper support as language feature, but it seems to be a very far away feature still.

C++20 might get contracts, but C++17 just got ratified, so who knows.

D, Ada 2012 and SPARK do already support contracts.

Re:The tragedy of 100% code coverage (2016) by cbanek on Tuesday May 09, 2017 @12:18AM (#14297706)
Indeed. It's almost as if testing should be built into the language itself. (I'm always a big fan of including self-test code in projects)
Re:The tragedy of 100% code coverage (2016) by WalterBright on Tuesday May 09, 2017 @03:29AM (#14298316)
And it is built in in D:

http://dlang.org/spec/unittest.html

It's been very effective at improving the overall quality of the code. And because of CTFE (Compile Time Function Execution) and static assert, many code correctness tests can even be run at compile time.

Re:The tragedy of 100% code coverage (2016) by vincnetas on Tuesday May 09, 2017 @12:44AM (#14297792)
What are the languages that have testing not as afterthought?
Re:The tragedy of 100% code coverage (2016) by pacaro on Tuesday May 09, 2017 @12:55AM (#14297819)
This depends on how you think of things. Some languages (Eiffel?) have pre- and post-conditions, which do some of this job. Much of the boilerplate testing done for dynamic languages is done by the compiler for static languages. The borrow checker in Rust is doing the work that a test suite and/or static analysis tool would be doing for C/C++
Re:The tragedy of 100% code coverage (2016) by realusername on Tuesday May 09, 2017 @12:52PM (#14301662)
Definitely Ruby in general, testing is a core of the language culture.
Re:The tragedy of 100% code coverage (2016) by pkroll on Tuesday May 09, 2017 @04:21PM (#14303479)
That doesn't mean testing isn't an afterthought in Ruby, the language. It means the culture built up a defense against writing buggy code.
Re:The tragedy of 100% code coverage (2016) by tracker1 on Tuesday May 09, 2017 @11:48PM (#14305793)
I think that's the nature of a dynamic/scripted language... it's just easier to handle in Ruby and JS than many other languages.
Re:The tragedy of 100% code coverage (2016) by V-2 on Tuesday May 09, 2017 @01:19PM (#14301966)
Re:The tragedy of 100% code coverage (2016) by thinkloop on Tuesday May 09, 2017 @12:46AM (#14297796)
> testing should be built into the language itself

I wonder what that would look like...

Re:The tragedy of 100% code coverage (2016) by saosebastiao on Tuesday May 09, 2017 @01:03AM (#14297847)
Rust has simple unit testing built into the language. And in Ada/Spark, tests can be a first class verification tool, alongside formal verification.

We should go a lot further though. IMO, a unit that does not pass a spec/test should cause a compile time error. Testing systems should facilitate and converge with formal verification. Where possible, property based testing should be used and encouraged. And debugger tools should be able to hone in on areas where the result diverges from expectations.

Re:The tragedy of 100% code coverage (2016) by chriswarbo on Tuesday May 09, 2017 @09:37AM (#14299839)
> IMO, a unit that does not pass a spec/test should cause a compile time error.

We can achieve this in dependently-typed languages like Idris. First we define a datatype 'Equal x y', which will represent the proposition that expression 'x' is equal to expression 'y':

    data Equal x y where
      Refl : (x : _) -> Equal x x
There are two things to note:

- There is only one way to construct a value of type `Equal x y`, which is to use the constructor we've called `Refl`.

- `Refl` only takes one argument, `x` (of arbitrary type `_`), and it only constructs values of type `Equal x x`. This is called "reflexivity", thus the name `Refl`.

Hence if we use the type `Expr x y` anywhere in our program, there is only one possible value we can provide that might typecheck (`Refl x`), and that will only typecheck if `Expr x x` unifies with `Expr x y`, which will only happen if `x` and `y` are the same thing; i.e. if they are equal. Thus the name `Equal`.

This `Equal` type comes in the standard library of all dependently typed languages, and is widely used. To use it for testing we just need to write a test, e.g. `myTest`, which returns some value indicating pass/fail, e.g. a `Boolean`. Then we can add the following to our program:

    myTestPasses : Equal myTest True
    myTestPasses = Refl myTest
This will only type-check if `Equal myTest myTest` (the type of `Refl myTest`) unifies with `Equal myTest True`, which will only be the case if `myTest` evaluates to `True`.
Re:The tragedy of 100% code coverage (2016) by tjalfi on Tuesday May 09, 2017 @01:16AM (#14297893)
Force.com doesn't allow you to deploy code to production unless at least 75% of the statements are executed[0]. I wish my employer had a similar requirement.

[0] https://developer.salesforce.com/docs/atlas.en-us.apexcode.m...

Re:The tragedy of 100% code coverage (2016) by cbanek on Tuesday May 09, 2017 @01:06AM (#14297849)
^ This.

And another example of something I'd want checked by the language is exception throwing / handling. It's another one of those places where code coverage won't help you unless you already know what you're looking for. Languages are getting better about it, but in general, handling errors is hard.

Re:The tragedy of 100% code coverage (2016) by Tarean on Tuesday May 09, 2017 @04:12AM (#14298444)
Pure languages like haskell might be the closest thing at the moment.

In haskell you embed domain specific languages when you require side effects:

    class MonadState s m where
        get :: m s
        put :: s -> m ()
This is like an interface specifying the basics of all stateful computations.

You can use different implementations for production and testing without changing any code so mocking is built into everything.

Re:The tragedy of 100% code coverage (2016) by mcv on Tuesday May 09, 2017 @07:03AM (#14298988)
Didn't Smalltalk do something like that? I thought it had TDD built into its standard IDE.
Re:The tragedy of 100% code coverage (2016) by bpicolo on Tuesday May 09, 2017 @10:26AM (#14300252)
Dynamic languages can kinda cheat here. Python has mocking built in to the core lib.