News for hackers. Stuff that matters.
Re:Near Future of Programming Languages [pdf] by YZF on Sunday October 29, 2017 @11:06PM (#15582880)
One area I haven't seen any good solutions for is the interleaving of tests with production code. I think we need better ways to express tests without cluttering the production code and excessive mocking. What I'd like to do, and really can't in any language/tooling that I know of, is to extract some arbitrary subset of the code and surround it with tests or a test harness. I'd also like to specify various injection points for tests right as I'm writing the code in a way that doesn't affect the production code and contributes to its readability. Perhaps this is pieces of server and client code or various services all tested together. When I refactor the code I want the tests to "refactor" with the code. I don't want to need to rewrite my tests... Automatic test generation is another interesting area (beyond the basic table or algorithm stuff we might do in tests today).

It's almost like the "problem" isn't the languages themselves, it's the tooling around the languages. I want to be able to do a lot more than "compile", "run", with my code... I can imagine machine learning driven tooling being able to automate a lot of the mechanical aspects of code writing beyond the simple generate a closing bracket that an IDE can do...

Re:Near Future of Programming Languages [pdf] by WalterBright on Monday October 30, 2017 @02:17AM (#15583380)
It many not be exactly what you want, but D supports unit tests embedded directly in the production code:

  https://dlang.org/spec/unittest.html
It's been a real game changer for us in improving the quality of the code.
Re:Near Future of Programming Languages [pdf] by geogriffin on Monday October 30, 2017 @12:00AM (#15583015)
William Byrd's work in program synthesis is an interesting take on this, where an IDE can, guided by test cases written for a function, actually auto-complete code itself whilst writing the function, or tell the programmer when they have something wrong by violating a test case. Of course it is impractical right now, but a good step nonetheless.. It (Barliman) is demoed near the end of this video: https://youtu.be/OyfBQmvr2Hc
Re:Near Future of Programming Languages [pdf] by zmonx on Monday October 30, 2017 @04:57AM (#15583757)
This is indeed an extremely interesting approach.

It was pioneered in the context of logic programming and Prolog by Ehud Shapiro in his 1982 PhD thesis "Algorithmic Program Debugging". The thesis was published as an ACM Distinguished Dissertation by MIT press and is available online from:

http://cpsc.yale.edu/sites/default/files/files/tr237.pdf

Together with Leon Sterling, Ehud Shapiro later wrote a very important introductory Prolog book called "The Art of Prolog".

Re:Near Future of Programming Languages [pdf] by mrkgnao on Monday October 30, 2017 @09:01AM (#15584660)
MagicHaskeller[0] (which seems to sadly be offline at present) is a similar project that infers Haskell functions from properties like

   reverse "abcde" = "edcba"
Other, older projects include Exference[1] and Djinn[2], in decreasing order of power.

Also, this is really similar to how Idris and Agda work, except they use expressive types to generate the code (using the Emacs modes) rather than test cases.

[0]: http://nautilus.cs.miyazaki-u.ac.jp/cgi-bin/MagicHaskeller.c...

[1]: https://github.com/lspitzner/exference/

[2]: http://www.hedonisticlearning.com/djinn/

Re:Near Future of Programming Languages [pdf] by mpweiher on Monday October 30, 2017 @03:27AM (#15583537)
> interleaving of tests with production code

> tests without cluttering the production code and excessive mocking

IMHO, tests should be an integral part of production code. In MPWTest[1], tests are typically expressed on the class side of the class under test, in a category called Testing, though it's easy to override that. This solves 2 of my major annoyances with xUnit-style testing: dual class/test hierarchies and (with static typing) the need to add public interface for the tests, which don't have privileged access.

Mocking in particular and stubbing should be eliminated as much as possible, but this is not a programming language issue[2], more a "not making architectural assumptions too early" issue.

[1] https://github.com/mpw/MPWTest

[2] http://blog.metaobject.com/2014/05/why-i-don-mock.html

Re:Near Future of Programming Languages [pdf] by yogthos on Monday October 30, 2017 @12:49AM (#15583154)
I recommend taking a look at Clojure Spec https://clojure.org/about/spec or Racket Contracts https://docs.racket-lang.org/guide/contracts.html
Re:Near Future of Programming Languages [pdf] by polymeris on Monday October 30, 2017 @01:32AM (#15583274)
I get clojure.spec is primary a way to define a schema and validate against it, and that you can use it for automatic test generation, too. But I feel I fail to grok the whole extent of the possibilities it offers. Does it address any of the other things YZF mentions? Specially, the "prevent excessive mocking" part?
Re:Near Future of Programming Languages [pdf] by yogthos on Monday October 30, 2017 @09:12AM (#15584730)
I think this is a good example of Spec being used to guide the solution http://gigasquidsoftware.com/blog/2016/05/29/one-fish-spec-f...

I don't think Spec alone removes the need for mocking, using Clean Architecture is probably more important for that https://8thlight.com/blog/uncle-bob/2012/08/13/the-clean-arc...

If you structure your app so that any IO lives at the edges, it becomes possible to test all your business logic without mocking. Spec can help at that point by providing generative testing.

Since Spec

Re:Near Future of Programming Languages [pdf] by angled on Monday October 30, 2017 @01:59AM (#15583330)
Re:Near Future of Programming Languages [pdf] by enobrev on Sunday October 29, 2017 @11:49PM (#15582992)
I can imagine something like a code-collapse indicator next to every method in a class (or function or code block, whatever), which would expand tests that can be run on-the-fly - even while you type. Kind of like how comments can be collapsed in some IDEs.

Technically, these could very well be simple commented annotations that point to a separate .test file or something, but are shown in-line by your favorite IDE or code-editor.

Re:Near Future of Programming Languages [pdf] by tiuPapa on Monday October 30, 2017 @02:29AM (#15583416)
Doesn't Rust and cargo do this kinda well?
Re:Near Future of Programming Languages [pdf] by zbraniecki on Monday October 30, 2017 @02:32AM (#15583423)
I really like how rust does unit test in line and in docs.
Re:Near Future of Programming Languages [pdf] by tiuPapa on Monday October 30, 2017 @05:03AM (#15583769)
You can also have integration testing.
Re:Near Future of Programming Languages [pdf] by rexpop on Monday October 30, 2017 @01:54AM (#15583320)
Forgive me, this is going to sound snarky but I promise I am earnest:

> I'd also like to specify various injection points for tests right as I'm writing the code

You mean, like methods/functions?

> extract some arbitrary subset of the code and surround it with tests

You mean, like modules/classes?

Re:Near Future of Programming Languages [pdf] by TeMPOraL on Monday October 30, 2017 @02:45AM (#15583444)
Kind of, but three big problems in practice are:

- runtime state you need to initialize if your code is written in a stateful manner

- other methods/functions in different modules/classes the code you want to test calls out to

- the fact that method/function and module/class separation is (and IMO always should be) primarily driven by needs of production architecture, not testing, means that it may not be perfect for testing needs

Add to that the failure of testing tools and methodologies (like "don't test private methods" meeting "separate out duplicated or complex code into private methods"), and I feel the problems are real.

----

Here's an idea that just popped into my head right now: how about "hash-based testing"? You take a code you want to test, like:

  private Integer foo(String bar, Frobnicator quux) {
    String frob = quux.invokeMagic(bar);
    return memberApi.transform(frob);
  }
and turn it into:

  Method cut = <<
  private Integer foo(String bar, Frobnicator quux) {
    String frob = MOCK(quux.invokeMagic(bar)) AS("frob");
    return MOCK(memberApi.transform(frob)) AS (frob.length() > 3);
  }>>;
  
  //continue testing cut()
The idea being, the compiler or whatever external tooling ensures that the original method in production code, and the inline-modified method in tests, are the same with respect to some equality/hashing function that always treats expressions "foo" and "MOCK(foo) AS(bar)" as equal.

This way, you end up being able to mock everything precisely, inline, in whatever way you see fit, with your tooling ensuring that the actual code stays in sync with the test, since whenever the original method changes in any meaningful way, you'll fail the code "equality" test.

Might be a stupid idea, I welcome comments.

(INB4 testing to implementation instead of the interface - if you have to mock anything, you're already testing to implementation, and this way you can inject testing alterations precisely, instead of having to turn your architecture inside-out to support IoC/DI/whatever the current testing-enabling OOP fad is.)

Re:Near Future of Programming Languages [pdf] by lmm on Monday October 30, 2017 @07:35AM (#15584267)
I just don't experience these problems since I moved to functional style; I find it hard to remember that they even existed. Separate state from logic, separate initialization from operation, and you never have problems with initializing state. If your logic needs to perform effectful operations, separate that out using a monad. Good code structure for production is the same as good code structure for testing, because the needs of code structure are the same in both cases.

Text substitution is impossible to reason about; magic test frameworks sound like a good idea because every kind of magic sounds like a good idea in isolation, but they'll always end up getting you in trouble. Plain old code, plain old functions, parameters, and modules, always win in the end, because every bit of brainpower you can save from understanding magic is brainpower you can spend understanding the details of your actual test instead.

Re:Near Future of Programming Languages [pdf] by TeMPOraL on Monday October 30, 2017 @08:27AM (#15584467)
I'm mostly writing functional-style too these days, even in imperative languages. So this was more of a thought experiment based on the testing I've done in the past or watch being done at my $dayjob.

That said, I didn't mean text substitution, but AST transformations (or rather, AST comparisons, since my example is defined in terms of AST equality with the MOCK magic thingie making the particular part of AST be ignored for the sake of comparison).

Re:Near Future of Programming Languages [pdf] by YZF on Monday October 30, 2017 @10:45PM (#15590770)
It's supposed to be this way but in practice it doesn't seem to work. With methods/functions it's a problem when they call other functions.

So really the idea that a good design is also testable is somewhat of an approximation of reality, It has some elements of truth but at some point making your code more testable actually makes it worse, often you need to make other compromises. Then the interactions of pieces of code typically happens in much more brittle system/integration tests. YMMV, this is just my experience.

A lot of interesting pointers in the replies, thanks!

Re:Near Future of Programming Languages [pdf] by tomelders on Sunday October 29, 2017 @11:56PM (#15583010)
I don’t do coding tests. I talk to the candidate about programming and I only require one interview. I haven’t made a hiring mistake in 6 years.
Re:Near Future of Programming Languages [pdf] by NotSammyHagar on Monday October 30, 2017 @12:18AM (#15583053)
okay. care to give any other info? Usually when someone says I haven't failed in 'some impressive challenge in a long time' it means your challenge wasn't that impressive. How much interviewing have you done, what were the hard calls you were asked to me, whats your scheme?

When I was at google, one of the hardest thing I did was look at the marginal intern review scores and tried to pick out the ones from the big pile whom we should look at again and who should not be looked at. There were all kinds of crazy ass stupid interview questions that in my opinion were not very useful to classify capability.

Re:Near Future of Programming Languages [pdf] by coldtea on Monday October 30, 2017 @05:09AM (#15583786)
>okay. care to give any other info? Usually when someone says I haven't failed in 'some impressive challenge in a long time' it means your challenge wasn't that impressive.

Or they couldn't recognize failure

Re:Near Future of Programming Languages [pdf] by closeparen on Monday October 30, 2017 @12:16AM (#15583046)
Uh, this is about automated testing of programs, not interviewing.
Re:Near Future of Programming Languages [pdf] by tomelders on Monday October 30, 2017 @10:17AM (#15585135)
I'm an idiot.
Re:Near Future of Programming Languages [pdf] by 35bge57dtjku on Monday October 30, 2017 @02:23AM (#15583396)
I'd love to know how you do that, but the answer is always, "I just ask them the right questions and can tell by their answers," which is so vague it's useless.