News for hackers. Stuff that matters.
Re:Observations on the testing culture of test-driven develo... by marsrover on Tuesday August 14, 2018 @08:26PM (#17763021)
Disclaimer: I work in statically typed languages.

I’ve studied and practiced TDD for years. I’ve even written articles to help beginners.

I now only practice it sparingly. Sometimes it makes more sense to write the test first and other times I already know exactly what I want and I know how to make it testable.

An even bigger issue I have is the huge reliance on mocking which leads to completely brittle tests and code that can withstand no change without all hell breaking lose.

Lately I spend more time on acceptance tests, which I do often write first. I use unit testing for non-trival logic.

Re:Observations on the testing culture of test-driven develo... by anonytrary on Tuesday August 14, 2018 @11:17PM (#17763507)
> I already know exactly what I want and I know how to make it testable. An even bigger issue I have is the huge reliance on mocking which leads to completely brittle tests and code. ... use unit testing for non-trival logic.

Couldn't have said it better myself. If you're unsure about the best form of your API, then TDD can be hindering. Tests are important but TDD can be an anti-pattern.

Re:Observations on the testing culture of test-driven develo... by u801e on Wednesday August 15, 2018 @12:12AM (#17763711)
> An even bigger issue I have is the huge reliance on mocking which leads to completely brittle tests and code that can withstand no change without all hell breaking lose.

In my opinion, unit tests should only cover code that doesn't have any side-effects or interact with external dependencies. For example, suppose you have a method that handles a HTTP request, extracts information from that request, create a database query, query the database, processes the results from the DB, generates a HTTP response, and then sends the response back to the client. You can easily factor out the parts that don't have external dependencies into their own methods. For example, you could pass in a HTTP request as a parameter to the method that extracts information from it and then returns that information.

If you unit test those methods, then you would have coverage for everything other than the actual interaction between the HTTP server and client and the interaction between the server and database. Those cases can then be covered with integration tests where an environment is set up that runs test instances of the HTTP client, server, and database.

Re:Observations on the testing culture of test-driven develo... by monocularvision on Wednesday August 15, 2018 @12:35AM (#17763791)
> In my opinion, unit tests should only cover code that doesn't have any side-effects or interact with external dependencies.

Almost sounds like functional code is the easiest to test code...

Re:Observations on the testing culture of test-driven develo... by canhascodez on Wednesday August 15, 2018 @01:24AM (#17763933)
I would like to believe this, so let's kick it about. I hypothesize that you just get to move state around: that having a pile of perfectly functional code means that you then have to setup/inject a lot of stateful stuff in order to actually test anything, and the complexity of the code/testcode system is conserved. Is there a good way to test this? Is the whole system easier or harder to understand if the stateful bits are isolated?
Re:Observations on the testing culture of test-driven develo... by u801e on Wednesday August 15, 2018 @08:31AM (#17765506)
In the example I gave, the state is held in the outer method. That is, the connections with the HTTP client and the database.

The methods that process the data from the request, generate the database query, process the data returned from the database and generate the HTTP response can all be tested by setting up the function parameters and asserting on the return value. Setting up mocks to do things like assert that the function calls another method with a particular set of parameters is not necessary.

As for testing interactions with the HTTP client and database, one can set up an environment that runs a actual HTTP client and an actual database server (populating it with test data). Testing against actual implementations is always better than testing against mocks.

Re:Observations on the testing culture of test-driven develo... by canhascodez on Thursday August 16, 2018 @03:45PM (#17777878)
It sounds like, "if you wish to make an apple pie from scratch, you must first invent the universe." But, hey, if it's good enough for the universe, who are we to try to improve on that?
Re:Observations on the testing culture of test-driven develo... by Retra on Wednesday August 15, 2018 @01:54AM (#17764022)
It depends where your state is. If it's all in one place, set-able in one language, then it's probably no worse. But if your state is spread out across 30 different config files for 12 different versions of software, spread among multiple machines... good luck getting it setup for reproducible tests.
Re:Observations on the testing culture of test-driven develo... by flukus on Wednesday August 15, 2018 @01:14AM (#17763908)
I'd say procedural more than functional, it drives you to avoid encapsulated state and that's about it, unless of course you want to define C's level of abstraction as functional :)
Re:Observations on the testing culture of test-driven develo... by kraftman on Tuesday August 14, 2018 @08:33PM (#17763052)
I found this set of articles great for using less mocks and creating less brittle tests: https://enterprisecraftsmanship.com/2015/07/06/how-to-do-pai...
Re:Observations on the testing culture of test-driven develo... by avinium on Tuesday August 14, 2018 @09:37PM (#17763226)
> Lately I spend more time on acceptance tests, which I do often write first. I use unit testing for non-trival logic.

What's the difference between acceptance tests and integration tests?

Re:Observations on the testing culture of test-driven develo... by arenaninja on Tuesday August 14, 2018 @10:30PM (#17763376)
It depends on where you work, but to me acceptance tests run on a browser (e.g.: automate site testing with puppeteer, selenium), while integration tests I would say tests running an API call that's allowed to hit a data store

At some places, the only acceptance testing that exists is user acceptance testing, where a user manually interacts with the application to sign off on updates after verifying that it meets the business requirements

Re:Observations on the testing culture of test-driven develo... by Denzel on Tuesday August 14, 2018 @11:47PM (#17763613)
You're right. The definitions of boundaries between acceptance vs. integration tests is fluid and depends upon the team context.

In our team, we define the terms less by tactics and more by intent.

It's our belief that you can generalize your idea on what acceptance testing means. The intent of an acceptance test is to verify that -- no matter how it's exercised -- the implementation performs the work and outputs the result I expected. Now, you can exercise this implementation with any interface. Of which, a browser is just one interface to your application layer. An HTTP API is yet another interface; a command-line utility is another; etc.

We actually have separate acceptance tests at the UI level and the API level. Sometimes even at the _feature_ level, in that it exercises a multi-step multi-result pathway through the application.

The engineer responsible for developing a unit of work may or may not implement API level acceptance tests (it's recommended); the reviewing engineer usually writes an automated API level acceptance test that must pass for the merge request to pass from code review -> QA/QC; then the QA/QC engineer implements and runs UI level acceptance tests that must pass to move from QA/QC -> Done.

We've found this process to be very effective in reducing our defect rate and improving the quality of our product.

Re:Observations on the testing culture of test-driven develo... by malkia on Wednesday August 15, 2018 @02:16AM (#17764080)
My understandings are:

Unit test: tests single function/class, uses fake or mocked data to test for expectations. Test coverage can be used to find spots missed in testing.

Integration test: test two or more things in isolation - interaction between two classes, functions, etc. OR more than one system (say micro-service) communicating with another. The rest of the systems are fake/mock or fetch/replay golden data.

Unit test should take milliseconds to execute.

I'm not actually sure what acceptance test would be?

Re:Observations on the testing culture of test-driven develo... by dionian on Tuesday August 14, 2018 @11:41PM (#17763589)
an IT can be used as an AT, but doesnt have to be
Re:Observations on the testing culture of test-driven develo... by Merad on Wednesday August 15, 2018 @01:20AM (#17763923)
> An even bigger issue I have is the huge reliance on mocking which leads to completely brittle tests and code that can withstand no change without all hell breaking lose.

In the .NET world, at least, I highly recommend the AutoFixture library. It's mainly meant for generating test data, but combined with your preferred mocking library it can auto generate mocks on the fly. For example, imagine that you're testing a class with 5 dependencies, but only one is relevant to the current test. AutoFixture will generate all the mock objects automatically for you, then let you capture the one you care about to set up its behavior for your test.

Re:Observations on the testing culture of test-driven develo... by u801e on Wednesday August 15, 2018 @12:05AM (#17763685)
> An even bigger issue I have is the huge reliance on mocking which leads to completely brittle tests and code that can withstand no change without all hell breaking lose.

In my opinion, unit tests should only cover code that doesn't have any side-effects or interact with external dependencies. For example, if you have a method that handles a HTTP request, extracts information from that request, queries a DB, processes the results from the DB

Re:Observations on the testing culture of test-driven develo... by u801e on Wednesday August 15, 2018 @06:26AM (#17764951)
Mods, you can delete the post above and this post. Somehow, it was posted before I finished typing it (the finished version is at this link https://news.ycombinator.com/item?id=17763711).
Re:Observations on the testing culture of test-driven develo... by rooam-dev on Wednesday August 15, 2018 @07:20AM (#17765205)
1. What does it mean to study TDD?

2. If acceptance tests are the highest level of tests, then focusing on them only may not be very efficient, mainly because they are expensive to run (see the test pyramid). If something doesn't work then most likely debugging is necessary, then it defeats the purpose a bit.