News for hackers. Stuff that matters.
Re:Firefox 55 and Selenium IDE by danidiaz on Monday August 21, 2017 @02:22AM (#15062354)
Test recorders are an antipattern. They lure you with their "no programming required!" siren call, but the generated scripts invaribly end up as a fragile, poorly abstracted mess.

GUI tests suites, especially those created with recorders, are prone to become unmaintainable beasts full of flaky tests.

This old interview with Bret Pettichord makes good points about the use of recorders: https://youtu.be/s_CUPs6xAWw?t=590

Re:Firefox 55 and Selenium IDE by slgt on Monday August 21, 2017 @04:57AM (#15062983)
Disagree 51%. Recording tests works well and saves tons of time in specific scenarios. It works especially well for "throw away" test cases that you need only for a few days, for example during development of a specific feature. So I consider your statement way too black and white.
Re:Firefox 55 and Selenium IDE by neves on Monday August 21, 2017 @02:32AM (#15062388)
All you said is true, but this doesn't stop Selenium IDE of being a valuable tool. You just need to know when to use it. Without it a lot systems in my company would have zero tests. Not even bad and fragile ones.
Re:Firefox 55 and Selenium IDE by bshacklett on Monday August 21, 2017 @02:36AM (#15062400)
One could argue that bad or fragile tests can be more harmful than having no tests at all.
Re:Firefox 55 and Selenium IDE by danidiaz on Monday August 21, 2017 @02:40AM (#15062412)
Exactly, there is a degreee of flakiness beyond which you don't gain much assurance from your tests and you spend more time fighting fires in the testsuite than actually testing your app.
Re:Firefox 55 and Selenium IDE by mariusmg on Monday August 21, 2017 @03:38AM (#15062643)
>You just need to know when to use it

When exactly is that ? Never ? That clusterfuck of generated code is useless.

Re:Firefox 55 and Selenium IDE by romaniv on Monday August 21, 2017 @11:38AM (#15065332)
UI-level tests are the only kind that reliably test the full stack of your application. No amount of unis testing will catch a missing button or a wrong config setting.

And while recorders present some challenges, manually coding UI-level tests make the maintainability issue an order of magnitude worse.

The best way to make web UI tests stable is to involve engineers and make sure that the pages are structured in the way that make automation and validation easy. Usually the changes required for this are trivial. When they're not, it often points out something in the UI that doesn't make sense anyway, like having no confirmations for user actions.

And in any case, automated UI tests are still better than manual UI test, which is what a lot of people are stuck with.

Re:Firefox 55 and Selenium IDE by danidiaz on Monday August 21, 2017 @01:14PM (#15065998)
I agree that is is important to structure the application to facilitate testing. But I find this surprising:

> And while recorders present some challenges, manually coding UI-level tests make the maintainability issue an order of magnitude worse.

In my experience, writing the tests manually for sufficiently "regular" applications ends up being more maitainable because you can put repeated logic in shared functions and classes, making it easier to adapt to changes in the GUI. Page Objects are an example of that: https://github.com/SeleniumHQ/selenium/wiki/PageObjects

Also, writing the tests manually lets you use "explicit waits". http://elementalselenium.com/tips/47-waiting Specifying exactly what we expect to happen tends to be more robust (and faster) than either using static sleeps or some kind of implicit wait. Recorders tend to have problems finding concrete things to wait for, especially for very dynamic pages.

(That said, I'm not against using something like Chrome DevTools to identify elements in the page, evaluate potential selectors, and so on. That is indeed useful.)

Re:Firefox 55 and Selenium IDE by camus2 on Monday August 21, 2017 @04:10AM (#15062781)
> Test recorders are an antipattern.

Selenium IDE is a useful tool for QA and a lot of user use case. Can people stop calling anything they don't like "antipattern"? All tools are useful when used correctly. Anything can be "an anti pattern" when used incorrectly.

Re:Firefox 55 and Selenium IDE by rgrieselhuber on Monday August 21, 2017 @11:54AM (#15065437)
I co-founded a company called Functionize (https://www.functionize.com/) and we found the same, but the problem is one of assumptions: should you be generating scripts in the first place?

What we ended up doing on our platform was discard the notion of generated scripts entirely. Everything becomes data, not source code. If you're willing to make the tradeoff to a cloud service vs. having the code (many businesses are), it's a great model. If you prefer having control over the code, it will drive you crazy.

Re:Firefox 55 and Selenium IDE by lesbaker on Monday August 21, 2017 @03:21PM (#15066986)
That's why I always used the selenium UI script as a starting point when I was doing web app QA. It was easier in my experience to take the generated output and eliminate unnecessary steps and clean up the selectors than to start from scratch.
Re:Firefox 55 and Selenium IDE by fulafel on Monday August 21, 2017 @11:23AM (#15065209)
Sounds like app specific recorders could be more robust and resilient to changes.