News for hackers. Stuff that matters.
Re:Sikuli: Automate Anything You See on Screen by dack on Thursday June 09, 2016 @03:21AM (#11868031)
I mean, we don't rely on pixels because we usually care more about behavior than we do the actual design. It's too brittle to rely on exact pixel locations in the face of added features and small design tweaks. Maybe it makes sense to have one or two of these types of visual checks, but they should be automated so that regenerating them is easy.
Re:Sikuli: Automate Anything You See on Screen by IanCal on Thursday June 09, 2016 @04:23AM (#11868195)
> I mean, we don't rely on pixels because we usually care more about behavior than we do the actual design.

True, although for some things I do still like a visual test. Your users may be used to clicking a button that looks a certain way in a certain place, and suddenly shifting that might be worthy of requiring a change to a test. You wouldn't want all your behavioural tests done like this though.

> Maybe it makes sense to have one or two of these types of visual checks, but they should be automated so that regenerating them is easy.

I agree. Perhaps there's a nice integration between the two? If you were able to give, say, a css selector and a template image rather than just one or the other to look for you'd be able to report these errors:

* I can't find the button I was looking for, but there is something that matches the css selector [blah] that looks like this: [image] instead of [image]. Do you want to update the image in the tests? Y/N

* I can't find the css selector [blah] but the button appears here [image] with [class] and [id], do you want to update the css selector in the tests? Y/N

If the program is actually still fine then it'd be a quick auto-update, and if it's not then these bits of information are probably the first bit of debug you'd want to do anyway to figure out what broke the tests.

Re:Sikuli: Automate Anything You See on Screen by fizzbatter on Thursday June 09, 2016 @01:26PM (#11870853)
> True, although for some things I do still like a visual test. Your users may be used to clicking a button that looks a certain way in a certain place, and suddenly shifting that might be worthy of requiring a change to a test. You wouldn't want all your behavioural tests done like this though.

A decent example of this would be that i've implemented frontend features before but did not notice an unintended pixel shift. I introduced a bug, albeit just visual, but a bug nonetheless - and i had no idea. Likewise, a minor visual issue like that can get through many stages of review and even deployment.

Pixel interaction may be terrible for early prototyping, but the idea has merit. Or, at the very least, perhaps it could identify by css/dom, and throw warnings for pixel alterations.

I like the idea of letting design people programmatically enforce standards (Though, no idea how to make them do that programmatically lol)

Re:Sikuli: Automate Anything You See on Screen by mmebane on Thursday June 09, 2016 @12:33PM (#11870469)
> Perhaps there's a nice integration between the two?

They both run in Java, so you can combine Sikuli with Selenium pretty easily [0], so long as you're not doing headless testing. I've played around with it a bit, using Selenium to get coordinates of a container element and then having SikuliX operate just on that region of the screen.

[0]: http://www.softwaretestinghelp.com/sikuli-tutorial-part-2/

Re:Sikuli: Automate Anything You See on Screen by tlarkworthy on Thursday June 09, 2016 @03:45AM (#11868100)
The visual matchers are fuzzy matchers, not pixel perfect.
Re:Sikuli: Automate Anything You See on Screen by jakozaur on Thursday June 09, 2016 @03:47AM (#11868108)
It can be some nice fuzzy algorithm with learning capabilities. E.g. if you change background, but button is in same place, you can just update the test.
Re:Sikuli: Automate Anything You See on Screen by mmebane on Thursday June 09, 2016 @12:07PM (#11870297)
From my experience, the fuzzy matching works well enough to handle picking out text or icons on interfaces with semi-transparent backgrounds that change on scrolling (e.g., due to a parallax effect). It's pixel-based and not scale-invariant, though, so it won't automatically handle multiple zoom levels/resolutions.
Re:Sikuli: Automate Anything You See on Screen by marxidad on Thursday June 09, 2016 @11:30AM (#11870047)
you can adjust the fuzziness