News for hackers. Stuff that matters.
Re:Show HN: Automatically create QA environments for your Gi... by mpotter on Tuesday December 03, 2013 @11:49AM (#6841413)
Dev here. Our team has grown to love the GitHub pull request workflow. We built Prune to solve one issue we consistently run into: how to QA on the various branches before merging. Happy to answer questions and thanks for any feedback!
Re:Show HN: Automatically create QA environments for your Gi... by jawns on Tuesday December 03, 2013 @12:02PM (#6841510)
Can you give more details about Visual Regression Testing? If you're comparing screenshots, how do you account for dynamic content?
Re:Show HN: Automatically create QA environments for your Gi... by mpotter on Tuesday December 03, 2013 @12:04PM (#6841524)
Sure! We use a web driver to simulate browsing in an actual client, so Javascript, etc. is executed as it normally would be.
Re:Show HN: Automatically create QA environments for your Gi... by 3am on Tuesday December 03, 2013 @12:21PM (#6841656)
WebDriver/Selenium? That's not really visual, like some kind of CV/OCR tool like Sikuli would be. It's a nice tool, I would be more inclined to use it if it supported more robust test environment provisioning (eg, if the pull request is a schema migration) and it gave more control over how to define my own tests, like Sauce Labs.

edit: never mind, I realized you're probably using the built in screenshot features with webdriver and just diffing the images.

Re:Show HN: Automatically create QA environments for your Gi... by jawns on Tuesday December 03, 2013 @12:05PM (#6841535)
But what about content that changes on each page load, like a timestamp or a rotating banner ad? Would that be flagged as a regression?
Re:Show HN: Automatically create QA environments for your Gi... by mpotter on Tuesday December 03, 2013 @12:34PM (#6841751)
Yeah, it's not perfect but right now we're solving this with tolerance thresholds on the image diffs.
Re:Show HN: Automatically create QA environments for your Gi... by maerek on Tuesday December 03, 2013 @12:45PM (#6841809)
Can you share any more details on how the image diffing is done? BBC has a github repo called Wraith that uses ImageMagick and PhantomJS to accomplish a similar task (without the awesomeness of on-demand testing environments). Always curious to learn more about how people are solving the GUI testing problem on the web.
Re:Show HN: Automatically create QA environments for your Gi... by dbarlett on Tuesday December 03, 2013 @01:31PM (#6842102)
There's also Depicted (dpxdt)[1] - more workflow, Python instead of Ruby.

[1] https://github.com/bslatkin/dpxdt

Re:Show HN: Automatically create QA environments for your Gi... by antjanus on Tuesday December 03, 2013 @12:14PM (#6841599)
I just wanted to know how this works. Our company uses a pretty neat setup where we run different different branches under a wildcard domain on a single machine.

Basically, we run a *.staging.domain.com; however, databases are always tied together. (This is for Wordpress purposes). We use [Genesis](https://github.com/genesis/) to achieve this effect.

So, do you guys work similarly to Travis-CI in that you run an instance, follow instructions in a file and how to provision and setup the site, and then instead of just running a unit test (like travis), you keep the instance running and expose it to public?

Do you use virtualization or docker instances? (Just curious!) How long do you keep them up and do you suspend the instances when it becomes disused? Will you be able to run a single instance at a time or many? (meaning, one branch at a time or will you have 20 environments setup for my 20 pull request).

In the future, will you support private repos?

Thanks! This looks REALLY badass! :)

Re:Show HN: Automatically create QA environments for your Gi... by adambouchard on Tuesday December 03, 2013 @11:53AM (#6841443)
looks awesome