News for hackers. Stuff that matters.
Re:Yahoo’s Engineers Move to Coding Without a QA Team by minimaxir on Friday December 11, 2015 @03:46PM (#10719873)
The stereotype that QA-is-pointless in Silicon Valley is persistent and actually annoying. There will always be issues that even the most comprehensive test suite will miss.

Startups still glorify Facebook's "Move Fast and Break Things" without noting that Facebook has backpedaled from that. After all, people expect startup software to have issues, so what's the harm? Technical debt? Pfft.

Engineers are not the best QA for their own code since they may be adverse to admitting errors in their own code. QA engineers are not as empathetic.

Disclosure: I am a Software QA Engineer in Silicon Valley.

Re:Yahoo’s Engineers Move to Coding Without a QA Team by dragonwriter on Friday December 11, 2015 @03:47PM (#10719885)
The QA function is not pointless.

Whether a distinct QA team is the best means of performing the QA function is, however, a separate question.

Re:Yahoo’s Engineers Move to Coding Without a QA Team by tostitos1979 on Friday December 11, 2015 @05:16PM (#10720493)
I was on a large, fast moving project, with dozens of components. Some genius decided that devs will do all QA, automated testing, blah blah. It was a disaster.

The insane deadlines required devs to write lots of poor-to-average quality code (tried code reviews and peer programming ... no time for that so it fell on the wayside). The automated testing done by devs was terrible but understandable. If you are up until 2am hacking out code (without any precise requirements), then why bother with testing? We ended up having one -somewhat- central component that had "gating tests". Everyone stuck their tests there. That made things worse since that one component was the one that kept seeing failed tests. The PMs did frantic "user-like" testing before demos. You can imagine how fun that was.

In my opinion, the decision to not have a dedicated team of testers was the big mistake in all of this. When you have many teams, many components and no precise requirements, you need an independent QA team to coordinate and prevent people from passing the buck. In a time critical project (what projects are not time critical today?), you don't have precise requirements and devs have to "sling" code. I accept this reality. But I don't except the "no QA will make you mature devs" stupidity. If I was being a mature dev, I would refuse to code until the requirements were clear. None of this 2 week agile-scrum nonsense.

Oh .. and one other big thing. The project was a cloud project that needed to be up 24/7 while we were developing it (for beta users). It was like going from one outage to the next. What a disaster!

So what I learned is this: not only have a QA team, but have a 24 by 7 QA team for the kind of project I was on. Note... not all projects are the same!

Re:Yahoo’s Engineers Move to Coding Without a QA Team by dragonwriter on Friday December 11, 2015 @06:09PM (#10720789)
Actually, it sounds like the main problem for the kind of project you were on was realistic timelines, not a QA team.

And I don't think an independent QA team helps to coordinate people and avoid buck-passing, IME, the more different teams are essential to delivering a piece of software to the customer, the more opportunities for buck passing, and the higher level of management the buck passing occurs at.

Re:Yahoo’s Engineers Move to Coding Without a QA Team by shortstuffsushi on Friday December 11, 2015 @06:06PM (#10720776)
> tried code reviews and peer programming ... no time for that so it fell on the wayside

You don't think that could be related? If you had a sane development cycle, that may have helped. Having QA would likely have helped as well, but a healthy dev cycle is a good start.

Re:Yahoo’s Engineers Move to Coding Without a QA Team by Too on Saturday December 12, 2015 @02:52AM (#10722119)
That sounds like a problem with lack of resources and bad planning, not just lacking qa. If you were to choose between a) hire 3 devs to offload your work so that you can focus on your tasks better, or b) hire 3 qa guys that will keep throwing back the shit you produce because you are overloaded. I don't think any qa would help in such a situation
Re:Yahoo’s Engineers Move to Coding Without a QA Team by alexlajeunesse on Friday December 11, 2015 @04:25PM (#10720153)
Where I work teams are made up of a product designer, UX designer or two, devs, and QAs. The QAs are responsible for high level automated tests as well as manual testing for the parts that are hard to automate. It makes communication between devs/qa/design much faster as we all sit together.
Re:Yahoo’s Engineers Move to Coding Without a QA Team by otterley on Saturday December 12, 2015 @11:31AM (#10723227)
You can be sure that any system at Yahoo! or anywhere else that is the bread and butter of the operation (ad systems, accounting, fraud tracking) still involves a software QA team.

Facebook could afford to "move fast and break things" when they weren't taking money from anyone. Now that they're a real company, that kind of attitude can get them adverse attention from the SEC if it causes an accounting discrepancy or from trial lawyers if it upsets their advertisers.

Re:Yahoo’s Engineers Move to Coding Without a QA Team by kumarski on Saturday December 12, 2015 @11:36AM (#10723247)
Facebook has an amazing 'dogfooding their own product' scenario that's hard to replicate for any other company.
Re:Yahoo’s Engineers Move to Coding Without a QA Team by curun1r on Friday December 11, 2015 @07:19PM (#10721100)
The thing that I think is missed in the "getting rid of QA" debate is the size of releases. When you're deploying continuously, the size of what's changing drops substantially. Adding QA as a gating step in the process between developers and production is inefficient and, worse yet, QA will be overwhelmed by the number of rounds of QA they're forced to perform. It's not uncommon for QA regression testing to take days so when there are multiple releases per day, you can see the problem with putting QA between development and production.

That's not to say that dedicated QA doesn't have a purpose. They can and should be testing the live code on a regular basis (weekly or daily, depending on how long the testing cycle takes). But automated tests are what's responsible for ensuring that nothing arrives in production in a completely-broken state and it's assumed that the benefits of continually shipping software will outweigh the downsides of occasionally having subtle bugs in production. And even with QA involved, bugs will make it through the process. At that point, the expedited process of pushing code to production becomes a huge win. Testing strategies often aim for the best MTBF, but when that comes at the cost of MTTF, it's not always a good thing. We've had bugs that were fixed in production less than 10 minutes after the bug is filed.

The other point that gets missed is that people assume that there's no manual QA happening and a developer's careless change just goes to production and wreaks havoc. This ignores the code review process, which is crucial to delivering quality software in a continuous deployment scenario. On my team, changes require 2 +1s before being merged into master, subjected to continuous integration again and eventually deployed to production. Moreover, if any engineer reviewing the code isn't sure they fully understand the change or otherwise wants to see the code running, a single command that runs in under 10 min will spin up an environment in AWS using the code in the pull request so that they can do any manual testing they need to feel comfortable adding their +1. When their done, a single command cleans up that environment.

The thing to keep in mind when designing a testing strategy is the context your software runs in. I would not advocate this testing strategy for code that runs in a vehicle where a bug could cause physical destruction. Likewise, I wouldn't use it for an application with access to highly-sensitive medical or financial information where a leak or data corruption could mean millions of dollars in losses/fines. But for the majority of internet software, the stakes just aren't that high and the gains from a streamlined development process will outweigh the losses from bugs that find their way into production.

Disclosure: I manage a team that deploys continually, usually upwards of 20 times per day. We're responsible for our own QA and have a significantly lower defect rate than other teams in the company with a more traditional QA strategy. However we still draw on QA resources when we feel like we're pushing something risky.

Re:Yahoo’s Engineers Move to Coding Without a QA Team by loco5niner on Friday December 11, 2015 @04:39PM (#10720235)
emphatic... were you going for empathetic?
Re:Yahoo’s Engineers Move to Coding Without a QA Team by minimaxir on Friday December 11, 2015 @04:40PM (#10720241)
Yes, fixed.