The architecture of Stack Overflow [video]

Re:The architecture of Stack Overflow [video] by BrandonY on Monday January 13, 2014 @04:39PM (#7053581)

User community as testers presents some interesting pros and cons.

Pros:

* Tests are self-updating. Add a new feature: tests come in for free. Change a feature: tests automatically update. Fail to document a change: tests fail.

* Tests are unusually thorough

* Eventually consistent testing. If nobody ever complains, it probably wasn't a bug worth fixing.

Cons:

* Tests cannot be run offline. Feature must be committed and deployed before tests can be run.

* Potentially large quantity of false positives (bad bug reports)

* Potentially large quantity of false negatives (nobody notices particular bug, release considered good)

* Does not work for non-user-visible features

So basically you trade the reliability of your tests for a substantial build/release speedup. Some users experience each bug, but they are the users who are actively using the meta-community and have signed up to experience more bugs. Still, lack of pre-release unit testing must radically increase the importance of VERY careful code reviews.

Not the decision I would have made, but definitely has the sorts of advantages that a small team of engineers drool would drool over.

[ Parent | on HN ]

Re:The architecture of Stack Overflow [video] by sklivvz1971 on Monday January 13, 2014 @05:15PM (#7053888)

Remember that our community writes bug reports but also vets bug reports. We rarely have to deal with bad reports. Interestingly, large quantities of false negatives are a non-issue.

[ Parent | on HN ]

Re:The architecture of Stack Overflow [video] by welegan on Monday January 13, 2014 @06:31PM (#7054344)

Presumably the same reason why they don't have a ton of bad questions on stack overflow: their community scoring would apply just as much to bug reports

[ Parent | on HN ]