Re:You Only Need to Test with 5 Users (2000)
by QuinnWilton on Sunday July 14, 2019 @08:21PM
(#20437172)
I'm been building an API Security Scanner, and one of the things it does is just fuzz every endpoint with garbage in each parameter to look for stacktraces, errors, etc.
Moreso than any of the security tests I've written, that fuzzing has broken every enterprise API our customers have thrown at it.
Even a sane person can find a lot if they never touched the implementation of a specific feature.
On one of my teams, we had a hard rule where all features must be tested by 2 other team members (and if multiple people worked on a feature, none of them can be the testers). Something like every other case found something questionable, if not an outright broken edge case, that the developer(s) completely missed.
Re:You Only Need to Test with 5 Users (2000)
by StavrosK on Monday July 15, 2019 @09:42AM
(#20440702)
I remember I implemented 2FA for a previous company, and our QA person managed to lock himself out of the account while enabling it. I asked him how that's possible, and he said "well I went to enable 2FA, it gave me the recovery codes, then it asked for a 2FA code so I entered one of the recovery ones, but now I ran out".
I thought this was so dumb it was brilliant QA.
It's one of the reasons why I don't trust myself to test things fully, we write the software with all sorts of assumptions in our heads and subconsciously steer away from doing silly things - in that context it's really difficult to aim at a point a zero-knowledge user would hit.