News for hackers. Stuff that matters.
Re:Android devices can be fatally hacked by malicious Wi-Fi ... by dlubarov on Friday April 07, 2017 @03:02PM (#14062560)
The platform team does do code review and automated tests, but occasionally something slips through, so they QA release branches as well.

Is there any major app (say, at least 1 million SLOC) which has such good automated tests that they're comfortable shipping master without any manual QA? I'd be really surprised if there is.

Even if a team is totally committed to automating all their testing, there are some big challenges with doing so at scale:

- You run into all kinds of rare bugs (with AOSP, the emulator, host OS, etc.). When I worked at Square our biggest app had ~5 hours of automated tests, and these rare bugs made it very hard to keep our builds stable. I imagine thoroughly testing AOSP would take an order of magnitude longer.

- If you want to test on a variety real phones (which seems critical for AOSP), then testing every master commit is cost prohibitive.

- It's hard to automatically detect graphical glitches. You can take screenshots at certain times and validate that they're pixel perfect, but you might still miss a glitch in between screenshots, e.g. during an animation.

There are no comments on this comment yet.