News for hackers. Stuff that matters.
Re:Android devices can be fatally hacked by malicious Wi-Fi ... by dlubarov on Friday April 07, 2017 @04:14AM (#14058042)
> There have been cases where AOSP releases on master have failed to compile or require binary blobs.

I wouldn't expect master to be stable, for Android or any other project. If you were setting up a new Linux machine and compiling the kernel yourself, surely you wouldn't build master? AOSP has release branches and RCs like android-7.1.1_r38 which will have gone though a bunch of QA.

Most users wouldn't like stock AOSP though, since the built-in apps are very primitive.

(I'm at Google, but don't work on Android.)

Re:Android devices can be fatally hacked by malicious Wi-Fi ... by adynatos on Friday April 07, 2017 @05:22AM (#14058323)
but you should expect it. any sane development environment will require a code review and build verification before code is submitted to master. there's only a problem when that system is overriden because of impossible deadlines, mandated crunch times and other stupid business practices.
Re:Android devices can be fatally hacked by malicious Wi-Fi ... by dlubarov on Friday April 07, 2017 @03:02PM (#14062560)
The platform team does do code review and automated tests, but occasionally something slips through, so they QA release branches as well.

Is there any major app (say, at least 1 million SLOC) which has such good automated tests that they're comfortable shipping master without any manual QA? I'd be really surprised if there is.

Even if a team is totally committed to automating all their testing, there are some big challenges with doing so at scale:

- You run into all kinds of rare bugs (with AOSP, the emulator, host OS, etc.). When I worked at Square our biggest app had ~5 hours of automated tests, and these rare bugs made it very hard to keep our builds stable. I imagine thoroughly testing AOSP would take an order of magnitude longer.

- If you want to test on a variety real phones (which seems critical for AOSP), then testing every master commit is cost prohibitive.

- It's hard to automatically detect graphical glitches. You can take screenshots at certain times and validate that they're pixel perfect, but you might still miss a glitch in between screenshots, e.g. during an animation.