Relevant thread: https://twitter.com/CopperheadOS/status/772592323112869888
If you read the thread I linked, you'll see that Google doesn't really bother to ensure open source Android builds actually work at all: They post them but are only concerned with proprietary flavors being functional. Which means a lot of work has to be done to make open source Android even build in the first place.
Then, most apps are built on proprietary Google APIs, not open source Android APIs. So you either have to use something like micro-g to let you use Google services without the proprietary on-device code, or create your own replacement APIs that just claim to be Google, but aren't, which Amazon does and Microsoft experimented with when they made Android apps installable on some test builds of Windows Mobile. Or you have to substantially rewrite the apps not to use proprietary Google APIs.
So you have to do a lot to make open source Android anything actually work and do what people expect it to.
Ah, the favorite "a million eyes fallacy".
The truth is: there are very few (if any) people capable to do a full audit on the Android source code to see if something nefarious is happening. The source code for Android is over 20 GB. Good luck that "anyone" who can check it out an be able to see ... well ... anything.
When significant flaws were discovered in OpenSSL, it turned out there were only a handful people in the world who could potentially do a full analysis of the code base. And the resulting audit ended up costing millions, took several years and resulted in the re-implementation/forking of the code base.
I know that OpenSSL isn’t a part of Android, but it shows the difficulty of auditing a large complex project.
Believing that anyone can checkout Android code and easily discover if Google (or anyone) has done something nefarious is a delusion of the highest order. The code base would need to be continuously audited by a team of experts in various domains.
i.e. Android is completely open; if Google were to try to put something nefarious in it, anyone who'd checked out the Android source would be able to see that nefarious thing and remove it; if Google kept putting it back in, nerds would be able to fork Android to take it away from Google.
This has become much less true over the years than it used to be, of course, as much of what makes Android Android has migrated out of the core OS into less open intermediate layers like apps and services. This transition helps Google push updates out to users more quickly, since cell network operators proved to be pretty slow about pushing out updates to Android proper, but from Google's perspective it also has the side benefit of making "open Android" (e.g. AOSP) pretty incomplete unless you pair it with Google's proprietary services.