It was really fun to read the forums and see how, day by day, they managed to get closer. Since it wasn’t really crucial IP to begin with, we were rooting for the little guys to see how close they would get, secure in the knowledge that our algorithm was solid. :-)
The final exploit that granted them access was due to a supplier who replaced an earlier validated random generator with something not quite as random, which enabled replay attacks.
I’m curious why there was no auth required for his calls.
* Make the right requests to one endpoint he found and retrieve company financials, number of hits to every google service, the name of every application running in every datacenter, etc.
* With the above two things, you know the location of services and every RPC endpoint on them, and all access control configs. You can take your sweet time to audit the 10's of millions of lines of code to find vulnerabilities and get to attack as an authenticated (albeit low privilege) user. A lot of stuff is open to all authenticated internal users.
* For example, you could take down any google service by quitting all the application servers at the same time by calling the right debugging RPC. You'd be caught obviously tho.
That said, it looks like in this case he would been able to turn on access to google3, through a special flag that is probably meant for internal GAE apps that work on source code. Presumably, the flag allows the app to use a GAE proxy that authenticates to Piper and provides a filesystem-like interface. It's not clear what would have happened at this point, because it is likely there exists a finer grained quota for source access than just all of GAE. These kinds of new RPC traffic would have been visible in Dapper and Census data, plus there are booby traps everywhere aka defense in depth.
The articles about Piper and co. have made it clear that it ran on top of Bigtable first and then Spanner, i.e. in the production network, spread over ten locations. There's file level auditing and access control. It would be very dumb of them not to have intrusion and anomaly detection in place.
36k is not a small bounty for an RCE, but I feel like this is more critical to Google than the highest Android payout, for which they pay up to 200k for: https://www.google.com/about/appsecurity/android-rewards/
For example, with a decent remote android exploit, I could distribute a patched Google Play Services to all vulnerable handsets which disables updates and then listens to my own command and control infrastructure for further actions.
I can now hold the phones hostage and extort google for money to regain control of them.
But I think the same could be accomplished with the access he had, or worse, but would have taken a lot more work. He also would have needed to avoid detection too. His access sounds more troubling than the Aroura attacks they had years ago.
And, you can't quit all application servers at the same time. These are distributed, self-healing, and highly-redundant -- meaning that there are thousands of copies of each service, and the system will bring up new copies to replace the ones you kill, without loosing service (though you could, potentially, affect quality of service).
Even the control mechanisms that allow all this magic to happen are run on the same platform, meaning that you can't affect the managers in this way either.
The sort of things you can do are: affect billing and reporting, find sensitive company data, and, potentially, execute code remotely (though that will probably be in a container, and not have access to much else).
Grabbing source code is problematic. There is more than one repo -- in an org this size there are probably millions. The code is huge, so downloading it will take forever, and then you'd have to read it. Finally, it's written in dozens of programming languages, some, like Golang, are unreadable to anyone but experts.
Google is using a monorepo.