Have a bunch of pages with no real content (but have millions of pages). Everytime someone tries to load a page, do some intensive task (ex: mining bitcoins). If you just make it appealing to GoogleBot and no one else, you get free computational resources.

Sorry, I still don't get it.

How does not charging for outgoing network traffic make computation free? You'd still be paying for everything else, eg the instances themselves, datastore storage, read/write datastore calls, using the logs API, which means mining bitcoins wouldn't be free.

OP's concern isn't with network traffic, it's with GAE compute time. Googlebot keeps causing instances to run.

If requests initiated by Googlebot were free to run, you could make a giant website full of garbage and use each free request to spend 50ms mining bitcoin.

If the mining are done at Google Cloud Storage, initiated by a google search bot, can't Google then identify and handle such abuse? I assume Google already scans for multiple types of abuse, such as sites that spread malware.

Google shouldn't really have to do this. Replace GoogleBot with BingBot or GCE with AWS and you still have the same problem. A website operator should be working to make sure search crawlers don't consume too many resources given that the bots follow rules.

Otherwise you'd have a team at every cloud provider trying to figure out how to manage bots.

Now you're suggesting that Google basically devotes a team to detecting "crawler-free-quota abuse", when the real solution needs to handles crawlers from many different sources that aren't all Google.

Is that really how it works?