News for hackers. Stuff that matters.

A comment on Gmail and Drive - a new way to send files. Parent comment

Re:Gmail and Drive - a new way to send files by threedaymonk on Tuesday November 27, 2012 @07:14PM (#4840133)
In a discussion about this at work a few months back, someone suggested to me that part of Google's motivation might be to increase the cost of entry for search competitors. No longer can you just scrape a page and index it: not one of the 22 KB served at that URL actually contains the text of the page. To scrape one of those awful pages, you need to JavaScript interpreter and a DOM - a headless browser, really. Unless you're Google: they can just index the back-end directly

I'm not entirely convinced, but I've heard worse conspiracy theories that turned out to be true.

[ Parent | on HN ]
Re:Gmail and Drive - a new way to send files by itsadok on Tuesday November 27, 2012 @11:46PM (#4841067)
You can read the entire blog from the RSS feed. Can't say that about Google+, though.

http://feeds.feedburner.com/OfficialGmailBlog

[ Parent | on HN ]
Re:Gmail and Drive - a new way to send files by uptown on Tuesday November 27, 2012 @10:06PM (#4840802)
Getting a headless browser up and running really isn't that huge of a hurdle if you're capable enough to build a search engine whose intent is to be competitive with Google.
[ Parent | on HN ]
Re:Gmail and Drive - a new way to send files by threedaymonk on Wednesday November 28, 2012 @09:04AM (#4842539)
It's not technically hard, no. However, there's an overhead in running a headless browser, of memory, CPU, time, and bandwidth, so it might cost substantially more.

You don't need a browser for everything, but working out which pages need a headless browser and which you can get away with scraping sounds like an interesting problem.

[ Parent | on HN ]