The most plausible reason I've seen was (I believe) in 'don't make me think' - The ideal search engine would return one result - exactly what you're looking for. If google isn't consistently return the relevant result in the top 10, it's already failed as a search engine. Research has shown that(iirc) <10% of users go past the first page of results, and over half of clicks are on the top 5 entries.

he also states that an original reason was in order to optimize for speed rather than # of results, since people tend to modify keywords and research far more frequently than visiting the second page.

Disclaimer: I may be misattributing this, don't have the book with me.