I think search engines will always be major feats of engineering - the big difference between a spell-checker and a search engine is in the former the problem space rarely evolves.

An English dictionary, for example, only has a a small number of words added each year. A search engine is a different thing entirely: not only are new items added all the time, but the format of these items evolves. Today search engines need to deal with non-textual formats such as videos, images, and audio. This is a very difficult problem.

The fact that there are now APIs and libraries that let developers use things like latent semantic mapping without having to roll their own algorithms and solutions can certainly make writing an intelligent textual search engine a lot easier than it used to be.

But the key difference here is that what we demand from a 'search engine' (and by this I mean Google, or Bing) ten years will be much greater than what we demand now. We'll have new media to search for, new types of queries, new expectations. But a spell checker will always be a spell checker.

At least, that's my opinion. I'm sure someone will disagree!

Sure, I'll disagree :) When you have enough memory bandwidth to compute the relationships of all pages in just a few seconds, then you can re-index the entire corpus according to whatever criteria you want (PageRank or face detection or whatever) for each query. So you have some spiders that add content to the database, and re-index the database for every query.

Good point - I don't know if that solves the problem of getting different forms of media into the system in the first place though. I think as long as people create and new types of media there will be interesting problems to search for them. Searching video based off of text queries, for example, is a much harder problem than simple text search!

That assumes that memory bandwidth grows faster than sqrt(#pages)...