I don't know what the big search engines do, but storing the data in a database for the single purpose of parsing it later sounds a bit unecessary. If you are only using the database as storage, the file system will do a better job.