It's a pretty herculean task, moreso no than ever before. On top of the sheer volume of the internet, and the technical challenges of performant search, many sites are hostile to unknown scrapers, and automation.