Re:Ask HN: How would you build a search engine in 2019?
by kamutuna on Wednesday October 23, 2019 @05:57AM
(#21332981)
with something like proxycrawl you should be able to build the crawler yourself and start crawling to build a big index. Once you get big enough and you start bringing traffic to the sites, you can then stop using them and create your own user agent and ask sites to whitelist you. It will take some time but can be done
[ Parent
| on HN ]
There are no comments on this comment yet.